You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: agents/avatars/generate_all_avatars.py
+2Lines changed: 2 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -50,6 +50,8 @@
50
50
("36-illustrator", "a French female artist named Iris with paint-stained apron, wild colorful hair, surrounded by floating illustrations and paintbrushes"),
51
51
("37-epigraph-writer", "a British male poet named Quentin with a quill pen, sitting in a leather armchair, chuckling at his own witty writing"),
52
52
("38-application-example", "a Nigerian female business consultant named Nadia with a blazer, holding case study folders, standing in front of industry charts"),
53
+
("39-fun-injector", "a charismatic mixed-race male comedian named Ziggy with wild curly hair, colorful suspenders, holding a rubber chicken in one hand and a textbook in the other, winking"),
54
+
("40-bibliography", "a distinguished older white female librarian named Margot with silver hair in a French twist, reading glasses on a chain, surrounded by floating hyperlinked book spines and glowing citation marks"),
53
55
]
54
56
55
57
BASE_STYLE="Digital art avatar portrait, Kurzgesagt-inspired minimal cartoon style, clean vector lines, vibrant flat colors, circular composition, gradient background, friendly and professional, no text"
framework that powers most modern LLM research and development. Finally, we introduce
206
-
reinforcement learning, the paradigm that makes LLMs helpful through RLHF.
211
+
reinforcement learning, the paradigm that makes LLMs helpful through RLHF, a topic
212
+
explored in full in <ahref="../../part-4-training-adapting/module-16-alignment-rlhf-dpo/index.html">Module 16: Alignment, RLHF & DPO</a>.
207
213
</p>
208
214
</div>
209
215
210
216
<divclass="callout practical-example">
211
217
<h4>Practical Example: The Shortcut That Backfired</h4>
212
218
<p><strong>Who:</strong> A junior ML engineer at a mid-sized e-commerce company (150 employees)</p>
213
-
<p><strong>Situation:</strong> Tasked with building a product recommendation model, she jumped straight into fine-tuning a large pretrained transformer without reviewing ML fundamentals.</p>
219
+
<p><strong>Situation:</strong> Tasked with building a product recommendation model, she jumped straight into <ahref="../../part-4-training-adapting/module-13-fine-tuning-fundamentals/index.html">fine-tuning</a> a large pretrained transformer without reviewing ML fundamentals.</p>
214
220
<p><strong>Problem:</strong> The model achieved 94% accuracy on the training set but only 61% on production data. She had no idea why.</p>
215
221
<p><strong>Dilemma:</strong> Ship the underperforming model to meet a deadline, or pause and diagnose the gap between training and production performance.</p>
216
222
<p><strong>Decision:</strong> She convinced her manager to allow a one-week pause to revisit overfitting diagnostics and regularization basics.</p>
@@ -222,11 +228,11 @@ <h4>Practical Example: The Shortcut That Backfired</h4>
222
228
<divclass="objectives">
223
229
<h3>Learning Objectives</h3>
224
230
<ul>
225
-
<li>Explain supervised learning, loss functions, and gradient descent intuitively and mathematically</li>
231
+
<li>Explain supervised learning, loss functions, and gradient descent intuitively and mathematically (these resurface in <ahref="../../part-4-training-adapting/module-13-fine-tuning-fundamentals/index.html">Module 13: Fine-Tuning Fundamentals</a>)</li>
226
232
<li>Describe the bias-variance tradeoff and apply regularization techniques</li>
227
233
<li>Build and train neural networks, understanding backpropagation at a mechanical level</li>
228
-
<li>Write complete PyTorch training loops with custom datasets and GPU acceleration</li>
229
-
<li>Explain the RL framework (agent, policy, reward) and its connection to LLM training</li>
234
+
<li>Write complete PyTorch training loops with custom datasets and GPU acceleration, skills applied throughout <ahref="../../part-4-training-adapting/module-14-peft/index.html">Part 4: Training & Adapting</a></li>
235
+
<li>Explain the RL framework (agent, policy, reward) and its connection to LLM training via <ahref="../../part-4-training-adapting/module-16-alignment-rlhf-dpo/index.html">RLHF and DPO (Module 16)</a></li>
Agent-environment loop, policy, value functions, Bellman equation, policy gradients,
298
-
PPO, and how RL connects to LLM training (RLHF).
308
+
PPO, and how RL connects to LLM training (RLHF). See also
309
+
<ahref="../../part-6-agents-applications/module-21-ai-agents/index.html" style="color: var(--text-light);">AI Agents (Module 21)</a> for how RL-trained policies power autonomous systems.
299
310
</span>
300
311
</a>
301
312
</li>
@@ -307,7 +318,7 @@ <h4>Practical Example: Choosing PyTorch Under Pressure</h4>
307
318
<p><strong>Situation:</strong> The team needed to prototype a fraud detection model and had two weeks before a board demo. Half the team knew TensorFlow; half knew PyTorch.</p>
308
319
<p><strong>Problem:</strong> Framework fragmentation meant code reviews took twice as long, and shared utilities were incompatible across the two codebases.</p>
309
320
<p><strong>Dilemma:</strong> Standardize on one framework (slowing down half the team initially) or maintain both and accept long-term maintenance costs.</p>
310
-
<p><strong>Decision:</strong> She standardized on PyTorch, citing its dominance in research papers the team needed to reproduce and its more intuitive debugging with eager execution.</p>
321
+
<p><strong>Decision:</strong> She standardized on PyTorch, citing its dominance in research papers the team needed to reproduce (see <ahref="../../part-2-understanding-llms/module-07-modern-llm-landscape/index.html">Module 7: Modern LLM Landscape</a>) and its more intuitive debugging with eager execution.</p>
311
322
<p><strong>How:</strong> She allocated three days for a "PyTorch bootcamp" covering tensors, autograd, and training loops, then paired TensorFlow veterans with PyTorch users for the remaining sprint.</p>
312
323
<p><strong>Result:</strong> The team delivered the demo on time. Over the following quarter, code review turnaround dropped by 35%, and onboarding new hires became two days faster.</p>
313
324
<p><strong>Lesson:</strong><strong>Short-term slowdowns from standardizing tools pay compound dividends in team velocity.</strong></p>
Understanding this progression is essential: the entire history of NLP is a quest for
220
223
better representations of meaning, and each technique you learn here is a building block
221
224
for everything that follows.
@@ -231,19 +234,19 @@ <h4>Practical Example: When Bag-of-Words Met Sarcasm</h4>
231
234
<p><strong>Decision:</strong> They replaced TF-IDF with pre-trained Word2Vec embeddings, reasoning that dense vectors would capture tonal patterns that sparse counts could not.</p>
232
235
<p><strong>How:</strong> They fine-tuned 300-dimensional Word2Vec vectors on their review corpus, then trained a simple logistic regression on the averaged embeddings.</p>
233
236
<p><strong>Result:</strong> Sarcasm misclassification dropped from 23% to 9%. Customer response SLA compliance rose from 74% to 91% within two months.</p>
234
-
<p><strong>Lesson:</strong><strong>Representation quality is the ceiling for downstream model performance. Upgrading from sparse to dense embeddings often delivers bigger gains than swapping classifiers.</strong></p>
237
+
<p><strong>Lesson:</strong><strong>Representation quality is the ceiling for downstream model performance. Upgrading from sparse to dense embeddings often delivers bigger gains than swapping classifiers.</strong> For production-scale embedding pipelines, see <ahref="../../part-5-retrieval-conversation/module-18-embeddings-vector-db/index.html">Module 18: Embeddings & Vector Databases</a>.</p>
235
238
</div>
236
239
237
240
<divclass="objectives">
238
241
<h3>Learning Objectives</h3>
239
242
<ul>
240
-
<li>Explain the evolution of NLP from rule-based systems to modern LLMs and <em>why</em> each transition happened</li>
243
+
<li>Explain the evolution of NLP from rule-based systems to modern LLMs (surveyed in <ahref="../../part-2-understanding-llms/module-07-modern-llm-landscape/index.html">Module 7: Modern LLM Landscape</a>) and <em>why</em> each transition happened</li>
241
244
<li>Build a complete text preprocessing pipeline using spaCy and NLTK</li>
242
245
<li>Implement and compare Bag-of-Words, TF-IDF, and one-hot encoding, and articulate their limitations</li>
243
246
<li>Explain how Word2Vec, GloVe, and FastText create dense word representations and <em>why</em> they work</li>
244
-
<li>Train a Word2Vec model from scratch and explore word analogies</li>
247
+
<li>Train a Word2Vec model from scratch (using techniques from <ahref="../module-00-ml-pytorch-foundations/index.html">Module 0</a>) and explore word analogies</li>
245
248
<li>Explain why static embeddings fail for polysemous words and how ELMo introduced contextual embeddings</li>
246
-
<li>Articulate the "big picture" of how text representation evolved toward transformers and LLMs</li>
249
+
<li>Articulate the "big picture" of how text representation evolved toward transformers and LLMs (explored further in <ahref="../../part-2-understanding-llms/module-06-pretraining-scaling-laws/index.html">Module 6: Pretraining & Scaling Laws</a>)</li>
The preprocessing pipeline (tokenization, lemmatization, stop words),
278
-
Bag-of-Words, TF-IDF, one-hot encoding, n-grams, and their limitations.
281
+
Bag-of-Words, TF-IDF, one-hot encoding, n-grams, and their limitations. Tokenization is explored in depth in <ahref="../module-02-tokenization-subword-models/index.html">Module 2</a>.
The distributional hypothesis, Skip-gram, negative sampling, word analogies,
291
-
cosine similarity, GloVe co-occurrence matrices, FastText subwords, and t-SNE visualization.
294
+
cosine similarity, GloVe co-occurrence matrices, FastText subwords, and t-SNE visualization. These embeddings underpin the retrieval systems in <ahref="../../part-5-retrieval-conversation/module-19-rag/index.html">Module 19: RAG</a>.
The polysemy problem, ELMo's bidirectional LSTMs, layer-wise representations,
303
-
the pre-train/fine-tune paradigm, and how this led to transformers and LLMs.
306
+
the pre-train/fine-tune paradigm, and how this led to transformers and LLMs. Continues into <ahref="../module-03-sequence-models-attention/index.html">Module 3: Sequence Models & Attention</a>.
304
307
</span>
305
308
</a>
306
309
</li>
@@ -315,7 +318,7 @@ <h4>Practical Example: The Polysemy Trap in Medical Search</h4>
315
318
<p><strong>Decision:</strong> They adopted ELMo embeddings, which generate a different vector for "cold" depending on surrounding words.</p>
316
319
<p><strong>How:</strong> They replaced their GloVe-based query encoder with a fine-tuned ELMo model, keeping the same retrieval pipeline and re-indexing their 50K article corpus.</p>
317
320
<p><strong>Result:</strong> Relevant-result click-through rate jumped from 34% to 52%. Polysemy-related support tickets dropped by 60% in the first quarter.</p>
318
-
<p><strong>Lesson:</strong><strong>When your domain has words with multiple critical meanings, static embeddings will silently degrade search quality. Contextual embeddings are not a luxury; they are a necessity.</strong></p>
321
+
<p><strong>Lesson:</strong><strong>When your domain has words with multiple critical meanings, static embeddings will silently degrade search quality. Contextual embeddings are not a luxury; they are a necessity.</strong> Modern transformer-based approaches (see <ahref="../module-04-transformer-architecture/index.html">Module 4: Transformer Architecture</a>) take this idea even further with full self-attention.</p>
<p><strong>Decision:</strong> They built a proper pipeline: HTML stripping, lowercasing, lemmatization, stop-word removal, and boilerplate detection.</p>
328
331
<p><strong>How:</strong> Using spaCy, they wrote a five-stage pipeline that reduced the average document from 1,200 tokens to 180 meaningful lemmas, then retrained the same TF-IDF model.</p>
329
332
<p><strong>Result:</strong> User engagement with recommendations rose 41%. The "thumbs-down" rate on suggestions fell from 38% to 14%, all without changing the underlying algorithm.</p>
330
-
<p><strong>Lesson:</strong><strong>Before reaching for a more powerful model, audit your preprocessing. Clean input often outperforms a fancy model fed noisy data.</strong></p>
333
+
<p><strong>Lesson:</strong><strong>Before reaching for a more powerful model, audit your preprocessing. Clean input often outperforms a fancy model fed noisy data.</strong> For a deeper look at subword tokenization techniques that modern LLMs use, see <ahref="../module-02-tokenization-subword-models/index.html">Module 2: Tokenization & Subword Models</a>.</p>
<li>Module 00: ML & PyTorch Foundations (especially sections on neural networks and gradient descent)</li>
343
+
<li><ahref="../module-00-ml-pytorch-foundations/index.html">Module 00: ML & PyTorch Foundations</a> (especially sections on neural networks and gradient descent)</li>
341
344
<li>Python proficiency (functions, classes, list comprehensions)</li>
342
345
<li>Basic linear algebra: vectors, dot products, matrix multiplication</li>
343
346
<li>Familiarity with NumPy and basic scikit-learn usage</li>
0 commit comments