LLM Curriculum: 7 Levels of Mastery
From foundational principles to subword tokens, multi-head attention, backpropagation, LoRA, and agentic architectures. Each lesson features interactive check quizzes and direct links to simulation sandboxes.
Level 1 — Foundations
0 / 5 CompletedCore concepts, architectures, training vs inference, and scaling laws.
What Is an LLM?
Understand what a Large Language Model actually is, how it processes sequences, and why it is fundamentally an autoregressive token predictor.
What Makes an LLM Large?
Explore the three dimensions of scale: parameters, dataset volume (tokens), and compute (FLOPs).
AI vs ML vs Deep Learning vs LLM
Disentangle the hierarchy of Artificial Intelligence, Machine Learning, Deep Learning, and Large Language Models.
Training vs Inference
Understand the fundamental divide between the compute-heavy training phase and the generation-focused inference phase.
Parameters vs Hyperparameters
Learn the distinction between parameters learned by the model and hyperparameters set by researchers.
Level 2 — Text & Representations
0 / 5 CompletedFrom human characters to subwords, token IDs, vocabularies, and vector embeddings.
What Is Tokenization?
Learn how neural networks convert unstructured text strings into discrete subword integer tokens.
Tokens vs Words
Understand the rule of thumb for token-to-word ratios and why code and non-English text consume more tokens.
Token IDs and Vocabulary
Follow the conversion from string tokens to their integer IDs in the model vocabulary table.
Vocabulary Design & Trade-offs
Learn why vocabulary size is a critical architectural trade-off between sequence length and parameter memory.
Vector Embeddings: Meaning in High Dimensions
Discover how integer token IDs are transformed into continuous vector embeddings that capture semantic relationships.
Level 3 — Transformers & Attention
0 / 10 CompletedSelf-attention, Query-Key-Value mechanics, multi-head projection, and residual blocks.
Why Transformers? (The Breakthrough of 2017)
Learn why the 2017 paper "Attention Is All You Need" replaced Recurrent Neural Networks (RNNs) and LSTMs.
Positional Information: Why Order Matters
Understand how transformers inject word order information using sinusoidal encodings or Rotary Position Embeddings (RoPE).
Self-Attention: The Engine of Context
Visualize how each token in a sentence looks at all other tokens to update its own contextual representation.
Query, Key, Value (Q, K, V) Explained
Understand the database retrieval analogy: Query = what I am looking for, Key = what I offer, Value = the actual content.
Attention Scores and Scaling
Follow the math behind the scaled dot-product attention equation step by step.
Multi-Head Attention: Multiple Perspectives
Discover why models split attention into multiple parallel heads to track grammar, facts, and co-references simultaneously.
Feed-Forward Networks (FFN): The Knowledge Store
Learn how multi-layer perceptron blocks inside transformer layers store factual associations.
Residual Connections: Enabling Deep Networks
Understand how skip connections (Add & Norm) allow gradients to flow unimpeded through hundreds of layers.
Layer Normalization & RMSNorm
Discover how normalizing activation scales keeps numerical stability during billion-parameter training.
The Complete Transformer Decoder Block
Assemble all pieces: RMSNorm, Multi-Head Attention, Residuals, FFN, and Output Projections.
Level 4 — Generation & Sampling
0 / 8 CompletedLogits, Softmax normalization, temperature scaling, Top-K, Top-P, and autoregression.
Logits: Raw Unnormalized Scores
Understand the unnormalized real numbers produced by the model before probability conversion.
Softmax: Turning Logits into Probabilities
Master the Softmax equation: exponentiation of logits followed by sum normalization.
Next-Token Prediction & Autoregression
See the complete generation loop: predict, sample, append, repeat.
Greedy Decoding vs Random Sampling
Learn why always choosing the highest probability token leads to repetitive and robotic text.
Temperature: Controlling Randomness and Creativity
Learn how the temperature parameter scales logits before Softmax to sharpen or flatten the distribution.
Top-K Sampling: Truncating the Long Tail
Discover how Top-K filtering prunes unlikely tokens to eliminate bizarre hallucinations.
Top-P (Nucleus) Sampling: Adaptive Filtering
Learn why nucleus sampling is superior to fixed Top-K when probability distributions vary in confidence.
The Production Sampling Pipeline
Combine Temperature, Top-K, and Top-P into the unified production generation pipeline.
Level 5 — Training & Optimization
0 / 4 CompletedCross-entropy loss, backpropagation, gradients, learning rate schedules, and batches.
Pretraining Datasets: Curating Trillions of Tokens
Explore where LLM data comes from: Common Crawl, Wikipedia, GitHub, arXiv, and filtering pipelines.
Cross-Entropy Loss: Measuring Prediction Error
Learn how Cross-Entropy Loss mathematically measures the gap between the predicted probability and the true target token.
Backpropagation: Tracing Gradients in Reverse
See how the chain rule of calculus computes gradients for billions of weights from loss.
Gradient Descent & Optimizers (AdamW)
Learn how AdamW updates model weights using gradients, learning rates, and momentum.
Level 6 — Fine-Tuning & Adaptation
0 / 3 CompletedInstruction tuning, RLHF, LoRA low-rank decomposition, PEFT, and RAG trade-offs.
What Is Fine-Tuning? (From Base Model to Assistant)
Discover the transformation from a raw text completer to a helpful instruction-following AI assistant.
LoRA & PEFT: Parameter-Efficient Fine-Tuning
Learn how Low-Rank Adaptation (LoRA) allows fine-tuning on consumer GPUs by decomposing weight updates.
Fine-Tuning vs RAG: The Architectural Choice
Understand when to retrieve documents (RAG) vs when to modify model behavior and tone (Fine-Tuning).
Level 7 — Applications & AI Systems
0 / 3 CompletedVector databases, RAG retrieval pipelines, tool calling, autonomous agents, and system design.
RAG: Retrieval-Augmented Generation Architecture
Learn the complete end-to-end RAG pipeline from document ingestion to vector search and augmented synthesis.
Tool Calling & Function Execution
Understand how models output structured function calls (JSON) to interact with external APIs and calculators.
Modern LLM Application Architecture
The blueprint of enterprise AI: UI, Prompt Firewalls, Semantic Caching, RAG, Routing, and Observability.