DPO turns pairwise preferences into binary cross-entropy on reference-relative log odds, using the KL-regularized RLHF optimum to make the policy itself an implicit reward model.

Direct Preference Optimization

KTO turns binary desirable/undesirable labels into a reference-relative utility loss: push a labeled output's policy/reference log-ratio above or below a KL-derived baseline, with saturating gradients.

Kahneman-Tversky Optimization

A process reward model scores intermediate reasoning steps instead of only terminal answers, giving denser verifier feedback for reranking and search while remaining a learned proxy.

Process Reward Models: Step-Level Verifiers for Reasoning

When an imperfect preference proxy is optimized past its validation regime, policy mass shifts toward reward-model errors; KL, ensembles, LCBs, and monitoring slow this down but do not make the proxy true.

Reward Hacking: Overoptimizing Preference Proxies

RLHF trains a reward model from pairwise preferences, then reweights a reference policy toward high learned reward while a KL penalty limits distribution shift.

RLHF: Reward Modeling + KL-Regularized Policy Optimization

How attention becomes practical at long context: KV caching for decoding, grouped-query attention, and IO-aware kernels like FlashAttention.

Efficient Attention at Scale: KV Cache, GQA & FlashAttention

A fused, tiled attention implementation that avoids materializing the full T x T matrix by using an online softmax, reducing memory traffic and speeding up long-context training/inference.

FlashAttention: IO-Aware Attention

How multi-head, grouped-query, and multi-query attention differ by the number of key/value heads, and why reducing H_kv shrinks the decoding KV cache by H_kv / H_q.

Grouped-Query Attention: Sharing KV Heads

Normalize one token/example vector across features: LayerNorm centers and scales, while RMSNorm keeps RMS-based scaling without mean-centering.

Layer Normalization & RMSNorm

How frontier LLMs stretch context windows: positional extrapolation (RoPE scaling) plus KV cache memory tricks (GQA, paging, quantization, compression).

Long Context Engineering: RoPE Scaling, KV Compression & Memory Optimization

How transformers represent order: sinusoidal encodings, learned embeddings, and relative-position methods like RoPE.

Positional Encoding

Why deep networks can keep useful features alive: each layer learns a correction to the identity instead of rewriting the whole representation from scratch.

Residual Connections & Skip Connections

A positional encoding that rotates queries and keys so attention depends on relative position via phase differences.

Rotary Position Embeddings (RoPE)

The core transformer operation: compute attention weights from query-key dot products, then mix values to copy information across a sequence.

Scaled Dot-Product Attention & Transformer Layers

How Mamba-style state-space models and hybrids trade a growing KV cache for a fixed recurrent state, then use selectivity and local attention to recover useful memory.

SSM Hybrids: Fixed-State Sequence Models for Long Context

Why modern transformer MLPs often use a learned multiplicative gate: one projection proposes a token-local write while another projection controls how much of each channel reaches the residual stream.

SwiGLU: Gated MLP Blocks in Transformers

How text becomes token IDs: segmentation, BPE/unigram tokenizers, and the tradeoffs that shape cost and capability.

Tokenization & Vocabulary Design

Backpropagation applies reverse-mode autodiff to neural networks so one scalar loss can train many parameters.

Backpropagation

Derivatives of composed functions multiply: small changes propagate through a chain of dependencies.

Chain Rule

A computation graph breaks a calculation into nodes so values flow forward and sensitivities flow backward.

Computation Graphs

The derivative is an instantaneous rate of change: the slope you get when a secant line becomes a tangent line.

Derivatives

A function maps inputs to outputs; understanding domain, codomain, graphs, and composition sets up everything in calculus.

Functions

Reverse-mode autodiff computes gradients by sending cotangents backward through a computation graph.

Reverse-Mode Automatic Differentiation

The practical toolkit for making big models cheaper: quantize weights/activations, distill teachers into students, adapt with low-rank updates (LoRA), and use sparsity (MoE).

Efficiency: Quantization, Distillation, LoRA & Sparse MoE

Train a smaller student to mimic a stronger teacher by matching soft probability distributions (often with temperature), transferring 'dark knowledge' beyond hard labels.

Knowledge Distillation: Learning from Teachers

Reduce parameter count by zeroing or removing weights. Unstructured sparsity needs sparse kernels for speed; structured pruning removes whole channels/heads to shrink dense tensor shapes.

Pruning: Removing Unnecessary Weights

Reduce memory and bandwidth by storing weights/activations in low-bit integers (INT8/INT4) with careful scaling to limit accuracy loss.

Quantization: Compressing Models to Integers

Conditional computation: a router picks a few experts per token. You can increase total expert parameters while keeping activated expert FFN compute small, but distributed systems may pay in communication and scheduling.

Sparse Mixture of Experts: Routing, Load Balancing & Expert Parallelism

Denoise noise into data: the diffusion forward process, score matching, and modern sampling via reverse-time dynamics and flow matching.

Diffusion, Score-Based Models & Flow Matching

Learn flow matching as velocity-label regression: straight conditional paths use x_t=(1-t)x0+t x1 but supervise the full target u_t=x1-x0.

Flow Matching & Rectified Flows

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

Normalizing Flows: Tractable Density via Invertible Transforms

Learn the score field grad_x log p(x) without normalizing constants. Denoising score matching turns diffusion training into simple regression on noise.

Score Matching & Score-Based Generative Models

A latent-variable model trained by maximizing an evidence lower bound; the gap is KL(q_phi(z|x) || p_theta(z|x)), so the encoder is learned inference rather than just compression.

Variational Autoencoders

KL divergence is a directional expected log-probability mismatch between distributions; it explains cross-entropy training, variational inference, and KL-regularized alignment.

KL Divergence (Relative Entropy)

The span is everything you can build by mixing vectors; a basis is a non-redundant set that spans the whole space and gives unique coordinates.

Basis and Span

The dot product measures alignment: it connects angles, lengths, and projections, and underlies cosine similarity in ML.

Dot Product

A set of vectors is linearly independent if none can be built from the others; independence is what makes coordinates unique.

Linear Independence

A linear transformation is a function that preserves addition and scaling; in coordinates it becomes a matrix, determined by what it does to a basis.

Linear Transformations

A norm is a notion of vector length; different norms change geometry, distances, and what 'small' means in optimization.

Norms

A vector space is a set of objects you can add and scale, where those operations behave consistently.

Vector Spaces

How inference settings reshape the next-token distribution into actual model behavior: temperature, nucleus sampling, and why decoding is a control knob.

Decoding & Sampling: Temperature, Top-p & Inference-Time Control

A systems mental model for LLM inference: prefill vs decode, TTFT vs TPOT, batching/scheduling, and why KV cache memory dominates.

LLM Serving at Scale: Prefill, Decode & Continuous Batching

Serving MoE turns sparse compute into a scheduling problem: routing skew can create stragglers and token-dispatch communication can bottleneck, motivating scheduling and, in systems such as MegaScale-Infer, disaggregated expert-parallel serving.

MoE Serving & Scheduling: Token Dispatch, All-to-All, Disaggregated Parallelism

How embedding retrieval selects a small weighted evidence set for generation, and why retrieval coverage, answer correctness, citations, and freshness are different quantities.

Retrieval-Augmented Generation: External Memory for Generation

Draft several tokens with a fast model, score draft prefixes with the target model in parallel, then use modified rejection/residual sampling so the sampled distribution matches target-model decoding.

Speculative Decoding: Lossless Multi-Token Generation

How a schema automaton or parser state turns next-token logits into constraint-valid generation by masking invalid continuations, while leaving truth and task success outside the formal guarantee.

Structured Decoding: Token Masks From Schema Automata

Adam is an adaptive optimizer that combines momentum (EMA of gradients) with per-parameter RMS normalization (EMA of squared gradients).

Adam Optimizer

Normalize activations using batch statistics to stabilize deep training (and understand the train vs inference mismatch).

Batch Normalization

How to stop one extreme batch from blowing up training: clip the gradient norm, bound the update, and keep moving in roughly the same direction.

Gradient Clipping & Explosion Prevention

Gradient descent turns local slope information into an iterative update rule for reducing a loss.

Gradient Descent

Replace one-hot labels with soft targets to reduce overconfidence, improve calibration, and sometimes boost generalization.

Label Smoothing & Soft Targets

Schedule shapes that change the scalar learning-rate scale over training, with sourced CLR/range-test and SGDR cosine-restart examples plus caveated warmup/decay teaching patterns.

Learning Rate Schedules: Warmup, Decay & Cycling

How 2D loss slices, Hessian curvature, SAM-style neighborhood loss, and a toy 2/eta stability line expose local sensitivity during optimization.

Loss Landscapes, Sharpness & Flat Minima

Why plain gradient descent is noisy, how momentum smooths and accelerates it, and why the same trick still sits under many modern optimizers.

SGD & Momentum: The Workhorses of Optimization

Why shrinking weights is not the same as adding an L2 penalty inside Adam, and how AdamW restores the intended regularization behavior.

Weight Decay & AdamW: Decoupled Regularization

How to pick weight scales so activations/gradients stay stable: Xavier/Glorot, He/Kaiming, and width-scaling ideas like muP.

Weight Initialization: Xavier, He & muP

Bayesian inference updates a prior distribution over unknowns into a posterior by multiplying by the likelihood and normalizing.

Bayesian Inference

Cross-entropy is the target-weighted surprise of a model distribution; in deep learning it is the bridge from likelihood to a differentiable training loss.

Cross-Entropy

A distribution is the law of a random variable: it says how probability mass or density lands on the values the variable can take.

Distributions

Maximum likelihood fits parameters by making the observed data most probable; for classifiers it becomes negative log-likelihood, cross-entropy, and a KL fit to the empirical distribution.

Maximum Likelihood

Events are subsets of a sample space, and probabilities obey a few axioms; from there you get conditional probability, independence, and Bayes' rule.

Probability Basics

A random variable is a function from outcomes to numbers; its distribution lets you compute expectations, variances, and likelihoods.

Random Variables

How models turn inputs into vectors whose geometry can expose useful factors, contextual meaning, and similarity structure.

Representation Learning & Embedding Geometry

Sparse autoencoders learn a reusable dictionary of feature directions so dense model activations can be explained by a small set of interpretable latent factors.

Sparse Autoencoders: Feature Dictionaries for Mechanistic Interpretability

A limit where wide neural networks behave like kernel methods: the model linearizes around initialization and training becomes kernel regression.

Neural Tangent Kernel (NTK) & Infinite-Width Limits

Test error can peak at the interpolation threshold then fall again as models get larger: why modern overparameterized nets still generalize.

Overparameterization & Generalization (Double Descent)

How filtering, deduplication, source weights, and curriculum turn a token budget into an effective pretraining distribution.

Pretraining Data Mixtures: Designing the Token Distribution

Empirical power laws that predict how loss and capability improve with parameters, data, and compute, and how to choose compute-optimal training runs.

Scaling Laws & Emergent Abilities

Test-time compute spends extra inference budget on sampling, verification, and selection; it helps when the generator can produce good candidates and the verifier ranks them reliably.

Test-Time Compute: Spending Inference Budget on Search

Tree search spends inference budget on partial reasoning prefixes, using local verifier scores, frontier expansion, and max backups to decide which branches deserve more thought.

Tree Search Reasoning: Allocating Inference Budget Across Prefixes

Adversarial min-max ideas appear in adversarial training and some alignment techniques

GANs & Adversarial Divergence Minimization

Frontier LMs heavily rely on superposition: neurons implement many overlapping features

Superposition, Sparse Features & Monosemanticity

Probing is one of the main tools to understand what GPT-like models know and where that knowledge lives

Probing, Linear Classifier Probes & Activation Analysis

These frameworks study GPT-style models, including Llama-3 and Claude-3, by identifying concrete circuits

Transformer Circuits, Induction Heads & Mechanistic Interpretability

PAC/VC theory gives language and bounds for generalization and sample complexity

Theoretical Foundations: PAC Learning, MDL & Information Bottleneck

Moves mechanistic interpretability from "one-off archaeology" to repeatable pipeline—essential for scaling to frontier models

Automated Circuit Discovery: Patching, Attribution & Decomposition at Scale

Inference-time control without fine-tuning—enforce format/length/constraints, style shifts, reduce/induce refusals at deployment

Activation Steering: Feature-Guided Interventions for Inference-Time Control

CLIP-style encoders are front-end for multimodal LLMs (how images become tokens) and conditioning backbone for text-to-image diffusion

Multimodal Foundations: Vision Encoders, Contrastive Learning & Cross-Attention Fusion

Every modern neural network is trained via backprop—it is the fundamental algorithm enabling gradient-based learning

Backpropagation & Automatic Differentiation

ICL is arguably THE signature capability of large language models—task adaptation without fine-tuning

In-Context Learning: Learning Without Weight Updates

Wasserstein distance explains why WGANs are more stable than vanilla GANs—it provides gradients even when distributions do not overlap

Optimal Transport & Wasserstein Distance

PPO is THE algorithm behind RLHF—understanding it explains how preference data becomes model behavior

PPO: Proximal Policy Optimization

CFG is why Stable Diffusion/DALL-E/Midjourney produce high-quality, on-prompt images

Classifier-Free Guidance in Diffusion

Reveals that neural networks are "right for the wrong reasons"—decision boundaries are brittle

Adversarial Examples & Robustness

Challenges classical learning theory: generalization can happen AFTER interpolation, not before

Grokking: Delayed Generalization

First simple tool for "seeing inside" transformers—reveals layer-by-layer computation

Logit Lens: Probing Intermediate Representations

Powers CLIP, which enabled zero-shot image classification and text-to-image (via embeddings)

Contrastive Learning & InfoNCE

Large models REQUIRE distributed training—no single GPU can hold GPT-4 class models

Distributed Training: Data, Tensor & Pipeline Parallelism

Beam search is standard for machine translation and structured generation tasks

Beam Search & Structured Decoding

Classic regularizer that prevents co-adaptation of features

Dropout: Stochastic Regularization

Unifies discriminative and generative modeling: classifier logits ARE energy differences

Energy-Based Models & Score Functions

Fisher gives the natural metric on probability distributions—not Euclidean distance in parameters

Fisher Information & Information Geometry

Natural gradient is coordinate-invariant—it gives the same update regardless of parameterization

Natural Gradient & Riemannian Optimization

Foundation of modern NLP: BERT, GPT, LLaMA all use self-supervised pretraining

Self-Supervised Learning: Labels from Structure

Modern deep networks are often overconfident—90% confidence doesn't mean 90% accuracy

Calibration & Temperature Scaling

How Claude is trained—principles replace pure human preference labeling

Constitutional AI: Principles-Based Alignment

Explains why different geometries suit different problems—simplex needs KL, not Euclidean

Bregman Divergence & Mirror Descent

Unifies SVMs, Gaussian processes, kernel regression, and NTK under one framework

Find an idea, then follow its edges.

Checking saved route