Domain
Generative Models
How generative models are built and trained: latent-variable models and the ELBO, normalizing flows, diffusion and score-based models, and flow matching.
Preview the recommended route
Start here. Make one prediction, then note what carries over to the next notebook.
x_tε_θdenoisex_0LearnerWhich notebook should I read first?
Open the first notebookRead the first notebook, make one prediction in its demo, then go on to the next.
Recommended route
Start here, then follow the prerequisites forward.
Read these in order: each notebook comes after the notebooks in this domain that it builds on, and the label beside it gives its level. A prerequisite with no page here yet is marked as outside reading. A notebook is left out of the route only when a prerequisite has neither a page here nor a named outside reading; the full list below marks it.
- 01Variational Autoencoders
A latent-variable model trained by maximizing the evidence lower bound (ELBO). The bound falls short of the log-likelihood by the KL divergence from the encoder to the true posterior, so the encoder learns approximate inference, not just compression.
Advanced20 mincodedemoafter Maximum Likelihood, Bayesian Inference, KL Divergence (Relative Entropy) - 02Diffusion, Score-Based Models & Flow Matching
How diffusion models generate: a fixed forward process adds Gaussian noise, a network learns to predict that noise, and a sampler runs the process in reverse.
Advanced25 mincodedemoafter Maximum Likelihood, Variational AutoencodersDiffusion, Score-Based Models & Flow Matching builds directly on Variational Autoencoders.
- 03Normalizing Flows: Tractable Density via Invertible Transforms
Invertible maps that turn a simple density into a richer one: exact log-likelihood from the change-of-variables formula, and layer designs that keep the Jacobian determinant cheap.
Advanced22 mincodedemoafter Maximum Likelihood, Variational Autoencoders - 04Flow Matching & Rectified Flows
Flow matching trains a velocity field by regression: on the straight path x_t = (1-t)x0 + t x1 the label is x1 - x0, and the learned field is the average label E[x1 - x0 | x_t].
Advanced34 mincodedemoafter Diffusion, Score-Based Models & Flow Matching, Normalizing Flows: Tractable Density via Invertible TransformsFlow Matching & Rectified Flows builds directly on Normalizing Flows.
- 05Score Matching & Score-Based Generative Models
Learn the score, the gradient of log p(x), without the normalizing constant. Denoising score matching turns this into regression on the added noise, which is how diffusion models are trained.
Advanced21 mincodedemoafter Maximum Likelihood, Diffusion, Score-Based Models & Flow Matching
All notebooks
Every published notebook in this domain.
Variational Autoencoders
A latent-variable model trained by maximizing the evidence lower bound (ELBO). The bound falls short of the log-likelihood by the KL divergence from the encoder to the true posterior, so the encoder learns approximate inference, not just compression.
Diffusion, Score-Based Models & Flow Matching
How diffusion models generate: a fixed forward process adds Gaussian noise, a network learns to predict that noise, and a sampler runs the process in reverse.
Normalizing Flows: Tractable Density via Invertible Transforms
Invertible maps that turn a simple density into a richer one: exact log-likelihood from the change-of-variables formula, and layer designs that keep the Jacobian determinant cheap.
Flow Matching & Rectified Flows
Flow matching trains a velocity field by regression: on the straight path x_t = (1-t)x0 + t x1 the label is x1 - x0, and the learned field is the average label E[x1 - x0 | x_t].
Score Matching & Score-Based Generative Models
Learn the score, the gradient of log p(x), without the normalizing constant. Denoising score matching turns this into regression on the added noise, which is how diffusion models are trained.