Generative Physics
Diffusion, flow matching, and the geometry of generation
Generation as Gradient Flow
How do we generate samples from a complex distribution? The insight of modern generative models: define a continuous path from noise to data, then learn to traverse it.
Whether diffusion, flow matching, or score-based methods, the core idea is the same: generation is physics, with data acting as an attractor.
Score Functions
The score of a distribution is the gradient of its log-probability:
s(x) = ∇x log p(x)
The score points toward regions of higher probability. If we follow the score, we flow uphill in probability space — toward the modes of the distribution.
The visualization shows the score field for a mixture of two Gaussians. Arrows point toward the two modes.
Diffusion Models
Diffusion models define a forward process that gradually adds noise to data, then learn to reverse it:
dx = [f(x,t) - g(t)²∇log pt(x)]dt + g(t)dw̄
The key insight: learning the score at each noise level is enough to reverse the diffusion process and generate samples.
Flow Matching
Flow matching takes a more direct approach: learn the velocity field that transports noise to data along straight paths.
xt = (1-t)x0 + tx1
v(xt, t) = x1 - x0
Rectified flows extend this by iteratively straightening the learned paths, enabling few-step generation.
The Big Picture
All these methods share a common structure:
- → Define a path between noise and data
- → Learn the vector field along that path
- → Integrate to generate samples
The differences lie in the choice of path and the training objective. Score matching targets the gradient of log probability; flow matching directly regresses on velocities.