Normalizing Flows: Exact Likelihood via Invertible Transforms

Why It Matters for Modern Models

Discrete normalizing flows give exact log-likelihoods, unlike GANs, and sample by transforming a base draw without Markov chain sampling, which energy-based models typically use. Continuous flows require numerical ODE integration for sampling and likelihood evaluation.
Continuous normalizing flows are the model class that flow matching trains, so flows are the direct ancestor of flow-matching generators (Lipman et al., 2023).
The requirements of invertibility and a cheap Jacobian determinant explain the shape of flow architectures such as coupling and autoregressive layers.

Questions to Think About

A VAE optimizes a lower bound on log p(x) while a flow computes it exactly; what does the flow pay for that exactness?
How does flow matching avoid Jacobian determinants during training, and why is the flow it learns not, in general, the optimal transport map?

Interactive Demo

The interactive demo for this concept is in the newer notebook.

Open the notebook demo

Core Math

If you prefer intuition first, try the demo in the newer notebook, then come back for the math.

Key Equation

\log p_x(x) = \log p_z(f^{-1}(x)) + \log \left| \det \frac{\partial f^{-1}}{\partial x} \right|

Normalizing flows transform a simple base distribution $p_z(z)$ , usually a standard Gaussian, through an invertible, differentiable function $f$ :

x = f(z), \quad z \sim p_z

The change-of-variables formula gives the exact log-likelihood:

\log p_x(x) = \log p_z(f^{-1}(x)) + \log \left| \det \frac{\partial f^{-1}}{\partial x} \right|

Composition. For $f = f_K \circ \cdots \circ f_1$ with $z_k = f_k(z_{k-1})$ , $z_0 = z$ and $z_K = x$ , the log-determinants add up:

\log p_x(x) = \log p_z(z_0) - \sum_{k=1}^K \log \left| \det \frac{\partial f_k}{\partial z_{k-1}} \right|

The minus sign appears because these are the Jacobians of the forward maps, from noise toward data; it is the same formula as above. Coupling layers make each Jacobian triangular, so its determinant is just the product of the diagonal (Dinh et al., 2017).

Continuous flows. In the limit of many small steps, a flow becomes an ordinary differential equation $dz/dt = v_\theta(z, t)$ , whose log-density changes by $-\operatorname{tr}(\partial v_\theta / \partial z)$ along the path (Chen et al., 2018). Flow matching trains such a velocity field by regression, without integrating the equation during training (Lipman et al., 2023).