Generative Models

Normalizing Flows: Tractable Density via Invertible Transforms

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

status: publishedimportance: importantdifficulty 4/5math: undergraduateread: 18mlive demo

Back to Generative Models Next: Diffusion, Score-Based Models & Flow Matching

Editorial generative-model illustration of an invertible grid warp with local volume-correction patches.

Concept Structure

Normalizing Flows: Tractable Density via Invertible Transforms

01Intuition

Start with the picture, metaphor, or geometric mechanism.

02Math

Make the objects explicit and connect them with notation.

03Code

Mirror the equations with runnable implementation details.

04Interactive Demo

Manipulate the mechanism and watch the idea respond.

2prerequisites

2next concepts

2related links

Learning map

Normalizing Flows: Tractable Density via Invertible Transforms

BeforeMaximum LikelihoodNow4/4 sections readyTryManipulate one control and predict the visible change.NextDiffusion, Score-Based Models & Flow Matching

Object flow

4/4 sections readyAsk about this Research room

ConceptNormalizing Flows: Tractable Density via Invertible TransformsGenerative Models EquationNormalizing Flows: Tractable Density via Invertible Transforms equati...Exact equation object CodeNormalizing Flows: Tractable Density via Invertible Transforms code w...Exact code witness DemoNormalizing Flows: Tractable Density via Invertible Transforms intera...Visualization object ClaimNormalizing flows compose invertible maps to turn a simple initial de...Exact claim check SourceVariational Inference with Normalizing FlowsExact source object

ConceptNormalizing Flows: Tractable Density via Invertible TransformsGenerative Models

1 source attachedLocal snapshot ready

concept:generative-models/normalizing-flows

Codewitness nearby Predictbefore reveal Roomobject handoff

Conceptual Bridge

What should feel connected as you move through this page.

Carry inMaximum Likelihood

Bring the mental model from Maximum Likelihood; this page will reuse it instead of restarting from zero.

Work hereNormalizing Flows: Tractable Density via Invertible Transforms

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

Carry outDiffusion, Score-Based Models & Flow Matching

The next edge should feel earned: use the demo prediction here before following Diffusion, Score-Based Models & Flow Matching.

Test the linkManipulate one control and predict the visible change.Then continue to Diffusion, Score-Based Models & Flow Matching

01IntuitionStart with the picture, metaphor, or geometric mechanism.02MathMake the objects explicit and connect them with notation.03CodeMirror the equations with runnable implementation details.04Interactive DemoManipulate the mechanism and watch the idea respond.

Intuition

Build the mental picture first so the rest of the page has something to attach to.

Section prompt

Normalizing flows are "reversible warps."

Start from a simple base distribution (like a standard Gaussian). Then apply an invertible transformation that bends and stretches space into a richer model distribution.

Because the transform is invertible, you get:

Exact sampling: sample $z$ from the base, push forward to $x=f(z)$ .
Tractable log-likelihood under the flow model: map $x$ back to $z=f^{-1}(x)$ and account for how volumes change.

The price is architectural: you must design $f$ so it is invertible and the needed map direction plus Jacobian determinant are cheap to compute.

Math

Translate the story into symbols, assumptions, and a derivation you can inspect.

Section prompt

Equation 1p_x(x)=p_z\big(f^{-1}(x)\big)\left|\det\frac{\partial f^{-1}}{\partial x}\right|,Equation 2\log p_x(x)=\log p_z\big(f^{-1}(x)\big)+\log\left|\det\frac{\partial f^{-1}}{\partial x}\right|.

Change of variables

Let $x=f(z)$ where $f$ is invertible and $z\sim p_z$ . Then:

p_x(x)=p_z\big(f^{-1}(x)\big)\left|\det\frac{\partial f^{-1}}{\partial x}\right|,

so:

\log p_x(x)=\log p_z\big(f^{-1}(x)\big)+\log\left|\det\frac{\partial f^{-1}}{\partial x}\right|.

The determinant term is the volume correction. If the inverse map squeezes a region of data space into a smaller region of latent space, density must increase; if it expands the region, density must decrease. This is the tractable-density advantage of flows, but also the constraint: a flexible transform is useful for likelihood work only when the needed map direction and log-determinant stay tractable.

Composition of flows

If $f=f_K\circ\cdots\circ f_1$ maps latent variables forward, with $z_k=f_k(z_{k-1})$ and $z_K=x$ , then the inverse-Jacobian log-determinants add as a negative sum of forward log-determinants:

\log p_x(x)=\log p_z(z_0)-\sum_{k=1}^K\log\left|\det\frac{\partial f_k}{\partial z_{k-1}}\right|.

This follows from the determinant of a product of Jacobians. Later flow architectures often engineer triangular or otherwise cheap Jacobians, which can turn a high-dimensional determinant into a sum of simpler per-dimension terms.

Code

Keep the implementation aligned with the notation so the algorithm is legible.

Section prompt

Code witness 1import numpy as np # A tiny 2D linear flow: x = A z + b, with z ~ N(0, I) A = np.array([[1.2,...python

import numpy as np

# A tiny 2D linear flow: x = A z + b, with z ~ N(0, I)
A = np.array([[1.2, 0.3], [0.1, 0.9]])
b = np.array([0.5, -0.2])
Ainv = np.linalg.inv(A)

def log_pz(z):
    d = z.shape[0]
    return -0.5 * float(z @ z) - 0.5 * d * np.log(2 * np.pi)

x = np.array([1.0, 0.0])
z = Ainv @ (x - b)
log_px = log_pz(z) + np.log(abs(np.linalg.det(Ainv)))

print("det(A):", round(float(np.linalg.det(A)), 3))
print("z:", np.round(z, 3))
print("log p_x(x):", round(float(log_px), 3))

Interactive Demo

Use direct manipulation to connect the explanation to a moving system.

Section prompt

The demo below asks you to predict the density effect before revealing the Jacobian correction. The key invariant is that exact likelihood comes from two terms: the base density at the inverse point and the log-volume correction from the inverse Jacobian.

Live Concept Demo

Explore Normalizing Flows: Tractable Density via Invertible Transforms

The stage is code-native and interactive. Use it to test the explanation against the mechanism.

difficulty 4/5undergraduatecode-aligned

Demo Prediction Checkpoint

Manipulate one control and predict the visible change.

Commit to what Normalizing Flows: Tractable Density via Invertible Transforms should make visible before reading the result.

After The First Pass

Turn the concept into an inspected object.

Once the invariant is visible in the intuition, math, code, and demo, use these panels to inspect the mechanism visually, check source support, practice the idea, and attach a grounded research question.

Mechanism Storyboard

See the idea move before the page explains it

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

Prediction open01 / Intuition

Prediction lens

Start with the picture, metaphor, or geometric mechanism.

Commit first

Before reading further, choose the kind of change Normalizing Flows: Tractable Density via Invertible Transforms should make visible.

Visual Inquiry

Make the image answer a mathematical question

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

4/4 stages readyLive demo connected

Visual cueWhich visible object should carry the first intuition?

Inspection depth2/4

Prediction

Which visible object should carry the first intuition?

Commit first

Pick the cue that should make Normalizing Flows: Tractable Density via Invertible Transforms easier to reason about before the page gives the answer.

Source Grounding

Canonical references for the mechanism on this page.

paper · 2015Variational Inference with Normalizing FlowsRezende and Mohamed

Grounds the change-of-variables density transformation and the term normalizing flow.

Open source

Claim Review

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

Status1 substantive review recorded

Claims without a substantive review badge still need exact source-support review.

Sources1 reference

rezende-2015-normalizing-flows

Witnesses4 local objects

Use equation, code, and demo objects to check whether the source support is operational.

Substantively reviewedNormalizing flows compose invertible maps to turn a simple initial density into a richer one; for a tractable finite flow, the log density at a point is computed exactly by evaluating the base log density at the corresponding preimage and adding the inverse-Jacobian log determinant.Claim metadata: source checked

Rezende and Mohamed define a normalizing flow as transforming a probability density through a sequence of invertible mappings, state that repeated change-of-variables moves an initial density through that sequence, and derive finite-flow density updates using Jacobian determinants. The page equations, code, and demo instantiate inverse-form base-density lookup plus log-volume correction.

Sources: Variational Inference with Normalizing FlowsRezende and Mohamed frame flows for variational approximate posteriors and density transformation; they do not by themselves justify broad claims about modern data modeling, sampling quality, stability, or universal exact-likelihood practicality.A bounded review summary is present; still check caveats and exact source scope.

Rezende and Mohamed support normalizing flows as invertible-map sequences that transform a simple initial density into a richer one, derive finite change-of-variables density updates using inverse/forward Jacobian determinants, and give the composed log-density formula as base log density minus summed forward logdet terms. Local math, affine code, and demo instantiate preimage lookup plus inverse-Jacobian log-volume correction.

Reviewer: codex+oracle+codex-5.3; reviewed 2026-05-08

source-span-rezende-2015-normalizing-flows math-object-1 math-object-2 code-witness-1 interactive-demo

Source support candidates

paper 2015Variational Inference with Normalizing Flows

Grounds the change-of-variables density transformation and the term normalizing flow.

Mechanism witnesses

Equation 1

p_x(x)=p_z\big(f^{-1}(x)\big)\left|\det\frac{\partial f^{-1}}{\partial x}\right|,

Equation 2

\log p_x(x)=\log p_z\big(f^{-1}(x)\big)+\log\left|\det\frac{\partial f^{-1}}{\partial x}\right|.

Code witness 1import numpy as np # A tiny 2D linear flow: x = A z + b, with z ~ N(0, I) A = np.array([[1.2,...Demo stateLive mechanism probe

Practice Loop

Try the idea before it explains itself

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

Readiness0/3 checks ready

Predict

Before touching the demo, predict one visible change that should happen in Normalizing Flows: Tractable Density via Invertible Transforms.

Hint 1

Reveal when your model needs a nudge.

Hint 2

Reveal when your model needs a nudge.

Hint 3

Reveal when your model needs a nudge.

Your answer canvas

Local checks

Claim

A concrete answer is on the canvas.

Mechanism

The answer names why the claim should hold.

Bridge

It touches the page context or a neighboring idea.

Misconception check

Object research drawerClose

ConceptNormalizing Flows: Tractable Density via Invertible TransformsGenerative Models

Code witness comparisonNormalizing Flows: Tractable Density via Invertible Transforms code witness 1A = np.array([[1.2, 0.3], [0.1, 0.9]])Prediction before revealNormalizing Flows: Tractable Density via Invertible Transforms interactive demoManipulate one control and predict the visible change.

Grounded room questionWhat is the smallest example that makes Normalizing Flows: Tractable Density via Invertible Transforms click without losing the math?Local snapshot ready

Research Room

Attach the question to an exact object

Pick the concept, equation, source, code witness, claim, misconception, or demo state before asking for help. The handoff stays grounded to that object.

Next local actionNo local draft saved yet

Open the draft below to save one note and next action in this browser.

conceptGenerative Models

Normalizing Flows: Tractable Density via Invertible Transforms

Anchored question

What is the smallest example that makes Normalizing Flows: Tractable Density via Invertible Transforms click without losing the math?

Local action draftNo local draft saved yetExpand only when ready to capture one local next action

Local action draft

This draft stays locally in this browser for concept:generative-models/normalizing-flows.

Draft noteNext action

No local draft saved.

Evidence to inspect

Source ids to inspect: rezende-2015-normalizing-flows
Definition, prerequisite, and contrast concept links
The equation or code witness that makes the concept operational
One demo state that shows the invariant instead of a slogan

What would resolve this

The learner can state the mechanism in their own words
The learner can name the prerequisite that would repair confusion
The learner can predict how the mechanism changes under one perturbation

Grounded AI handoff

I am working in Continuous Function's research reading room. Object: concept - Normalizing Flows: Tractable Density via Invertible Transforms Object key: concept:generative-models/normalizing-flows Context: Generative Models Anchor id: concept/concept-notebook/generative-models/normalizing-flows Open question: What is the smallest example that makes Normalizing Flows: Tractable Density via Invertible Transforms click without losing the math? Evidence to inspect: - Source ids to inspect: rezende-2015-normalizing-flows - Definition, prerequisite, and contrast concept links - The equation or code witness that makes the concept operational - One demo state that shows the invariant instead of a slogan What would resolve this: - The learner can state the mechanism in their own words - The learner can name the prerequisite that would repair confusion - The learner can predict how the mechanism changes under one perturbation Answer as a careful research tutor: stay source-grounded, separate verified evidence from assumptions, name the relevant math objects, and end with one next action.

Open source object

concept/concept-notebook/generative-models/normalizing-flows
concept:generative-models/normalizing-flows

Learning Map

Before / Now / Try / Next

BeforeMaximum Likelihood

NowIntuition → Math → Code → Demo

TryManipulate one control and predict the visible change.

NextDiffusion, Score-Based Models & Flow Matching

Intuitionready
Mathready
Codeready
Interactive Demoready

Object Companion

Ask beside the selected object

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

Your question

GoalComfortStyleStuck on

Context prompt

You are my AI learning companion for Continuous Function. Current context: Generative Models concept. Learning surface: Normalizing Flows: Tractable Density via Invertible Transforms. What this page says: Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable. Current section: Intuition, math, code, and interactive demo. Suggested next step: Manipulate one control and predict the visible change.. Learner goal: Understand the idea. Learner comfort level: New to this. Preferred explanation style: Visual first. Task: Explain the central idea in plain language, then restate it with the exact math objects from the page. Answer in a way that helps me learn: ask one clarifying question only if needed, use intuition before notation, and end with one thing I should try on the page.

Domain

Generative Models

generative-modelsnormalizing-flowschange-of-variablesjacobian

Prerequisites

Maximum Likelihood Variational Autoencoders

Leads To

Diffusion, Score-Based Models & Flow Matching Flow Matching & Rectified Flows

gans Variational Autoencoders

Within this domain