∇Continuous Function
Concept notebookChecking saved investigationReading browser-local route memory before showing a continuation.

Normalizing Flows: Tractable Density via Invertible Transforms

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

published · difficulty 4/5 · 18 min read

01

01

Intuition

Build the mental picture first so the rest of the page has something to attach to.

PredictName the object in plain language, then predict what should change.Leave with one reusable mental picture before notation appears.

Normalizing flows are "reversible warps."

Start from a simple base distribution (like a standard Gaussian). Then apply an invertible transformation that bends and stretches space into a richer model distribution.

Because the transform is invertible, you get:

  • Exact sampling: sample zzz from the base, push forward to x=f(z)x=f(z)x=f(z).
  • Tractable log-likelihood under the flow model: map xxx back to z=f−1(x)z=f^{-1}(x)z=f−1(x) and account for how volumes change.

The price is architectural: you must design fff so it is invertible and the needed map direction plus Jacobian determinant are cheap to compute.

Section prompt

Copy-only prompts — each action copies a page-grounded prompt to your clipboard. Nothing is sent by this site.

02

02

Math

Translate the story into symbols, assumptions, and a derivation you can inspect.

InspectTrack the same object through the notation and check each symbol.Leave with the invariant the equations preserve.
Equation 1Inspect equation objectp_x(x)=p_z\big(f^{-1}(x)\big)|\det\frac{\partial f^{-1}}{\partial x}|,Equation 2Inspect equation object\log p_x(x)=\log p_z\big(f^{-1}(x)\big)+\log|\det\frac{\partial f^{-1}}{\partial x}|.Equation 3Inspect equation object\log p_x(x)=\log p_z(z_0)-\sum_{k=1}^K\log|\det(\partial f_k)/(\partial z_{k-1)}|.

Change of variables

Let x=f(z)x=f(z)x=f(z) where fff is invertible and z∼pzz\sim p_zz∼pz​. Then:

px(x)=pz(f−1(x))∣det⁡∂f−1∂x∣,p_x(x)=p_z\big(f^{-1}(x)\big)\left|\det\frac{\partial f^{-1}}{\partial x}\right|,px​(x)=pz​(f−1(x))​det∂x∂f−1​​,

so:

log⁡px(x)=log⁡pz(f−1(x))+log⁡∣det⁡∂f−1∂x∣.\log p_x(x)=\log p_z\big(f^{-1}(x)\big)+\log\left|\det\frac{\partial f^{-1}}{\partial x}\right|.logpx​(x)=logpz​(f−1(x))+log​det∂x∂f−1​​.

The determinant term is the volume correction. If the inverse map squeezes a region of data space into a smaller region of latent space, density must increase; if it expands the region, density must decrease. This is the tractable-density advantage of flows, but also the constraint: a flexible transform is useful for likelihood work only when the needed map direction and log-determinant stay tractable.

Composition of flows

If f=fK∘⋯∘f1f=f_K\circ\cdots\circ f_1f=fK​∘⋯∘f1​ maps latent variables forward, with zk=fk(zk−1)z_k=f_k(z_{k-1})zk​=fk​(zk−1​) and zK=xz_K=xzK​=x, then the inverse-Jacobian log-determinants add as a negative sum of forward log-determinants:

log⁡px(x)=log⁡pz(z0)−∑k=1Klog⁡∣det⁡∂fk∂zk−1∣.\log p_x(x)=\log p_z(z_0)-\sum_{k=1}^K\log\left|\det\frac{\partial f_k}{\partial z_{k-1}}\right|.logpx​(x)=logpz​(z0​)−k=1∑K​log​det∂zk−1​∂fk​​​.

This follows from the determinant of a product of Jacobians. Later flow architectures often engineer triangular or otherwise cheap Jacobians, which can turn a high-dimensional determinant into a sum of simpler per-dimension terms.

Section prompt

Copy-only prompts — each action copies a page-grounded prompt to your clipboard. Nothing is sent by this site.

03

03

Code

Keep the implementation aligned with the notation so the algorithm is legible.

TraceMatch variables to symbols before reading the implementation.Leave with a runnable witness for the math.
Code witness 1import numpy as np # A tiny 2D linear flow: x = A z + b, with z ~ N(0, I) A = np.array([[1.2,...python
import numpy as np

# A tiny 2D linear flow: x = A z + b, with z ~ N(0, I)
A = np.array([[1.2, 0.3], [0.1, 0.9]])
b = np.array([0.5, -0.2])
Ainv = np.linalg.inv(A)

def log_pz(z):
    d = z.shape[0]
    return -0.5 * float(z @ z) - 0.5 * d * np.log(2 * np.pi)

x = np.array([1.0, 0.0])
z = Ainv @ (x - b)
log_px = log_pz(z) + np.log(abs(np.linalg.det(Ainv)))

print("det(A):", round(float(np.linalg.det(A)), 3))
print("z:", np.round(z, 3))
print("log p_x(x):", round(float(log_px), 3))
Section prompt

Copy-only prompts — each action copies a page-grounded prompt to your clipboard. Nothing is sent by this site.

04

04

Interactive Demo

Use direct manipulation to connect the explanation to a moving system.

ManipulateChange one control and predict the visible response before reveal.Leave with the observed invariant or a repaired model.

Live Concept Demo

Explore Normalizing Flows: Tractable Density via Invertible Transforms

The stage is code-native and interactive. Use it to test the explanation against the mechanism.

difficulty 4/5undergraduatecode-aligned
Demo inquiry checkpoint

Manipulate one control and predict the visible change.

01Choose lensTrace a quantity
02ObserveDemo state pending
03GroundName the equation, invariant, or control that explains it.
04CarryNext: Diffusion, Score-Based Models & Flow Matching

Choose what to inspect in Normalizing Flows: Tractable Density via Invertible Transforms. This shared fallback is an observation guide, not evidence of learning.

Loading interactive demo...

The demo below asks you to predict the density effect before revealing the Jacobian correction. The key invariant is that exact likelihood comes from two terms: the base density at the inverse point and the log-volume correction from the inverse Jacobian.

Section prompt

Copy-only prompts — each action copies a page-grounded prompt to your clipboard. Nothing is sent by this site.

4/4 sections ready

Concept: Normalizing Flows: Tractable Density via Invertible Transforms

What is the smallest example that makes Normalizing Flows: Tractable Density via Invertible Transforms click without losing the math?

BeforeMaximum LikelihoodNow4/4 sections readyTryManipulate one control and predict the visible change.NextDiffusion, Score-Based Models & Flow Matching
Object contextGenerative Models
StateGenerative Models
1 source attachedChecking local snapshot
concept:generative-models/normalizing-flows
Math3 equation objectsp_x(x)=p_z\big(f^{-1}(x)\big)|\det\frac{\partial f^{-1}}{\partial x}|,CodeNormalizing Flows: Tractable Density via Invertible Transforms code w...A = np.array([[1.2, 0.3], [0.1, 0.9]])DemoNormalizing Flows: Tractable Density via Invertible Transforms predic...Manipulate one control and predict the visible change.
ConceptLearner lens

Normalizing Flows: Tractable Density via Invertible Transforms

What is the smallest example that makes Normalizing Flows: Tractable Density via Invertible Transforms click without losing the math?

Mode questionCan I say the mechanism back in one sentence before I reveal anything?

Start with the prediction checkpoint, then compare the reveal to the mental model.

Take this move

Study modes

Keep the object fixed; change the lens.
01PredictCommit before reveal.02ExplainName the invariant.03Ground1 source link nearby.04ExperimentChange one knob.05CarrySave the next question.

Route back through the notebook

Carry the same object through intuition, math, code, and demo.

4/4 sections ready
Carry inMaximum Likelihood

Bring the mental model from Maximum Likelihood; this page will reuse it instead of restarting from zero.

Work hereNormalizing Flows: Tractable Density via Invertible Transforms

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

Carry outDiffusion, Score-Based Models & Flow Matching

The next edge should feel earned: use the demo prediction here before following Diffusion, Score-Based Models & Flow Matching.

Test the linkManipulate one control and predict the visible change.Then continue to Diffusion, Score-Based Models & Flow Matching

After The First Pass

Turn the concept into an inspected object.

The lower panels are one second act: keep the object fixed, inspect it visually, check source boundaries, practice transfer, then attach the research question.
ConceptNormalizing Flows: Tractable Density via Invertible TransformsGenerative Models
01MechanismInspect motionSee how the idea changes form before adding more explanation.02VisualChoose a cueBind one visible mark to one mathematical role.03SourcesCheck boundary1 source link attached to the object.04ClaimAudit evidenceTreat the page claim as something to verify.05PracticeAttempt transferWrite before feedback so the model has something to repair.06RoomAsk groundedAttach the question to the same exact object.

Mechanism Storyboard

See the idea move before the page explains it

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

Demo notes open01 / Intuition
Editorial generative-model illustration of an invertible grid warp with local volume-correction patches.
Prediction lens

Start with the picture, metaphor, or geometric mechanism.

Commit first

Before reading further, choose the kind of change Normalizing Flows: Tractable Density via Invertible Transforms should make visible.

Visual Inquiry

Make the image answer a mathematical question

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

4/4 stages readyDemo notes connected
Prediction

Which visible object should carry the first intuition?

Commit first

Pick the cue that should make Normalizing Flows: Tractable Density via Invertible Transforms easier to reason about before the page gives the answer.

Source Grounding

Canonical references for the mechanism on this page.

Object - ConceptNormalizing Flows: Tractable Density via Invertible TransformsQuestion

What is the smallest example that makes Normalizing Flows: Tractable Density via Invertible Transforms click without losing the math?

concept:generative-models/normalizing-flows
Boundary

sources: rezende-2015-normalizing-flows

Check

Open the closest source note before trusting the local explanation.

Evidence

1 selected-object source shown first; 1 reference total.

Next move

Audit the claim boundary, then ask from the same selected object.

selected object source · paper · 2015Variational Inference with Normalizing FlowsRezende and Mohamed
Located CF editorial boundary

Grounds the change-of-variables density transformation and the term normalizing flow.

Used here as

Rezende and Mohamed define a normalizing flow as transforming a probability density through a sequence of invertible mappings, state that repeated change-of-variables moves an initial den...

Caveat

Rezende and Mohamed frame flows for variational approximate posteriors and density transformation; they do not by themselves justify broad claims about modern data modeling, sampling qual...

Open source

Claim Review

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

Object - ConceptNormalizing Flows: Tractable Density via Invertible TransformsQuestion

What is the smallest example that makes Normalizing Flows: Tractable Density via Invertible Transforms click without losing the math?

concept:generative-models/normalizing-flows
Boundary

sources: rezende-2015-normalizing-flows

Check

Treat every claim as provisional until source support and a local witness agree.

Evidence

1 structured claim check on this concept.

Next move

Run the prediction or practice transfer before asking for a grounded review.

1 CF editorial source-scope review recorded

Publisher-side editorial review is not independent replication. Claims without it still need exact source-support review. 1 reference and 3 local witnesses are available for inspection.

Normalizing flows compose invertible maps to turn a simple initial density into a richer one; for a tractable finite flow, the log density at a point is computed exactly by evaluating the base log density at the corresponding preimage and adding the inverse-Jacobian log determinant.
Registered CF editorial boundary
Variational Inference with Normalizing FlowsGrounds the change-of-variables density transformation and the term normalizing flow.
Used here as

Rezende and Mohamed define a normalizing flow as transforming a probability density through a sequence of invertible mappings, state that repeated change-of-variables moves an initial density through that se...

Local witness
source-span-rezende-2015-normalizing-flowsmath-object-1math-object-2code-witness-1interactive-demo
Equation 1
px(x)=pz(f−1(x))∣det⁡∂f−1∂x∣,p_x(x)=p_z\big(f^{-1}(x)\big)\left|\det\frac{\partial f^{-1}}{\partial x}\right|,
Equation 2
log⁡px(x)=log⁡pz(f−1(x))+log⁡∣det⁡∂f−1∂x∣.\log p_x(x)=\log p_z\big(f^{-1}(x)\big)+\log\left|\det\frac{\partial f^{-1}}{\partial x}\right|.
Caveat

Rezende and Mohamed frame flows for variational approximate posteriors and density transformation; they do not by themselves justify broad claims about modern data modeling, sampling quality, stability, or u...

Review stateCF editorial source-scope reviewClaim metadata: source checkedPublisher-side editorial review only; not independent replication. Check caveats and exact source scope.

Rezende and Mohamed support normalizing flows as invertible-map sequences that transform a simple initial density into a richer one, derive finite change-of-variables density updates using inverse/forward Jacobian determinants, and give the composed log-density formula as base log density minus summed forward logdet terms. Local math, affine code, and demo instantiate preimage lookup plus inverse-Jacobian log-volume correction.

Reviewer: codex+oracle+codex-5.3; reviewed 2026-05-08

Practice notebook

Use the idea, then test it somewhere new

Invertible transforms trained with change-of-variables: tractable transformed densities and sampling, with architectural constraints to make the needed Jacobian determinants tractable.

AttemptNo learning claim inferred
Object - ConceptNormalizing Flows: Tractable Density via Invertible TransformsQuestion

What is the smallest example that makes Normalizing Flows: Tractable Density via Invertible Transforms click without losing the math?

concept:generative-models/normalizing-flows
Boundary

sources: rezende-2015-normalizing-flows

Check

Use one state from Normalizing Flows: Tractable Density via Invertible Transforms to explain what changes, why it changes, and which assumption the explanation needs.

Evidence

No learner move yet; no learning state is inferred.

Next move

Write first, use only the help you need, then try a new case without it.

Explain

Use one state from Normalizing Flows: Tractable Density via Invertible Transforms to explain what changes, why it changes, and which assumption the explanation needs.

Hint 1

Reveal when your model needs a nudge.

Hint 2

Reveal when your model needs a nudge.

Hint 3

Reveal when your model needs a nudge.

Grounded object roomClose
Selected object routeAsk from this object; carry one invariant back.sources: rezende-2015-normalizing-flows
  1. ObjectConceptNormalizing Flows: Tractable Density via Invertible Transforms
  2. PredictBefore revealNormalizing Flows: Tractable Density via Invertible Transforms predic...
  3. WitnessCompare codeNormalizing Flows: Tractable Density via Invertible Transforms code w...
  4. RoomAsk groundedChecking local snapshot
ConceptNormalizing Flows: Tractable Density via Invertible TransformsGenerative Models
Code witness comparisonNormalizing Flows: Tractable Density via Invertible Transforms code witness 1A = np.array([[1.2, 0.3], [0.1, 0.9]])Prediction before revealNormalizing Flows: Tractable Density via Invertible Transforms predictionManipulate one control and predict the visible change.
Grounded room questionWhat is the smallest example that makes Normalizing Flows: Tractable Density via Invertible Transforms click without losing the math?Checking local snapshot

Research Room

Attach the question to an exact object

Pick the concept, equation, source, code witness, claim, misconception, or demo state before asking for help. The handoff stays grounded to that object.
Next local actionNo local draft saved yet

Open the draft below to save one note and next action in this browser.

conceptGenerative Models

Normalizing Flows: Tractable Density via Invertible Transforms

Anchored question

What is the smallest example that makes Normalizing Flows: Tractable Density via Invertible Transforms click without losing the math?

Source boundaryInspect source ids: rezende-2015-normalizing-flowsStable content-object key attached
Role lenses for this object

These are fixed, deterministic perspectives derived from the selected object. They do not represent people, community contributions, or independent review.

Learner evidence requestAsk what would make "Normalizing Flows: Tractable Density via Invertible Transforms" feel predictable rather than familiar.
Assumption

Source ids rezende-2015-normalizing-flows must support the exact object, not just the surrounding topic.

Source-checking summary

Treat this as a mechanism object: connect the definition to one equation, code witness, or demo before broadening the discussion.

Proposed experiment

Ask the learner to perturb one representation, then check whether the same invariant survives in math, code, and demo.

Next action

The learner can state the mechanism in their own words

Evidence4 checks
PredictionChecking carried observation
ActionReady for one action
AILearner handoff ready
Open source object
01PredictionChecking browser-local route memory
02EvidenceChecking for a carried observation
03BoundaryInspect source ids: rezende-2015-normalizing-flows
04Next moveSave one next action
Local action draftNo local draft saved yetExpand only when ready to capture one local next action
Local action draft

This draft stays locally in this browser for concept:generative-models/normalizing-flows.

No local draft saved.
Evidence to inspect
  • Source ids to inspect: rezende-2015-normalizing-flows
  • Definition, prerequisite, and contrast concept links
  • The equation or code witness that makes the concept operational
  • One demo state that shows the invariant instead of a slogan
What would resolve this
  • The learner can state the mechanism in their own words
  • The learner can name the prerequisite that would repair confusion
  • The learner can predict how the mechanism changes under one perturbation
Object-attached AI handoff

I am working in Continuous Function's research reading room. Object: concept - Normalizing Flows: Tractable Density via Invertible Transforms Object key: concept:generative-models/normalizing-flows Context: Generative Models Anchor id: concept/concept-notebook/generative-models/normalizing-flows Open question: What is the smallest example that makes Normalizing Flows: Tractable Density via Invertible Transforms click without losing the math? Evidence to inspect: - Source ids to inspect: rezende-2015-normalizing-flows - Definition, prerequisite, and contrast concept links - The equation or code witness that makes the concept operational - One demo state that shows the invariant instead of a slogan Deterministic role lenses for this object: - Boundary: fixed perspectives, not people, community contributions, or independent review - Source-checking summary: Treat this as a mechanism object: connect the definition to one equation, code witness, or demo before broadening the discussion. - Proposed experiment: Ask the learner to perturb one representation, then check whether the same invariant survives in math, code, and demo. - Teach/transfer move: Turn the mechanism into one sentence that predicts a neighboring concept. - Assumptions: - Source ids rezende-2015-normalizing-flows must support the exact object, not just the surrounding topic. - The stable content-object key lets local drafts, prompts, and route memory attach without changing the source page. - The concept explanation is local atlas prose until checked against its math, code, and source support. - Prerequisite gaps should become a repair route, not a reason to leave the object vague. - Role-lens requests: - Learner: ask for "Ask what would make "Normalizing Flows: Tractable Density via Invertible Transforms" feel predictable rather than familiar." | assumption: Source ids rezende-2015-normalizing-flows must support the exact object, not just the surrounding topic. | next action: The learner can state the mechanism in their own words - Researcher: ask for "Source ids to inspect: rezende-2015-normalizing-flows" | assumption: The stable content-object key lets local drafts, prompts, and route memory attach without changing the source page. | next action: The learner can name the prerequisite that would repair confusion - Experimenter: ask for "Choose one variable or condition to perturb before asking for an explanation." | assumption: The concept explanation is local atlas prose until checked against its math, code, and source support. | next action: The learner can predict how the mechanism changes under one perturbation - Professor: ask for "Find the smallest transferable rule a learner could reuse without the AI." | assumption: Prerequisite gaps should become a repair route, not a reason to leave the object vague. | next action: Teach or transfer: Turn the mechanism into one sentence that predicts a neighboring concept. What would resolve this: - The learner can state the mechanism in their own words - The learner can name the prerequisite that would repair confusion - The learner can predict how the mechanism changes under one perturbation Answer as a careful research tutor: stay source-grounded, separate verified evidence from assumptions, name the relevant math objects, and end with one next action. Current deterministic role lens for this object: - Role lens: Learner - Evidence request: Ask what would make "Normalizing Flows: Tractable Density via Invertible Transforms" feel predictable rather than familiar. - Assumption to keep visible: Source ids rezende-2015-normalizing-flows must support the exact object, not just the surrounding topic. - Proposed experiment: Ask the learner to perturb one representation, then check whether the same invariant survives in math, code, and demo. - Next action: The learner can state the mechanism in their own words

concept/concept-notebook/generative-models/normalizing-flows concept:generative-models/normalizing-flows