A generative adversarial network trains a generator against a discriminator that distinguishes generated samples from data. Min–max objectives of the same form appear in adversarial training for robustness, where a model is trained against worst-case perturbations of its inputs (Madry et al., 2018).
Concept 8 of 100GANsGenerative Models · Study phase 4: Generative modeling families
Question
Why does the Jensen–Shannon divergence give vanishing gradients when the data and model distributions have non-overlapping supports, and how does the Wasserstein distance avoid this?
How to study this pagePredict · Check · Generalize · Continue
PredictSay what you expect.
Before you move a control in the demo, say what you expect to change.
CheckCheck against the math and the papers.
Compare what the demo shows with the key equation and with the papers cited below.
GeneralizeName what stays the same.
Identify the part of the idea that survives a change of notation or model, so you can recognize it in another paper or system.
ContinueFollow the connections.
Use the prerequisites and related concepts at the end of the page to fill the next gap.
Why It Matters for Modern Models
A generative adversarial network trains a generator against a discriminator that distinguishes generated samples from data. Min–max objectives of the same form appear in adversarial training for robustness, where a model is trained against worst-case perturbations of its inputs (Madry et al., 2018).
Adversarial losses are still part of image generators: the autoencoder used by latent diffusion is trained with a patch-based adversarial loss (Rombach et al., 2022).
Questions to Think About
What do the discriminator decision surfaces look like around the data during training?
Interactive Demo
Core Math
If you prefer intuition first, try the demo above, then come back for the math.
For a fixed generator the optimal discriminator is D∗(x)=pdata(x)+pG(x)pdata(x). With D∗ plugged in, the generator objective equals 2JS(pdata∥pG)−log4, so the generator minimizes the Jensen–Shannon divergence between the data and model distributions (Goodfellow et al., 2014).
WGAN replaces JS with the Earth-Mover (Wasserstein-1) distance, which requires the discriminator (called a critic) to be 1-Lipschitz (Arjovsky et al., 2017).
Which limitations does it address, or leave to a later idea?
Fix mode collapse: a limitation of GANs that Diffusion addresses. Open Diffusion
GAN training is a min-max game that can be unstable and can drop modes of the data. Diffusion models replace it with a regression loss; Dhariwal and Nichol (2021) found that they surpassed the best GANs on image quality while covering more of the data distribution, at the cost of many network evaluations per sample.
Where does it break down?
Non-overlapping supports: where GANs breaks down, taken up in Maximum Likelihood. Open Maximum Likelihood
When the generator's distribution and the data distribution do not overlap, the Jensen–Shannon divergence that the original GAN minimizes is constant at log 2, so a near-optimal discriminator gives the generator almost no gradient (Arjovsky & Bottou, 2017). WGAN replaced it with the Wasserstein distance, which keeps changing smoothly as the two distributions move closer (Arjovsky et al., 2017).
Where else does the same idea appear?
Learned critic: an analogy between GANs and RLHF. Open RLHF
GANs learn a discriminator that acts like a learned loss; RLHF learns a reward model that acts like a learned loss. In both, a generator/policy is optimized against a learned evaluator that can be exploited.
Learned loss gaming: the same technique appears in Reward Hacking. Open Reward Hacking
GANs train a generator against a learned discriminator; RLHF trains a policy against a learned reward model. Both create adversarial dynamics where the generator/policy can exploit weaknesses in the learned evaluator.
Saddle point game: an analogy between GANs and Sharpness. Open Sharpness
GAN training is a min-max game whose solutions are saddle points rather than minima. Gradient methods can circle around such points instead of converging to them, one source of the oscillation seen in GAN training.
Two learning rates: the same technique appears in Adam. Open Adam
GANs are commonly trained with Adam, and many setups give the generator and the discriminator different learning rates. Heusel et al. (2017) call this the two time-scale update rule and prove that, under mild assumptions, it converges to a local Nash equilibrium, a result that carries over to Adam.
MLE-trained classifier: the same technique appears in Maximum Likelihood. Open Maximum Likelihood
GAN discriminator maximizes likelihood of correct real/fake labels via logistic cross-entropy. MLE classification repurposed as training signal for generator.
Which ideas are its mathematical duals?
Adversarial ↔ Denoising: Diffusion is its mathematical dual. Open Diffusion
Both turn noise into data. A GAN does it in one pass of a generator trained against a discriminator; a diffusion model does it over many denoising steps, with a denoiser trained by plain regression. Trading the adversarial game for regression makes training more stable, at the cost of slower sampling.
Implicit vs explicit density: VAEs is its mathematical dual. Open VAEs
GANs define a density only implicitly: they produce samples but give no value of p(x). VAEs define an explicit density and are trained on a lower bound on log p(x), which can be estimated but not computed exactly. They are two approaches to the same generative goal.