8Generative Models

⚔GANs & Adversarial Divergence Minimization

Canonical Papers

Goodfellow et al.2014NeurIPS

Arjovsky et al.2017ICML

Original GAN objective:

\min_G \max_D \; \mathbb E_{x\sim p_{\text{data}}}[\log D(x)] + \mathbb E_{z\sim p(z)}[\log(1 - D(G(z)))]

At optimum, with optimal discriminator $D^*$ , this minimizes the Jensen–Shannon divergence between model and data.

WGAN replaces JS with Earth-Mover (Wasserstein-1) distance, with Lipschitz constraints on $D$ .

Key Equation

\min_G \max_D \; \mathbb E_{p_{\text{data}}}[\log D(x)] + \mathbb E_{p(z)}[\log(1 - D(G(z)))]

Adversarial min-max ideas appear in adversarial training and some alignment techniques
GAN-like training still influential in high-fidelity image/video generation

What is still poorly explained in textbooks and papers:

Why JS divergence leads to vanishing gradients when supports don't overlap, and how Wasserstein distances fix this
Geometric visualizations of discriminator decision surfaces over latent manifolds

Explore this concept from different angles — like a mathematician would.

↔️ Mathematical Dual

≈ Analogy

🔄 Same Technique

⚠️ Breaks When

🔧 Invented to Fix