Lie Groups & Equivariant Networks

Why It Matters for Modern Models

Rotations and translations form Lie groups: groups of transformations that vary smoothly with continuous parameters. A network is equivariant when transforming its input transforms its output in a matching way: shifting an image shifts a convolutional feature map. Building a symmetry the task is known to have into the network lets it share weights across transformed inputs instead of learning each case separately (Bronstein et al., 2021).
Group-equivariant CNNs extend this weight sharing to rotations and reflections, adding expressive capacity without adding parameters; they reached state-of-the-art results at the time on rotated MNIST and CIFAR-10 (Cohen & Welling, 2016).
Molecular and protein models often build symmetry in: AlphaFold 2's structure module uses an attention operation that is invariant to global rotations and translations (Jumper et al., 2021). AlphaFold 3 instead omitted these constraints from its diffusion module and found they were not required (Abramson et al., 2024).

Questions to Think About

Why is translation on a grid the easiest symmetry to build in, while rotations and scalings need group convolutions or steerable filters?
How does the Lie algebra describe a continuous symmetry through its infinitesimal generators, for example a rotation by a small angle θ written as I + θA with A skew-symmetric?

Core Math

Key Equation

f(\rho_{\text{in}}(g)\, x) = \rho_{\text{out}}(g)\, f(x)

A Lie group $G$ is a group that is also a smooth manifold, with smooth multiplication and inversion. Rotations, translations and scalings all form Lie groups.

A function $f$ is equivariant to $G$ if transforming the input transforms the output in a matching way:

f(\rho_{\text{in}}(g)\, x) = \rho_{\text{out}}(g)\, f(x) \quad \text{for all } g \in G

where $\rho_{\text{in}}$ and $\rho_{\text{out}}$ say how $g$ acts on inputs and on outputs. Invariance is the special case in which $\rho_{\text{out}}(g)$ is the identity.

Group convolution generalizes ordinary convolution from translations to a group $G$ , integrating over the group with its invariant (Haar) measure (Cohen & Welling, 2016; Cohen, Geiger & Weiler, 2019):

[f \star \psi](g) = \int_G f(h)\, \psi(g^{-1}h)\, dh

Lie algebra. The Lie algebra $\mathfrak{g}$ holds the infinitesimal generators of $G$ , and the exponential map $\exp: \mathfrak{g} \to G$ turns them into group elements. For the rotation group $SO(3)$ the Lie algebra is the space of $3 \times 3$ skew-symmetric matrices: for a suitably normalized generator $A$ , $\exp(\theta A)$ is a rotation by angle $\theta$ , and for small $\theta$ , $\exp(\theta A) \approx I + \theta A$ .