Geometric Deep Learning
Symmetry as inductive bias
Why Geometry Matters
The structure of your data should inform the structure of your model. If rotating an image shouldn't change what object it contains, your network should reflect that symmetry.
Geometric deep learning makes this precise: we design networks that are equivariant to group transformations of the input.
Equivariance
A function f is equivariant to a transformation g if transforming the input leads to a predictably transformed output:
f(g · x) = g · f(x)
Rotate the input using the slider below. Watch how an equivariant feature detector moves with the rotation — it detects the same feature, just in the transformed location.
CNNs as Translation Equivariance
Convolutional neural networks are equivariant to translations. A feature detected at position (x, y) will be detected at position (x+Δx, y+Δy) when the input is shifted.
This is why CNNs work so well for images: we don't need to learn separate detectors for "cat in top-left corner" and "cat in bottom-right corner".
The same convolutional kernel is applied at every spatial location, encoding translation equivariance into the architecture.
Group Theory
Symmetries form mathematical structures called groups. Common examples:
| Group | Symmetry | Application |
|---|---|---|
| SO(2) | Rotation | Aerial imagery |
| SE(3) | Rotation + Translation | Molecules, robotics |
| Sn | Permutation | Sets, graphs |
Graph Neural Networks
GNNs are equivariant to node permutations. Reordering the nodes of a graph doesn't change what the network computes — the output permutes accordingly.
hv(k) = UPDATE(hv(k-1), AGG({hu(k-1) : u ∈ N(v)}))
The aggregation function (sum, mean, max) is permutation-invariant, making the overall network permutation-equivariant.