Continuous Function

Foundation reference

Transformer Circuits, Induction Heads & Mechanistic Interpretability

Circuit analysis traces a specific behavior of a trained transformer to particular attention heads and MLPs. Wang et al. (2023) traced indirect-object identification in GPT-2 small to a circuit of 26 attention heads.

Concept 13 of 100RepresentationsStudy phase 5
Open the demoAll concepts
Key equation
rl+1=rl′+MLP(rl′),rl′=rl+∑hheadh(rl)r_{l+1} = r_l' + \mathrm{MLP}(r_l'), \quad r_l' = r_l + \sum_h \mathrm{head}_h(r_l)
Reading map and next steps

Study guide

Where to start

Concept 13 of 100CircuitsRepresentations · Study phase 5: Representation & interpretability
Question

How do the QK and OV matrices of two attention heads combine to implement an algorithm such as induction?

Open the demoGo to the mathAll conceptsNext concept
How to study this pagePredict · Check · Generalize · Continue
PredictSay what you expect.

Before you move a control in the demo, say what you expect to change.

CheckCheck against the math and the papers.

Compare what the demo shows with the key equation and with the papers cited below.

GeneralizeName what stays the same.

Identify the part of the idea that survives a change of notation or model, so you can recognize it in another paper or system.

ContinueFollow the connections.

Use the prerequisites and related concepts at the end of the page to fill the next gap.

Why It Matters for Modern Models

  • Circuit analysis traces a specific behavior of a trained transformer to particular attention heads and MLPs. Wang et al. (2023) traced indirect-object identification in GPT-2 small to a circuit of 26 attention heads.
  • Knowing the mechanism behind a behavior makes it possible to test, edit or monitor that mechanism directly, which is why interpretability is studied as a tool for safety research.

Questions to Think About

  • Which circuit motifs recur across models, beyond the few that have been studied in detail?

Interactive Demo

Core Math

If you prefer intuition first, try the demo above, then come back for the math.

Key Equation
rl+1=rl′+MLP(rl′),rl′=rl+∑hheadh(rl)r_{l+1} = r_l' + \mathrm{MLP}(r_l'), \quad r_l' = r_l + \sum_h \mathrm{head}_h(r_l)

Treat the residual stream rr as a shared channel: each attention head and each MLP reads from it and adds its output back:

rl′=rl+∑hheadh(rl),rl+1=rl′+MLP(rl′)r_l' = r_l + \sum_h \mathrm{head}_h(r_l), \qquad r_{l+1} = r_l' + \mathrm{MLP}(r_l')

Each head splits into a QK circuit, WQ⊤WKW_Q^\top W_K, which sets the attention pattern (which positions it attends to), and an OV circuit, WOWVW_O W_V, which sets what it copies from those positions. Once the pattern is fixed, the head acts linearly on the stream (Elhage et al., 2021).

Induction heads implement the pattern-completion rule

[A][B]…[A]→[B][A][B]\dots[A] \rightarrow [B]

in two steps: an earlier previous-token head writes "the token before me was [A]" into the position of [B]; the induction head at the current [A] then attends to that position (the token after the earlier [A]) and copies [B] to the output.

Olsson et al. (2022) found that induction heads form at the same point in training as a sudden improvement in in-context learning, and give preliminary, indirect evidence that they may be the mechanism behind most in-context learning in large models.

Key Papers

A Mathematical Framework for Transformer Circuits

Elhage et al.2021Transformer Circuits Thread (Anthropic)
Read paper →

In-Context Learning and Induction Heads

Olsson et al.2022Transformer Circuits Thread (Anthropic)
Read paper →

Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small

Wang et al.2023ICLR
Read paper →

Connections

Prerequisites

AttentionEmbeddingsSuperpositionProbing

Leads to

Sparse AutoencodersCircuit DiscoveryIn-Context Learning

Where to go next

Sparse Autoencoders builds directly on Circuits. The other ideas that build on this page are listed under Leads to, above.

Sparse Autoencoders
Other connections (5)

Where does it break down?

  • Feature learning dominates: where NTK breaks down, taken up on this page. Open NTK

    NTK assumes training stays near initialization (kernel/linearized regime). Induction heads emerge from representation reorganization and new circuit formation—behavior NTK-style predictions miss.

Where else does the same idea appear?

  • Circuits: an analogy between Circuits and Circuit Discovery. Open Circuit Discovery

    Induction heads are a specific circuit (copy from past). Circuit discovery generalizes: finding any computational subgraph implementing a behavior.

  • Copy via attention: the same technique appears in Attention. Open Attention

    An induction head completes patterns of the form [A][B] … [A] → [B]. A previous-token head in an earlier layer writes each token's predecessor into its position; the induction head then attends from the current token A to the token that followed an earlier A, and copies it (Olsson et al., 2022). Attention does the copying.

  • Algorithmic feature: an analogy between Circuits and Embeddings. Open Embeddings

    Induction heads are "algorithmic features"—circuits that implement pattern-completion algorithms. Understanding them bridges circuit-level interpretability and representation-level analysis.

  • Interventions: the same technique appears in Circuit Discovery. Open Circuit Discovery

    Both look for causal structure by intervening on activations. Elhage et al. (2021) first described induction heads by reverse-engineering small attention-only models, and Olsson et al. (2022) used ablations, among other evidence, to argue that they drive in-context learning.