Activation Steering: Feature-Guided Interventions for Inference-Time Control
Activation steering changes how a model behaves at inference time without fine-tuning, for example to follow format or length instructions, shift style, or make refusals more or less likely (Stolfo et al., 2025; Arditi et al., 2024).
Concept 29 of 100Activation SteeringRepresentations · Study phase 5: Representation & interpretability
Question
Why does a small steering strength nudge the model while a large one pushes its hidden states into regions never seen in training, where outputs become incoherent?
How to study this pagePredict · Check · Generalize · Continue
PredictSay what you expect.
Before you move a control in the demo, say what you expect to change.
CheckCheck against the math and the papers.
Compare what the demo shows with the key equation and with the papers cited below.
GeneralizeName what stays the same.
Identify the part of the idea that survives a change of notation or model, so you can recognize it in another paper or system.
ContinueFollow the connections.
Use the prerequisites and related concepts at the end of the page to fill the next gap.
Why It Matters for Modern Models
Activation steering changes how a model behaves at inference time without fine-tuning, for example to follow format or length instructions, shift style, or make refusals more or less likely (Stolfo et al., 2025; Arditi et al., 2024).
Steering is also a causal test: if adding a direction produces a behavior, that direction is involved in producing it, which helps check hypotheses about circuits.
Activation scaling multiplies existing activation vectors by learned scalars instead of adding new directions; Stoehr et al. (2024) report that it matches steering vectors in effectiveness with far fewer learned parameters.
With sparse-autoencoder features, the steering direction comes with a human-readable description, which makes the intervention easier to interpret (Soo et al., 2025).
Because no weights change, a steering vector can be switched on, off or rescaled for each request; the cost is that strong steering can degrade output quality.
Questions to Think About
Does steering along a named sparse-autoencoder feature change only that concept, or can it also affect other behaviors that share parts of the same direction?
Why can two steering vectors added together cancel or amplify each other instead of combining cleanly?
How should the steering strength be chosen when too little has no effect and too much breaks generation quality?
If steering can bring out a behavior the model does not show by default, what does that say about what the model has learned, and what does it not prove?
Interactive Demos
Core Math
If you prefer intuition first, try the demos above, then come back for the math.
Key Equation
hℓ,tsteer=hℓ,t+αvℓ
Activation steering changes a model's behavior at inference time by editing its hidden states, without changing its weights.
Adding a steering vector at one layer:
hℓ,tsteer=hℓ,t+αvℓ
where hℓ,t is the hidden state at layer ℓ and position t, vℓ is the steering direction and α sets its strength.
A steering vector from contrasting examples (difference of means):
vℓ=E[hℓ∣desired]−E[hℓ∣undesired]
Contrastive activation addition (CAA) builds vℓ this way from pairs of examples that differ only in the target behavior (Rimsky et al., 2024). Instruction vectors are built in the same way from inputs with and without an instruction (Stolfo et al., 2025).
Steering along a sparse-autoencoder feature:
hℓ,tsteer=hℓ,t+δWdecek
where ek selects feature k, so Wdecek is that feature's decoder direction. Adding it to h directly keeps the part of h that the SAE fails to reconstruct; encoding h, raising zk and decoding would discard that part.
Key Papers
Steering Llama 2 via Contrastive Activation Addition
Which limitations does it address, or leave to a later idea?
Targeted intervention: a limitation of Activation Steering that Sparse Autoencoders addresses. Open Sparse Autoencoders
SAE features make steering more targeted. Instead of adding a direction found by contrasting two sets of examples, one can add or remove a specific feature that the SAE has identified and that has a human-readable description (Templeton et al., 2024).
Inference interventions: a limitation of Circuit Discovery that Activation Steering addresses. Open Circuit Discovery
Once a feature or circuit that drives a behaviour has been found, a natural next step is to change the behaviour without retraining. Activation steering does this by adding a vector to the hidden states at inference time.
Where does it break down?
Feature entanglement: where Activation Steering breaks down, taken up in Sparse Autoencoders. Open Sparse Autoencoders
SAE features are not guaranteed to be perfectly monosemantic or causally isolated. Steering along imperfect features can cause unintended side effects on behaviour, or push the hidden states into regions the model never saw in training.
Where else does the same idea appear?
Linear directions: the same technique appears in Embeddings. Open Embeddings
Both rely on many concepts being represented as directions in activation space. Steering adds such a direction, for example a "refusal direction", to the activations (Arditi et al., 2024): the same geometry, used to change behaviour instead of reading it.
Inference-time control: an analogy between Activation Steering and Decoding & Sampling. Open Decoding & Sampling
Decoding changes behavior by reshaping the output distribution (logits → sampling); steering changes it by editing hidden states. Both control a model at inference time without retraining, and both degrade the output when pushed too far: a very high temperature gives incoherent text, and a very strong steering vector moves hidden states into regions the model never saw in training.