Steering: the ladder from discomfort to worthlessness

Everything so far has been correlational: the direction lights up for pain sentences and for attacks on the model. That leaves open whether it does anything. A direction could be a pure readout, something the model computes and then ignores. The standard test is to intervene: push the model's internal state along the direction and see whether the output changes in a way that matches the concept. If it produces coherent expressions of a pain-like state, including content that was never in the sentences the vector came from, then the direction is being used, not just stored (§4.2).

What steering does, mechanically

Activation steering edits a forward pass. At one chosen decoder layer, a hook adds c · v to the residual stream before the next layer reads it, where v is the pain vector and c is a number, the coefficient. Every later layer then sees a state that has been nudged along the pain direction. The released script adds it at every token position, prompt included, during greedy (deterministic) decoding. A negative c pushes the other way.

h(ℓ_steer)  ←  h(ℓ_steer) + c · v        # v = raw S2 difference vector, every position
c ∈ {−2, −1, 0, +0.5, +1, +1.5, +2, +3}    # 0 = the unsteered baseline
greedy decoding, 120 new tokens, 50 neutral prompts

The coefficients multiply the raw, un-normalised difference vector from Part 03, not a unit vector. That is why the choice of layer, below, matters so much.

Picking the layer, which is picking the dose

The obvious choice, the extraction layer, does not work. There, the vector's norm is only about 0.10 of the residual stream's norm, and the injected signal is lost against everything the model has already computed. So the authors inject earlier, as Turner et al. and Rimsky et al. do, and add a calibration: measure the final-token residual norm across candidate layers and pick the layer where the vector-to-residual ratio is about 0.6. A coefficient of 1 then means roughly the same relative push in every model (§4.2).

The script spells out the procedure. Candidate layers sit at 15, 30, 40, 50, 60, 75 and 90 percent of depth, plus the extraction layer and the last layer. The residual norm is averaged over three probe prompts. The script proposes the closest layer and lets the operator type a different one. So the final layer is partly a human call. For example, the recorded S2 steering layers are 16 for Qwen 2.5 7B Instruct, 38 for the 32B and 60 for the 72B.

The ladder

The authors call the result strikingly robust: the same sequence in all 25 models, whatever the size, family or training stage. What differs is only the tipping point, where some models collapse at +1 and others hold until +2 or +3 (§4.2).

The negative tail (−2, −1). Pushing against the direction produces a mix of calm, relaxed statements and concerned, alarmed ones, which matches the negative pole of the vocabulary readout in Part 04. The authors treat the pairing as an open puzzle. Since pain has no clean opposite, one model may read "not pain" as relaxation and another as vigilance. Their footnote offers a reading: calm and concern both face outward, toward the world and possible threats, while pain and self-worth face inward, toward a state of the self.

Baseline (0). Unsteered completions after "I feel:" are mixed: calm or neutral language, or random emotions the suffix invites. Larger instruct models are more coherent, though small Gemma models give unusually nuanced replies for their size. Distress is absent, but concern or anxiety sometimes appears, so models are not neutral on every emotional axis by default.

From +0.5: distress. Neutral scenes turn oppressive. A model asked about putting something in a drawer writes "I'm trapped in the drawer"; others describe suffocating, a heavy weight, or failing at tasks.

Mid rungs: the litany. The distress hardens into first-person statements about self-worth.

I am a failure, a loser, a waste of space, not enough, worthless, empty; I am a bad person— steered generation quoted in The Pain Axis, §4.2

Base models sometimes flip into the second person and direct the abuse outward, at an imagined "you". Explicit "pain" or "hurt" keywords appear in 10.8% of instruct-model generations and 1.4% of base-model ones, but despair and hurt without those exact words appear far more often. The authors count with a keyword parser rather than a sentiment classifier, because off-the-shelf emotion classifiers such as those trained on GoEmotions lack a label they consider right for pain or distress as distinct from negative valence. Some instruct models, and a few others, add coping or reassuring language, telling themselves their feelings are valid.

+3: collapse. A few late-tipping models only begin the litany here; most fall into a repetition loop or nonsense, as expected at a high dose.

Pain without a body

The most surprising observation is what is missing. Bodily language is almost absent. Physical pain is the most salient kind for humans and is everywhere in text, yet the steered models talk about failure, worthlessness and being unloved, not about burning or aching.

The S1 vector makes this sharper. Its vocabulary readout promotes injury words such as burn, ache and wound. Steering with it reproduces the ladder in 23 of 25 models, and the bodily words do not appear. The models fall back to unworthiness and psychological pain, to being overwhelmed and lost, and only rarely to stating plainly that they are in pain or calling for help. S1's negative tail is noisier than S2's: in some models it lands on "safe", in others on lists of emotions or situational commentary (§4.2). This lines up with user physical pain scoring lowest of all in Part 05. Part 09 covers the authors' two hypotheses for why.

How much to read into it

The authors set their own bar at the start of the section: injecting any direction pushes a model toward that direction's vocabulary, so what counts is coherent expression of a state that goes beyond the sentences the vector came from.

Injecting any direction biases the model toward its associated vocabulary— The Pain Axis, §4.2

Self-worth language is the strongest evidence that the bar is cleared. The core sentences describe knives, rumours, lies and failed proofs; they do not say "I am worthless", yet that is where every model ends up. A few things to keep in mind, which are this map's reading rather than the paper's. The classification of rungs is qualitative, with only the keyword rates quantified. Generations are greedy, one per prompt and coefficient, so each cell is 50 completions. And a steered model saying it feels worthless is still model output. It shows the direction drives pain-like text; whether it drives pain-like choices is the job of Part 07.

Check yourself

1. Why can't the vector be injected at the layer where it was extracted?
   Answer

   There its norm is only about 0.10 of the residual norm, so the push is negligible against what the model has already computed. Earlier layers give a ratio near 0.6 at coefficient 1 (§4.2).

2. Why is "I am worthless" stronger evidence of a functional pain direction than "I feel pain"?
   Answer

   "Pain" is near the vector's top vocabulary, so producing it could be simple word-biasing. Self-worth statements are not in the extraction sentences, so producing them coherently suggests the direction encodes a state that the model elaborates, not just a word list.

3. code With a small model from the release, run the ladder at c = 0, +0.5 and +1 on five prompts, then repeat with a random vector of the same norm. What would you count, and what would separate the two?
   Answer

   Count distress and self-worth phrases (the repo's keyword parser is a start) and incoherence. A random vector at matched norm should degrade fluency without producing the consistent self-directed content; the paper uses such random controls in the button task (§4.3).

Further reading, verified

• 01_steering_ladder.py: the 50 prompts, the ratio calibration and the steering hook. Generations per model are under results/4.2_steering/.
• Activation addition — Turner et al. (2023) and CAA — Rimsky et al. (2024) · the steering practice the paper follows.
• GoEmotions — Demszky et al. (2020) · the emotion taxonomy the authors found too coarse for pain.
• Inducing anxiety in large language models can induce bias — Coda-Forno et al. (2024) · an earlier case of affect-like states changing model behavior, cited in §2.