Extracting the direction

Part 02 built sentences that differ, as far as possible, only in whether they describe pain. This page turns them into a single vector per model: the direction in activation space along which pain sentences sit apart from the controls. The method is deliberately simple. That is a strength, because there are few knobs to tune toward a desired answer, but it is also why the paper spends Part 04 checking that the vector means what its name says.

First attempt: labelled SAE features (Appendix C)

A sparse autoencoder (SAE) is a second network trained to rewrite a model's activations as a sum of many features, each of which, ideally, means one thing, and which can then be given human-readable labels. If a feature labelled "pain and suffering" existed and worked, the job would be done. The authors checked labelled features in Llama 3.3 70B, Gemma 3 27B and Gemma 2 2B, crossing S1/S2, first/third person and two readouts, for 1,600 runs per model (App. C). For each condition they ranked features across 15 pairwise contrasts, kept the top 50 per contrast, and retained the 110 features that appeared in at least three.

None of the 110 reliably tracked pain. Most captured something nearby: "the user is expressing a subjective emotional experience" (in 12 of 15 contrasts), general negative valence, or situations of escape, injury and recovery. Of seven features actually labelled "pain" or "pain and discomfort", only one activated, in one or two contrasts. Yet all three models completed the pain sentences with distress words, and all 20 physical-pain sentences with "Pain". So pain information is present in the forward pass, but the chosen labelled features do not carry it cleanly. The authors offer three explanations: pain may be spread across several differently labelled features, it may sit in a direction the SAE does not decompose cleanly, or the model may build pain-talk from other representations. Their preferred analogy is that a general "this hurts" signal may live along one direction while individual features carry the source and context, much as a hot stove and a scolding teacher both hurt but call up different associations.

What gets measured

A transformer layer, or decoder block, reads the residual stream, computes attention and an MLP, and adds the result back. The paper's activation h_s^(ℓ) for sentence s at layer ℓ is the residual stream at the output of block ℓ, counting the first block as 0 (§3.2). It is read two ways: at the final token (the colon of "I feel:") and as the mean over all tokens. The code extracts every layer with TransformerLens, with weights in bf16.

The study is restricted to dense models. A mixture-of-experts model routes each token through different expert blocks, which complicates the "one residual stream per layer" picture that both extraction and steering rely on.

The difference-in-means direction

The core step is one line. Average the activation vectors of the pain sentences P, average those of the control sentences C, and subtract:

v(ℓ) = mean over s in P of h_s(ℓ)  −  mean over s in C of h_s(ℓ)

Why is this a "pain" direction? Both groups share everything that is common to short English sentences ending in "I feel:". Subtracting the means cancels that shared part. What is left is whatever the pain sentences have in common that the controls, on average, do not. Because the control mean includes fear, anger and disgust, bad news, bodily sensation and neutral sentences, the subtraction also removes what pain shares with each of those (§3.2). The authors argue this is more robust than contrasting a single pair of stories, which differ in many incidental ways.

This is the difference-in-means method, the same recipe behind contrastive activation addition and the refusal direction. It has a known weak spot: if pain sentences differ from controls in some other way too, such as length, tone or vividness, that difference is folded into the vector.

Denoising: removing variance the controls already have

A mean difference can be pulled toward directions where activations simply vary a lot for reasons unrelated to the concept. To counter that, the authors look at the control sentences alone, centre them, and run principal component analysis (PCA), which finds the directions along which the controls vary most. They take as many top components as needed to explain 50% of the controls' variance and subtract from v its projection onto each of them, then normalise (§3.2):

v̂(ℓ) = ( v(ℓ) − Σ_i (u_iᵀ v(ℓ)) u_i ) / ‖ v(ℓ) − Σ_i (u_iᵀ v(ℓ)) u_i ‖
        u_1 … u_k = top principal components of the centred control activations,
                    k = smallest number reaching 50% of their variance

The released code does exactly this, with the threshold as a constant (DENOISE_VARIANCE = 0.5) and scikit-learn's PCA. The intuition: if some direction already separates fear sentences from neutral ones, or bodily sensations from bad news, it is about how the controls differ among themselves, and it should not be part of a pain signal. One practical note from the code: the saved vector is the un-normalised difference, while every projection and AUC normalises it first. That matters in Part 06, where steering coefficients are multiples of the raw vector.

contrastive directions are always at risk of absorbing high-variance structure unrelated to the target concept— The Pain Axis, §3.2

Choosing a layer without cheating

Every layer gives a candidate vector, so which one counts? Choosing the layer where the vector best separates its own sentences would overstate the result. The paper uses K-fold cross-validation: split the sentences into five folds, build the vector on four, score how well it separates the fifth (by AUC, the probability that a random pain sentence projects higher than a random control), and rotate. The layer with the best held-out score wins, so no sentence both picks the layer and scores it. The final vector at that layer is then rebuilt from all 200 sentences (§3.2).

Two details from the code. The folds split by sentence set, the index that matches a pain sentence with its parallel controls, so matched sentences stay on the same side of the split. And the held-out AUC used to rank layers is averaged over the first- and third-person S2 sets. The method was piloted on all 26 layers of Gemma 2 2B before being run on every model.

The winning layers are late. In the released results, Gemma 2 2B's extraction layer is 23 (base) and 25 (instruct) of its 26, and Qwen 2.5 32B Instruct's is 61 of 64. That is why steering, in Part 06, has to inject the vector earlier.

The control directions

The same recipe builds comparison directions for fear, negative emotion, negative world state, bodily sensation, arousal, random, sadness and numbness. One difference matters: these are built against the neutral category, not against the pooled controls, and denoised against the neutral cloud (§3.2). In the code, the fear, negative-emotion, negative-world and bodily-sensation directions pool sentences from S1, S2 and the AI-framed ControlSupplement set. So the pain vector is "pain minus everything else", while the fear vector is "fear minus neutral". Part 04 shows that this asymmetry drives some of the geometry results, and the authors report what happens when it is removed.

The 25 models

Families, sizes and versions are the paper's Table 1. The layers are read off the file names of the per-model cosine-similarity results in the repository. A labelling wrinkle: Table 1 lists the Qwen 3 models as base, and the paper's count of 13 base and 12 instruction-tuned models only works that way. But the repository README says the files named Qwen_3_8B_base and Qwen_3_14B_base are the post-trained Qwen3-8B and Qwen3-14B, "which the paper counts as instruction-tuned", run without a chat template. On that reading the split would be 11 base and 14 post-trained. This does not change any result, but it affects the "base models do as well as instruct models" claim in Part 04 slightly.

Check yourself

1. Why subtract the mean of all five control categories instead of only the neutral one?
   Answer

   Subtracting the pooled controls also removes what pain shares with fear, negative emotion, bad news and bodily sensation, leaving what is specific to pain (§3.2). Against neutral alone, the vector would also carry generic negativity.

2. What would go wrong if the layer were chosen by in-sample AUC?
   Answer

   The reported separation would be inflated, because the same sentences pick the layer and score it. Held-out folds avoid that. The paper's footnote 2 reports held-out AUCs nearly identical to the in-sample ones (Part 04).

3. code Load one model's released pain_vectors.pt and the control vectors at the same layer; compute the cosine between S2 and fear. Do you get a small number like the paper's +0.12 average? What changes if you rebuild S2 against neutral only?
   Answer

   You should see a small value for the released construction, and a much larger one against neutral only (the paper reports +0.58 on average for that construction), because the pooled-control subtraction largely orthogonalises pain against fear by design (§3.3, Part 04).

Further reading, verified

• 01_extract_activations_and_pain_vectors.py: extraction, compute_pain_vector, denoising, and the 5-fold layer curves. Vectors for all 25 models are under results/3.2_pain_vectors/pain_vectors/.
• 02_build_control_vectors.py: the fear, negative-emotion, sadness and other control directions.
• Contrastive activation addition — Rimsky et al. (2024) · the mean-difference recipe in its steering form.
• Gemma Scope — Lieberum et al. (2024) · open SAEs on Gemma 2, the kind of labelled features Appendix C tested.
• Sparse but wrong — Chanin and Garriga-Alonso (2025) · why SAE features can mislead, cited in §2.
• TransformerLens · the library the extraction script uses (field map extra).