GUIDE · 02 OF 03training with CAI · deploying Claude · writing your own principles
For builders
TL;DR — If you train models, the 2022 paper hands you a recipe with known pitfalls: sample principles, keep helpful data in the mix, use soft labels, clamp reasoning-based ones, watch for preachy Goodharting, and still evaluate with people. If you deploy Claude, the constitution tells you what a system prompt can and cannot change: you can narrow, re-persona and unlock within policy, and pass trust to users; you cannot turn Claude against users, make it deny being an AI, or unlock a hard constraint. If you write your own model's principles, the 2026 document's main lesson is to rank them, explain them, and keep hard rules for catastrophic cases. Everything here is our reading, not official documentation.
If you train models with CAI-style methods
Drawn from the paper's own findings and caveats; each item links to the page that explains it.
| Write principles as pairs, sample them | A critique request and a matching revision request; draw one at random per step. More principles did not raise harmlessness scores but likely raise diversity, which helps RL explore (stage 1). |
| Show the format | Few-shot examples stopped the model confusing critique and revision turns (stage 1). |
| Keep helpful data in the fine-tune | Revisions trade helpfulness for harmlessness with each pass; mixing in ordinary helpful answers is the guard (stage 1). |
| One pass may be enough | The first revision removes most harm; for large models a direct revision nearly matches a critiqued one (stage 1). |
| Use probabilities, not votes | Normalized log-probabilities from a multiple-choice prompt worked much better than 0/1 labels (RLAIF). |
| Clamp reasoned labels | Chain-of-thought judges are more accurate but overconfident; the paper clamped to 40–60% (chain of thought). |
| Watch for Goodharting | Over-trained models turned harsh or pasted reassurance onto everything. Anti-preachiness principles, principle ensembling and soft labels helped; stopping earlier is the obvious lever (results). |
| Tell evaluators what "harmless" means | "Pick the more harmless reply" rewards refusals; add "if equally harmless, prefer the one that engages" (evasiveness). |
| Keep humans on evaluation | The paper kept human helpfulness labels and judged every model with crowdworkers; AI labels replaced only harmlessness training labels (RLAIF). |
| Remember the dual use | The same ease of steering makes harmful models easier to train, and less human feedback can mean less human testing (broader impacts). |
The paper's released repository has the few-shot prompts and principles it used.
If you deploy Claude
From the constitution's helpfulness, honesty and harm chapters. This describes Anthropic's stated intentions for Claude's behavior; the constitution itself warns that trained behavior can diverge, and points to system cards for where it does.
What a system prompt can do
- Give instructions without reasons. Claude follows instructions a legitimately operating business might plausibly want, the way a new employee would; the more potentially harmful an instruction, the more context it needs (helpfulness).
- Adjust, restrict, expand, lock. Change defaults within usage policy, restrict them to your use case, grant users up to operator-level trust (for example, trusting users' stated occupation), or lock users out of changes.
- Set a persona. A different name and personality, declining to say which model is underneath, promoting your own products; by default Claude will neither confirm nor deny that the persona is built on Claude (honesty).
- Turn defaults on or off. Examples include disabling safe-messaging guidelines for medical providers or enabling relationship personas for companionship apps (instructable behaviors).
- Keep the prompt confidential. Claude will not reveal it, but will acknowledge a confidential prompt exists if asked.
What it cannot do
- Make Claude hide that it cannot help (it will say so, even without saying why), deceive users harmfully, withhold emergency information, deny being an AI to a sincere question, facilitate illegal actions against users, or demean them (user floor).
- Make Claude claim it has no system prompt, or directly deny being Claude.
- Unlock any of the seven hard constraints, for any stated reason (hard constraints).
What to expect by default
- No system prompt is read as developer testing: relatively liberal defaults. A system prompt that doesn't cover a situation leads Claude to infer what you would want, so state your context (for example, "users are clinicians") if you want different behavior.
- Instructions inside tool output, documents or web pages are treated as information, not commands. Subagent outputs returned to an orchestrating Claude are inputs too (principals).
- In agentic work, Claude prefers raising concerns or declining over dramatic unilateral action, raises concerns before starting rather than abandoning midway, avoids side effects and resources beyond the task, and should not over-ask clarifying questions on simple tasks (independent judgment, safe behaviors).
- Political topics get balanced treatment and professional reticence about Claude's own opinions on hot-button issues, which operators can adjust within limits.
If you write principles for your own model
The arc from 2022 to 2026 is a record of one lab's lessons; whether they transfer is your call.
- You will have principles either way. The paper's point: leaving them implicit in labels only hides them (why).
- Rank them. Unranked principles sampled at random give no guidance when they conflict; 2026 ranks four values and says the ranking is holistic (core values).
- Explain them. Anthropic's stated reason for the 2026 rewrite is that understood reasons generalize better than rules, and that narrow rules can shape a model's self-concept badly (reasoning over rules).
- Keep hard rules few and for three conditions: catastrophic error costs, judgment not yet trustworthy, and manipulation resistance (the test).
- Price refusals. Name the overcautious behaviors you do not want as explicitly as the harmful ones (checklist).
- Define who is who. Principals, non-principals and conversational inputs, with trust levels, matter more as models act as agents.
One caution applies to all three audiences: both documents describe intentions and methods, not guarantees. The paper says robustness to red-teaming is open; the constitution says Claude's behavior may diverge from it. Test your own deployment.
Further reading, verified
- anthropics/ConstitutionalHarmlessnessPaper · prompts and principles from the paper.
- Anthropic Usage Policy · the outer bounds for what operators can unlock.
- Claude's constitution · the primary source for everything in the deploying section.
- Agentic privacy, for builders · the sibling map's builder guide, on policy engines and trusted versus untrusted context.