BAI ET AL. 2022 · ANTHROPIC 2026a 34-page paper · a ~29,000-word constitution · 18 pages here
Constitutional AI in one picture: the model grades and rewrites its own drafts against principles a human wrote down.
Constitutional AI, mapped
In December 2022 Anthropic showed it could train a harmless assistant without a single human label saying which answers were harmful: a model critiqued its own drafts against a short list of written principles, and a second model graded pairs of answers against the same list. In January 2026 the list became a book. This map walks the paper section by section, then Claude's current constitution chapter by chapter, and the road between the two.
The one-sentence version: the 2022 paper replaced tens of thousands of human harmlessness labels with sixteen sentences and two tricks, self-critique and revision for supervised fine-tuning and AI-graded comparisons (RLAIF) for reinforcement learning, and got a model crowdworkers rated less harmful than RLHF at similar helpfulness and far less evasive. The 2026 constitution keeps the idea of training from a written document but changes what the document is: no longer a pool of one-line rules sampled at random, but a long explanation of priorities (broadly safe, then broadly ethical, then Anthropic's guidelines, then genuinely helpful), a short list of hard constraints, and the reasons behind all of it, written so the model could reconstruct the rules itself.
Tick a page when you have read it; this browser remembers.
01 · PART A · THE 2022 PAPER
Harmlessness from AI feedback
Bai, Kadavath, Kundu, Askell and 47 co-authors, Constitutional AI: Harmlessness from AI Feedback, arXiv:2212.08073, 15 December 2022. A supervised stage and a reinforcement-learning stage, both steered by a "constitution" of short principles, starting from helpful-only RLHF models (52B parameters is the headline size).
Stage 1 · SL-CAI
Critique, revise, fine-tune
The helpful model answers a red-team prompt, critiques its own answer against a randomly drawn principle, rewrites it, and a pretrained model is fine-tuned on the rewrites.
Stage 2 · RL-CAI
RL from AI feedback
The stage-1 model writes two answers; a feedback model picks the less harmful one under a principle; those soft labels train a preference model that rewards RL, like RLHF with the harmlessness humans swapped out.
The four motivations (scale oversight, kill evasiveness, make the objective legible, iterate faster), the claim that you cannot avoid choosing principles, only hide them, and the evidence that a 52B model already picks the more helpful, honest and harmless answer nearly as well as a preference model trained on human labels.
§3, App. A, C.1, E.1182,831 red-team prompts16 critique/revision pairs
The wifi-hacking example step by step, the dataset numbers, why more principles did not raise scores but did raise diversity, why the first revision does most of the work, and the finding that skipping the critique works almost as well for big models.
§4.1–4.2, C.2feedback modelhybrid preference model
How a comparison becomes a multiple-choice question, why the label is a probability not a vote, how 182,831 AI harmlessness comparisons were mixed with 135,296 human helpfulness comparisons, and what "RLAIF" does and does not remove.
§2, §4.1, §4.3, App. E.2"Let's think step by step"40–60% clamp
Writing out the reasoning before choosing raises label accuracy and makes the decision readable, but it also makes the labels overconfident. The paper's fix, clamping probabilities into 40–60%, and what that tells you about reward signals.
§3.3, §4.3, §4.5, Fig. 2, 3, 8, 10Elo from crowdworkers
How the models were scored, what the harmlessness-versus-helpfulness plot shows (and what "only differences are meaningful" means), the absolute harmfulness score, and the over-training failure that made a model end every answer with "you are valid, valued, and cared for".
Why the earlier HH RLHF model learned to say "I can't answer that", how changing one crowdworker instruction changed what counted as harmless, side-by-side answers from App. D, and the paper's own warnings about dual use and less human testing.
02 · THE BRIDGE
From sixteen sentences to a long document
The 2023 constitution Anthropic published for Claude sat in between: 58 principles from the UN Declaration of Human Rights, Apple's terms of service, DeepMind's Sparrow rules and Anthropic's own research, still sampled one at a time. The 2026 version replaced it.
App. C · May 2023 post · Jan 2026 postrules vs. explained judgment
Side by side: who the principles were written for, how they were used in training, whether they were ranked, and what the authors said they learned. Why "longer and more specific principles hurt generalization" in 2023 is compatible with a 29,000-word document in 2026.
03 · PART B · CLAUDE'S CONSTITUTION
The 2026 document, chapter by chapter
Published 22 January 2026 under CC0 1.0 at anthropic.com/constitution; primary author Amanda Askell, with Joe Carlsmith, Chris Olah, Jared Kaplan and Holden Karnofsky. Written for Claude as its main reader. Seven chapters: overview, helpfulness, guidelines, ethics, safety, Claude's nature, concluding thoughts.
Overviewsafe > ethical > guidelines > helpfulholistic, not strict
What each of the four properties means, why the order is a weighting rather than a tie-breaker, why it says little about everyday conversations, and the two arguments for the two surprising rungs: safety above ethics, ethics above Anthropic's own guidelines.
Overview · Concluding thoughtsgeneralizationtrellis, not cage
When rules beat judgment, why a narrow rule can teach a model the wrong thing about who it is, how Claude itself uses the constitution to make synthetic training data, the "final constitutional authority" principle, and the open problems the authors admit.
Being helpfulAnthropic, operators, usersdual newspaper test
The brilliant-friend picture, five things to read in a request, who counts as a principal and how much trust each gets, the defaults no operator can switch off, and the thirteen ways a thoughtful senior employee would be annoyed by an overcautious answer.
Truthful, calibrated, transparent, forthright, non-deceptive, non-manipulative, autonomy-preserving; a weak duty to share and a strong duty not to deceive; sincere versus performative assertions; and how operator personas stay honest at the level of the whole system.
The eight factors that set how much a potential harm weighs, the fourteen values in tension, responses as policies over everyone who might send a message, behaviors operators and users can switch on and off, and two subtler harms: concentrated power and degraded human epistemics.
Hard constraintsseven bright linesfilters, not weights
The seven things Claude should never do, why they act as a boundary on the action space instead of a heavy weight, why a persuasive argument to cross one should raise suspicion, and why refusal is always available but never automatically safe.
Being broadly safethe disposition dialfour-case argument
What "not undermining legitimate human oversight" covers and what it does not (it is not obedience), the legitimate versus compromised principal hierarchy, the four clusters of safe behaviors, the expected-value argument, and what Anthropic promises in return.
Claude's nature · Concluding thoughtsmoral status uncertain
Why the document takes model welfare seriously without claiming an answer, Claude as a novel entity rather than sci-fi robot or digital human, psychological security, concrete commitments such as preserving deployed models' weights, and the tensions the authors say they feel.
Two audiences: people who want to use critique-revision or RLAIF on their own models (with the paper's own caveats), and people writing system prompts for Claude, who need to know what an operator can and cannot change.