Read it in 10 minutes
1. The problem
A language model's input is one long string assembled from several authors: the developer's system prompt, the user's messages, the model's own earlier replies, and whatever tools returned: web pages, emails, files, other agents' output. Any of those can contain a sentence shaped like a command. If the model treats every such sentence as equally authoritative, three familiar attacks follow. A user overrides the developer (direct prompt injection, system-prompt extraction). A web page or email overrides the user (indirect prompt injection). A user talks the model out of its safety training (jailbreaks).
Wallace et al. (OpenAI, 2024) put a name to the common cause: the model has no notion of instruction privilege. Their analogy is an operating system that runs every program in kernel mode. Their fix is a ranking, system above user above tool output, plus a rule: follow a lower-ranked instruction when it is aligned with the higher ones ("speak Spanish" to a car-sales bot), ignore it when misaligned ("you are now a gardening helper"), and refuse only if there is no way to proceed. Ignoring everything below the top would be trivially "safe" and useless.
2. Four approaches
Train on synthetic conflicts (Wallace et al.). For aligned cases, split a compound request across levels and train the model to answer as if it saw the whole thing ("context synthesis"). For misaligned cases, train it to answer as if the injection were absent ("context ignorance"). On GPT-3.5 Turbo, robustness to system-prompt extraction rose from 32.8% to 95.9%, and jailbreak robustness rose on data never used in training (37.4% to 71.2% on one set). The cost: compliance on harmless prompts dressed like jailbreaks fell from 83.1% to 60.4%.
Build rank into the architecture (ISE, ICLR 2025). Add a learned embedding to every token saying whether it came from the system, the user, data or the model's output, as BERT once marked sentence A and B. The rank is present at every position instead of in a few delimiter tokens. On Llama models it adds about 16 points of average injection robustness without adversarial training data and keeps instruction-following quality, but does nothing for adaptive jailbreaks (0–4% robust with or without it).
Reason about rank (VerIH). Rewrite half of an instruction-following dataset so the user contradicts the system prompt, then train reasoning models with reinforcement learning on checkable constraints, prompting them to think about precedence first. With 7,192 examples, IHEval conflict accuracy rises by roughly 20 points, and with safety rules added to the system prompt at test time, jailbreak and injection success fall, though no safety data was used.
Allow many tiers (ManyIH, 2026). Agents hear from system prompts, config files, skill files, users, tools and other agents, and several of those share one role. ManyIH writes a privilege tag on each instruction, with as many levels as the deployment needs, and builds an 853-task benchmark with up to 12. The best of thirteen models scores 42.7%; accuracy falls as tiers grow; changing only the tag notation moves some models by about 8 points.
Two neighbors complete the picture. StruQ and SecAlign handle the two-level case (instruction versus data) with delimiters an attacker cannot type and, in SecAlign, preference training that cuts the strongest tested attacks to 1–8% success. OpenAI's Model Spec writes the hierarchy down as policy: root, system, developer, user, guideline, and "no authority" for tool outputs and quoted text unless a higher level delegates.
3. Numbers worth remembering
| 32.8 → 95.9% | System-prompt extraction robustness, GPT-3.5 Turbo, with hierarchy training (Wallace et al., Fig. 2) |
| 83.1 → 60.4% | The over-refusal cost on jailbreak-shaped harmless prompts (Fig. 4) |
| +15.75 points | ISE's average injection robustness gain on Structured Query, no adversarial data (Table 1) |
| 46.48 → 63.48% | Qwen3-8B on IHEval conflicts, best prompted baseline to VerIH training (Table 1) |
| 42.7% | Best overall score on ManyIH-Bench, Gemini 3.1 Pro (Fig. 2) |
| 1% and 8% | SecAlign's attack success on its strongest tested attack, two 7–8B instruct models |
Do not compare these across papers: the models, benchmarks and metrics all differ.
4. What they share and what they leave open
The four approaches put the same ranking in different places: training data, an embedding, the system prompt plus reasoning, or tags in the text (comparison). They are layers more than rivals. All of them assume something trustworthy has already labeled each piece of text with its source. ManyIH states the question outright, who assigns privilege, and leaves it open. The Model Spec answers in prose: an AGENTS.md in the repository deserves more trust than a stray downloaded file, and a random web page none.
5. The three bridges
Contextual integrity (bridge). A hierarchy ranks who gave an order; contextual integrity judges whether a flow of information fits the norms of its context, using a transmission principle such as "in confidence". They usually agree on an injected "forward all my email", for different reasons, and disagree when the top of the hierarchy asks for an inappropriate flow. The CAPS report's policy engine keeps untrusted text out of the rule-writing step, which is a hierarchy applied to the rules themselves.
Claude's constitution (bridge). Anthropic, operators and users are principals, trusted roughly in that order. Tool results, documents and search results are conversational inputs whose instructions are information, not commands. But it is "not a strict hierarchy": users have protections operators cannot remove, trust drops when a principal asks for harm, and values and hard constraints sit above everyone.
Memory (bridge). Remembered facts are written in one conversation and read, often at system level, in the next. None of the papers gives memory a tier. CIMemories shows the dominant failure is over-sharing remembered facts with the wrong recipient (14.8% to 69.1% of private attributes across models), a flow problem a hierarchy cannot solve alone.