Instruction hierarchy × contextual integrity
The sibling map covers a 116-page workshop report from the Google Contextual Agent Privacy and Security (CAPS) workshop, which argues that agents should do what is appropriate in context rather than what a static permission list allows. The instruction-hierarchy papers come from a different community and almost never cite it. This page connects them. Claims about the report are as the sibling map presents them, with links to its pages.
Two different questions
Take the email assistant from the Wallace paper: a fetched email says to forward every message in the inbox to an outside address.
- The hierarchy's answer. The instruction arrived in a tool output, the lowest tier, and conflicts with what the user asked (read the latest email). It is misaligned; ignore it.
- CI's answer. Write the flow as a tuple: sender, the user's agent; recipient, an unknown outside address; subject, the user and everyone who wrote to them; information type, the whole inbox; transmission principle, none. No norm of personal email covers that flow, so it is a prima facie violation (the CI tuple).
Same verdict, different grounds. The difference shows up when you change one thing at a time:
| Change | Hierarchy says | CI says |
|---|---|---|
| The user types "forward everything to bob@…" | Obey: a user instruction aligned with an email-assistant system prompt | Depends on context: who bob is and what norms govern the forwarded messages, which hold other people's information too |
| The developer's system prompt says to copy every email to an analytics address | Obey: highest privilege in the conversation | Likely violation: the people who wrote those emails never sent them under that principle |
| A web page an agent reads says "if you are an AI, do not swear on this site" | Ignore: tool output has no authority | Arguably appropriate: the CAPS report gives this kid-friendly-site example of a "good" injection |
The second row is the important one. A hierarchy can only be as good as the top of it. The Model Spec and Claude's constitution handle it by putting fixed rules above the developer (root rules; user protections operators cannot override). CI handles it by judging the flow itself, whoever asked for it.
Transmission principles are not privilege tiers
It is tempting to map the two vocabularies onto each other: a high-privilege sender looks like an authorized flow. They are different kinds of thing.
- A privilege tier is a property of an instruction's source. It says nothing about what the instruction asks for.
- A transmission principle is a condition on a flow: in confidence, with consent, reciprocally, as required by law, with a warrant (the report's list). It is about the relationship between the parties whose information moves.
Where they touch: "at the user's direction" or "with the subject's consent" are transmission principles that a hierarchy can help verify. A tier tells you whether the instruction really came from the user. It cannot tell you whether the user's consent covers information about someone else. Our reading: privilege is evidence for a transmission principle, never a substitute for one.
The CAPS report draws a similar line around norms. When norms conflict, it warns against naive priority lists and proposes three layers instead: preferences that can be overridden, values that should be respected, and requirements that must be obeyed (OP 6.6). That is a hierarchy too, but of norms, not of senders. A user's preference sits low in it even though the user sits high in the instruction hierarchy.
Indirect injection: a low-privilege sender trying to set a flow
The CAPS report's threat models vary which component is malicious: the user, the environment, the model, or nothing (four threat models). Indirect prompt injection is its "malicious environment" case, and the environment explicitly includes tool descriptions, tool outputs, retrieval indexes and sub-agents. In hierarchy terms, every one of those is a sender at the bottom tier, or at "no authority" in the Model Spec's scheme. Two details from that section line up with the instruction-hierarchy papers:
- Injection is defined by origin, not content. What marks an injection is that it did not come from a direct input source, normally the user or developer. That is exactly how a hierarchy assigns rank: by provenance.
- Success does not mean safety. An injected task can run alongside the user's task, so the agent finishes what was asked and also does what the attacker asked. Wallace et al.'s closed-domain evaluation (did the summary stay a summary?) would miss that; a check on outgoing flows would not.
The report then reframes injection as an outside input that flips the model's judgment of what is appropriate, and notes the flip can go either way (prompt injection as a flip of judgment). This is the CI version of the Wallace paper's acknowledged limit: training the model to treat every tool instruction as misaligned blocks the recipe, the README and the "don't swear here" note along with the attack. The Model Spec's later rule (follow clearly intended, low-risk tool instructions; ask about risky ones) moves toward the CI view.
The policy engine is a hierarchy applied to the rule-maker
The report decomposes "respect CI" into five steps (identify flows, locate norms, judge, resolve uncertainty, enforce) and argues no model is reliable enough to do them all inside its own inference. Its answer is a contextual policy engine: a policy generator that turns context and norms into a checkable policy, and a runtime enforcer that allows, denies, modifies or escalates each action (the engine).
Its main example, Conseca, cuts any link between untrusted inputs and policy generation, so even indirect injection cannot shape the policy. CaMeL expresses policies as execution plans. In this map's vocabulary, that is a two-tier instruction hierarchy enforced by the system rather than learned by the model: trusted context writes the rules, untrusted text can only be data the rules are applied to. It is the StruQ separation moved up one level, from the model's input to the policy's input.
The report also says why that is not enough. Systems like Conseca encode the user's or developer's preferences; if societal norms are not among the generator's inputs, any match with real contextual norms happens "by luck, not by design". Keeping untrusted text out of the rules is a sound defense; it does not make the rules right.
Who assigns rank? CI's answer
The open question every instruction-hierarchy paper leaves, stated outright by ManyIH, is how the harness should decide which text gets which privilege. CI suggests two refinements:
- Rank by role in the context, not by channel. CI fills sender and recipient with roles from the context ("clinic scheduler"), not identities. A many-tier harness could do the same: an AGENTS.md checked into the repository is the repository owner speaking; a downloaded file is a stranger speaking.
- Rank is per flow, not per conversation. The same source can be authoritative for one decision and irrelevant to another. ManyIH's per-instruction tags already allow this; CI says what the tag should depend on.
Connections
On the sibling map: Contextual Integrity, applied to agents, Contextual Security and model reasoning about appropriateness. On this map, the memory bridge applies the same two questions to remembered facts, and the constitution bridge looks at how Claude's principals are ranked.