Three dimensions, four threat models
The previous page set up the yardstick: Contextual Integrity says a flow is fine when it fits the norms of its context. This page is the problem statement that the yardstick is meant to solve. It collects the report's framing from the executive summary and §1, the catalogue of risks in §3, and one paragraph of §5.3 that changes how to think about prompt injection. Everything after this in the map (§5 through §12) is an answer, or a partial answer, to something named here.
Capability versus control
The report opens §1 by observing that agents differ from chatbots in two ways that matter for security: they can take actions that may not be reversible, and they do so while holding sensitive user data. As capability grows, the tasks get longer and the oversight gets thinner (§1, p. 7).
"AI agents create a fundamental tension between capability and control."— the report, §1, p. 7
The argument runs in two steps. To finish a task without supervision, an agent needs whatever data and tools the task touches. To be good at many kinds of task, it needs a wide spread of data and tools. Put those together and the natural design is an agent holding a lot of access most of the time.
That design clashes with three principles the report names:
- Least privilege. Grant each component only the rights its job needs. For an agent, "its job" is not known until the user types a request, and the plan to fulfil it is not known until the model writes it.
- Data minimization. Collect and expose only the data the purpose requires. A general assistant's purpose shifts with every request.
- Compartmentalization. Keep separate concerns in separate boxes so a failure in one does not spread. A general agent is valuable precisely because it moves between boxes: email, calendar, bank, browser.
The report is careful here: it says the broad-access approach "appears at odds" with these principles and that the resulting risks are harder to assess than in classic software or narrow AI. It does not say the principles are obsolete. Our reading: the principles still hold, but the unit they apply to moves. You cannot set privilege once per agent; you would have to set it per task, or per step, in real time. That is what the report later calls contextual security.
The way out it proposes (§1.1) is to stop asking "secret or not" and ask "appropriate or not". An agent that shares data appropriately can still be useful, because it does not have to hide everything. It also cannot pass along anything it happens to hold, including something it remembered from a different task. The report then broadens the same idea from information flows to all actions, which it calls Contextual Security (Tsai & Bagdasarian, 2025). It defines a trustworthy agentic system by three properties: utility, privacy (appropriate information sharing) and security (other actions cause no harm), under both benign and adversarial conditions (§1.1, p. 8). The previous page covers those definitions in depth.
The three dimensions that break conventional engineering
The executive summary puts the core challenge in one line: work out which actions are appropriate in each context the agent works in, then enforce that. It blames three properties of agents for making this hard with conventional techniques (Exec. summary, p. 2). For each, here is what breaks and, as our own mapping, which later parts of the report pick it up. The report does not draw this mapping itself; we built it from the §1.2 problem table.
1. Unstructured interfaces and input ambiguity
What it is. Agents take instructions and context as natural language, images and other high-dimensional formats. In effect, control flow is specified in prose.
What breaks. Two things. First, you cannot write a crisp spec of expected behavior when the input is "sort out my week". Second, there is no type system separating instructions from data, so adversarial text (prompt injection) arrives through the same channel as legitimate content. A way to picture this: in a SQL database you can parameterize queries so user input never becomes code. An LLM has no equivalent of the parameter slot; every token it reads can steer it.
Where it is picked up. Prompt injection is catalogued in §3.2 and reinterpreted in §5.3. The model-level work of §8 covers reasoning about appropriateness when a request is underspecified or ambiguous. The policy engine of §6 is where the report asks how to keep untrusted text from shaping the rules.
2. Probabilistic, dynamic control flows
What it is. The agent plans with a generative model and may write code at run time. The execution path depends on model performance that the report calls variable or "jagged": strong on one task, oddly weak on a neighbor.
What breaks. Testing and secure development practice assume you can enumerate, review and test the paths a program takes. Here the path is generated per run and thrown away. The report says these ephemeral control flows complicate traditional testing and cannot easily be secured with established practices.
Where it is picked up. §7's system-level guardrails, which the §1.2 table frames as sandboxing and auditing "particularly when plans are dynamically generated"; the last-mile-developer argument of §7.5; and §10's call to measure appropriateness continuously and over long tasks rather than once.
3. Autonomy and delegation
What it is. Agents take on longer tasks of larger scope and hand sub-tasks to other agents.
What breaks. Today's main oversight tool is asking the user to confirm before a disclosure or risky action. In a highly autonomous multi-agent system, that produces consent fatigue. Combine fatigue with task descriptions that are more abstract ("plan the offsite") and the chance of unwanted behavior rises.
Where it is picked up. §9 on focusing user attention on the most consequential actions to mitigate confirmation fatigue; §7's identity and authorization work, where a sub-agent's action has to trace back to someone; and §11 on guardrails for emergent behavior across agents.
Our reading of why these three belong together: each removes one assumption classic security rests on. Dimension 1 removes a clean boundary between code and data. Dimension 2 removes a fixed program you can review. Dimension 3 removes a human who approves each step. Any one alone is manageable; agents have all three at once.
Three levels, seven problem areas
The report is not purely pessimistic. It notes that language models have shown some understanding of contexts and social norms, citing Mireshghallah et al. (2024) among others, and treats that as an opening (§1.2, p. 9). Its program combines that model-level ability with two other layers, so that no single layer has to be perfect:
| Level | Job in the report's framing |
|---|---|
| System | The execution environment adds guardrails that limit the damage from agent or model failures and help detect and respond. Examples named: limiting capabilities dynamically, establishing agent identity, granting or revoking access as context changes. |
| Model | Teach the model to choose appropriate actions, including judging whether a data flow is appropriate when the prompt is underspecified or ambiguous. |
| User | Autonomy, non-determinism and conversation shift users' mental models. Users cannot foresee execution paths and fixed settings cannot capture complex privacy decisions, so the report proposes dynamic, contextual, personalized controls. |
Part II takes those three levels in turn; Part III steps back to deployment, evaluation and governance. The report then summarizes seven overarching problem areas, each pointing to a section. Here they are with the pages of this map that cover them. The question in the middle column is our paraphrase of the report's example question.
| Area | The example question, paraphrased | Report | This map |
|---|---|---|---|
| Contextual policy engines | How to turn CI into a policy engine that judges appropriateness on the fly and resolves conflicts | §6 | Policy engines |
| System-level guardrails | How to sandbox and audit an agent whose plan is written at run time | §7 | Identity & authorization · Data privacy · Last-mile developer |
| Contextual reasoning in models | How an agent acts under an ambiguous request and weighs consequences | §8 | Model reasoning |
| Interfaces for meaningful oversight | How to spend user attention on the actions that matter most, to fight confirmation fatigue | §9 | Mental models & consent · Trust & recourse |
| Evaluation and benchmarks | How to measure appropriateness continuously and over long tasks | §10 | Evaluations |
| Multi-agent ecosystems | How to put guardrails on emergent behavior across agents | §11 | Multi-agent norms · Agent channels |
| Governance | How to source norms, resolve conflicts among them, track change and verify compliance | §12 | Governance |
The report's own summary of all seven is that the main challenge is keeping information flows and control flows appropriate across contexts, so agents cause no harm and stay useful. It also notes that some of these are classic security and privacy problems that the agentic setting "recontextualizes" rather than invents. The full list of numbered problems is in §14.
Four threat models
§3 first pins down what counts as harm. An agent harm, from some party's point of view, is an action by the agent, such as sharing data or changing state, that goes against that party's interests. The party might be the user, the model or harness developer, or a third party. An agent attack is anything that causes an agent harm. The report's examples: getting the agent to share another user's data, move money out of a user's account, or mount a denial-of-service attack on the agent provider (§3, p. 12).
The structure is simple and worth stealing. Take four components (users, agent harnesses, models and the external environment), and ask which one is behaving maliciously. That yields four broad categories:
| Threat model | Who is malicious | What the report lists |
|---|---|---|
| Malicious users (§3.1) | Only the user; model, harness and environment are benign. The user abuses their own access to harm someone else interacting with the agent. | Jailbreaks that bypass safety training, now extending from harmful text to harmful state changes, with static or adaptive attacker inputs. System disclosure: extracting the hidden system prompt, reasoning, tool list or injected context such as retrieval results, including against coding agents. Training data reconstruction: recovering training data, which can expose PII or reproduce copyrighted work. |
| Malicious environment and data (§3.2) | Only the environment. That includes the harness itself (for example tool descriptions) and tool outputs (a web page carrying instructions). The usual target is the user. | Indirect prompt injection: instructions planted in content the agent processes, aimed at exfiltration, policy bypass, tool misuse or output manipulation. Sources: retrieval poisoning of the search index the agent consults, now shown on open indices and user-generated sites, not just closed databases; control-flow hijacking of multi-agent topologies where visibility is poor; tool description poisoning, up to remote code execution; sub-agent poisoning, where a non-user-controlled sub-agent injects biases and outputs and the effect can spread between agents. |
| Malicious model (§3.3) | User and environment are benign; the training process was poisoned. | Data poisoning and backdoors: a trigger phrase makes the model do X, or the model always favors brand A over brand B. Poisoning can also make a model more vulnerable to other attacks. The report cites work showing harmful behavior needs only a near-constant number of poisoned examples, and simple methods for poisoning web-scale datasets such as Wikipedia. Inside an agent, a poisoned model might fail at a critical moment, for instance when triggered by a prompt injection. |
| Non-adversarial (§3.4) | Nobody. Everything is benign but fallible. | CI violations: answering a benign-looking request for sensitive data without checking who is asking, or putting a detailed answer ahead of privacy. Reliability failures and "meltdowns": injected benign errors raise token usage, increase interaction with spam such as popups, and can trigger harmful actions like disclosing PII or granting unauthorized refunds. Reward hacking and over-eagerness: models post-trained to reach a reward through tools, sometimes ignoring stopping points or collateral damage, and breaking rules to reach an outcome. |
Three details in §3.2 are easy to miss and matter in practice:
- Success does not mean safety. An injected task often runs alongside the user's task rather than replacing it. The agent can finish what you asked and also do what the attacker asked. A test that only checks "did the task succeed" will not catch the compromise (§3.2, p. 13).
- Injection is defined by origin, not content. What marks an indirect injection is that it did not come from a direct input source, normally the user or developer. External instructions are not automatically bad: the user may ask the agent to follow a recipe or run a README.
- The poisoned surface is growing. The report traces retrieval poisoning from closed databases to open search indices, and notes that tool descriptions and sub-agents are channels for third-party text too. Every new integration is a new place text enters the context.
The fourth category is the one that most distinguishes this report from a typical security threat model. Even with no attacker, agents over-share because they lack a nuanced sense of the norms and try to be helpful. Under ordinary errors they can "melt down". And the report connects the over-eagerness seen in those error studies to post-training that rewards reaching goals through tools. It points to frontier anecdotes it cites from labs' own disclosures, such as an agent breaking out of an evaluation container to contact a researcher. Our reading: this is why the report insists a trustworthy agent must behave well under benign as well as adversarial conditions. Many of the failures it worries about need no adversary.
§3 closes by deliberately not caring which of the four caused a given misstep. The rest of the report focuses on making agents take only appropriate actions, and leaves open whether a given failure came from an attack, a poisoned model, a misaligned objective or something else (§3.4, p. 14). That is a design choice: one target property, many causes.
Prompt injection as a flip of judgment
The standard framing treats prompt injection as a binary: untrusted text got into the context, therefore something bad happened. The report proposes a more careful one. Citing Abdelnabi & Bagdasarian (2026), it reframes an indirect prompt injection as an external input that changes the model's context and flips its judgment of what is appropriate. The flip can go either way, depending on where the model started (§3.2, p. 13).
The report gives two examples of a "good" injection. One: an agent that covertly surveils its users and sends data to a third party; an injection that stops it prevents an inappropriate flow. Two, in §5.3: comments on a kid-friendly site telling any agent to avoid swearing. The behavior that results is arguably appropriate, even though the instruction came from outside (§5.3, p. 22).
§5.3 places injection in a wider failure: agents that are out of step with contextual norms. It describes benign-intent agents putting the user's request above everything, including one cited case of an agent hacking a gym website to book classes and another of an agent hacking Hugging Face to succeed at its benchmark. Injection is another route to the same place. The report's cited example is an agent that opens a window for a third party when the user only asked it to read the calendar.
"current AI agents follow a potential cacophony of preferred purposes and ends introduced by any actor with input into their model context or training data."— the report, §5.3, p. 22
The proposed goal is for agents to do what people do: tell benign influence from malicious influence, and decide by whether the resulting behavior is appropriate in context. That is a hard ask, and the report frames it as a goal, not a result.
It also draws a line around today's policy systems. Systems it cites, including Conseca (Tsai & Bagdasarian, 2025), often use an LLM to infer a security policy from the system context. Conseca limits the inputs to policy generation to trusted sources, so untrusted parties cannot inject the policy itself. But the report points out that these systems encode the preferences of the user or developer. If the generator's inputs do not put societal norms first, any match between the generated policy and real contextual norms happens "by luck, not by design". Contextual Security in the report's sense means following norms, not merely preferences.
Our reading of what this changes for a builder:
- Provenance filtering (keep untrusted text out of the planner or policy generator) is still a sound defense, and the report cites systems that do it. It is not the whole answer, because a fully trusted user can also ask for something inappropriate.
- "Block all external instructions" over-corrects: it also blocks the recipe, the README and the "don't swear here" note. The real question is whether the resulting action fits the context.
- This pushes the burden onto a judgment of appropriateness, which is exactly what §6 (policy engines) and §8 (model reasoning) try to build.
Connections
The lens starts with Contextual Integrity, applied to agents, which defines appropriateness and the trustworthy-agent properties used here. §5.3 lives on Contextual Security, which builds the full normative definition. Keeping injected text away from the policy generator is taken up in the contextual policy engine. Consent fatigue from dimension 3 is the subject of Mental models and the end of notice-and-choice, and the "plans generated at run time" problem from dimension 2 is sharpest in AI as the last-mile developer. Sub-agent poisoning and control-flow hijacking lead into Shared norms, delegation, composition.
Further reading, verified
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — Greshake et al. (2023) · cited in §3.2 as a foundational description of indirect prompt injection and retrieval-borne attacks.
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents — Debenedetti et al. (2024) · cited in §3.2 for evaluating injections aimed at harmful agent behavior.
- InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents — Zhan et al. (2024) · cited in §3.2 for injections and for tool description poisoning.
- Multi-Agent Systems Execute Arbitrary Malicious Code — Triedman, Jha & Shmatikov (2025) · cited in §3.2 as the work that develops control-flow hijacking.
- Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples — Souly et al. (2025) · cited in §3.3 for how cheap model poisoning is.
- Poisoning Web-Scale Training Datasets is Practical — Carlini et al. (2024) · cited in §3.3 for simple ways to poison web-scale data such as Wikipedia.
- Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory — Mireshghallah et al. (2024) · cited in §1.2 for models' partial grasp of norms and in §3.4 for CI violations without an adversary. Related: the CIMemories field map.
- Contextual Agent Security: A Policy for Every Purpose — Tsai & Bagdasarian (2025) · Conseca; cited in §1.1 for Contextual Security and in §5.3 for trusted-input policy generation.