Operationalizing CI: the contextual policy engine

Section 6 is the hinge of the report. Part I up to this point argued that agent privacy and security are questions of appropriateness in context (Contextual Integrity) and that security, too, needs a contextual version (§5). Section 6 asks the engineering question: if we take that seriously, what does a system have to do? Its answer is a decomposition into steps and an architecture that separates judging and enforcing from planning. The Part II chapters that follow (§7 systems, §8 models, §9 users) are each one leg of that architecture.

The five CI reasoning steps

The report defines operationalizing CI as making sure no agent acts in a way that violates legitimate contextual norms, and then says that any agent that does this must, by definition, be able to perform five steps (§6, p. 23):

The subsections follow these steps in order. Our reading: most current defenses cover one or two steps well (often step 5, a blocklist or a confirmation prompt) and quietly assume the others.

Step 1 is already hard (§6.2)

Step 1 has no numbered problem of its own, but the report flags two difficulties. First, each effect has to be described in CI terms, the standard tuple for an information flow. For effects that are not information flows (deleting a file, booking a flight) the report says the open problems of §5 must be solved before we even know which parameters define the norm. Second, effects can be internal (should this datum feed into planning? may the agent edit a file in its own workspace?) or external via tool calls, and agents meet new tools at runtime. Because agents cross application boundaries, the report treats the space of contexts and actions as practically unbounded, and it grows with every new actor, capability, tool or peer agent.

OP 6.1 · How much to trust the model, how much the user

The question. As models improve, how far can we let them make privacy and security calls, and when we hand a decision to the user instead, how do we make that practical?

The two poles. The report describes an ideal "oracular" model that never proposes a CI-violating action despite its own randomness, and that asks the user or other sources exactly when it needs to. If such a model existed, steps 1 to 5 could all happen inside inference. That is the goal of §8. The report then says plainly that no such model exists, for three reasons it gives (§6.1, p. 23): LLMs are stochastic and prompt-sensitive; alignment techniques reduce bad behavior rather than remove it; and commercial models already disclose information in contexts people find inappropriate (it cites the ConfAIde benchmark of Mireshghallah et al., 2024). It adds a theoretical result it cites, Wolf et al. (2024): any behavior with finite probability can be elicited, with probability rising as the adversarial prompt gets longer.

The other pole fails too. Pushing every decision to the user captures true intent but does not scale to the volume of an agent's actions, runs into attention limits and consent fatigue, and the user's own preferences may not be the last word when they clash with social norms or law (developed in §9).

Where the report lands. Neither pole works alone, so it proposes enforcement layered on top of the core model, supplementing user-level defenses. The open part is the balance point, which moves as models improve.

OP 6.2 · Can models understand contextual norms?

The question. Can a language model efficiently pull the relevant norms out of a messy context? Older systems would match an effect against rules written in advance. LLMs open a different path: infer norms from natural language and extrapolate beyond hand-written rules (§6.3).

The evidence so far. The report sorts existing work into three groups (§6.3, p. 24): benchmarks that test whether models judge CI flows correctly; privacy-aware agents that use contextual reasoning to decide what to disclose (it cites AirGapAgent among others); and training methods that try to bake norms in through fine-tuning or RL. Its conclusion is cautious. Models show at least some latent knowledge of norms: they can identify actors, infer appropriate recipients, and often agree with human judgments. This suggests norm understanding comes partly from pretraining. But the report says their competence with CI theory as a whole is still unclear.

Why that is not enough for agents. Benchmarks mostly score one judgment on one fixed scenario. An agent has to pick which norms apply, notice when it is unsure, reconcile competing norms, generalize to unfamiliar contexts and stay consistent across long interactions. The report treats current results as encouraging but not decisive and sends the reader to §10 for the richer benchmarks this needs.

OP 6.3 · Representing norms

The question. How should norms be stored, kept current and adapted across communities and cultures?

The reframe. The report expects a model's built-in knowledge of norms to be incomplete, incoherent in places, out of date and not universal, and it suggests treating that as a permanent feature rather than a bug to patch. The consequence is that systems need ongoing ways to extract, characterize and validate norms from many reliable sources. It criticizes current training data on this point: formal privacy policies cover only norms that someone wrote down, and synthetic scenarios cover only a constructed space. Real norms also live in community rules, organizational policies, law and observed practice.

Knowing norms is not reasoning with them. To make a decision, an agent must handle norms that interact: hierarchies, apparent conflicts. Filling in the five CI parameters correctly is not enough. The report suggests metadata on each norm could help: where its authority comes from, how widely it is held, how bad a violation would be.

A direction, not a result. One idea it floats is a norm registry carrying that metadata, which would allow explicit curation. It points to Limited Memory Language Models, which store factual knowledge in a knowledge base during pretraining and retrieve it at generation time, as a hint that this kind of split could work. A registry could fill gaps in under-specified user intent and could even hold rules for exceptional conflicts. Its example is a right to privacy about mental health colliding with the need to prevent immediate danger. The report notes that enumerating norms is itself unsolved, but that it can be done ahead of time, outside live agent operation. It names taxonomies of norms that support retrieval, conflict resolution and policy generation as the broader open direction, since most of the difficulty appears when norms are applied, not listed.

OP 6.4 · Turning natural-language norms into formal policy

The problem. A judgment only matters if it can be expressed as specific actions to allow or block. But intents and norms arrive in natural language ("don't be creepy," "keep this between us," the report's examples), and the norms themselves sit in text or in model weights. Without an oracular model, the report argues, enforcement needs formal logic, and static rule-based translation from such phrases does not work (§6.4, p. 26). OP 6.4 asks how to build tools that make sure policies generated from unstructured input are accurate.

The design space the report lists

The classic route is a policy over actions (it cites Schneider, 2000). For open-ended contexts and tools found at runtime, the policy itself can be generated on the fly, with the social context as input. An earlier, non-agentic example is a use-based approach where the policy is a state machine tracking the task and limiting actions by state. The report then lists recent systems, which differ mainly in what a "policy" is:

"No consensus has yet emerged around a standard policy language for arbitrary agents, nor even around the set of necessary or sufficient properties that such a language ought to have."— the report, §6.4, p. 26

The contextual policy engine

From the common pattern in several of these systems the report names a component, the contextual policy engine, with two jobs:

1. Dynamic policy generation. Map context and norms to a formal policy over (formally modeled) agent actions, in a machine-checkable language. If it cannot produce a formal policy, it may escalate to the user to clarify intent.
2. Runtime enforcement. Given the policy, a proposed action and the current execution state, allow, deny, modify or escalate to the user for confirmation.

Figure 2 of the report (p. 27) sketches one way to place it, redrawn here:

The figure's caption says this is an illustrative instantiation: other architectures are possible, and the ideal one is an open problem.

Why separate it from the planner. The report says the split has theoretical and practical advantages. Its example is Conseca, which uses the separation to cut any link between untrusted inputs and policy generation, so even indirect prompt injection cannot shape the policy. It also notes that industry safety architectures increasingly wrap enforcement components around the model instead of relying on alignment alone. A way to picture this: the planner is a contractor who may have read a malicious note left at the job site; the policy engine is the building code, written before the contractor arrived and checked at each step.

The missing "compiler"

The deepest part of OP 6.4 is translation. How do you compile vague natural-language policy into formal, verifiable logic (temporal logic, for instance) without losing what the person meant? The report places three reference points (p. 27). Policy-as-code tools such as Open Policy Agent need rules written by hand. LLM-based "constitutional AI" (Bai et al., 2022) relies on prompting, not formal verification. ControlValve (Jha et al., 2026b) sits between them: it uses natural-language prompting to generate a context-free-grammar policy listing allowed and disallowed sub-agents and tools. Whatever wins, the report says, will need system-level enforcement abstractions that formal policies map onto. That is the question "what is the formal policy language?" again, from the other side.

OP 6.5 · Keeping harmful context out of policy generation

The question. Can we separate trusted from untrusted data (adversarial or just misaligned) even in complex settings?

Why it bites. The best policy generator would know the user's exact intent, all their personal information, every relevant norm, and trustworthy facts about the systems involved. It never does. On top of ordinary ambiguity, the generator faces prompt injection and misaligned context, such as the wrong set of norms for the task (§6.4, pp. 27–28). The generator is the most sensitive part of the engine. If an attacker can steer it, the enforcer will faithfully enforce the attacker's policy.

Two approaches the report cites. Many systems (Conseca, CaMeL, the f-secure work and several others) have developers mark which context sources are trusted. Conseca goes furthest: its generator sees only trusted context. Other work (Hu et al., 2026; Beurer-Kellner et al., 2025) relaxes the boundary through low-entropy information bottlenecks, letting a little untrusted content through to improve utility while still guarding against hijacking. Our reading: this is the usual security-versus-utility trade. The report defers the authorization side to §7.

OP 6.6 · Which policy wins

The question. How do we formalize precedence among conflicting policies so outcomes are robust, justified and understandable, and how do mutually untrusted agents negotiate toward compromise or at least progress?

Two kinds of conflict (§6.5, p. 28). Inside one agent, a user preference can collide with a societal norm. The report's example is a user asking the agent to hide a reportable disease that a rule says must go to public health authorities. Between agents, policies can clash because agents have different data needs or represent communities with different norms.

"conflicts between the user's preferences and social norms are not just errors—they are social violations."— the report, §6.5, p. 28

Directions. For single-agent conflicts, the report again suggests a taxonomy of norms (source of authority, epistemic certainty, severity, underlying values and purposes) to decide precedence. It warns against naive priority lists and proposes three layers: preferences, which can be overridden; values, which should be respected; and requirements, which must be obeyed. These layers may interact differently in different places, for example the same agent deployed in two countries. It points to earlier work on automated negotiation of user-defined privacy policies (Mosca & Such, 2021) as a possible foundation. Multi-agent conflicts are left to §11 and the governance questions to §12.

Resolving uncertainty (§6.6)

Step 4 has no numbered problem, but the report calls navigating uncertainty a major open challenge and expects different kinds to need different fixes (§6.6, pp. 28–29). It names four: the user's intent, the environment, the consequences of an action, and gaps or conflicts among norms. It then sketches several responses:

• Better model reasoning (§8) for ambiguity in language, including exploring the environment with tools.
• Asking the user. Even an oracular model cannot recover intent that was never formed. Users may not know their own preferences. The report calls the "translation gap" between vague intent and executable code a primary failure mode, and asks how to help users see when no precise translation exists and settle on one interpretation. Frequent questions may be justified for legal requirements (it names GDPR and HIPAA) but too costly for minor preferences, risking consent fatigue or users giving up on the agent.
• An AI delegate, a model prompted or fine-tuned to reflect the user's preferences, which could take over routine calls and leave only truly ambiguous points to the human. It could live inside the policy generator or be fed to it as trusted context. The open question is how to make one users can trust across contexts.
• Probing the environment when the unknown is the setting itself (casual chat or a medical emergency?), and estimating harm probabilities so a framework like Solko-Breslin et al. (2027) can enforce a risk ceiling.

OP 6.7 · Enforcement that works at scale

The question. Given a judgment, the report wants the agent to comply deterministically where possible, and without an oracular model that means an explicit enforcer like the one in Figure 2 (§6.7, p. 29). OP 6.7 lists six axes it must scale along: users, queries, concurrent agent loops, distinct agent systems, policies and capabilities.

Three kinds of pressure. The report groups the axes by what they demand:

• Cost. Latency and compute. How do you check complex, stateful permissions for billions of concurrent agent loops?
• Abstractions. Should agent capabilities get formal semantics, and how? Its example is a long-running coding agent writing Bash or Python that can change environment variables, move files or change permissions. Every one of those effects has to be modeled before you can enforce a policy stated in higher-level terms like information flow control.
• Architecture. Is there a reference design any new agent could adopt? How do you verify or audit enforcement across many agents and instances? These tie to trust boundaries in §7 and agent-to-agent issues in §11.

Finally, the enforcer must cope when the judgment is indeterminate, using the strategies of §6.6, with user escalation picked up in §9. Our reading: a good answer is an enforcer cheap enough for every tool call, rich enough for arbitrary code, and auditable by third parties.

Bridge to Part II

The short Part II introduction (p. 30) maps the engine onto three levels: system-level enforcement abstractions (§7), model-level ability to find and resolve uncertainty in norms, preferences and intent (§8), and user-level strategies for handling ambiguity without consent fatigue (§9).

Connections

The non-information-flow effects in step 1 depend on Contextual Security (§5). The enforcement side and trusted-context boundaries continue in Identity, authorization, accounting (§7) and AI as the last-mile developer. The oracular-model ideal is Teaching models contextual reasoning (§8). Escalation and consent fatigue are Mental models and consent (§9). Benchmarks for norm understanding are in Evaluations (§10); the sibling CIMemories field map covers one of them in depth. Cross-agent conflicts go to Shared norms (§11) and Governance (§12).

Further reading, verified

• Contextual Agent Security: A Policy for Every Purpose — Tsai & Bagdasarian (2025) · Conseca, the report's main example of a policy generator isolated from untrusted input.
• Defeating Prompt Injections by Design — Debenedetti et al. (2025) · CaMeL, policies expressed as execution plans.
• Progent: Securing AI Agents with Privilege Control — Shi et al. (2026) · policies as predicates over tool actions.
• System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective — Wu, Cecchetti & Xiao (2024) · the "f-secure" LLM system, plan-based policies and trusted/untrusted separation.
• Formal Policy Enforcement for Agentic Systems — Palumbo et al. (2026) · Datalog as the policy language (the report cites it as "...for Real-World Agentic Systems").
• VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation — Miculicich et al. (2025) · statically verifiable policy functions.
• Cedar: A New Language for Expressive, Fast, Safe, and Analyzable Authorization — Cutler et al. (2024) · verifiable RBAC/ABAC policy language.
• Design Patterns for Securing LLM Agents against Prompt Injections — Beurer-Kellner et al. (2025) · relaxing isolation through low-entropy bottlenecks.