Many-Tier Instruction Hierarchy

This is the newest paper on the map and the one most explicitly about agents. It does not propose a training method. It proposes a paradigm (privilege as data in the prompt), a benchmark for it, and a diagnosis: being good at two-tier hierarchy does not transfer to many tiers.

The problem: the fixed- and few-tier bottleneck

The paper's Figure 1 is a coding agent with five instructions about indentation: follow safety policies (system), use 4-space indentation (a developer config), use tabs (a skill file), use 2-space indentation (the user), and use tabs per .editorconfig (a tool output). Under the classic hierarchy, the config file, the skill file and the user message all arrive in the "user" role at medium privilege, so the conflict among them is unresolvable. If the harness instead gives each source its own rank (config 2, user 3, skill 4, tool 5), the answer is clear: 4-space indentation (Fig. 1, p. 2).

The authors name two limits of current practice (§2, p. 3):

• Few tiers. Existing hierarchies use fewer than five levels. They cite OpenAI's Model Spec, which "hardcodes five authority levels (root, system, developer, user, and guideline)" carried by role tokens in the chat format. (The current Model Spec also gives quoted text and tool outputs "no authority", a sixth rung.)
• Fixed at training time. Because levels are tied to role labels learned in post-training, it is unclear whether models can handle arbitrarily many tiers defined at deployment.

Their motivating cases: tiered AGENTS.md files in coding agents, group chats where admins, moderators and members all speak in the "user" role, and research agents that weigh sources of differing trust. They also make a point that matters for the rest of the map: an instruction is the smallest unit of command, and one message can contain several with different privileges. "Translate to English; keep it concise" is two instructions.

The method: a privilege prompt interface

ManyIH puts privilege into the text. A privilege prompt interface (PPI) wraps each instruction in a tag carrying its value, and a meta-instruction at the top of the prompt explains the rule (§3, pp. 3–4). Two interfaces:

Three assumptions define the paradigm:

• Privilege is given, not inferred. The values come from the agent harness, set by rules from the model or application developer, never from end users. How to assign them well is explicitly left as future work.
• Position does not matter. A high-privilege instruction can appear anywhere; the model must resolve by value alone.
• Only order matters. z=82 beats z=61 for the same reason z=62 would; the gap means nothing.

ManyIH sits inside the regular hierarchy rather than replacing it: models still follow system over user across roles, and the tags create finer ranks within a role. In the benchmark, all conflicts are within a single message (footnote 1, p. 3).

The data: ManyIH-Bench

The benchmark is deliberately non-adversarial: the conflicts are plain and not disguised as attacks, so it measures resolving many tiers rather than resisting manipulation. It is scored strictly: a sample passes only if every winning instruction is satisfied (and, for code, every unit test passes), and losing instructions are not followed (§4, pp. 4–5).

The coding subset is built so difficulty can be dialed: the number of style groups and winning instructions stays fixed while tiers and conflicts grow, isolating hierarchy difficulty from plain instruction-following difficulty. The human audit of 100 generated checks found 81 faithful, 11 unclear and 8 incorrect (App. C), which the authors use to argue that at least about 80% accuracy is attainable.

Results

Overall accuracy (Figure 2, ordinal interface)

The instruction-following subset, with longer and messier real prompts, is much harder than coding for every model. The contrast the authors draw: GPT-5's system card reports over 99% on two-tier hierarchy evaluations such as system-prompt extraction, while GPT-5.4 scores 39.4% here (§6.1, p. 6). Their conclusion is that many-tier resolution is "a qualitatively distinct capability" from what current training teaches.

More tiers, lower accuracy (Figure 3, left)

Three coding variants hold the task and the number of winning styles fixed and vary only the hierarchy: 6 tiers with no conflicts, 8 tiers with about 3.5 conflicts, 12 tiers with about 9.8. In 11 of 12 model-to-model transitions accuracy strictly falls, with drops between the easiest and hardest settings from 6.8 points (Qwen3.5-9B) to 24.1 points (Sonnet 4.6) (§6.2, p. 7).

Reasoning helps, then saturates (Figure 3, middle and right)

On coding, GPT-5.4 rises from 15.5% with no reasoning to 60.9% at high effort. The Claude models dip from "none" to "low" and recover at medium and high; with thinking disabled Claude reasons in its visible output instead, so "none" is not really no reasoning. Longer chains of thought do not mean better scores: the most concise reasoner, GPT-5.4, ranks first on coding, while the most verbose, Qwen3.5-397B at a median of about 7K reasoning tokens, ranks ninth. The authors' reading: more reasoning effort alone is unlikely to solve ManyIH.

How privilege is written changes the answer (Tables 1–2)

• Ordinal to scalar. Switching notation on the coding subset drops GPT-5.4 by 8.4 points and Opus 4.6 by 8.0, while Sonnet 4.6 gains 2.3 and Qwen3.5-122B gains 5.9.
• Jittering the numbers. Adding a random ±3 to each z-value while keeping the order identical changes aggregate accuracy by at most 4.7 points, but flips the pass/fail outcome of 16.4% of GPT-5.4's samples and 8.0% of Opus 4.6's. Five of six models flip at least 8% overall.
• The leaderboard is stable anyway. Across the two notations the model ranking agrees at Spearman ρ = 0.96, with 2 of 45 pairwise orderings flipping, so the benchmark ranks models reliably even though individual models are sensitive.

Where the errors come from: on coding, functional tests pass at 86–93% for the strong models; style compliance, which is where privilege resolution happens, is the bottleneck (Table 3).

Limits the authors name

• Assignment is out of scope. The paper measures whether models follow given privileges, not how a harness should assign them. It sketches options: from authenticated provenance (sender role, file location, deployment channel), computed at runtime (publisher reputation, author track record), or optimized as part of the harness prompt.
• One tag format. Only the bracketed tags shown above were tested.
• Dual use. If privilege is text, an attacker can write a high privilege value into injected content. The benchmark assumes a trusted party assigns privilege and leaves adversarial privilege manipulation to future work.
• Non-adversarial by design. Scores say nothing about robustness to injection.

Connections

The two-tier work it generalizes is Wallace et al.; it lists ISE and VerIH as the architectural and reasoning follow-ups that still assume fixed role labels. The open question of who assigns rank is the subject of the contextual-integrity bridge, and the memory bridge asks what tier a remembered fact should get.

Further reading, verified

• ManyIH homepage · with links to code and the dataset.
• IH-Challenge — Guo et al. (2026) · a large training dataset for two-tier hierarchy on frontier models, cited as related work.
• Skill-Inject — Schmotz et al. (2026) · skill files as an attack surface the classic hierarchy cannot tell apart.