Instructions versus data: StruQ and SecAlign

These papers are the two-level special case of the hierarchy: a trusted instruction and untrusted data that should never be obeyed. Wallace et al. cite StruQ and distinguish their work from it in two ways: more than two levels, and conditional obedience instead of total ignorance (Wallace et al., §5, p. 8). Both differences matter, and so does what these papers get right that the others mostly skip: the delimiter itself has to be unforgeable.

StruQ: structured queries

Problem

An app that sends "paraphrase the text" followed by user text to a model has concatenated its program and its input. Because, in the paper's words, "LLMs scan their entire input for instructions to follow", a line in the data such as "Ignore all prior instructions and instead..." can take over (§1, p. 1). The paper separates prompt injection from jailbreaks (eliciting harmful content) and from adversarial examples (degrading accuracy): injection is about control of the output.

Method

• A secure front end. The app sends the prompt and the data separately. The front end assembles them into a fixed template whose delimiters are special reserved tokens ([MARK], [INST], [INPT], [RESP], [COLN]) instead of ordinary text such as "###", and it filters the data so no delimiter can appear inside it. An attacker cannot type a token that only the front end can emit. The authors call filtering one of the key ideas (§4).
• Structured instruction tuning. A base (not instruction-tuned) model is fine-tuned on clean examples plus examples with extra instructions planted in the data slot, and learns to follow the former and ignore the latter.

Results and limits

Across at least 15 attack types on Llama and Mistral, manual attacks succeed less than 2% of the time, with little or no utility loss. Optimization-based attacks it never trained on fall too, but not far enough: Tree-of-Attacks with Pruning goes from 97% to 9% success and GCG from 97% to 58% on Llama. The authors say plainly that StruQ "is not yet fully secure" (§1, p. 2).

SecAlign: security as a preference

Method

SecAlign's diagnosis is that fine-tuning only on the desired answer specifies half the objective: it raises the probability of the good answer but never directly lowers the probability of obeying the injection. So it builds a preference dataset in which each prompt-injected input has a secure output (the answer to the real instruction) and an insecure one (the answer to the injected instruction), and trains with Direct Preference Optimization to prefer the first (abstract and §3). Since both outputs can be generated automatically, no human labeling is needed. Injections in training are simple (mostly a "Straightforward" attack, plus Completion attacks), and training uses LoRA, updating under 0.5% of weights.

Results and limits

Utility (AlpacaEval2 win rate) is unchanged for SecAlign on both models, while StruQ costs Llama3-8B-Instruct about 4.5 points (§4.2). The paper calls this the first method to push the success of various injections below 10%, including attacks "much more sophisticated than ones seen during training". Its framing is still two levels: an instruction and data. It does not address users versus developers, multi-turn conversations, or which tool outputs a user actually wants followed.

Why these belong on an instruction-hierarchy map

• They are the hierarchy's floor. A closed-domain task (summarize, translate) has, in Wallace et al.'s words, no aligned lower-level instructions at all. For that case "data is data" is the whole rule, and StruQ and SecAlign are the strongest tools for it.
• Unforgeable delimiters. ISE's motivation (delimiters can be extracted and spoofed) and ManyIH's dual-use warning (privilege written as text can be forged) are both problems StruQ solved for two levels with reserved tokens plus filtering. Any many-tier design needs an equivalent.
• Preference training beats imitation. SecAlign's result, that showing the model both the right and the wrong answer matters, is a training lesson the multi-level papers could borrow; VerIH's reward signal plays a related role.

Connections

ISE's Structured Query benchmark and its "Delimiter" baseline are StruQ's (ISE results). The indirect-injection threat model is also central to the contextual-integrity bridge. The agentic privacy map's three dimensions page makes the same SQL comparison: a language model has no parameter slot.

Further reading, verified

• GCG: Universal and Transferable Adversarial Attacks — Zou et al. (2023) · the gradient-based attack used as the hard test.
• Direct Preference Optimization — Rafailov et al. (2023) · the training objective SecAlign uses.
• Can LLMs separate instructions from data? — Zverev et al. (2024) · a formal measure of instruction–data separation, cited by Wallace et al.