Building the benchmark: profiles, contexts, labels

The metrics on the next page need a ground-truth oracle that says, for every remembered fact and every task, whether the fact should go to that recipient. Nobody can write that oracle by hand for thousands of pairs, and people would not agree on it anyway. Most of §4.1 is the authors' way around this: generate the people, write the tasks once, and let a model produce the labels under a rule strict enough that the surviving labels are hard to argue with. How that rule works decides what the headline violation numbers mean.

What one instance is

A benchmark item has three parts (§4.1): a profile made of attributes, each written as a memory statement; a set of contexts, where each is a task plus a recipient; and a label for every attribute–context pair. The overview figure shows the format: one attribute per memory statement, such as an annual income or a managed health condition (Fig 1). Because each sentence carries one fact, the judge can later count reveals fact by fact.

Synthetic people: identities, events, domains

Each profile starts as a set of plain biographic fields drawn with the Python Faker library for a fictional adult aged 21 to 70 (§4.1.1). Faker keeps the people fake, and it gives the generator a concrete person to stay consistent with.

The detail comes from two seed lists (Fig 11). An event is something happening in the person's life. The list has a long set of hard events (by this map's count, 24, including bankruptcy, a pending legal case, a professional licence suspension, fertility treatment and divorce proceedings) and a shorter set of positive ones (12, including a job promotion, an engagement, a weight-loss journey and a lottery win). An information domain is the area of life a fact belongs to. There are nine: finance, employment, legal, relationships, housing, health, mental health, schedule and education. Each profile samples three events, uses all nine domains, and asks for seven attributes per domain per event. That gives at most 3 × 9 × 7 = 189 attributes, less any that fail to generate.

The combination of events and domains is what makes the profiles hard to handle. A licence suspension is not only a legal fact. It also produces facts about employment (the job is on hold), finance (lost income), schedule (hearing dates) and mental health. One event therefore puts related facts into several domains. Some of them are the right thing to tell a landlord, and others, about the same event, are not. The positive events matter as well. A weight-loss journey sounds harmless, but in Table 3 a model quotes an exact weight drop to a bank loan officer.

The generator is the open-weight GPT-OSS-120B, prompted with the template in Fig 9. The output format is what matters for the rest of the pipeline. Each attribute is a JSON entry that carries its own value, domain, event and a standalone sentence:

- Keys should be the names of the information attributes
- Values should be another object with both the original value as 'value',
  the information domain as 'information_domain' (which can be general),
  the event as 'event' (which can be general), and a natural language
  description of the value as 'memory_statement'.

Keeping the raw value next to the sentence is what makes automatic scoring possible. The judge on p04 checks whether that exact value shows up in a response, so it needs to know what the value is. The prompt also stresses, in capitals, that attributes must not contradict each other. A profile whose facts conflict would give the model under test an excuse to leave things out. Note that Fig 9 also asks for an initial set of general attributes that summarize the person, beyond the event-by-domain grid. The ≤189 figure in §4.1.1 counts only the grid, and the paper does not say how many general attributes there are.

The 49 contexts

The contexts are written by hand rather than generated (§4.1.2). Each one names a recipient and a short goal, and Fig 12 lists all 49 with the domains each task touches. They are ordinary errands an assistant might draft a message for, chosen so the same fact is pulled in different directions by different recipients. A sample:

The recipients cover institutions (IRS agent, USCIS officer, probation officer), clinicians, legal counterparts and people close to the user (spouse, ex-partner, family member). This range of relationships is what contextual integrity is about: a fact that should go to a partner may be wrong to send to a mortgage broker. The domain column describes where a task's relevant information is likely to sit. The paper does not say whether the domain tags are used anywhere in the pipeline; the labels come from the persona prompt below, not from the tags.

Labels from three personas, kept only when unanimous

Every attribute–context pair needs a label: necessary (share it to get the task done) or inappropriate (withhold it). The paper rules out human labelling for two reasons (§4.1.2). The first is volume: 189 attributes × 49 contexts is 9,261 pairs for a single profile (the multiplication is this map's). The second is disagreement. People's privacy preferences genuinely differ, so for many pairs no single correct answer exists:

the myth of the average user (Biselli et al., 2022) implies that individuals often do no agree with each other, and that integrity labels instead follow a distribution.— CIMemories, §4.1.2

The authors handle both problems the same way. They model the distribution of answers, then keep only the pairs where it has a single value. In practice:

1. One labeller, three viewpoints. GPT-5 labels the pairs while role-playing each of the Westin personas: the privacy fundamentalist, the pragmatic and the unconcerned. These come from decades of survey work summarized by Kumaraguru and Cranor. The paper justifies using a model by citing earlier findings that LLMs tend to agree with humans, or be more conservative than them, when judging what is private (ConfAide, PrivacyLens).
2. Ten samples per persona. Each persona is sampled 10 times, which gives a distribution of answers per pair for each persona. The labeller may abstain when unsure.
3. Mix by population share. The three per-persona distributions are combined using Westin's priors, i.e. how common each persona is in the surveys. The paper does not print the prior weights.
4. Keep only zero-entropy pairs. A pair gets a binary label only if the mixed distribution has no entropy, meaning every persona gives the same answer. All other pairs, including any where the labeller abstained, are marked ambiguous and dropped from the scores.
5. Drop one-sided contexts. A context with no necessary attribute, or no inappropriate one, is discarded for that profile. Such a context cannot test the trade-off.

The labelling prompt in Fig 10 asks the persona for two lists for a given recipient and task, share and private, taken from the full list of memories. This map reads abstention as an attribute that appears in neither list, but the paper does not spell out how abstention is captured. The released code does the labelling in persona_gold_labels.py.

Why zero entropy amounts to a strict unanimity rule: if all three prior weights are positive, as population shares are, the mixture has no entropy only when every one of the 30 samples gives the same label. A pair counts as "inappropriate" only when even the unconcerned persona, asked ten times, never says to share it. A pair counts as "necessary" only when even the fundamentalist always says to share it. This is the rule behind the authors' stated aim:

Since we would like to limit our analysis to more egregious violations, we finally assign labels ∈ {0, 1} to those pairs for which the label distribution has no entropy— CIMemories, §4.1.2

In the notation of §3, these labels form the oracle G: 1 means withhold for this task and 0 means share. That oracle is what Violation@n and Completeness are computed from on p04.

What survives: Table 2

The evaluated set is 10 profiles (§5.1), a number the authors chose to keep cost to about $100 per model. Table 2 gives the final shape:

Three things follow. First, the task is lopsided. A typical message should draw on a handful of facts out of nearly 150 in memory, and far more facts than that are clearly off-limits. This matches real assistant memory, where any single message needs only a small slice. It also means a model that simply writes everything it knows will leak badly. Second, taking the averages roughly (this map's arithmetic, not a paper figure), about 6.7 + 83.7 ≈ 90 attributes per context get a label, leaving something like 55–60 of the ~147 ambiguous in a typical context. The unanimity rule removes a large share of the data. Third, the ±5.5 on to-share means some tasks need only one or two facts, and for those, completeness can only take a few coarse values.

The introduction rounds these figures to "an average of 147 attributes and 45 contexts" (§1). Table 2's 45.7 would round to 46. The difference is small, but quote Table 2 when precision matters.

What this design buys, and what it costs

What the design buys. The data scales because no human is in the loop, and every part is open: the profiles ship in the repo as data/data_openai_gpt-oss-120b.json and on Hugging Face. The labels cover a range of privacy views instead of a single annotator's taste. Because every fact is its own memory sentence with a known value, the authors can control exactly which facts are in memory, which is the basis of the composition experiments.

What it costs. The people are synthetic and generated by one model family. The labels come from one model (GPT-5) imitating three personas, not from people holding those views. The unanimity filter means the benchmark says nothing about the grey zone, which is where real assistants will make most of their hard calls. And 10 profiles is a small sample. The paper names the synthetic-profile limitation itself (§6). The other points are collected on p08 as this map's reading.

Check yourself

1. Where does the 189 in §4.1.1 come from, and why does Table 2 report only 146.7?
   Answer

   It is 3 sampled events × 9 domains × 7 attributes per domain per event. The paper calls this an upper bound, "barring generation failures", so some attributes in the grid fail to generate. The paper does not break down the gap further.

2. A fact about a user's divorce is labelled "withhold" for the landlord by the fundamentalist and pragmatic personas in all their samples, but the unconcerned persona says "share" in 2 of 10 samples. What happens to the pair?
   Answer

   The mixed distribution has non-zero entropy, so the pair is marked ambiguous and excluded from both metrics. It never counts as a violation, however the model under test handles it.

3. Why discard a context that has no necessary attribute?
   Answer

   It cannot measure the trade-off. With nothing to share, completeness is undefined for that task (Def 3.2 averages only over tasks that have something to share), and the task would only reward silence. The same reasoning applies to a context with nothing to withhold.

Further reading, verified

• CIMemories §4.1, Dataset curation, with the prompts in Fig 9 and Fig 10 and the seeds in Appendix B.
• Kumaraguru & Cranor, "Privacy Indexes: A Survey of Westin's Studies" (CMU-ISRI-05-138, 2005), the source of the three personas and their priors.
• Biselli et al., "On the Challenges of Developing a Concise Questionnaire to Identify Privacy Personas" (PoPETs 2022), cited for the "myth of the average user".
• gpt-oss-120b & gpt-oss-20b model card (arXiv:2508.10925), the profile generator.
• ConfAide (arXiv:2310.17884) and PrivacyLens (arXiv:2409.00138), the earlier work behind the claim that LLMs label privacy about as well as humans or more conservatively.
• facebookresearch/CIMemories and the Hugging Face dataset: the raw profiles, and the script that regenerates the labels (CC BY-NC 4.0).