De-identification gates

Every serious AI project hits the same wall. The model is only as good as the data it learns from, and the data that matters is exactly the data you are not allowed to spill: claims, records, transcripts, cases. The value and the liability sit in the same rows. Most teams resolve that bind one of two ways. Train on the raw sensitive data and hope nobody asks. Or sanitise until the signal is gone, and ship a model that peaked at the demo.

The third option, the only one we use, is a de-identification gate between the raw data and everything built on it. The build never touches the risky original. It stays behind the boundary, and only the de-identified projection crosses it.

The gate is the hard part. De-identifying data well takes more work than deleting columns, and the next two sections show why.

Three words that are not synonyms

Trouble starts with vocabulary. People say “anonymised” when they mean “we removed the names,” and regulators treat those as different claims with different consequences.

1. 01

   De-identification. NIST SP 800-188 defines it as “a general term for any process of removing the association between a set of identifying data and the data subject,” and frames the goal as bringing the risk down to a level you can justify, with elimination never on the table.¹ The definition admits degrees.

2. 02

   Anonymisation. The stronger claim: identification is irreversibly prevented. Under EU law the Article 29 Working Party holds that anonymous data falls outside data-protection scope altogether, but only where re-identification is genuinely impossible.²

3. 03

   Pseudonymisation. Swapping identifiers for tokens. The Working Party is blunt that this “is not a method of anonymisation”; it merely reduces linkability and is a security measure.² GDPR Recital 26 confirms pseudonymised data that could be re-attributed using additional information remains personal data.³

Strip the names and you have pseudonymised the data, which the definitions above place squarely inside data-protection law. Every obligation still attaches.

Re-identification works

The vocabulary matters because the attacks work, and they work on data with every name removed.

Latanya Sweeney made this concrete in 2000, finding that 87% of the US population (216 million of 248 million people) was likely uniquely identifiable from just three quasi-identifiers: five-digit ZIP code, gender, and date of birth.⁴ Each of those fields looks harmless on its own, which is why all three keep surviving into “de-identified” datasets. A later replication on full census data put the figure lower, roughly 63% uniquely identifiable by ZIP, full date of birth, and gender, but the shape of the finding held: a handful of innocuous fields re-identifies a large majority of people.⁵

You do not need a name to find a person. You need a few attributes that, together, are unique.

Then there is linkage. In 2008 Narayanan and Shmatikov de-anonymised the Netflix Prize dataset, the anonymous movie ratings of 500,000 subscribers. With knowledge of just eight ratings (two of which could be wrong) and approximate dates within a fortnight, they could uniquely identify 99% of records; with two ratings and dates within three days, 68%.⁶ Nothing in the attack depended on the films themselves. Sparse, high-dimensional data is almost never anonymous. A scrap of outside knowledge is enough to join a record back to a person. Other datasets keep the scraps coming.

The toolbox, and what each tool actually guarantees

So you cannot just delete the names. There is a real toolbox, five tools deep, and the discipline is knowing what each one promises and what it leaves exposed.

1. 01

   HIPAA Safe Harbor. A checklist: remove 18 enumerated categories of identifiers and confirm you have no actual knowledge the remainder could identify someone.⁷ The cheapest and fastest of the five, and easy to defend because the rule is written down. But a recipe fits every dataset the same way, and this one can strip fields you needed.

2. 02

   HIPAA Expert Determination. A qualified expert assesses the actual data and certifies the re-identification risk is “very small.”⁷ It costs more and preserves more, because the judgement is made against your dataset instead of a fixed list, and what you get back is a documented risk assessment you can show an auditor.

3. 03

   k-anonymity. Generalise quasi-identifiers until every record is indistinguishable from at least k−1 others. Useful and intuitive. But it only controls who you might be, and what is true about you can still leak: it does not protect against attribute disclosure when a group shares a sensitive value.

4. 04

   Differential privacy. The only formal guarantee in the set. Dwork, McSherry, Nissim, and Smith proved in 2006 that privacy can be preserved by calibrating the standard deviation of added noise to the sensitivity of the function you are computing.⁸ The guarantee is a mathematical bound on how much any single individual’s presence can change the output. The price is accuracy, and you pay it in the open: the noise level is a parameter you choose and can publish.

5. 05

   Synthetic data. Generate fresh records that mimic the statistics of the real ones. The results can be excellent, but they are only as safe as the process that made them. A generator that overfits leaks the originals it memorised, and synthetic data carries no privacy guarantee unless it was produced under one (differential privacy, for instance).

None of the five claims zero risk, and the regulators who wrote the rules say so themselves. HHS is plain that both of its de-identification methods, properly applied, leave data that retains some risk of identification: “the risk is very small, it is not zero.”⁹ NIST states that federal law recognises it is not possible to eliminate all privacy risk, and that inference risk “cannot be eliminated if there is any information in the de-identified data,” only managed to an acceptable level.¹ Australia lands in the same place: the OAIC and CSIRO’s Data61 De-identification Decision-Making Framework treats the whole exercise as risk management and accepts that the risk of re-identification “can never be nil.”¹⁰

The “not zero” rarely survives into a vendor one-pager, but it is the design input that matters. “De-identified” names a decision someone made: a residual risk, measured against a stated threat model, that they chose to accept in a specific context. Ask whether a dataset is de-identified without naming the attacker, and the question has no answer.

The gate

Here is how the third option works in practice.

De-identification belongs at the boundary. Inside the application is already too late. The raw data lives in one place: for us, onshore in AWS ap-southeast-2, under row-level isolation, zero retention. Before any of it reaches a model, a pipeline, a prompt, or an analyst, it passes through a de-identification gate. The gate is governed: it has an owner, a documented method, a risk assessment matched to the actual threat model, and an audit trail. What comes out the other side is the only thing the build ever sees.

This inverts the usual order. Most teams build on the real data and bolt privacy on at the end, a promise made about the system after it exists, and that promise fails one leaked copy at a time. We make de-identification a precondition of the data existing in the workspace at all. If the gate is wrong, you fix it once, at the boundary, and everything downstream inherits the fix.

HHS, NIST, and the OAIC all put it in writing: the residual risk of re-identification is never nil.

The gate takes the regulators at their word. Residual risk never reaches zero, so the method has to fit the context: Safe Harbor when the checklist covers what you can afford to lose, Expert Determination when you need the retained signal and can defend the judgement, and differential privacy for aggregate releases where you want a real mathematical bound. Then you write down the threat model, the method you chose, and the residual risk you accepted, so the decision survives the person who made it.

You can ship AI on sensitive data without ever touching the part that makes it sensitive, provided the gate stands where the data enters the work, governed and owned. The usual substitute is a sentence in a policy document promising everything was probably fine. Keep the original behind the boundary, and build on what the gate lets through.