The masking layer matters more than the model

Once sensitive data reaches a model provider, no later decision can bring it back. The layer that detects and masks that data before any model call deserves more design time than model selection. In the AI systems I build, designing the masking layer was among the hardest parts of the work, harder than choosing the models behind it.

Key takeaways

• A masking layer detects and replaces sensitive data before any text reaches a language model, local or external.
• Masking belongs at the boundary of the pipeline, as one service in front of every model call, not code repeated inside each feature.
• Detection does not need a large model. GLiNER2-PII covers 42 PII entity types with 0.3 billion parameters.
• Detection is never complete: the strongest configuration compared in the PIIBench paper reaches an F1 of 0.6455 across 82 entity types, so design for leakage.
• A few hundred labelled examples from your own traffic tell you more than any vendor benchmark.

Why does the masking layer matter more than the model?

Model selection is a reversible decision. A pipeline built behind an abstraction can swap one provider for another or move to a local model without touching business logic. Sending a customer's name, address and order history to the wrong place is not reversible. The data has left the building, it may be logged, cached or used for training, and no later architecture change undoes that.

The asymmetry is why I treat masking as the core of any AI system that touches operational data. The model decides how good the answers are. The masking layer decides whether the system may run inside a real company at all. GDPR obligations follow the data, not the model, so the question a data protection officer will ask is not which model I chose but what exactly left the infrastructure.

The other reason is practical. Models improve every quarter with no effort on my side, because vendors and open-source teams keep releasing better ones. The masking layer improves only when I work on it. Nobody will ship me a better understanding of which fields in my own systems are sensitive or which partners put phone numbers in comment fields.

What does a masking layer actually do?

A masking layer performs three steps: it detects spans of sensitive data in the text about to be sent to a model, replaces them with placeholders, and restores the original values in the response before a human sees it.

Detection covers two different problems. Structured fields are easy: the schema already says which column holds a name or a bank account, so masking is a lookup, not a guess. Free text is the hard part. A support ticket or an invoice line can contain a phone number, an identification number or an address in any format and any language, and finding those spans is a named entity recognition task, not a string match.

Replacement, in the design I would recommend, uses deterministic placeholders within a request, so the same person resolves to the same token everywhere. A model that sees CUSTOMER_1 ordering and CUSTOMER_1 complaining can reason about the connection; one that sees two unrelated tokens cannot.

A consistent token is also a pseudo-identifier. With enough surrounding context, a stable CUSTOMER_1 can be linked back to a person, so the mapping should be scoped as narrowly as the task allows and long-lived tokens should be treated as data that still needs protection.

Restoration means the mapping between placeholders and real values never leaves the infrastructure: the model only ever sees tokens, and unmasking happens on the way back, under the same access rules as the rest of the system.

Where does the masking layer sit in the pipeline?

One service in front of every model call is the placement that works. Every feature that talks to a model, whether it classifies tickets, checks invoices or answers a colleague's question, routes its request through the same masking gateway, and nothing else in the codebase holds an API key to a model provider.

Centralising the layer buys three things. First, one place to test: a single suite and one labelled dataset cover every feature at once. Second, one place to improve: when a new leak pattern shows up, the fix lands in one service instead of five prompt builders. Third, one audit log: the gateway records exactly what left the infrastructure and in what form, the record a GDPR conversation needs.

The alternative, masking logic living inside each feature, decays fast. Different features drift to different rules, a new feature ships in a hurry without any masking, and no one can answer what the system as a whole sends out.

Can a small model handle the detection?

Detection does not require a large generative model. GLiNER2-PII, published in May 2026, is a 0.3 billion parameter model extracting 42 types of personally identifiable information across languages, trained on a constraint-generated synthetic corpus of 4,910 annotated texts rather than real people's text. On the SPY benchmark it reports higher span-level F1 than the systems it was compared against, the OpenAI Privacy Filter and three GLiNER-based detectors. A model of that size runs on modest hardware inside your own network, where a masking component should run.

A second result points at where the effort should go. On PIIBench, a corrected multi-source dataset with 82 entity types, direct token-classification fine-tuning of DeBERTa reached an F1 of 0.6455 on a test set of 100,002 records, beating a source-conditioned hierarchical model and a curriculum-based variant from the same author. The conclusion of that paper matches my experience: diverse task-specific training data moves detection quality more than architectural sophistication does.

Using a large hosted model to detect PII defeats the purpose. The detector would receive the raw sensitive text, precisely the exposure the masking layer exists to prevent. The detector has to be the one component you can run entirely inside your own boundary. My own later measurement of GLiNER2-PII on public data confirmed the strengths and found the weak spot, surnames; the numbers are in why I pre-register a kill-gate for every experiment.

What slips through, and how do you measure it?

An F1 of 0.6455 reported in that paper across 82 entity types means a substantial share of entity mentions are missed or mislabelled. Narrower, well-represented types score much higher, but the honest reading is that free-text detection leaks, and the design has to survive that.

Leakage is not only a missed entity. Three unmasked details that look harmless on their own, a town, a delivery window and an unusual surname, identify a person more reliably than any single field does.

Benchmark scores are measured on someone else's data. Mine has Bulgarian names in Cyrillic and in several Latin transliterations, national identifiers whose format collides with other numbers, and addresses buried in free-text comments. No public benchmark contains that mix, so no public score predicts my leakage. The same gap between benchmark scores and production behaviour shows up everywhere in AI systems, not only in PII detection: see the demo is the easy part.

The measurement is small and boring. Take a few hundred real examples from your own traffic, hand-label the sensitive spans, and compute recall per entity type. Recall matters more than F1 here: a false positive masks too much and costs a little answer quality, a false negative sends personal data to a provider. The errors are not symmetric, so tune thresholds toward over-masking.

For categories where a miss is unacceptable, the fallback is structural, not statistical: route those requests only to a locally hosted model, or keep those fields out of model input entirely. I do not have a method that reaches zero leakage, and I distrust claims of one.

What does this look like in practice?

In the systems I work on, several features share one masking layer: text-heavy request handling, document reconciliation, and an internal service that lets colleagues' assistants query company data. Each of them sends text to a language model, and none of them is allowed to do so directly.

Designing that layer was among the hardest parts of the whole effort. I consider the time well spent, because it is the reason the features may run in production at all. The building blocks are ordinary: schema-driven masking for structured fields, a named entity recognition model for free text, one gateway in front of every model call, and tracing on each request. The mapping between placeholders and real values stays inside the infrastructure, and what any given assistant may see is decided by the role of the person using it, not by the model.

Not everything worked on the first attempt. The structured-field side was straightforward, because the schemas were known. The free-text side was not. The spans I trust least are those where a harmless-looking string carries identity: organisation names shaped like personal names, addresses written into comment fields, and numbers whose meaning depends entirely on context. The per-entity recall measurement described above now exists, and its first run turned that impression into numbers.

FAQ

Do I still need a masking layer if the model runs on my own hardware?

Yes. A local model removes the external provider, but prompts still end up in logs, traces and evaluation datasets. Masking limits what spreads into those secondary stores. It also keeps the pipeline portable if part of the workload later moves to an API model.

Is regex enough for PII detection?

For structured fields and strictly formatted identifiers, yes: regex is fast and auditable. For free text it is not enough: names, addresses and mixed-language content do not follow patterns. The practical setup is layered: schema-based masking first, regex for formatted identifiers, an NER model for everything else.

Does masking hurt the quality of the model's answers?

Little, provided placeholders are deterministic: the model reasons about CUSTOMER_1 and ORDER_17 as consistent entities, and real values return during unmasking. Quality drops when the masked value carried meaning the task needed, such as a city name in a routing question, so masking rules must be scoped per task.

What is the difference between masking and anonymisation?

Anonymisation is irreversible: once applied, nobody can recover the original values, which suits published datasets. Masking is reversible inside a trusted boundary: the mapping to real values is kept, and authorised users see restored output. Assistants that act on real operations need masking: an answer about ORDER_17 is useless if nothing can say which order that is.

References

• Urchade Zaratiana, Ash Lewis, George Hurn-Maloney. GLiNER2-PII: A Multilingual Model for Personally Identifiable Information Extraction, 2026. arxiv.org/abs/2605.09973
• Pritesh Jha. Fine-Tuning Over Architectural Complexity: Broad-Coverage PII Detection on PIIBench with DeBERTa, 2026. arxiv.org/abs/2605.25816