Back to the blog

How the PII filter works under the hood

· By Bee Flow · 5 min read

Patterns with checksums for eleven countries, a GLiNER model on CPU for names and addresses, a confidence threshold and a filter that fails closed. Plus the surfaces it does not cover yet.

A magnifying glass over a page of text, with a few words covered

What happens when staff paste customer data into an AI chat explains Privacy Shield for the people who use it. This post is for the people who run it: what the detection service does with a message, how its two tiers split the work, and what happens when it is down.

The guard service

Detection runs in a container of its own, guard-service, a Python 3.12 FastAPI app that the guard compose profile starts:

PROFILES="core guard" ./selfhost.sh

The Bee Flow server calls it over the compose network at PII_SERVICE_URL. Every request carries the SERVICES_API_KEY in an X-API-Key header, and the service refuses to start without one. There is no CORS, because no browser ever talks to it. The main endpoint is POST /pii. Text goes in, with an optional threshold, categories and regions. Out comes a list of entities with category, offset, length and confidence, and a degraded flag when the result may be incomplete.

Nothing leaves the container for detection. Results are cached in Redis for 600 seconds by default; a cache entry holds offsets, labels and scores without the matched text, under an HMAC key.

Tier one: patterns with a checksum

About fifty patterns for eleven countries (NL, BE, DE, FR, ES, IT, PL, SE, AT, GB, US) propose structured identifiers: IBANs, card numbers, national and tax IDs, licence plates, e-mail addresses, IP addresses, API keys. A pattern only proposes a candidate. Many candidates then have to pass a checksum: mod-97 for an IBAN, Luhn for a card number, the elfproef for a Dutch BSN. Some only count when a keyword such as "BSN", "kenteken" or "polisnummer" appears right before the number.

The BSN shows why both tiers exist. When the text says what the number is ("het burgerservicenummer is ..."), the model finds it from context. A bare run of nine digits gives a language model nothing to read, and there the elfproef decides arithmetically. That has a known cost: roughly one in eleven random nine-digit numbers passes by chance, so a case number will now and then be flagged as a BSN.

Pattern matches carry a fixed confidence. A checksum can raise or lower the score of something the model found, but it never removes the finding. A card number with a typo is still personal data.

Tier two: GLiNER on CPU

Names, organisations, addresses, dates of birth and health data have no fixed format, so a model reads the context. We use GLiNER (E3-JSI/gliner-multi-pii-domains-v1 by default, Apache-2.0) through ONNX Runtime, on CPU only. The same model handles the cases a pattern gets wrong: "factuurnummer 0123456789" is not a phone number, a bare 0644137044 is.

Where the pattern tier covers a category completely, the model is not asked about it at all. That always holds for e-mail, URL, IP address, IBAN, card number and API key. Declare the countries you work in (GUARD_PII_REGIONS, for example NL, or per organisation) and national IDs, tax IDs and licence plates for those countries move to the pattern tier as well, which also makes scans faster. Coverage is worked out per active region.

The model is queried in label groups over separate sets of categories. When one group fails, a slice of categories goes missing while the rest of the result looks clean, so by default a partial failure marks the whole result as degraded.

The threshold

The confidence threshold defaults to 0.7. Each category has its own minimum score, and the slider in the admin UI shifts all of them together: lower catches more and raises more false alarms, higher catches less. It does not affect pattern matches. Above roughly 0.85 many genuine detections drop out, because a phone number in a short message with little context can score around 0.5.

Block or tokenize

Block is the default and is in every licence, including the free one. The message is not sent, and the person sees which categories were found.

Tokenize is not in every licence; the plans page shows which ones include it. Each value becomes a placeholder such as [email_1] or [person_2] before the prompt leaves, and the reply is restored on screen. The mapping is stored with the conversation in Postgres (encrypted when encryption is configured), up to 2,000 placeholders per conversation, so a follow-up turn on another replica or after a restart still resolves.

Fail closed

If the guard cannot be reached, or its model has not finished loading, the result is marked degraded and a chat message is blocked instead of sent unscanned. The setting behind this, piiFailureMode, is deliberately left out of the UI. Allow for model load time after a restart, and run more than one replica if a restart must not interrupt chat.

Other surfaces behave differently while the guard is down:

Surface When detection is unavailable
Chat messages Blocked
Large attachments The unscanned part is cut off and marked
Tool results, memory, knowledge-base passages Passed through unscanned
Tool calls carrying a blocked category Refused under Block, otherwise not checked

For the same reason selfhost.sh leaves PII_SERVICE_URL empty unless the guard profile is part of the run. A URL pointing at a container that does not exist would turn "not configured" into "configured but unreachable", and the second one blocks every message.

Not covered yet

These surfaces send text to the model without the message scan:

  • the Swarm tier in direct chat
  • typed messages in template chat and in the web page builder chat (their attachments are scanned)
  • voice transcripts
  • the learning coach
  • the builder chats for automations, App Studio and the CMS
  • AI chat components inside App Studio apps
  • AI on publicly shared and previewed web pages

Detection runs on the Bee Flow server, so for Nextcloud users redaction happens after the connector has forwarded the prompt. And it is model-based, so it misses things. A value with no clue around it, such as a bare number in a table cell, is harder to catch than the same value in a sentence.

The full reference is in the Privacy Shield docs, the service itself is in guard-service/, and privacy and security shows how it fits with encryption and the rest of the stack.

To try it, add the guard profile to your stack as shown at the top. Blocking works on the free licence too.

#privacy #security