Python SDK /
PII redaction for Python. Sync and async support, country inference, ~9.5 ms per page — or ~4.8 ms with the optional RE2 prefilter.
Installation
$ pip install euredactThe core install has no dependencies. The optional euredact[fast] extra adds an RE2 scan prefilter: one DFA pass per 1 KB window reports which patterns can match there, so the rest are skipped. Measured at ~2.0x on a 2,000-character page. Output is unchanged by construction — the prefilter only decides which patterns are worth running, and each survivor is then run over the whole text.
If you installed euredact[fast] at 0.3.1, upgrade. In that release the windowed scan could miss SECRET patterns wider than the window — a PEM private key passed through unmasked. The pure-Python default and the Node.js package were never affected. Fixed in 0.3.2.
redact()
Main entry point. Detects and redacts PII from a text string. Returns a RedactResult with the cleaned text and a list of detections.
def redact(
text: str,
*,
countries: list[str] | None = None,
mode: str = "rules",
referential_integrity: bool = False,
detect_dates: bool = False,
cache: bool = True,
) -> RedactResult| Parameter | Type | Default | Description |
|---|---|---|---|
text | str | — | Input text to scan. |
countries | list[str] | None | None | ISO country codes the document is declared to belong to. Since 0.3.2 this scores detection rather than gating it: every pattern runs whatever you declare, and a match attributed outside the set is flagged out_of_scope rather than dropped. Declaring it is still worth 99.72% recall against 99.50% blind. |
country_hint | list[str] | None | None | A prior that resolves ambiguity without narrowing scope or flagging anything out of scope — use it when you know the likely origin but do not want out_of_scope semantics. |
context | DocumentContext | None | None | Shares country evidence across the chunks of one document, so a chunk carrying no country signal is still scored with what the rest of the document established. Pair with chunk_offset. Caching is disabled while a context is in use, since the result no longer depends on the text alone. |
chunk_offset | int | 0 | Character offset of this chunk within the whole document, so evidence spans recorded on a context point into the full text. |
mode | str | "rules" | Detection mode (currently only "rules"). |
referential_integrity | bool | False | Replace PII with consistent labels (NAME_1, BANK_ACCOUNT_1) instead of generic [TYPE] labels. |
detect_dates | bool | False | Include DOB/date-of-death detection. Off by default because it requires context. |
cache | bool | True | Cache results for identical inputs. |
Example
import euredact
result = euredact.redact(
"Mijn BSN is 111222333 en IBAN NL91ABNA0417164300.",
countries=["NL"],
)
print(result.redacted_text)
# "Mijn BSN is [NATIONAL_ID] en IBAN [BANK_ACCOUNT]."
print(result.detections)countries takes a list. Since 0.3.3 a bare string raises TypeError, on every entry point, in both SDKs. Before that, countries="NL" was iterated character by character into the codes "N" and "L"; neither resolves, so the call declared nothing and every detection came back flagged out_of_scope while the redacted text still looked correct. A pipeline filtering on that field — which these docs tell you to do — kept none of them. A wrong country code still only warns; a wrong type has no correct reading to fall back on.
redact_batch()
Batch redaction. More efficient than calling redact() in a loop because it loads configs once.
def redact_batch(
texts: list[str],
**kwargs,
) -> list[RedactResult]| Parameter | Type | Default | Description |
|---|---|---|---|
texts | list[str] | — | List of input texts to redact. |
**kwargs | | — | Same keyword arguments as redact() (countries, mode, referential_integrity, detect_dates, cache). |
Example
import euredact
texts = [
"BSN 111222333",
"IBAN NL91ABNA0417164300",
]
results = euredact.redact_batch(texts, countries=["NL"])
for r in results:
print(r.redacted_text)aredact()
Async version of redact(). Offloads CPU work to a thread pool. Same keyword arguments and return type.
async def aredact(
text: str,
**kwargs,
) -> RedactResultExample
import asyncio
import euredact
async def main():
result = await euredact.aredact("BSN 111222333")
print(result.redacted_text)
asyncio.run(main())aredact_batch()
Async batch redaction with controlled concurrency.
async def aredact_batch(
texts: list[str],
*,
max_concurrency: int = 4,
**kwargs,
) -> list[RedactResult]| Parameter | Type | Default | Description |
|---|---|---|---|
texts | list[str] | — | List of input texts to redact. |
max_concurrency | int | 4 | Maximum number of concurrent tasks. |
**kwargs | | — | Same keyword arguments as redact(). |
redact_iter()
Lazy iterator for large datasets. Yields results one at a time without loading everything into memory.
def redact_iter(
texts: Iterable[str],
**kwargs,
) -> Iterator[RedactResult]Example
import euredact
texts = ["BSN 111222333", "IBAN DE89370400440532013000"]
for result in euredact.redact_iter(texts):
print(result.redacted_text)add_custom_pattern()
Register a custom regex pattern. Matches are reported with the given name as the entity type.
def add_custom_pattern(
name: str,
pattern: str,
) -> None| Parameter | Type | Default | Description |
|---|---|---|---|
name | str | — | Entity type name for matches (e.g., "EMPLOYEE_ID"). |
pattern | str | — | Regular expression pattern to match. |
available_countries()
Returns a sorted list of supported ISO country codes.
import euredact
print(euredact.available_countries()) # ["AT", "BE", "BG", ...]EuRedact class
For isolated instances with separate caches and custom patterns. Useful when different parts of your application need different configurations.
from euredact import EuRedact
instance = EuRedact()
instance.add_custom_pattern("CASE_REF", r"CASE-\d{8}")
result = instance.redact(
"See CASE-20260401",
countries=["NL", "BE"],
)
print(result.redacted_text)
# "See [CASE_REF]"The EuRedact instance exposes the same methods: redact(), redact_batch(), aredact(), aredact_batch(), redact_iter(), and add_custom_pattern().
Return Types
RedactResult
A dataclass returned by all redaction functions.
| Parameter | Type | Default | Description |
|---|---|---|---|
redacted_text | str | — | The text with PII replaced by labels like [NATIONAL_ID], [BANK_ACCOUNT], etc. |
detections | list[Detection] | — | List of detected PII spans. |
source | str | "rules" | Detection source used. |
degraded | bool | False | Whether results may be incomplete due to an internal issue. |
inferred_countries | tuple[tuple[str, float], ...] | — | New in 0.3.2. The countries the engine concluded the document belongs to, each with a score in 0-1, most likely first. Populated whether or not you passed countries. |
evidence | tuple[CountryEvidence, ...] | — | New in 0.3.2. The individual signals behind inferred_countries, so an attribution can be audited rather than taken on faith. |
detection_mode | str | "inferred" | New in 0.3.2. "declared" when you passed countries, "inferred" otherwise (including when you passed only country_hint). |
Detection
A frozen dataclass (immutable and hashable) representing a single PII detection.
| Parameter | Type | Default | Description |
|---|---|---|---|
entity_type | EntityType | str | — | The type of PII detected (e.g., NATIONAL_ID, BANK_ACCOUNT). |
start | int | — | Start character offset in the original text. |
end | int | — | End character offset in the original text. |
text | str | — | The matched PII text. |
source | DetectionSource | — | Detection source ("rules" or "cloud"). |
country | str | None | — | ISO country code the detection is associated with. |
confidence | str | "high" | Confidence level of the detection. |
country_confidence | float | 0.0 | New in 0.3.2. How strongly the document supports the country on this detection, 0-1. Separate from confidence, which is about the match itself: a pattern can fire unambiguously while its country attribution stays a guess. |
out_of_scope | bool | False | New in 0.3.2. True when the detection was attributed to a country outside the countries you declared. It is still redacted and still returned — filter on this field if you only want in-scope entities. |
CountryEvidence
One signal supporting a country attribution. New in 0.3.2.
| Parameter | Type | Default | Description |
|---|---|---|---|
country | str | — | The ISO country code this signal points at. |
source | str | — | What produced the signal, e.g. "iban_prefix", "phone_prefix", "tld". |
log_odds | float | — | How much this signal moves the score. Signals accumulate, so several weak ones can outweigh a single strong one. |
span | tuple[int, int] | — | Where in the text the signal was found. Offsets are absolute when chunk_offset is passed. |
DocumentContext
New in 0.3.2. Country inference reads signals out of the text, so a document processed in chunks loses accuracy at exactly the chunks that carry no signal of their own — a page of bare reference numbers between two pages full of Dutch addresses. A DocumentContext carries the evidence across chunks so the whole document is scored as one.
from euredact import DocumentContext, redact
ctx = DocumentContext()
offset = 0
for chunk in chunks:
result = redact(chunk, context=ctx, chunk_offset=offset)
offset += len(chunk)
print(result.redacted_text)
# Every signal the document produced, spans pointing into the full text.
print(ctx.evidence())Pass chunk_offset or the evidence spans will all point into the start of the text. Caching is disabled while a context is in use, since the result depends on the document rather than on the chunk alone.
Reuse a context only across chunks of the same document. A context shared between unrelated documents mixes their countries, which cannot cause a miss — a context influences scoring only, never which spans are found — but can attribute a value to the wrong national scheme.
Custom Patterns
Register custom regex patterns to detect domain-specific identifiers. Custom patterns are always active regardless of the countries parameter.
import euredact
euredact.add_custom_pattern("EMPLOYEE_ID", r"EMP-\d{6}")
result = euredact.redact("Contact EMP-123456 for details")
print(result.redacted_text)
# "Contact [EMPLOYEE_ID] for details"Priority order:validated patterns > custom patterns > regex-only patterns.
For isolated pattern registrations, use separate EuRedact instances.
Secret Detection
euRedact automatically detects secrets and API keys using two strategies.
Known-prefix detection
Matches tokens with recognized prefixes from common services:
Entropy-based detection
Flags 32+ character high-entropy strings found near context keywords such as key, token, secret, password, and their translations in 12 EU languages.
Performance
0.3.6 is about 13% slower than 0.3.2 per page, and about 14% slower per record. The extra work is rule and vocabulary growth from 0.3.5 and 0.3.6 — support-desk terms in nine languages, wider currency lists, more month spellings. The record figure moves more than the page figure because 0.3.5 began treating a value that fills an entire field of a delimited row as context, and a 300-character record is exactly that shape.
If you are coming from 0.3.3, you will find this faster. 0.3.3 fixed a real defect — Python’s \bis Unicode-aware where JavaScript’s is ASCII-only, so a national ID written against a non-ASCII letter (ЕГН7523169263, PESELŁ44051401359) was redacted by the Node SDK and silently missed by Python — but it paid for the fix by rewriting every \bin all 303 patterns into a three-branch union, which cost roughly 4×. 0.3.4 chose the boundary per occurrence instead and gave the time back without giving back the recall.
For throughput, install euredact[fast]: the RE2 prefilter roughly halves per-page latency. Node is unaffected by any of this at ~1.6 ms per page — its \b always behaved this way.
Supported Countries
31 European countries supported out of the box.
Entity Types
These are the 27 types this package emits, across all supported countries, backed by 346 pattern definitions and 44 checksum validators. IBAN is still accepted as a legacy alias on input, but detections are emitted as BANK_ACCOUNT.
27 is the Rules Engine layer. The forthcoming AI layer adds 13 more — names, addresses, medical information and the other categories that have no fixed format for a pattern to match, plus the GDPR Article 9 special-category types. What euRedact detects lists all 40 with the layer responsible for each, so a count of 40 elsewhere on this site is not a different number from this one.
View source on GitHub
Browse the code, report issues, or contribute.