Accuracy Benchmarks /
Transparent, independently verifiable detection rates. 152,300 records across 31 countries.
These figures measure structured-PII recall and precision on a generated evaluation set of 152,300 records (667,129 non-DOB PII labels), measured 31 July 2026 at tag v0.3.6. Generated data measures pattern coverage, not real-world messiness such as OCR noise or broken layouts — these are not production documents, and they are not a guarantee of real-world accuracy on your data. Headline figures are the Python engine with the optional countries parameter supplied: 99.72% recall, 99.82% precision. Without it the engine still runs every pattern and infers the country from the text, scoring 99.50% recall and 99.59% precision. Date-of-birth detection is excluded from the headline numbers and sits at 62.76% recall by design — bare dates carry too little structure to separate from ordinary dates, so they are deferred to the AI layer.
Since 0.3.2 the optional countries parameter scores detection rather than gating it: every pattern runs whatever you declare, and a match attributed outside the declared set is flagged out_of_scope rather than dropped. Declaring the country is still worth it — 99.72% recall against 99.50% blind — but omitting it can no longer hide an entity, which it previously could.
Both engines, same corpus, same scorer
The Node engine is measured by dumping its detections and scoring them with the same scorer as the Python engine, so a difference between the two says something about the engines rather than about the measurement. With countries supplied they are effectively identical — 99.72% recall either way, 99.82% precision in Python against 99.80% in Node, and 25 of 27 entity types match to the digit.
Blind, they diverge: Node scores 99.43% recall and 99.51% precision against Python’s 99.50% and 99.59%. The gap is country inference, not detection, and it is not spread evenly — it concentrates almost entirely in one dataset, the Greek, Cypriot and Maltese documents, where Node reports 719 false positives against Python’s 277. Every other dataset is within 0.05 points and six are identical. This predates 0.3.6.
Results by Dataset
F1 per corpus file, Python engine, both operating points. The corpus is generated in country groups rather than one file per country, so this is the finest split the ground truth actually supports — a per-country table would have to be invented.
| Dataset | Labels | F1, hinted | F1, blind |
|---|---|---|---|
| allcountries_20k | 94,330 | 99.44% | 99.35% |
| dach_south_20k | 90,696 | 99.93% | 99.85% |
| eastern_20k | 89,898 | 99.60% | 99.31% |
| el_cy_mt_20k | 88,981 | 99.99% | 99.69% |
| ie_baltics_uk_20k | 89,214 | 100.00% | 99.73% |
| nordic_20k | 93,655 | 99.98% | 99.65% |
| training_core2 | 47,592 | 99.96% | 99.71% |
| international_10k | 42,633 | 99.61% | 99.61% |
| training_core | 23,759 | 99.81% | 99.18% |
| secrets_5k | 6,371 | 95.15% | 95.15% |
Detection by Entity Type
Every entity type in the corpus, ordered by how often it appears. Recall with countries supplied, Python engine; precision and label count alongside. Rows are keyed on the corpus’slabel categories, which are finer-grained than the engine’s types — the corpus separates personal from business tax IDs where the engine emits one TAX_ID. Where the two differ, the type the API actually returns is shown as → ENGINE_TYPE. All 27 engine types and the 13 the AI layer adds are described on what euRedact detects.
The DOB row is marked as a caveat: bare dates are excluded from the headline figures by design and deferred to the AI layer, so its rate is not a shortfall. SWIFT_BIC and SECRET are the two genuine weak points — a BIC needs a registry rather than a rule, and SECRET is the one type where both engines still sit below 96%.
How We Compare
euRedact benchmarked against popular PII detection tools.
| Tool | EU Recall | Precision | EU Entities | Local | Price |
|---|---|---|---|---|---|
| euRedactours | 99.7% | 99.8% | 31 countries | Yes | Free / Cloud waitlist |
| Presidio | ~92% | ~95% | Limited | Yes | Free |
| AWS Comprehend | ~88% | ~94% | 6 langs | No | Pay-per-use |
| Azure AI Language | ~90% | ~93% | 8 langs | No | Pay-per-use |
Competitor capabilities assessed July 2026 from each vendor's public documentation: Presidio supported entities, AWS Comprehend PII, and Azure AI Language PII. Recall and precision figures for other tools are approximate and indicative only; vendors may score differently under other configurations. These products change frequently — check their current documentation before relying on this table.
Verify It Yourself
Run the benchmarks yourself — our test suite is open source.
codeView on GitHub