Redacteu
/Benchmarks

euRedact Cloud Benchmarked

euRedact Cloud and four other systems on the published benchmark: 900 synthetic IT, HR and medical documents in five languages, 27,831 annotated items of personal data. What each system finds, and what it leaves readable.

All 900 documents

F2 counts recall twice, because a missed item leaks. “Found” is recall with overlap matching: an item counts when any part of it was redacted. “Left readable” is the share of items with less than 75% of their letters and digits hidden. The Article 9 and 10 column is the share of those items masked by the same strict rule.

SystemMeasuredF2FoundPrecisionLeft readableArt. 9/10 masked
euRedact2026-11-140.92194.2%84.8%7.3%78.5%
GLiNER2-PII2026-11-020.70668.7%70.3%33.0%1.9%
MS Presidio2026-11-020.65663.2%77.2%44.6%14.1%
bards.ai v32026-11-020.63258.7%71.0%43.9%0.7%
Mistral Small 42026-11-020.56153.0%73.4%48.5%5.5%

F2 is each system’s figure as published in the benchmark’s held-out review; the other columns pool the three sets’ counts. Pooled the same way, F2 is 0.690 for GLiNER2-PII and 0.608 for bards.ai v3. euRedact’s release frozen before these sets were opened (1 November, measured 2 November with the other four) scored F2 0.919, found 82% of the Article 9 and 10 items and left 7.8% readable; the table shows the release of 10 November.

Per type of personal data

The benchmark’s own 35 types, grouped as it groups them, with the Article 9 and 10 data first. “Items” is how many of that type the 900 documents hold; the other four systems are shown by what they found. GLiNER2-PII was not asked for the Article 9 and 10 types.

TypeItemseuRedactFound by the four others
FoundLeft readableGLiNER2-PIIMS Presidiobards.ai v3Mistral Small 4
Article 9 and 10: special categories and offences
HealthHEALTH5,18886.9%17.9%1.6%37.1%0.9%10.3%
Criminal offenceCRIMINAL_OFFENCE36264.6%41.7%2.8%39.0%0.8%5.8%
Genetic dataGENETIC23689.4%17.8%6.8%57.2%3.4%20.8%
Religion or beliefRELIGION_BELIEF21180.6%28.0%10.4%63.5%8.5%15.6%
Biometric referenceBIOMETRIC20761.8%47.8%26.6%64.7%9.2%20.3%
Trade-union membershipTRADE_UNION17786.4%22.0%18.1%77.4%3.4%1.7%
Political opinionPOLITICAL_OPINION15478.6%34.4%22.1%40.9%9.7%8.4%
Sex life or orientationSEX_LIFE_ORIENTATION14348.9%53.8%2.8%46.2%0.7%10.5%
Ethnic originETHNIC_ORIGIN13491.8%11.9%27.6%73.9%20.2%25.4%
Names
Person namePERSON7,03499.4%0.7%94.9%93.6%97.2%66.5%
Username or handleUSERNAME88395.3%4.8%88.8%62.8%62.5%64.9%
Occupation
Job titleJOB_TITLE3,24496.9%3.6%84.8%10.2%23.5%25.5%
Places
CityCITY1,56596.7%3.3%96.4%99.0%83.3%68.2%
Street addressSTREET_ADDRESS655100.0%0.1%98.9%99.9%93.1%94.7%
PostcodePOSTCODE645100.0%0.3%99.4%98.1%92.6%95.2%
Contact details
E-mail addressEMAIL903100.0%0.0%99.6%99.0%100.0%88.6%
Phone numberPHONE407100.0%0.3%100.0%94.3%99.0%96.3%
Government identifiers
Tax identifierTAX_ID246100.0%1.2%83.3%78.5%97.6%88.2%
Vehicle plateVEHICLE_PLATE14494.4%5.6%36.1%73.6%94.4%93.8%
National identifierNATIONAL_ID135100.0%0.0%84.4%88.1%96.3%95.6%
Passport numberPASSPORT127100.0%0.0%98.4%89.0%96.1%97.6%
Driving licenceDRIVERS_LICENSE121100.0%0.0%91.7%86.0%92.6%95.9%
Financial
Financial amountFINANCIAL_AMOUNT1,90195.8%4.4%90.8%0.5%54.1%55.8%
IBANIBAN212100.0%0.0%91.5%99.1%96.2%93.4%
Payment cardPAYMENT_CARD151100.0%0.0%27.2%99.3%24.5%80.1%
Network identifiers
IP addressIP_ADDRESS523100.0%0.0%99.2%99.6%96.8%76.9%
MAC addressMAC_ADDRESS212100.0%0.0%25.0%100.0%90.6%82.1%
Personal URLPERSONAL_URL170100.0%0.6%0.0%98.2%9.4%90.6%
Other identifiers
Employee identifierEMPLOYEE_ID53196.2%4.3%86.1%69.7%83.0%88.5%
Device serialDEVICE_SERIAL23625.4%74.6%42.4%60.2%67.8%83.5%
Customer identifierCUSTOMER_ID18798.4%1.6%83.4%57.8%86.1%88.2%
Professional identifierPROFESSIONAL_ID166100.0%0.0%65.1%81.3%93.4%95.8%
Medical record numberMEDICAL_RECORD_NUMBER143100.0%0.0%55.3%76.2%97.9%99.3%
Credentials
Secret or API keySECRET15599.4%7.1%89.7%71.6%72.3%91.6%
Dates
Date of birthDATE_OF_BIRTH323100.0%0.6%100.0%93.2%99.7%99.1%

Per set and per language

Three sets of 300 documents, and sixty documents per language in each set. F2, with the share left readable under it.

SystemPer setPer language
ITHRMedicalGermanEnglishFrenchDutchPolish
euRedact0.9315.8% readable0.9455.6% readable0.9009.5% readable0.9295.9% readable0.9197.2% readable0.9247.6% readable0.9118.7% readable0.9247.2% readable
GLiNER2-PII0.80916.5% readable0.78920.7% readable0.52752.2% readable0.68732.7% readable0.70230.4% readable0.67835.0% readable0.69732.8% readable0.68633.9% readable
MS Presidio0.70733.0% readable0.68938.4% readable0.59856.7% readable0.67040.7% readable0.65547.1% readable0.65045.7% readable0.66442.7% readable0.64047.2% readable
bards.ai v30.74127.8% readable0.69234.9% readable0.44960.8% readable0.60743.5% readable0.63340.3% readable0.60944.6% readable0.61043.3% readable0.58147.8% readable
Mistral Small 40.64537.8% readable0.61241.7% readable0.46560.2% readable0.56348.0% readable0.56448.1% readable0.55748.9% readable0.55249.5% readable0.56948.0% readable

How to read these figures

When and how. euRedact was measured on 14 November 2026 through the production service with the public SDK 1.0.0 (model release of 10 November). The other four were measured on 2 November 2026, with every configuration fixed before the sets were opened. One run per system.

The data. Synthetic documents, no customer data, written and annotated by language models under one guideline family; agreement before adjudication was 0.85–0.97. None of these documents is in euRedact’s training data. The answer key’s guideline is not euRedact’s own set of definitions: where the two disagree, euRedact loses points.

The other systems. MS Presidio with EU recognizers and term lists, configured in six rounds against 65 development documents from the benchmark’s earlier sets, none of them among the 900. GLiNER2-PII with its model card’s labels plus job title and money amount. Mistral Small 4 with a prompt that lists its 34 types, each with a definition, and asks for structured output. bards.ai v3 decoded as its model card does.

Which systems. Compared systems are limited to those that can be self-hosted or are operated under EU jurisdiction only. Services subject to the US CLOUD Act are out of scope.

Precision. Part of what euRedact redacts beyond the annotations is masked on purpose, such as departments and tests in a medical context; that is one of the reasons its precision is 85% rather than higher.

The all-sets F2 is the figure the benchmark publishes for each system. Pooling the three sets’ counts gives 0.690 for GLiNER2-PII and 0.608 for bards.ai v3; the per-set and per-type figures are exact.

Rerun it. A 60-document sample with its gold standard, the harness, the configurations and our scores are on GitHub (source-available, for internal use). All 900 documents and their annotations are pinned by hash, and euRedact’s full-set predictions are committed by hash, so they can’t be changed after publication. The figures on this page were exported from the benchmark repository at commit 92863c1 on 2026-11-16.

Run It on Your Documents

euRedact was measured through the same service a free account uses. The harness on GitHub takes an API key and runs the 60-document sample on the free plan; your own documents go in the same way, and no text or result is kept. Or see a redaction first in the live demo, on a sample or text you made up, no account needed.