euRedact Cloud Benchmarked
euRedact Cloud and four other systems on the published benchmark: 900 synthetic IT, HR and medical documents in five languages, 27,831 annotated items of personal data. What each system finds, and what it leaves readable.
All 900 documents
F2 counts recall twice, because a missed item leaks. “Found” is recall with overlap matching: an item counts when any part of it was redacted. “Left readable” is the share of items with less than 75% of their letters and digits hidden. The Article 9 and 10 column is the share of those items masked by the same strict rule.
| System | Measured | F2 | Found | Precision | Left readable | Art. 9/10 masked |
|---|---|---|---|---|---|---|
| euRedact | 2026-11-14 | 0.921 | 94.2% | 84.8% | 7.3% | 78.5% |
| GLiNER2-PII | 2026-11-02 | 0.706 | 68.7% | 70.3% | 33.0% | 1.9% |
| MS Presidio | 2026-11-02 | 0.656 | 63.2% | 77.2% | 44.6% | 14.1% |
| bards.ai v3 | 2026-11-02 | 0.632 | 58.7% | 71.0% | 43.9% | 0.7% |
| Mistral Small 4 | 2026-11-02 | 0.561 | 53.0% | 73.4% | 48.5% | 5.5% |
F2 is each system’s figure as published in the benchmark’s held-out review; the other columns pool the three sets’ counts. Pooled the same way, F2 is 0.690 for GLiNER2-PII and 0.608 for bards.ai v3. euRedact’s release frozen before these sets were opened (1 November, measured 2 November with the other four) scored F2 0.919, found 82% of the Article 9 and 10 items and left 7.8% readable; the table shows the release of 10 November.
Per type of personal data
The benchmark’s own 35 types, grouped as it groups them, with the Article 9 and 10 data first. “Items” is how many of that type the 900 documents hold; the other four systems are shown by what they found. GLiNER2-PII was not asked for the Article 9 and 10 types.
| Type | Items | euRedact | Found by the four others | ||||
|---|---|---|---|---|---|---|---|
| Found | Left readable | GLiNER2-PII | MS Presidio | bards.ai v3 | Mistral Small 4 | ||
| Article 9 and 10: special categories and offences | |||||||
| HealthHEALTH | 5,188 | 86.9% | 17.9% | 1.6% | 37.1% | 0.9% | 10.3% |
| Criminal offenceCRIMINAL_OFFENCE | 362 | 64.6% | 41.7% | 2.8% | 39.0% | 0.8% | 5.8% |
| Genetic dataGENETIC | 236 | 89.4% | 17.8% | 6.8% | 57.2% | 3.4% | 20.8% |
| Religion or beliefRELIGION_BELIEF | 211 | 80.6% | 28.0% | 10.4% | 63.5% | 8.5% | 15.6% |
| Biometric referenceBIOMETRIC | 207 | 61.8% | 47.8% | 26.6% | 64.7% | 9.2% | 20.3% |
| Trade-union membershipTRADE_UNION | 177 | 86.4% | 22.0% | 18.1% | 77.4% | 3.4% | 1.7% |
| Political opinionPOLITICAL_OPINION | 154 | 78.6% | 34.4% | 22.1% | 40.9% | 9.7% | 8.4% |
| Sex life or orientationSEX_LIFE_ORIENTATION | 143 | 48.9% | 53.8% | 2.8% | 46.2% | 0.7% | 10.5% |
| Ethnic originETHNIC_ORIGIN | 134 | 91.8% | 11.9% | 27.6% | 73.9% | 20.2% | 25.4% |
| Names | |||||||
| Person namePERSON | 7,034 | 99.4% | 0.7% | 94.9% | 93.6% | 97.2% | 66.5% |
| Username or handleUSERNAME | 883 | 95.3% | 4.8% | 88.8% | 62.8% | 62.5% | 64.9% |
| Occupation | |||||||
| Job titleJOB_TITLE | 3,244 | 96.9% | 3.6% | 84.8% | 10.2% | 23.5% | 25.5% |
| Places | |||||||
| CityCITY | 1,565 | 96.7% | 3.3% | 96.4% | 99.0% | 83.3% | 68.2% |
| Street addressSTREET_ADDRESS | 655 | 100.0% | 0.1% | 98.9% | 99.9% | 93.1% | 94.7% |
| PostcodePOSTCODE | 645 | 100.0% | 0.3% | 99.4% | 98.1% | 92.6% | 95.2% |
| Contact details | |||||||
| E-mail addressEMAIL | 903 | 100.0% | 0.0% | 99.6% | 99.0% | 100.0% | 88.6% |
| Phone numberPHONE | 407 | 100.0% | 0.3% | 100.0% | 94.3% | 99.0% | 96.3% |
| Government identifiers | |||||||
| Tax identifierTAX_ID | 246 | 100.0% | 1.2% | 83.3% | 78.5% | 97.6% | 88.2% |
| Vehicle plateVEHICLE_PLATE | 144 | 94.4% | 5.6% | 36.1% | 73.6% | 94.4% | 93.8% |
| National identifierNATIONAL_ID | 135 | 100.0% | 0.0% | 84.4% | 88.1% | 96.3% | 95.6% |
| Passport numberPASSPORT | 127 | 100.0% | 0.0% | 98.4% | 89.0% | 96.1% | 97.6% |
| Driving licenceDRIVERS_LICENSE | 121 | 100.0% | 0.0% | 91.7% | 86.0% | 92.6% | 95.9% |
| Financial | |||||||
| Financial amountFINANCIAL_AMOUNT | 1,901 | 95.8% | 4.4% | 90.8% | 0.5% | 54.1% | 55.8% |
| IBANIBAN | 212 | 100.0% | 0.0% | 91.5% | 99.1% | 96.2% | 93.4% |
| Payment cardPAYMENT_CARD | 151 | 100.0% | 0.0% | 27.2% | 99.3% | 24.5% | 80.1% |
| Network identifiers | |||||||
| IP addressIP_ADDRESS | 523 | 100.0% | 0.0% | 99.2% | 99.6% | 96.8% | 76.9% |
| MAC addressMAC_ADDRESS | 212 | 100.0% | 0.0% | 25.0% | 100.0% | 90.6% | 82.1% |
| Personal URLPERSONAL_URL | 170 | 100.0% | 0.6% | 0.0% | 98.2% | 9.4% | 90.6% |
| Other identifiers | |||||||
| Employee identifierEMPLOYEE_ID | 531 | 96.2% | 4.3% | 86.1% | 69.7% | 83.0% | 88.5% |
| Device serialDEVICE_SERIAL | 236 | 25.4% | 74.6% | 42.4% | 60.2% | 67.8% | 83.5% |
| Customer identifierCUSTOMER_ID | 187 | 98.4% | 1.6% | 83.4% | 57.8% | 86.1% | 88.2% |
| Professional identifierPROFESSIONAL_ID | 166 | 100.0% | 0.0% | 65.1% | 81.3% | 93.4% | 95.8% |
| Medical record numberMEDICAL_RECORD_NUMBER | 143 | 100.0% | 0.0% | 55.3% | 76.2% | 97.9% | 99.3% |
| Credentials | |||||||
| Secret or API keySECRET | 155 | 99.4% | 7.1% | 89.7% | 71.6% | 72.3% | 91.6% |
| Dates | |||||||
| Date of birthDATE_OF_BIRTH | 323 | 100.0% | 0.6% | 100.0% | 93.2% | 99.7% | 99.1% |
Per set and per language
Three sets of 300 documents, and sixty documents per language in each set. F2, with the share left readable under it.
| System | Per set | Per language | ||||||
|---|---|---|---|---|---|---|---|---|
| IT | HR | Medical | German | English | French | Dutch | Polish | |
| euRedact | 0.9315.8% readable | 0.9455.6% readable | 0.9009.5% readable | 0.9295.9% readable | 0.9197.2% readable | 0.9247.6% readable | 0.9118.7% readable | 0.9247.2% readable |
| GLiNER2-PII | 0.80916.5% readable | 0.78920.7% readable | 0.52752.2% readable | 0.68732.7% readable | 0.70230.4% readable | 0.67835.0% readable | 0.69732.8% readable | 0.68633.9% readable |
| MS Presidio | 0.70733.0% readable | 0.68938.4% readable | 0.59856.7% readable | 0.67040.7% readable | 0.65547.1% readable | 0.65045.7% readable | 0.66442.7% readable | 0.64047.2% readable |
| bards.ai v3 | 0.74127.8% readable | 0.69234.9% readable | 0.44960.8% readable | 0.60743.5% readable | 0.63340.3% readable | 0.60944.6% readable | 0.61043.3% readable | 0.58147.8% readable |
| Mistral Small 4 | 0.64537.8% readable | 0.61241.7% readable | 0.46560.2% readable | 0.56348.0% readable | 0.56448.1% readable | 0.55748.9% readable | 0.55249.5% readable | 0.56948.0% readable |
How to read these figures
When and how. euRedact was measured on 14 November 2026 through the production service with the public SDK 1.0.0 (model release of 10 November). The other four were measured on 2 November 2026, with every configuration fixed before the sets were opened. One run per system.
The data. Synthetic documents, no customer data, written and annotated by language models under one guideline family; agreement before adjudication was 0.85–0.97. None of these documents is in euRedact’s training data. The answer key’s guideline is not euRedact’s own set of definitions: where the two disagree, euRedact loses points.
The other systems. MS Presidio with EU recognizers and term lists, configured in six rounds against 65 development documents from the benchmark’s earlier sets, none of them among the 900. GLiNER2-PII with its model card’s labels plus job title and money amount. Mistral Small 4 with a prompt that lists its 34 types, each with a definition, and asks for structured output. bards.ai v3 decoded as its model card does.
Which systems. Compared systems are limited to those that can be self-hosted or are operated under EU jurisdiction only. Services subject to the US CLOUD Act are out of scope.
Precision. Part of what euRedact redacts beyond the annotations is masked on purpose, such as departments and tests in a medical context; that is one of the reasons its precision is 85% rather than higher.
The all-sets F2 is the figure the benchmark publishes for each system. Pooling the three sets’ counts gives 0.690 for GLiNER2-PII and 0.608 for bards.ai v3; the per-set and per-type figures are exact.
Rerun it. A 60-document sample with its gold standard, the harness, the configurations and our scores are on GitHub (source-available, for internal use). All 900 documents and their annotations are pinned by hash, and euRedact’s full-set predictions are committed by hash, so they can’t be changed after publication. The figures on this page were exported from the benchmark repository at commit 92863c1 on 2026-11-16.
Run It on Your Documents
euRedact was measured through the same service a free account uses. The harness on GitHub takes an API key and runs the 60-document sample on the free plan; your own documents go in the same way, and no text or result is kept. Or see a redaction first in the live demo, on a sample or text you made up, no account needed.