Optimization of the mainzelliste software for fast privacy-preserving record linkage
Open accessAbstract
BACKGROUND: Data analysis for biomedical research often requires a record linkage step to identify records from multiple data sources referring to the same person. Due to the lack of unique personal identifiers across these sources, record linkage relies on the similarity of personal data such as first and last names or birth dates. However, the exchange of such identifying data with a third party, as is the case in record linkage, is generally subject to strict privacy requirements. This problem is addressed by privacy-preserving record linkage (PPRL) and pseudonymization services. Mainzelliste is an open-source record linkage and pseudonymization service used to carry out PPRL processes in real-world use cases. METHODS: We evaluate the linkage quality and performance of the linkage process using several real and near-real datasets with different properties w.r.t. size and error-rate of matching records. We conduct a comparison between (plaintext) record linkage and PPRL based on encoded records (Bloom filters). Furthermore, since the Mainzelliste software offers no blocking mechanism, we extend it by phonetic blocking as well as novel blocking schemes based on locality-sensitive hashing (LSH) to improve runtime for both standard and privacy-preserving record linkage. RESULTS: The Mainzelliste achieves high linkage quality for PPRL using field-level Bloom filters due to the use of an error-tolerant matching algorithm that can handle variances in names, in particular missing or transposed name compounds. However, due to the absence of blocking, the runtimes are unacceptable for real use cases with larger datasets. The newly implemented blocking approaches improve runtimes by orders of magnitude while retaining high linkage quality. CONCLUSION: We conduct the first comprehensive evaluation of the record linkage facilities of the Mainzelliste software and extend it with blocking methods to improve its runtime. We observed a very high linkage quality for both plaintext as well as encoded data even in the presence of errors. The provided blocking methods provide order of magnitude improvements regarding runtime performance thus facilitating the use in research projects with large datasets and many participants.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Individuals who participate multiple times will be automatically allocated to the same study arm as before via a privacy-preserving record linkage service.
“Individuals who participate multiple times will be automatically allocated to the same study arm as before. This process is ensured by a privacy-preserving record linkage service which identifies double entries based on personal data and the IP address (Mainzelliste; Rohde et al., 2021).”
From The efficacy of automated feedback after internet-based depression screening: Study protocol of the German, three-armed, randomised controlled trial DISCOVER (2021) · cites this paper as Rohde et al., 2021
The Background and Methods (‘Record linkage in the mainzelliste’; pages unavailable) describe Mainzelliste matching new records against previous records using identifying data and assigning duplicates the same pseudonym. They do not say that it uses IP addresses or allocates repeat participants to their previous study arm. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Space/time trade-offs in hash coding with allowable errors
Bloom, B. (1970) · CACM · doi:10.1145/362686.362692 · Cited by 8,424 · see citing works
(1999) · Cited by 3,186 · see citing works
Christen, P. (2012) · Springer eBooks · doi:10.1007/978-3-642-31164-2 · Cited by 1,111 · see citing works
A Survey of Indexing Techniques for Scalable Record Linkage and Deduplication
Christen, P. (2011) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.127 · Cited by 685 · see citing works
BMC Medical Informatics and Decision Making
McConnell, P. I., Dash, R., Chilukuri, R., Pietrobon, R. S., Johnson, K. S., Annechiarico, R. P., & Cuticchia, A. J. (2008) · Cited by 678 · see citing works
Privacy-preserving record linkage using Bloom filters
Schnell, R., Bachteler, T., & Reiher, J. (2009) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-9-41 · Cited by 343 · see citing works
A taxonomy of privacy-preserving record linkage techniques
Vatsalan, D., Christen, P., & Verykios, V. S. (2013) · Information Systems · doi:10.1016/j.is.2012.11.005 · Cited by 295 · see citing works
Cohort Profile: The Swiss Childhood Cancer Survivor Study
Kuehni, C. E., Rueegg, C. S., Michel, G., Rebholz, C. E., Strippoli, M. F., Niggli, F. K., Egger, M., von der Weid, N., & for the Swiss Paediatric Oncology Group (SPOG) (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyr142 · Cited by 167 · see citing works
Privacy-Preserving Record Linkage for Big Data: Current Approaches and Research Challenges
Vatsalan, D., Sehili, Z., Christen, P., & Rahm, E. (2017) · Springer eBooks · doi:10.1007/978-3-319-49340-4_25 · Cited by 163 · see citing works
Self-citationVatlach, S., Maas, C., & Poets, C. F. (2014) · Orphanet Journal of Rare Diseases · doi:10.1186/1750-1172-9-9 · Cited by 119 · see citing works
A RESTful interface to pseudonymization services in modern web applications
Lablans, M., Borg, A., & Ückert, F. (2015) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-014-0123-5 · Cited by 85 · see citing works
Self-citationTran, N. K., Vatsalan, D., & Christen, P. (2013) · Proceedings of the ... ACM International Conference on Information & Knowledge Management · doi:10.1145/2505515.2508207 · Cited by 53 · see citing works
A framework for accurate, efficient private record linkage
Durham, E. (2012) · Cited by 46 · see citing works
An optimal code for patient identifiers
Faldum, A., & Pommerening, K. (2005) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2005.03.004 · Cited by 34 · see citing works
Gibberd, A. J., Supramaniam, R., Dillon, A., Armstrong, B. K., & O’Connell, D. L. (2016) · BMC Cancer · doi:10.1186/s12885-016-2322-1 · Cited by 34 · see citing works
An Architecture for Translational Cancer Research As Exemplified by the German Cancer Consortium
Lablans, M., Schmidt, E., & Ückert, F. (2017) · JCO Clinical Cancer Informatics · doi:10.1200/cci.17.00062 · Cited by 33 · see citing works
Self-citationThe EpiLink Record Linkage Software
Tittarelli, A., Tagliabue, G., Maghini, A., Fabiano, S., Crosignani, P., Tessandori, R., & Contiero, P. (2005) · Methods of Information in Medicine · doi:10.1055/s-0038-1633924 · Cited by 32 · see citing works
A fast and efficient Hamming LSH-based scheme for accurate linkage
Karapiperis, D., & Verykios, V. S. (2016) · Knowledge and Information Systems · doi:10.1007/s10115-016-0919-y · Cited by 31 · see citing works
Contiero, P., Tittarelli, A., Tagliabue, G., Maghini, A., Fabiano, S., Crosignani, P., & Tessandori, R. (2005) · PubMed · Cited by 29 · see citing works
Register für seltene Erkrankungen
Storf, H., Schaaf, J., Kadioglu, D., Göbel, J., Wagner, T. O. F., & Ückert, F. (2017) · Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz · doi:10.1007/s00103-017-2536-7 · Cited by 27 · see citing works
LSHDB: a parallel and distributed engine for record linkage and similarity search
Karapiperis, D., Gkoulalas-Divanis, A., & Verykios, V. S. (2016) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2016.7867099 · Cited by 21 · see citing works
OSSE – open source registry software solution
Muscholl, M., Lablans, M., Wagner, T. O. F., & Ückert, F. (2014) · Orphanet Journal of Rare Diseases · doi:10.1186/1750-1172-9-s1-o9 · Cited by 20 · see citing works
Self-citationSkripcak, T., Just, U., Simon, M., Büttner, D., Lühr, A., Baumann, M. A., & Krause, M. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2450833 · Cited by 19 · see citing works
Parallel Privacy-preserving Record Linkage using LSH-based Blocking
Franke, M., Sehili, Z., & Rahm, E. (2018) · International Conference on Internet of Things, Big Data and Security (IoTBDS) · doi:10.5220/0006682701950203 · Cited by 18 · see citing works
Self-citationZentralisierte Biobanken als Grundlage für die medizinische Forschung
Bernemann, I., Kersting, M., Prokein, J., Hummel, M., Klopp, N., & Illig, T. (2016) · Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz · doi:10.1007/s00103-015-2295-2 · Cited by 17 · see citing works
SOEMPI: A Secure Open Enterprise Master Patient Index Software Toolkit for Private Record Linkage.
Toth, C., Durham, E., Kantarcıoğlu, M., Xue, Y., & Malin, B. (2014) · PubMed Central · Cited by 14 · see citing works
Luo, Q., Yu, X. Q., Smith, D. P., Goldsbury, D. E., Cooke‐Yarborough, C., Patel, M. I., & O’Connell, D. L. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014259 · Cited by 11 · see citing works
Franke, M., Sehili, Z., & Rahm, E. (2019) · Proceedings of the VLDB Endowment · doi:10.14778/3352063.3352076 · Cited by 8 · see citing works
Self-citation
Flagged first, then by citations
2 of the 30 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [32]Tran K-N, Vatsalan D, Christen P. Geco: an online personal data generator and corruptor. In: Proceedings of the 22nd ACM International Conference on Information and Knowledge Management, pp. 2473–2476 (2013). ACM.
Cited by
Sorted by citations
SARS-CoV-2 activates ER stress and Unfolded protein responsePreprint
Rosa-Fernandes, L., Lázari, L. C., da Silva, J. M., Gomes, V. M., Machado, R. R. G., dos Santos, A. F., Araújo, D. B., Coutinho, J. V. P., Arini, G. S., Angeli, C. B., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.06.21.449284 · Cited by 35 · see citing works
Bhagwat, K., & Delhi, V. S. K. (2021) · Built Environment Project and Asset Management · doi:10.1108/bepam-04-2021-0068 · Cited by 27 · see citing works
Sikorski, F., König, H.-H., Wegscheider, K., Zapf, A., Löwe, B., & Kohlmann, S. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100435 · Cited by 15 · see citing works
“This process is ensured by a privacy-preserving record linkage service which identifies double entries based on personal data and the IP address (Mainzelliste; Rohde et al., 2021).”
BackgroundKussel, T., Brenner, T., Tremper, G., Schepers, J., Lablans, M., & Hamacher, K. (2022) · Journal of Translational Medicine · doi:10.1186/s12967-022-03671-6 · Cited by 13 · see citing works
Self-citationKohlmann, S., Sikorski, F., König, H.-H., Schütt, M., Zapf, A., & Löwe, B. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00070-0 · Cited by 11 · see citing works
Nesbitt, G., & Murphy, P. A. (2022) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2022.100973 · Cited by 10 · see citing works
Advancements in the Application of the Intersection of Medicine and Engineering in Cancer Research
Chen, H., Zhang, G.-M., Qian, Y., Yang, P., Li, X., Wang, J., Wang, J., Hu, Y.-X., Xiao, H., Zhong, Q., et al. (2025) · Cancer Nexus · doi:10.1002/cnx2.70003 · Cited by 5 · see citing works
An enhanced privacy-preserving record linkage approach for multiple databases
Han, S., Shen, D., Nie, T., Kou, Y., & Yu, G. (2022) · Cluster Computing · doi:10.1007/s10586-022-03590-7 · Cited by 4 · see citing works
Kussel, T., Brenner, T., Tremper, G., Schepers, J., Lablans, M., & Hamacher, K. (2022) · Research Square · doi:10.21203/rs.3.rs-1486673/v1 · Cited by 2 · see citing works
Self-citationSecUREmatch: Integrating Clerical Review in Privacy-Preserving Record Linkage
Rohde, F., Christen, V., & Rahm, E. (2025) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3722212.3725131 · Cited by 1 · see citing works
Self-citationSikorski, F., Löwe, B., Daubmann, A., & Kohlmann, S. (2025) · Journal of Medical Internet Research · doi:10.2196/59476 · Cited by 1 · see citing works
Mainzelliste – 10 Years of Pseudonymization, Record Linkage and Informed Consent ManagementPreprint
Tremper, G., Brenner, T., Amor, M. B., Kussel, T., & Lablans, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5348342
Self-citationMainzelliste: Ten years of pseudonymization, record linkage, and informed consent management
Tremper, G., Brenner, T., Amor, M. B., Kussel, T., & Lablans, M. (2025) · Patterns · doi:10.1016/j.patter.2025.101432
Self-citationMachine Learning Techniques for Linkage
Vatsalan, D., Asghar, H., & Kâafar, M. A. (2026) · Advances in information security · doi:10.1007/978-3-032-21932-9_5
- Authors
- Rohde, F., Franke, M., Sehili, Z., Lablans, M., & Rahm, E.
- Affiliations
- Leipzig University (DE)
- German Cancer Research Center (DE)
- Corresponding author
- Rohde, Florens
- Venue
- Journal of Translational Medicine
- Published
- 2021-01-15
- Volume
- 19
- Issue
- 1
- Pages
- 33-33
- ISSN
- 1479-5876 (ISSN-L)
- PMC
- PMC7809773
- OpenAlex
- W3122238638
- Cited by
- 14
- References
- 30
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 1 · 2022: 4 · 2021: 3
- FWCI
- 1.22
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Journal of Translational Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Qucosa (Saxon State and University Library Dresden) (submittedVersion, cc-by)
- Funders
- Bundesministerium für Bildung und Forschung (BMBF 01IS14014B)
- Projekt DEAL
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Data Quality and Management
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- privacy-preserving record linkage, pseudonymization, Bloom filter, locality-sensitive hashing, blocking schemes, linkage quality, runtime performance, name matching, biomedical data integration
- MeSH terms
- Algorithms, Humans, Medical Record Linkage, Software, Computer Security, Privacy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
