Comparing record linkage software programs and algorithms using real-world data.
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Linkage of medical databases, including insurer claims and electronic health records (EHRs), is increasingly common. However, few studies have investigated the behavior and output of linkage software. To determine how linkage quality is affected by different algorithms, blocking variables, methods for string matching and weight determination, and decision rules, we compared the performance of 4 nonproprietary linkage software packages linking patient identifiers from noninteroperable inpatient and outpatient EHRs. We linked datasets using first and last name, gender, and date of birth (DOB). We evaluated DOB and year of birth (YOB) as blocking variables and used exact and inexact matching methods. We compared the weights assigned to record pairs and evaluated how matching weights corresponded to a gold standard, medical record number. Deduplicated datasets contained 69,523 inpatient and 176,154 outpatient records, respectively. Linkage runs blocking on DOB produced weights ranging in number from 8 for exact matching to 64,273 for inexact matching. Linkage runs blocking on YOB produced 8 to 916,806 weights. Exact matching matched record pairs with identical test characteristics (sensitivity 90.48%, specificity 99.78%) for the highest ranked group, but algorithms differentially prioritized certain variables. Inexact matching behaved more variably, leading to dramatic differences in sensitivity (range 0.04-93.36%) and positive predictive value (PPV) (range 86.67-97.35%), even for the most highly ranked record pairs. Blocking on DOB led to higher PPV of highly ranked record pairs. An ensemble approach based on averaging scaled matching weights led to modestly improved accuracy. In summary, we found few differences in the rankings of record pairs with the highest matching weights across 4 linkage packages. Performance was more consistent for exact string matching than for inexact string matching. Most methods and software packages performed similarly when comparing matching accuracy with the gold standard. In some settings, an ensemble matching approach may outperform individual linkage algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Probabilistic record linkage algorithms can help facilitate the combination of patient records across health systems and data sets, reducing gaps in missing patient data [64,65].”
From Bias in medical AI: Implications for clinical decision-making (2024) · cites this paper as [65]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444 · cited 1×
“, using the expectation-maximization [EM] algorithm [10]).”
Fellegi, I. P., & Sunter, A. B. (1969) · Journal of the American Statistical Association · doi:10.1080/01621459.1969.10501049 · Cited by 2,475 · cited 2×
“Probabilistic weight determination uses statistical modeling based on estimated probabilities that records match, given equality or similarity of linking variables [9].”
Methodvan der Laan, M. J., Polley, E. C., & Hubbard, A. E. (2007) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1309 · Cited by 1,719
Agniel, D., Kohane, I. S., & Weber, G. M. (2018) · BMJ · doi:10.1136/bmj.k1479 · Cited by 391 · cited 1×
“This specific group underscores the importance of understanding real-world healthcare practices—such as the naming of newborn infants in inpatient settings as “Male” or “Female” with mothers’ surnames—when interpreting EHR data [27].”
BackgroundData linkage: a powerful research tool with potential problems
Bohensky, M. A., Jolley, D., Sundararajan, V., Evans, S., Pilcher, D. V., Scott, I., & Brand, C. A. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-346 · Cited by 234
Sayers, A., Ben-Shlomo, Y., Blom, A. W., & Steele, F. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyv322 · Cited by 186
Campbell, K. M., Deck, D., & Krupski, A. (2008) · Health Informatics Journal · doi:10.1177/1460458208088855 · Cited by 156
A guide to evaluating linkage quality for the analysis of linked data
Harron, K. L., Doidge, J., Knight, H. E., Gilbert, R. E., Goldstein, H., Cromwell, D. A., & van der Meulen, J. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyx177 · Cited by 156 · cited 1×
“Furthermore, evaluating linkage error using a gold standard—even an alloyed gold standard—provides important information on linkage quality [7].”
MethodData quality: A statistical perspective
Karr, A. F., Sanil, A. P., & Banks, D. L. (2005) · Statistical Methodology · doi:10.1016/j.stamet.2005.08.005 · Cited by 150 · cited 1×
“Nonetheless, in terms of the linking variables tested, both datasets seemed to be rather good quality, in part perhaps because of human health and financial incentives [24].”
BackgroundSelf-citationGUILD: GUidance for Information about Linking Data sets†
Gilbert, R. E., Lafferty, R., Hagger‐Johnson, G., Harron, K. L., Zhang, L., Smith, P. W. F., Dibben, C. J. L., & Goldstein, H. (2017) · Journal of Public Health · doi:10.1093/pubmed/fdx037 · Cited by 146
Research works printed but not linked: 2
- J McIllece, 2014, A simplified approach to administrative record linkage in the quarterly census of employment and wages., JSM Proceedings: Survey Research Methods, 4392
- D Krewski, 2005, The effect of record linkage errors on risk estimates in cohort mortality studies, Statistics Canada, 31, 13
Cited by
Bias in medical AI: Implications for clinical decision-making
Cross, J. L., Choma, M. A., & Onofrey, J. A. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000651 · Cited by 607
Dagan, N., Elnekave, E., Barda, N., Bregman-Amitai, O., Bar, A., Orlovsky, M., Bachmat, E., & Balicer, R. D. (2020) · Nature Medicine · doi:10.1038/s41591-019-0720-z · Cited by 113
Asher, J., Resnick, D. M., Brite, J., Brackbill, R. M., & Cone, J. E. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17186937 · Cited by 37
“[32] reviewed multiple software options for record linkage: R (Version 3.”
MethodA Context-Independent Ontological Linked Data Alignment Approach to Instance Matching
Barbosa, A., Bittencourt, I. I., Siqueira, S. W. M., Dermeval, D., & da Cruz, N. J. T. (2022) · International Journal on Semantic Web and Information Systems · doi:10.4018/ijswis.295977 · Cited by 24
“In Databases, the problem is known by record linkage (Gu et al., 2003; Karr et al., 2019), which aims to identify and link resources that are judged to represent the same real-world entity.”
BackgroundTahamont, S., Jelveh, Z., Chalfin, A. J., Yan, S., & Hansen, B. (2020) · Journal of Quantitative Criminology · doi:10.1007/s10940-020-09461-x · Cited by 25
Metadata
- Authors
- Karr, A. F., Taylor, M. T., West, S. L., Setoguchi, S., Kou, T. D., Gerhard, T., & Horton, D. B.
- Affiliations
- RTI International (US)
- Rutgers, The State University of New Jersey (US)
- Bristol-Myers Squibb (United States) (US)
- Corresponding author
- Karr, Alan F.
- ORCID
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-09-24
- Volume
- 14
- Issue
- 9
- Pages
- e0221459
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC6759179
- OpenAlex
- W2976642796
- Cited by
- 27
- References
- 27
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 6 · 2023: 2 · 2022: 5 · 2021: 6 · 2020: 4 · 2019: 1
- FWCI
- 2.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- The Medicine Forum (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Bristol-Myers Squibb
- Topics
- Physical Sciences › Computer Science › Information Systems › Data Quality and Management
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- data linkage, electronic health records, string matching, linkage quality
- MeSH terms
- Algorithms, Humans, Medical Record Linkage, Software, Databases, Factual, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Comparing record linkage software programs and algorithms using real-world data. https://app.cochat.ai/papers/10.1371/journal.pone.0221459
- The paper
- Karr, A. F., Taylor, M. T., West, S. L., Setoguchi, S., Kou, T. D., Gerhard, T., & Horton, D. B. (2019). Comparing record linkage software programs and algorithms using real-world data. PLOS ONE. https://doi.org/10.1371/journal.pone.0221459
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.