How Well Do Automated Linking Methods Perform? Lessons from US Historical Data
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
This paper reviews the literature in historical record linkage in the United States and examines the performance of widely used record-linking algorithms and common variations in their assumptions. We use two high-quality, hand-linked data sets and one synthetic ground truth to examine the direct effects of linking algorithms on data quality. We find that (i) no algorithm (including hand linking) consistently produces representative samples; (ii) 15 to 37 percent of links chosen by widely used algorithms are classified as errors by trained human reviewers; and (iii) false links are systematically related to baseline sample characteristics, showing that some algorithms may introduce systematic measurement error into analyses. A case study shows that the combined effects of (i)–(iii) attenuate estimates of the intergenerational income elasticity by up to 29 percent, and common variations in algorithm assumptions result in greater attenuation. As current practice moves to automate linking and increase link rates, these results highlight the important potential consequences of linking errors on inferences with linked data. We conclude with constructive suggestions for reducing linking errors and directions for future research. (JEL C45, C81, J62, N31, N32)
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Characterizing Selection Bias Using Experimental Data
Heckman, J., Ichimura, H., Smith, J., & Todd, P. (1998) · Econometrica · doi:10.2307/2999630 · Cited by 2,286
Intergenerational Mobility in the Labor Market
Solon, G. (1999) · Handbook of Labor Economics · doi:10.1016/s1573-4463(99)03010-2 · Cited by 1,109
Life-Cycle Variation in the Association between Current and Lifetime Earnings
Haider, S., & Solon, G. (2006) · American Economic Review · doi:10.1257/aer.96.4.1308 · Cited by 744
When the levee breaks: black migration and economic development in the American South
Hornbeck, R., & Naidu, S. (2014) · American Economic Review · doi:10.1257/aer.104.3.963 · Cited by 341
Identification and Robustness with Contaminated and Corrupted Data
Horowitz, J. L., & Manski, C. F. (1995) · Econometrica · doi:10.2307/2951627 · Cited by 337
Shocking behavior: random wealth in antebellum georgia and human capital across generations
Bleakley, H., & Ferrie, J. (2016) · The Quarterly Journal of Economics · doi:10.1093/qje/qjw014 · Cited by 125
“Obama, Katrina, and the Persistence of Racial Inequality”
Margo, R. A. (2016) · The Journal of Economic History · doi:10.1017/s0022050716000590 · Cited by 81
Metadata
- Authors
- Bailey, M. J., Cole, C., Henderson, M., & Massey, C.
- Venue
- Journal of Economic Literature
- Publisher
- American Economic Association
- Published
- 2020-12-01
- Volume
- 58
- Issue
- 4
- Pages
- 997-1044
- ISSN
- 0022-0515 (ISSN-L)
- OpenAlex
- W2944196176
- Cited by
- 175
- References
- 7
- FWCI
- 12.91
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.