Imputing missing race/ethnicity in pediatric electronic health records: reducing bias with use of U.S. Census location and surname data
Open accessAbstract
Objective: To assess the utility of imputing race/ethnicity using U.S. Census race/ethnicity, residential address, and surname information compared to standard missing data methods in a pediatric cohort. Data Sources/Study Setting: Electronic health record data from 30 pediatric practices with known race/ethnicity. Study Design: In a simulation experiment, we constructed dichotomous and continuous outcomes with pre‐specified associations with known race/ethnicity. Bias was introduced by nonrandomly setting race/ethnicity to missing. We compared typical methods for handling missing race/ethnicity (multiple imputation alone with clinical factors, complete case analysis, indicator variables) to multiple imputation incorporating surname and address information. Principal Findings: Imputation using U.S. Census information reduced bias for both continuous and dichotomous outcomes. Conclusions: The new method reduces bias when race/ethnicity is partially, nonrandomly missing.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The most widely researched imputation method was Bayesian Improved Surname Geocoding (BISG).
“The most widely researched imputation method was Bayesian Improved Surname Geocoding (BISG).22,45,46,50,65”
From The quality of social determinants data in the electronic health record: a systematic review (2021) · cites this paper as [65]
The paper’s BISG Method and Discussion sections show that it studies BISG as a race/ethnicity imputation method, and its introduction cites several earlier BISG studies. It does not compare how extensively BISG and other imputation methods have been researched, so it does not establish that BISG was the most widely researched. (LLM, conf=0.86)
Experimental — accuracy not yet measuredThree studies in the review found that casewise deletion decreased the quality of race/ethnicity data.
“However, 3 studies in our review found that casewise deletion decreased the quality of race/ethnicity data.23,50,65”
From The quality of social determinants data in the electronic health record: a systematic review (2021) · cites this paper as [65]
The Performance of Imputation Methods section reports that complete-case analysis, which excludes records with missing values, produced highly biased estimates of associations between race/ethnicity and outcomes. This supports the concern about casewise deletion, but the paper measures bias in analysis results rather than a decrease in the quality of the race/ethnicity data itself. (LLM, conf=0.84)
Experimental — accuracy not yet measuredSeveral studies looked at the use of imputation to increase the completeness of datasets and avoid casewise deletion.
“Several studies looked at the use of imputation, also referred to as indirect estimation, to increase the completeness of datasets and avoid casewise deletion.14,22,30,45,46,50,61,65,67–70”
From The quality of social determinants data in the electronic health record: a systematic review (2021) · cites this paper as [65]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Schafer, J. L. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800102 · Cited by 3,949 · see citing works
Multiple imputation of discrete and continuous data by fully conditional specification
van Buuren, S. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280206074463 · Cited by 2,827 · see citing works
Multiple Imputation of Missing Values
Royston, P. (2004) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0400400301 · Cited by 2,340 · cited 4× · see citing works
“In addition to parent-reported race and ethnicity, these variables were child surname, home address, gender, age and insurance type, presence of asthma, and presence of attention-deficit hyperactivity disorder (ADHD) as identified by ICD-9 codes specific to these conditions on the problem list (codes beginning with th…”
MethodInfluentialA critical look at methods for handling missing covariates in epidemiologic regression analyses.
Greenland, S., & Finkle, W. D. (1995) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117592 · Cited by 891 · cited 1× · see citing works
“The missing indicator method— using a missing category indicator for missing variables such as race/ethnicity —also produces biased estimates and (Greenland and Finkle 1995; Knol et al. 2010).”
MethodAgreement between administrative data and patients’ self-reports of race/ethnicity
Kressin, N. R., Chang, B.-H., Hendricks, A., & Kazis, L. E. (2003) · American Journal of Public Health · doi:10.2105/ajph.93.10.1734 · Cited by 234 · cited 1× · see citing works
“…2007; Fiks et al. 2012) but requires accurate racial/ethnic data, which are often completely or partially missing in EHR data (Bierman et al. 2002; Kressin et al. 2003; Hasnain-Wynia, Pierce, and Pittman 2004; Fremont et al. 2005; Bilheimer and Sisk 2008; Smith et al. 2010;Wynia, Ivey, and…”
BackgroundElliott, M. N., Morrison, P. A., Fremont, A., McCaffrey, D. F., Pantoja, P., & Lurie, N. (2009) · Health Services and Outcomes Research Methodology · doi:10.1007/s10742-009-0047-1 · Cited by 224 · see citing works
Self-citationDerose, S. F., Contreras, R., Coleman, K. J., Koebnick, C., & Jacobsen, S. J. (2013) · Medical Care Research and Review · doi:10.1177/1077558712466293 · Cited by 102 · see citing works
Smith, N., Iyer, R. L., Langer-Gould, A., Getahun, D. T., Strickland, D., Jacobsen, S. J., Chen, W., Derose, S. F., & Koebnick, C. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-316 · Cited by 62 · cited 1× · see citing works
“…accurate racial/ethnic data, which are often completely or partially missing in EHR data (Bierman et al. 2002; Kressin et al. 2003; Hasnain-Wynia, Pierce, and Pittman 2004; Fremont et al. 2005; Bilheimer and Sisk 2008; Smith et al. 2010;Wynia, Ivey, and Hasnain-Wynia 2010; Bergdall et al. 2012).”
Background
Flagged first, then by citations
Research works printed but not linked: 20
- Bergdall, A., S. Asche, N. Schneider, T. Kerby, K. Margolis, J. Sperl-Hillen, J. Sekenski, R. Pritchard, M. Maciosek, and P. O ' Connor. 2012. ' CB3-01: Comparison of Ethnicity and Race Categorization in Electronic Medical Records and by SelfReport. ' Clinical Medicine & Research 10 (3): 172. doi:10.3121/cmr.2012.1100.cb301. · DOI
- Bierman, A. S., N. Lurie, K. S. Collins, and J. M. Eisenberg. 2002. ' Addressing Racial and Ethnic Barriers to Effective Health Care: The Need for Better Data. ' Health Affairs (Project Hope) 21 (3): 91 -102.
- Bilheimer, L. T., and J. E. Sisk. 2008. ' Collecting Adequate Data on Racial and Ethnic Disparities in Health: The Challenges Continue. ' Health Affairs (Project Hope) 27 (2): 383 -91. doi:10.1377/hlthaff.27 .2.383. · DOI
- Elliott, M. N., A. Fremont, P. A. Morrison, P. Pantoja, and N. Lurie. 2008. ' A New Method for Estimating Race/Ethnicity and Associated Disparities Where Administrative Records Lack Self-Reported Race/Ethnicity. ' Health Services Research 43 (5 Pt 1): 1 722 -36. doi:10.1111/j.1475-6773.2008.00854.x. · DOI
- Elliott, M. N., K. Becker, M. K. Beckett, K. Hambarsoomian, P. Pantoja, and B. Karney. 2013. ' Using Indirect Estimates Based on Name and Census Tract to Improve the Ef fi ciency of Sampling Matched Ethnic Couples from Marriage License Data. ' Public Opinion Quarterly 77 (1): 375 -84. doi:10.1093/poq/nft007. · DOI
- Fiks, A. G., R. W. Grundmeier, B. Margolis, L. M. Bell, J. Steffes, J. Massey, and R. C. Wasserman. 2012. ' Comparative Effectiveness Research Using the Electronic Medical Record: An Emerging Area of Investigation in Pediatric Primary Care. ' Journal of Pediatrics 160 (5): 719 -24. doi:10.1016/j.jpeds.2012.01.039. · DOI
- Fiscella, K., and A. M. Fremont. 2006. ' Use of Geocoding and Surname Analysis to Estimate Race and Ethnicity. ' Health Services Research 41 (4 Pt 1): 1482 -500. doi:10.1111/j.1475-6773.2006.00551.x. · DOI
- Fremont, A. M., A. Bierman, S. L. Wickstrom, C. E. Bird, M. Shah, J. J. Escarce, T. Horstman, and T. Rector. 2005. ' Use of Geocoding in Managed Care Settings to Identify Quality Disparities. ' Health Affairs (Project Hope) 24 (2): 516 -26. doi:10.1377/hlthaff.24.2.516. · DOI
- Graham, J. W. 2012. Missing Data Analysis and Design . New York: Springer.
- Knol, M. J., K. J. M. Janssen, A. R. T. Donders, A. C. G. Egberts, E. R. Heerdink, D. E. Grobbee, K. G. M. Moons, and M. I. Geerlings. 2010. ' Unpredictable Bias WhenUsing the Missing Indicator Method or Complete Case Analysis for Missing Confounder Values: An Empirical Example. ' Journal of Clinical Epidemiology 63 (7): 728 -36. doi:10.1016/j.jclinepi.2009.08.028. · DOI
- Little, R. J. A. 1992. ' Regression with Missing X ' s: A Review. ' Journal of the American Statistical Association 87 (420): 1227 -37.
- Martino, S. C., R. M. Weinick, D. E. Kanouse, J. A. Brown, A. M. Haviland, E. Goldstein, J. L. Adams, K. Hambarsoomian, D. J. Klein, and M. N. Elliott. 2013. ' Reporting CAHPS and HEDIS Data by Race/Ethnicity for Medicare Benefi ciaries. ' Health Services Research 48 (2 Pt 1): 417 -34. doi:10.1111/j.14756773.2012.01452.x. · DOI
- McCaffrey, D. F., and M. N. Elliott. 2008. ' Power of Tests for a Dichotomous Independent Variable Measured with Error. ' Health Services Research 43 (3): 1085 -101. doi:10.1111/j.1475-6773.2007.00810.x. · DOI
- Molenberghs, G., and M. G. Kenward. 2007. Missing Data in Clinical Studies, Chichester . Hoboken, NJ: Wiley.
- National Research Council (U.S.), Panel on Handling Missing Data in Clinical Trials, National Research Council (U.S.), and Committee on National Statistics. 2010. The Prevention and Treatment of Missing Data in Clinical Trials . Washington, DC: National Academies Press.
- Olsen, L., D. Aisner, J. M. McGinnis, and Institute of Medicine (U.S.), and Roundtable on Evidence-Based Medicine. 2007. The Learning Healthcare System Workshop Summary . Washington, DC: National Academies Press.
- Rubin, D. B. 1987. Multiple Imputation for Nonresponse in Surveys . New York: Wiley.
- White, I. R., P. Royston, and A. M. Wood. 2011. ' Multiple Imputation Using Chained Equations: Issues and Guidance for Practice. ' Statistics in Medicine 30 (4): 377 -99. doi:10.1002/sim.4067. · DOI · Paper page
- Wolraich, M. L., I. D. Feurer, J. N. Hannah, A. Baumgaertel, and T. Y. Pinnock. 1998. ' Obtaining Systematic Teacher Reports of Disruptive Behavior Disorders Utilizing DSM-IV. ' Journal of Abnormal Child Psychology 26 (2): 141 -52.
- Wynia, M. K., S. L. Ivey, and R. Hasnain-Wynia. 2010. ' Collection of Data on Patients ' Race and Ethnic Group by Physician Practices. ' New England Journal of Medicine 362 (9): 846 -50. doi:10.1056/NEJMsb0910799. · DOI
Cited by
Sorted by citations
The quality of social determinants data in the electronic health record: a systematic review
Cook, L. A., Sachs, J., & Weiskopf, N. G. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab199 · Cited by 153 · see citing works
Zhou, Y., Shi, J., Stein, R., Liu, X., Baldassano, R. N., Forrest, C. B., Chen, Y., & Huang, J. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad066 · Cited by 33 · see citing works
RIDDLE: Race and ethnicity Imputation from Disease history with Deep LEarning
Kim, J.-S., Gao, X., & Rzhetsky, A. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006106 · Cited by 31 · see citing works
“Bayesian approaches to race and ethnicity imputation using census data have been proposed [6] and have been used for race and ethnicity imputation in EMR datasets [7].”
Method
- Authors
- Grundmeier, R. W., Song, L., Ramos, M. J., Fiks, A. G., Elliott, M. N., Fremont, A., Pace, W., Wasserman, R. C., & Localio, R.
- Affiliations
- Children's Hospital of Philadelphia (US)
- RAND Corporation (US)
- DARTNet Institute (US)
- University of Vermont (US)
- University of Pennsylvania (US)
- Corresponding author
- Grundmeier, Robert W.
- ORCID
- Venue
- Health Services Research
- Publisher
- Wiley
- Published
- 2015-03-11
- Volume
- 50
- Issue
- 4
- Pages
- 946-960
- ISSN
- 0017-9124 (ISSN-L)
- 1475-6773
- PMC
- PMC4545341
- OpenAlex
- W1985419223
- Cited by
- 55
- References
- 8
- Citations by year
- 2026: 4 · 2025: 8 · 2024: 7 · 2023: 10 · 2022: 9 · 2021: 4 · 2020: 3 · 2018: 5 · 2016: 3 · 2015: 2
- FWCI
- 2.97
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Health Services Research (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Agency for Healthcare Research and Quality (P30HS021645)
- U.S. Public Health Service (UB5MC20286)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- imputation bias, residential addresses, U.S. Census, multiple imputation, complete case analysis, dummy variables, racial and ethnic disparities, simulation study
- MeSH terms
- Adolescent, Age Factors, Asthma, Attention Deficit Disorder with Hyperactivity, Black or African American, Child, Child, Preschool, Data Collection, Ethnicity, Female, Hispanic or Latino, Humans, Infant, Infant, Newborn, Male, Names, Research Design, Sex Factors, Socioeconomic Factors, United States, Bias, Censuses, Racial Groups, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
