Cited by 31Machine Learning in HealthcareRace, Genetics, and SocietyArtificial Intelligence in Healthcare

RIDDLE: Race and ethnicity Imputation from Disease history with Deep LEarning

Open access
Kim, Ji-Sung, Gao, Xin, Rzhetsky, Andrey·2018·PLOS Computational Biology

Abstract

Anonymized electronic medical records are an increasingly popular source of research data. However, these datasets often lack race and ethnicity information. This creates problems for researchers modeling human disease, as race and ethnicity are powerful confounders for many health exposures and treatment outcomes; race and ethnicity are closely linked to population-specific genetic variation. We showed that deep neural networks generate more accurate estimates for missing racial and ethnic information than competing methods (e.g., logistic regression, random forest, support vector machines, and gradient-boosted decision trees). RIDDLE yielded significantly better classification performance across all metrics that were considered: accuracy, cross-entropy loss (error), precision, recall, and area under the curve for receiver operating characteristic plots (all p < 10-9). We made specific efforts to interpret the trained neural network models to identify, quantify, and visualize medical features which are predictive of race and ethnicity. We used these characterizations of informative features to perform a systematic comparison of differential disease patterns by race and ethnicity. The fact that clinical histories are informative for imputing race and ethnicity could reflect (1) a skewed distribution of blue- and white-collar professions across racial and ethnic groups, (2) uneven accessibility and subjective importance of prophylactic health, (3) possible variation in lifestyle, such as dietary habits, and (4) differences in background genetic variation which predispose to diseases.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1006106

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Several studies looked at the use of imputation to increase the completeness of datasets and avoid casewise deletion.

    “Several studies looked at the use of imputation, also referred to as indirect estimation, to increase the completeness of datasets and avoid casewise deletion.14,22,30,45,46,50,61,65,67–70”

    From The quality of social determinants data in the electronic health record: a systematic review (2021) · cites this paper as [68]

    The Introduction’s “Imputation” section (page unavailable) describes estimating missing race and ethnicity values and using them to generate an imputed dataset, supporting the dataset-completeness part of the claim. Our copy does not say that the study used imputation to avoid casewise deletion. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • Two additional studies looked at imputing race/ethnicity data with anonymized clinical datasets.

    “Finally, 2 additional studies looked at imputing race/ethnicity data with anonymized clinical datasets,14,68”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

2 of the 29 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. [17]Ojodu J, Hulihan MM, Pope SN, Grant AM, Centers for Disease C, Prevention. Incidence of sickle cell trait–United States, 2010. MMWR Morb Mortal Wkly Rep. 2014; 63(49): 1155– 8. 25503918 PMC4584538

Cited by

Sorted by citations

Authors
Kim, J.-S., Gao, X., & Rzhetsky, A.
Affiliations
  • Princeton University (US)
  • King Abdullah University of Science and Technology (SA)
  • University of Chicago (US)
Corresponding author
Rzhetsky, Andrey
Venue
PLOS Computational Biology
Published
2018-04-26
Volume
14
Issue
4
Pages
e1006106
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
OpenAlex
W2725551975
Cited by
31
References
29
Citations by year
2026: 3 · 2025: 4 · 2024: 4 · 2023: 6 · 2022: 4 · 2021: 3 · 2020: 3 · 2019: 3 · 2018: 1
FWCI
1.41
Type
journal article
Language
en
Open access
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Knowledge@UChicago (University of Chicago) (submittedVersion, cc-by-nc-nd)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
  • King Abdullah University of Science and Technology Repository (King Abdullah University of Science and Technology) (submittedVersion, other-oa)
  • arXiv (Cornell University) (cc-by)
  • University of Chicago (cc-by)
Funders
  • National Institute of Mental Health (1P50MH094267)
  • National Heart, Lung, and Blood Institute (R01HL122712)
  • Defense Advanced Research Projects Agency (W911NF1410333)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Race, Genetics, and Society
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
electronic health records, deep neural network, model interpretability, genetic ancestry
MeSH terms
Supervised Machine Learning, Ethnicity, Genetics, Population, Humans, Genetic Variation, Epidemiologic Factors, Neural Networks, Computer, Computational Biology, Genetic Predisposition to Disease, Racial Groups, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review