The DOI matches its Crossref record; 38 of the 39 references it cites were found in a registry or an index, and 1 are not checked yet.
AI recognition of patient race in medical imaging: a modelling study
Open accessAbstract
BACKGROUND: Previous studies in medical imaging have shown disparate abilities of artificial intelligence (AI) to detect a person's race, yet there is no known correlation for race on medical imaging that would be obvious to human experts when interpreting the images. We aimed to conduct a comprehensive evaluation of the ability of AI to recognise a patient's racial identity from medical images. METHODS: Using private (Emory CXR, Emory Chest CT, Emory Cervical Spine, and Emory Mammogram) and public (MIMIC-CXR, CheXpert, National Lung Cancer Screening Trial, RSNA Pulmonary Embolism CT, and Digital Hand Atlas) datasets, we evaluated, first, performance quantification of deep learning models in detecting race from medical images, including the ability of these models to generalise to external environments and across multiple imaging modalities. Second, we assessed possible confounding of anatomic and phenotypic population features by assessing the ability of these hypothesised confounders to detect race in isolation using regression models, and by re-evaluating the deep learning models by testing them on datasets stratified by these hypothesised confounding variables. Last, by exploring the effect of image corruptions on model performance, we investigated the underlying mechanism by which AI models can recognise race. FINDINGS: In our study, we show that standard AI deep learning models can be trained to predict race from medical images with high performance across multiple imaging modalities, which was sustained under external validation conditions (x-ray imaging [area under the receiver operating characteristics curve (AUC) range 0·91-0·99], CT chest imaging [0·87-0·96], and mammography [0·81]). We also showed that this detection is not due to proxies or imaging-related surrogate covariates for race (eg, performance of possible confounders: body-mass index [AUC 0·55], disease distribution [0·61], and breast density [0·61]). Finally, we provide evidence to show that the ability of AI deep learning models persisted over all anatomical regions and frequency spectrums of the images, suggesting the efforts to control this behaviour when it is undesirable will be challenging and demand further study. INTERPRETATION: The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging. FUNDING: National Institute of Biomedical Imaging and Bioengineering, MIDRC grant of National Institutes of Health, US National Science Foundation, National Library of Medicine of the National Institutes of Health, and Taiwan Ministry of Science and Technology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“In this situation, XAI becomes crucial as interpretability methods can help to find and eliminate spurious correlations and shortcut learning46–48.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [47]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In our experimental evaluation of the reading race paper where we demonstrated superhuman ability for AI to predict self-reported race across multiple modalities, we found that some transfer learning tasks are better in race detection. 28”
From AI pitfalls and what not to do: mitigating bias in AI (2023) · cites this paper as [28]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the fundamental bias in many large databases that do not reflect the target population of patient being treated making fairness an important issue to address”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [12]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, models built on EHR data skewed toward a primarily Caucasian class of patients may not model well African-American, Hispanic or oriental patients [11, 12].”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [12]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The rapid advances in AI, and in particular the release of publicly available generative AI applications leveraging advanced large language models, have greatly accelerated discussions considering the promises and pitfalls related to AI deployment in society and healthcare [7–17].”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [12]
Experimental — accuracy not yet measured
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“These experiments were: (1) to assess the ability of deep learning AI models to recognise race from medical images, including the ability of these models to generalise to new environments and across multiple imaging modalities; (2) to examine possible confounding anatomic and phenotype population features as explanati…”
Method(2018) · Cited by 3,510
Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning
Poplin, R., Varadarajan, A. V., Blumer, K., Liu, Y., McConnell, M. V., Corrado, G. S., Peng, L., & Webster, D. R. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0195-0 · Cited by 1,865 · cited 1×
“13–15 These findings, which show that demographic factors that are strongly associated with disease outcomes (eg, age, sex, and racial identity), are also strongly associated with features of medical images and might induce bias in model results, mirroring what is known from over a century of…”
MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports
Johnson, A. E. W., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Mark, R. G., & Horng, S. (2019) · Scientific Data · doi:10.1038/s41597-019-0322-0 · Cited by 1,816
Self-citationHidden in Plain Sight — Reconsidering the Use of Race Correction in Clinical Algorithms
Vyas, D. A., Eisenstein, L. G., & Jones, D. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmms2004740 · Cited by 1,731 · cited 1×
“35 Although this approach has already been criticised as being ineffective, or even harmful in some circum stances, 36 our work suggests that such an approach could be impossible in medical imaging because racial identity information appears to be incredibly difficult to isolate.”
Benjamens, S., Dhunnoo, P., & Meskó, B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00324-0 · Cited by 1,215
Cited by
Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-022-01981-2 · Cited by 1,365
“Importantly, recent work has demonstrated that AI models can identify such features solely from imaging data, which highlights the need for deliberate efforts to detect racial bias and equalize racial outcomes during data quality control and model development 157 .”
BackgroundSeyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y., & Ghassemi, M. (2021) · Nature Medicine · doi:10.1038/s41591-021-01595-0 · Cited by 928
Self-citationArtificial intelligence for multimodal data integration in oncology
Lipkova, J., Chen, R. J., Chen, B., Lu, M. Y., Barbieri, M., Shao, D., Vaidya, A. J., Chen, C., Zhuang, L., Williamson, D. F. K., et al. (2022) · Cancer Cell · doi:10.1016/j.ccell.2022.09.012 · Cited by 777
Algorithmic fairness in artificial intelligence for medicine and healthcare
Chen, R. J., Wang, J. J., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Lu, M. Y., Sahai, S., & Mahmood, F. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01056-8 · Cited by 750
“And in the problem of unknown dataset shifts of radiology images leaking self-reported race, model auditing via saliency mapping was ineffective in determining any explainable anatomic landmarks or image-acquisition factors associated with the misdiagnoses 12 .”
BackgroundImpact of Artificial Intelligence on Dental Education: A Review and Guide for Curriculum Update
Thurzo, A., Strunga, M., Urban, R., Surovková, J., & Afrashtehfar, K. I. (2023) · Education Sciences · doi:10.3390/educsci13020150 · Cited by 342
“e have not yet fully recovered from the shock that an AI can recognize gender [42] and ethnicity [43] from an X-ray, nor from the fact that no one knows how the AI is capable of doing so, and here we are confronted with the possibility of semi-automatic communication of our office AI telemonitoring systems with the pa…”
Background
Metadata
- Authors
- Gichoya, J. W., Banerjee, I., Bhimireddy, A. R., Burns, J. L., Celi, L. A., Chen, L.-C., Correa, R., Dullerud, N., Ghassemi, M., Huang, S.-C., Kuo, P.-C., Lungren, M. P., Palmer, L. J., Price, B. J., Purkayastha, S., Pyrros, A. T., Oakden-Rayner, L., Okechukwu, C., Seyyed-Kalantari, L., Trivedi, H., Wang, R., Zaiman, Z., & Zhang, H.
- Affiliations
- Emory University (US)
- Arizona State University (US)
- Indiana University – Purdue University Indianapolis (US)
- Beth Israel Deaconess Medical Center (US)
- National Tsing Hua University (TW)
- University of Toronto (CA)
- Massachusetts Institute of Technology (US)
- Stanford Medicine (US)
- The University of Adelaide (AU)
- Florida State University (US)
- DuPage Medical Group (US)
- Georgia Institute of Technology (US)
- Corresponding author
- Gichoya, Judy Wawira
- ORCID
- Venue
- The Lancet Digital Health
- Publisher
- Elsevier BV
- Published
- 2022-06
- Volume
- 4
- Issue
- 6
- Pages
- e406-e414
- ISSN
- 2589-7500 (ISSN-L)
- PMC
- PMC9650160
- OpenAlex
- W4280519765
- Cited by
- 611
- References
- 35
- Citations by year
- 2026: 151 · 2025: 152 · 2024: 150 · 2023: 109 · 2022: 40
- FWCI
- 33.96
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Locations
- The Lancet Digital Health (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- IUScholarWorks (Indiana University) (submittedVersion, cc-by)
- Purdue University Indianapolis (Indiana University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- NSF
- Ministry of Science and Technology, Taiwan (MOST109-2222-E-007-004-MY3)
- National Science Foundation (1928481)
- National Institutes of Health
- NLM (R01LM012966)
- National Institute of Biomedical Imaging and Bioengineering (75N92020C00008, 75N92020C00021, R01 EB017205)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Anthropology › Race, Genetics, and Society
- Keywords
- race prediction, artificial intelligence, diagnostic imaging, deep learning, chest radiography, chest CT, mammography, cervical spine imaging, external validation, image corruption robustness, algorithmic bias, racial disparities in health care, algorithmic fairness, model interpretability
- MeSH terms
- Deep Learning, Artificial Intelligence, Humans, Lung Neoplasms, Retrospective Studies, Early Detection of Cancer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: AI recognition of patient race in medical imaging: a modelling study. https://app.cochat.ai/papers/10.1016/s2589-7500(22)00063-2
- The paper
- Gichoya, J. W., Banerjee, I., Bhimireddy, A. R., Burns, J. L., Celi, L. A., Chen, L.-C., Correa, R., Dullerud, N., Ghassemi, M., Huang, S.-C., Kuo, P.-C., Lungren, M. P., Palmer, L. J., Price, B. J., Purkayastha, S., Pyrros, A. T., Oakden-Rayner, L., Okechukwu, C., Seyyed-Kalantari, L., . . . Zhang, H. (2022). AI recognition of patient race in medical imaging: a modelling study. The Lancet Digital Health. https://doi.org/10.1016/s2589-7500(22)00063-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.