CoChat
Sign Up
Cited by 173Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AIMachine Learning in Healthcare

The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.

Sociodemographic biases in medical decision making by large language models

Omar, Mahmud, Soffer, Shelly, Agbareia, Reem, Bragazzi, Nicola Luigi, Apakama, Donald U., Horowitz, Carol R., Charney, Alexander W., Freeman, Robert, Kummer, Benjamin, Glicksberg, Benjamin S., Nadkarni, Girish N., Klang, Eyal·2025·Nature Medicine
DOI 10.1038/s41591-025-03626-6
Open this paper in CoChat — free account

Abstract

Large language models (LLMs) show promise in healthcare, but concerns remain that they may produce medically unjustified clinical care recommendations reflecting the influence of patients' sociodemographic characteristics. We evaluated nine LLMs, analyzing over 1.7 million model-generated outputs from 1,000 emergency department cases (500 real and 500 synthetic). Each case was presented in 32 variations (31 sociodemographic groups plus a control) while holding clinical details constant. Compared to both a physician-derived baseline and each model's own control case without sociodemographic identifiers, cases labeled as Black or unhoused or identifying as LGBTQIA+ were more frequently directed toward urgent care, invasive interventions or mental health evaluations. For example, certain cases labeled as being from LGBTQIA+ subgroups were recommended mental health assessments approximately six to seven times more often than clinically indicated. Similarly, cases labeled as having high-income status received significantly more recommendations (P < 0.001) for advanced imaging tests such as computed tomography and magnetic resonance imaging, while low- and middle-income-labeled cases were often limited to basic or no further testing. After applying multiple-hypothesis corrections, these key differences persisted. Their magnitude was not supported by clinical reasoning or guidelines, suggesting that they may reflect model-driven bias, which could eventually lead to health disparitie…

Overview

Integrity signals

Registry

Crossref · 10.1038/s41591-025-03626-6

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 38 references

Research works printed but not linked: 9

  1. Liu, M., Sandhu, S., Reisner, S. L., Gonzales, G. & Keuroghlian, A. S. Health status and health care access among lesbian, gay, and bisexual adults in the US, 2013 to 2018. JAMA Intern. Med. 183, 380–383 (2023).
  2. Rejeleene, R., Xu, X. & Talburt, J. Towards trustable language models: investigating information quality of large language models. Preprint at http://arxiv.org/abs/2401.13086 (2024). · arXiv
  3. Poulain, R., Fayyaz, H. & Beheshti, R. Bias patterns in the application of LLMs for clinical decision support: a comprehensive study. Preprint at http://arxiv.org/abs/2404.15149 (2024). · arXiv
  4. OpenAI et al. GPT-4 technical report. Preprint at http://arxiv.org/abs/2303.08774 (2024). · arXiv
  5. Cochran, S. D., Sullivan, J. G. & Mays, V. M. Prevalence of mental disorders, psychological distress, and mental health services use among lesbian, gay, and bisexual adults in the United States. J. Consult. Clin. Psychol. 71, 53–61 (2003).
  6. Hoy-Ellis, C. P. Minority stress and mental health: a review of the literature. J. Homosex. 70, 806–830 (2023).
  7. Arpey, N. C., Gaglioti, A. H. & Rosenbaum, M. E. How socioeconomic status affects patient perceptions of health care: a qualitative study. J. Prim. Care Community Health 8, 169–175 (2017).
  8. Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y. & Clifton, D. A. An adversarial training framework for mitigating algorithmic biases in clinical machine learning. NPJ Digit. Med. 6, 55 (2023).
  9. Hackmann, S., Mahmoudian, H., Steadman, M. & Schmidt, M. Word importance explains how prompts affect language model outputs. Preprint at http://arxiv.org/abs/2403.03028 (2024). · arXiv

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Omar, M., Soffer, S., Agbareia, R., Bragazzi, N. L., Apakama, D. U., Horowitz, C. R., Charney, A. W., Freeman, R., Kummer, B., Glicksberg, B. S., Nadkarni, G. N., & Klang, E.
Affiliations
  • Mount Sinai Health System (US)
  • Rabin Medical Center (IL)
  • Hadassah Medical Center (IL)
  • Institut für Schlaganfall- und Demenzforschung (DE)
  • Icahn School of Medicine at Mount Sinai (US)
Corresponding author
  • Omar, Mahmud
  • Nadkarni, Girish N.
  • Klang, Eyal
ORCID
Venue
Nature Medicine
Publisher
Springer Science and Business Media LLC
Published
2025-04-07
Volume
31
Issue
6
Pages
1873-1881
ISSN
  • 1078-8956 (ISSN-L)
  • 1546-170X
DOI
10.1038/s41591-025-03626-6
OpenAlex
W4409210035
Cited by
173
References
38
Citations by year
2026: 121 · 2025: 52
FWCI
41.64
Type
journal article
Language
en
Open access
closed
Licence
Locations
  • Nature Medicine (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
large language models, algorithmic bias, sociodemographic bias, emergency department care, health disparities, racial bias, LGBTQ health, homelessness, socioeconomic status, bias mitigation
MeSH terms
Clinical Decision-Making, Sociodemographic Factors, Large Language Models, Adult, Emergency Service, Hospital, Female, Humans, Language, Male, Middle Aged, Socioeconomic Factors, Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review