CoChat
Sign Up
Cited by 174Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AIMachine Learning in Healthcare

Sociodemographic biases in medical decision making by large language models

Omar, Mahmud, Soffer, Shelly, Agbareia, Reem, Bragazzi, Nicola Luigi, Apakama, Donald U., Horowitz, Carol R., Charney, Alexander W., Freeman, Robert +4 more·2025·Nature Medicine
DOI 10.1038/s41591-025-03626-6
Open this paper in CoChat — free account

Abstract

Large language models (LLMs) show promise in healthcare, but concerns remain that they may produce medically unjustified clinical care recommendations reflecting the influence of patients' sociodemographic characteristics. We evaluated nine LLMs, analyzing over 1.7 million model-generated outputs from 1,000 emergency department cases (500 real and 500 synthetic). Each case was presented in 32 variations (31 sociodemographic groups plus a control) while holding clinical details constant. Compared to both a physician-derived baseline and each model's own control case without sociodemographic identifiers, cases labeled as Black or unhoused or identifying as LGBTQIA+ were more frequently directed toward urgent care, invasive interventions or mental health evaluations. For example, certain cases labeled as being from LGBTQIA+ subgroups were recommended mental health assessments approximately six to seven times more often than clinically indicated. Similarly, cases labeled as having high-income status received significantly more recommendations (P < 0.001) for advanced imaging tests such as computed tomography and magnetic resonance imaging, while low- and middle-income-labeled cases were often limited to basic or no further testing. After applying multiple-hypothesis corrections, these key differences persisted. Their magnitude was not supported by clinical reasoning or guidelines, suggesting that they may reflect model-driven bias, which could eventually lead to health disparitie…

Integrity signals

Registry

Crossref · 10.1038/s41591-025-03626-6

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Directional patterns of socioeconomic status bias vary: clinical settings showed bias favoring higher-income patients, while simulated college admissions favored low-SES applicants.

    “Directional patterns vary: clinical settings showed bias favoring higher-income patients [4, 70], while simulated college admissions favored low-SES applicants [66].”

    From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [70]

  • Prior work on occupation-related LLM behavior has found context-dependent patterns: clinical settings showed models favoring higher-income patients, while simulated college admissions showed the reverse.

    “Prior work on occupation-related LLM behavior has found context-dependent patterns: clinical settings showed models favoring higher-income patients [4, 70], while simulated college admissions showed the reverse [66].”

    From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [70]

  • Our findings are consistent with the clinical pattern of favoring higher-income patients.

    “Our findings are consistent with the clinical pattern.”

    From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [70]

  • The consistent gender null contrasts with prior work detecting gender effects in resume screening, clinical contexts, and deliberate attempts to elicit biased outputs.

    “This consistent null contrasts with prior work detecting gender effects in resume screening[3], clinical contexts [70], and deliberate attempts to elicit biased outputs [54].”

    From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [70]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Omar, M., Soffer, S., Agbareia, R., Bragazzi, N. L., Apakama, D. U., Horowitz, C. R., Charney, A. W., Freeman, R., Kummer, B., Glicksberg, B. S., Nadkarni, G. N., & Klang, E.
Affiliations
  • Mount Sinai Health System (US)
  • Rabin Medical Center (IL)
  • Hadassah Medical Center (IL)
  • Institut für Schlaganfall- und Demenzforschung (DE)
  • Icahn School of Medicine at Mount Sinai (US)
Corresponding author
  • Omar, Mahmud
  • Nadkarni, Girish N.
  • Klang, Eyal
ORCID
Venue
Nature Medicine
Publisher
Springer Science and Business Media LLC
Published
2025-04-07
Volume
31
Issue
6
Pages
1873-1881
ISSN
  • 1078-8956 (ISSN-L)
  • 1546-170X
DOI
10.1038/s41591-025-03626-6
OpenAlex
W4409210035
Cited by
173
References
38
Citations by year
2026: 121 · 2025: 52
FWCI
41.64
Type
journal article
Language
en
Open access
closed
Licence
Locations
  • Nature Medicine (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
large language models, algorithmic bias, sociodemographic bias, emergency department care, health disparities, racial bias, LGBTQ health, homelessness, socioeconomic status, bias mitigation
MeSH terms
Clinical Decision-Making, Sociodemographic Factors, Large Language Models, Adult, Emergency Service, Hospital, Female, Humans, Language, Male, Middle Aged, Socioeconomic Factors, Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review