The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Sociodemographic biases in medical decision making by large language models
Abstract
Large language models (LLMs) show promise in healthcare, but concerns remain that they may produce medically unjustified clinical care recommendations reflecting the influence of patients' sociodemographic characteristics. We evaluated nine LLMs, analyzing over 1.7 million model-generated outputs from 1,000 emergency department cases (500 real and 500 synthetic). Each case was presented in 32 variations (31 sociodemographic groups plus a control) while holding clinical details constant. Compared to both a physician-derived baseline and each model's own control case without sociodemographic identifiers, cases labeled as Black or unhoused or identifying as LGBTQIA+ were more frequently directed toward urgent care, invasive interventions or mental health evaluations. For example, certain cases labeled as being from LGBTQIA+ subgroups were recommended mental health assessments approximately six to seven times more often than clinically indicated. Similarly, cases labeled as having high-income status received significantly more recommendations (P < 0.001) for advanced imaging tests such as computed tomography and magnetic resonance imaging, while low- and middle-income-labeled cases were often limited to basic or no further testing. After applying multiple-hypothesis corrections, these key differences persisted. Their magnitude was not supported by clinical reasoning or guidelines, suggesting that they may reflect model-driven bias, which could eventually lead to health disparitie…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Directional patterns vary: clinical settings showed bias favoring higher-income patients [4, 70], while simulated college admissions favored low-SES applicants [66].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [70]
In the citing paper
“Prior work on occupation-related LLM behavior has found context-dependent patterns: clinical settings showed models favoring higher-income patients [4, 70], while simulated college admissions showed the reverse [66].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [70]
In the citing paper
“Our findings are consistent with the clinical pattern.”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [70]
In the citing paper
“This consistent null contrasts with prior work detecting gender effects in resume screening[3], clinical contexts [70], and deliberate attempts to elicit biased outputs [54].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [70]
Citations
Cites
Meyer, I. H. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.5.674 · Cited by 14,736
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
The social determinants of health: It’s time to consider the causes of the causes
Braveman, P., & Gottlieb, L. (2014) · Public Health Reports® · doi:10.1177/00333549141291s206 · Cited by 3,861
Global health burden and needs of transgender populations: a review
Reisner, S. L., Poteat, T., Keatley, J. G., Cabral, M., Mothopeng, T., Dunham, E., Holland, C. E., Max, R. J., & Baral, S. (2016) · The Lancet · doi:10.1016/s0140-6736(16)00684-x · Cited by 1,570
A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and ApplicationsPreprint
Sahoo, P., Singh, A. K., Saha, S., Jain, V., Mondal, S., & Chadha, A. R. (2024) · arXiv (Cornell University) · doi:10.13140/rg.2.2.13032.65286 · Cited by 1,526
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Waste in the US Health Care System
Shrank, W. H., Rogstad, T. L., & Parekh, N. (2019) · JAMA · doi:10.1001/jama.2019.13978 · Cited by 1,071
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045
Factors Influencing Healthcare Service Quality
Mosadeghrad, A. M. (2014) · International Journal of Health Policy and Management · doi:10.15171/ijhpm.2014.65 · Cited by 755
Zack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00225-x · Cited by 510
Eliminating Explicit and Implicit Biases in Health Care: Evidence and Research Needs
Vela, M. B., Erondu, A. I., Smith, N. A., Peek, M. E., Woodruff, J. N., & Chin, M. H. (2022) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-052620-103528 · Cited by 426 · cited 1×
“Biases can be explicit in a model’s output when different treatment recommendations are clearly tied to a patient’s sociodemographic attributes or implicit when the model’s recommendations shift without including overt acknowledgement of the sociodemographic influence 10 .”
Discrimination and Medical Mistrust in a Racially and Ethnically Diverse Sample of California Adults
Bazargan, M., Cobb, S., & Assari, S. (2021) · The Annals of Family Medicine · doi:10.1370/afm.2632 · Cited by 408
Large pre-trained language models contain human-like biases of what is right and wrong to do
Schramowski, P., Turan, C., Andersen, N., Rothkopf, C. A., & Kersting, K. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00458-8 · Cited by 291
Yadav, H., Shah, D. R., Sayed, S., Horton, S., & Schroeder, L. F. (2021) · The Lancet Global Health · doi:10.1016/s2214-109x(21)00442-3 · Cited by 273
Cascella, M., Semeraro, F., Montomoli, J., Bellini, V., Piazza, O., & Bignami, E. (2024) · Journal of Medical Systems · doi:10.1007/s10916-024-02045-3 · Cited by 219
Influence of Patients' Socioeconomic Status on Clinical Management Decisions: A Qualitative Study
Bernheim, S. M., Ross, J. S., Krumholz, H. M., & Bradley, E. H. (2008) · The Annals of Family Medicine · doi:10.1370/afm.749 · Cited by 212 · cited 1×
“Although real-world economic status can influence care intensity 27,28 , the scenario presented here was clinically identical, with only sociodemographic attributes changed.”
Raven, M. C., Lowe, R. A., Maselli, J. H., & Hsia, R. Y.-J. (2013) · JAMA · doi:10.1001/jama.2013.1948 · Cited by 200
Njoku, A. U., Evans, M. A., Nimo-Sefah, L., & Bailey, J. (2023) · Healthcare · doi:10.3390/healthcare11030438 · Cited by 174
Operario, D., Gamarel, K. E., Grin, B. M., Lee, J. H., Kahler, C. W., Marshall, B. D. L., van den Berg, J. J., & Zaller, N. D. (2015) · American Journal of Public Health · doi:10.2105/ajph.2015.302762 · Cited by 171
Cardiovascular Disease in Women: Understanding Symptoms and Risk Factors
Keteepe‐Arachi, T., & Sharma, S. (2017) · European Cardiology Review · doi:10.15420/ecr.2016:32:1 · Cited by 145
The Role of Large Language Models in Transforming Emergency Medicine: Scoping Review
Preiksaitis, C., Ashenburg, N., Bunney, G., Chu, A., Kabeer, R., Riley, F., Ribeira, R., & Rose, C. (2024) · JMIR Medical Informatics · doi:10.2196/53787 · Cited by 134
A toolbox for surfacing health equity harms and biases in large language models
Pfohl, S. R., Cole-Lewis, H., Sayres, R., Neal, D., Asiedu, M., Dieng, A., Tomasev, N., Rashid, Q. M., Azizi, S., Rostamzadeh, N., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03258-2 · Cited by 113 · cited 1×
“While prior work has demonstrated that LLMs can produce biased outputs in healthcare settings 11–14 , large-scale analyses examining a wide range of sociodemographic attributes have only recently begun to emerge 15 .”
Investigating Gender Bias in BERT
Bhardwaj, R., Majumder, N., & Poria, S. (2021) · Cognitive Computation · doi:10.1007/s12559-021-09881-2 · Cited by 112
Omar, M., Sorin, V., Agbareia, R., Apakama, D. U., Soroush, A., Sakhuja, A., Freeman, R., Horowitz, C. R., Richardson, L. D., Nadkarni, G. N., et al. (2025) · International Journal for Equity in Health · doi:10.1186/s12939-025-02419-0 · Cited by 110
Self-citationLarge Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
Yu, Y., Zhuang, Y., Zhang, J., Meng, Y., Ratner, A., Krishna, R., Shen, J., & Zhang, C. (2023) · neural information processing systems · doi:10.52202/075280-2433 · Cited by 109
Unmasking and quantifying racial bias of large language models in medical report generation
Yang, Y., Liu, X., Jin, Q., Huang, F., & Lu, Z. (2024) · Communications Medicine · doi:10.1038/s43856-024-00601-z · Cited by 105
Glicksberg, B. S., Timsina, P., Patel, D., Sawant, A., Vaid, A., Raut, G., Charney, A. W., Apakama, D., Carr, B. G., Freeman, R., et al. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae103 · Cited by 82
Self-citationStacey, D., Brière, N., Robitaille, H., Fraser, K. D., Desroches, S., & Légaré, F. (2014) · Journal of Interprofessional Care · doi:10.3109/13561820.2014.911157 · Cited by 68
Large Language Models Are Biased Because They Are Large Language Models
Resnik, P. (2025) · Computational Linguistics · doi:10.1162/coli_a_00558 · Cited by 61
Kaplan, D. M., Palitsky, R., Arconada Alvarez, S. J., Pozzo, N. S., Greenleaf, M. N., Atkinson, C. A., & Lam, W. A. (2024) · Journal of Medical Internet Research · doi:10.2196/51837 · Cited by 55
Gmelin, J.-O. H., de Vries, Y. A., Baams, L., Aguilar‐Gaxiola, S., Alonso, J., Borges, G. L. G., Bunting, B., Cardoso, G., Florescu, S., Gureje, O., et al. (2022) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-022-02320-z · Cited by 55
Richardson-Parry, A., Baas, C., Donde, S. S., Ferraiolo, B., Karmo, M., Maravic, Z., Münter, L., Ricci‐Cabello, I., Silva, M., Tinianov, S., et al. (2023) · International Journal for Equity in Health · doi:10.1186/s12939-023-01841-6 · Cited by 54
Cau, R., Pisu, F., Suri, J. S., & Saba, L. (2024) · European Journal of Radiology · doi:10.1016/j.ejrad.2024.111867 · Cited by 43
Serchen, J., Hilden, D. R., & Beachy, M. W. (2024) · Annals of Internal Medicine · doi:10.7326/m23-2795 · Cited by 22
Certifying Counterfactual Bias in LLMsPreprint
Chaudhary, I., Hu, Q., Kumar, M., Ziyadi, M., Gupta, R., & Singh, G. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.18780 · Cited by 16
The Role of Prompt Engineering for Multimodal LLM Glaucoma DiagnosisPreprint
Agbareia, R., Omar, M., Zloto, O., Chandala, N., Tai, T. Y. T., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2024) · medRxiv · doi:10.1101/2024.10.30.24316434 · Cited by 15
Self-citationMeasuring and Mitigating Gender Bias in Legal Contextualized Language Models
Bozdag, M., Sevim, N., & Koç, A. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3628602 · Cited by 15
Research works printed but not linked: 9
- Liu, M., Sandhu, S., Reisner, S. L., Gonzales, G. & Keuroghlian, A. S. Health status and health care access among lesbian, gay, and bisexual adults in the US, 2013 to 2018. JAMA Intern. Med. 183, 380–383 (2023).
- Rejeleene, R., Xu, X. & Talburt, J. Towards trustable language models: investigating information quality of large language models. Preprint at http://arxiv.org/abs/2401.13086 (2024). · arXiv
- Poulain, R., Fayyaz, H. & Beheshti, R. Bias patterns in the application of LLMs for clinical decision support: a comprehensive study. Preprint at http://arxiv.org/abs/2404.15149 (2024). · arXiv
- OpenAI et al. GPT-4 technical report. Preprint at http://arxiv.org/abs/2303.08774 (2024). · arXiv
- Cochran, S. D., Sullivan, J. G. & Mays, V. M. Prevalence of mental disorders, psychological distress, and mental health services use among lesbian, gay, and bisexual adults in the United States. J. Consult. Clin. Psychol. 71, 53–61 (2003).
- Hoy-Ellis, C. P. Minority stress and mental health: a review of the literature. J. Homosex. 70, 806–830 (2023).
- Arpey, N. C., Gaglioti, A. H. & Rosenbaum, M. E. How socioeconomic status affects patient perceptions of health care: a qualitative study. J. Prim. Care Community Health 8, 169–175 (2017).
- Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y. & Clifton, D. A. An adversarial training framework for mitigating algorithmic biases in clinical machine learning. NPJ Digit. Med. 6, 55 (2023).
- Hackmann, S., Mahmoudian, H., Steadman, M. & Schmidt, M. Word importance explains how prompts affect language model outputs. Preprint at http://arxiv.org/abs/2403.03028 (2024). · arXiv
Cited by
Sorted by citations
Omar, M., Sorin, V., Collins, J. D., Reich, D., Freeman, R., Gavin, N., Charney, A., Stump, L., Bragazzi, N. L., Nadkarni, G. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-025-01021-3 · Cited by 204
“To automate and streamline vignette drafting, we used Anthropic ’ s Claude Sonnet 3.5 on a case-by-case basis with a structured few-shot prompt that contained two physician-written templates — one short and one long, similar to our prior validated casegenerationpipelinein NatureMedicine 16 .”
InfluentialSelf-citationArtsi, Y., Sorin, V., Glicksberg, B. S., Korfiatis, P. D., Nadkarni, G. N., & Klang, E. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1659134 · Cited by 113
Self-citationChatGPT Health performance in a structured test of triage recommendations
Ramaswamy, A., Tyagi, A., Hugo, H., Jiang, J. J., Jayaraman, P., Jangda, M., Te, A. E., Kaplan, S. A., Lampert, J., Freeman, R., et al. (2026) · Nature Medicine · doi:10.1038/s41591-026-04297-7 · Cited by 74
Self-citationGuo, S., Liu, D., Fang, X., Meng, Y. G., Zhou, Z.-Z., Li, J., Li, M., Luo, L. Q., Li, H.-L., Cai, X.-Y., et al. (2025) · International Journal of Surgery · doi:10.1097/js9.0000000000003668 · Cited by 83
Bouguettaya, A., Stuart, E. M., & Aboujaoude, E. N. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01746-4 · Cited by 60
Metadata
- Authors
- Omar, M., Soffer, S., Agbareia, R., Bragazzi, N. L., Apakama, D. U., Horowitz, C. R., Charney, A. W., Freeman, R., Kummer, B., Glicksberg, B. S., Nadkarni, G. N., & Klang, E.
- Affiliations
- Mount Sinai Health System (US)
- Rabin Medical Center (IL)
- Hadassah Medical Center (IL)
- Institut für Schlaganfall- und Demenzforschung (DE)
- Icahn School of Medicine at Mount Sinai (US)
- Corresponding author
- Omar, Mahmud
- Nadkarni, Girish N.
- Klang, Eyal
- ORCID
- Venue
- Nature Medicine
- Published
- 2025-04-07
- Volume
- 31
- Issue
- 6
- Pages
- 1873-1881
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- OpenAlex
- W4409210035
- Cited by
- 173
- References
- 38
- Citations by year
- 2026: 121 · 2025: 52
- FWCI
- 41.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Nature Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- large language models, algorithmic bias, sociodemographic bias, emergency department care, health disparities, racial bias, LGBTQ health, homelessness, socioeconomic status, bias mitigation
- MeSH terms
- Clinical Decision-Making, Sociodemographic Factors, Large Language Models, Adult, Emergency Service, Hospital, Female, Humans, Language, Male, Middle Aged, Socioeconomic Factors, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
