The DOI matches its Crossref record; 47 of the 49 references it cites were found in a registry or an index; 13 of the 20 sources its claims cite were checked for support.
Evaluating and addressing demographic disparities in medical large language models: a systematic review
Open accessPublished version of a preprint
Abstract
Abstract Background Large language models are increasingly evaluated for use in healthcare. However, concerns about their impact on disparities persist. This study reviews current research on demographic biases in large language models to identify prevalent bias types, assess measurement methods, and evaluate mitigation strategies. Methods We conducted a systematic review, searching publications from January 2018 to July 2024 across five databases. We included peer-reviewed studies evaluating demographic biases in large language models, focusing on gender, race, ethnicity, age, and other factors. Study quality was assessed using the Joanna Briggs Institute Critical Appraisal Tools. Results Our review included 24 studies. Of these, 22 (91.7%) identified biases. Gender bias was the most prevalent, reported in 15 of 16 studies (93.7%). Racial or ethnic biases were observed in 10 of 11 studies (90.9%). Only two studies found minimal or no bias in certain contexts. Mitigation strategies mainly included prompt engineering, with varying effectiveness. However, these findings are tempered by a potential publication bias, as studies with negative results are less frequently published. Conclusion Biases are observed in large language models across various medical domains. While bias detection is improving, effective mitigation strategies are still developing. As LLMs increasingly influence critical decisions, addressing these biases and their resultant disparities is essential for ensuring fair artificial intelligence systems. Future research should focus on a wider range of demographic factors, intersectional analyses, and non-Western cultural contexts. Graphic Abstract
Overview
CoChat Summary
TL;DR
This systematic review of 24 studies found that demographic biases are pervasive in medical large language models, with 22 studies (91.7%) identifying bias. Gender and racial/ethnic biases were most common, while mitigation strategies such as prompt engineering and debiasing algorithms showed promise but remain inconsistently effective.
Key findings
- Among 24 included studies, 22 (91.7%) identified biases. Gender bias was reported in 15 of 16 studies (93.7%), and racial or ethnic bias in 10 of 11 studies (90.9%). Only two studies found minimal or no bias in certain contexts.
- Other biases included age, disability, socioeconomic status, sexual orientation, language, and cultural factors. Cultural, age, and intersectional disparities were apparent in all evaluated studies (100% of 3, 2, and 3 studies, respectively); socioeconomic and language biases were noted in 50% of 2 studies each.
- Mitigation strategies were implemented in 7 studies (29%): 4 used prompt engineering and 3 used debiasing algorithms. Six of the seven reported reduced disparities; Bhardwaj et al. reported a 63.9% reduction in gender bias metrics for BERT models.
- Examples included GPT-3.5-turbo predicting lower death rates for White patients (56.54%) compared with up to 62.25% for Black patients, and GPT-4 showing lower rates of advanced imaging recommendations for patients from underrepresented racial groups.
Methods
- Systematic review following PRISMA guidelines, registered with PROSPERO (CRD42024578467). Searched PubMed, Embase, Web of Science, APA PsycInfo, and Scopus for publications from January 1, 2018, to July 31, 2024.
- Included peer-reviewed studies evaluating demographic biases in LLMs applied to medical or healthcare tasks, focusing on gender, race, ethnicity, age, and other factors. Excluded non-LLM models, performance-only studies, and non-peer-reviewed materials. Two reviewers independently screened and assessed studies.
- From 863 identified articles, 257 duplicates were removed, 539 excluded through title/abstract screening, 67 underwent full-text review, and 24 met inclusion criteria. Quality was assessed using JBI Critical Appraisal Checklists (Diagnostic Test Accuracy for 3 studies; Analytical Cross-Sectional for 21 studies). Due to heterogeneity, a narrative synthesis was conducted.
Limitations
- The paper states potential publication bias, as studies with negative results are less frequently published. It also notes limited quantitative data on mitigation effectiveness, lack of standardized metrics for measuring bias reduction, and that most studies focused on cisgender men and women, with almost no work on non-binary gender identities.
- The paper states that few studies address sexual orientation, non-binary gender identities, and intersectional identities; geographical concentration in Western countries limits understanding of biases in diverse cultural contexts; and current literature mainly focuses on GPT models.
- Quality assessment found that 8 of 24 studies (33.3%) met all applicable criteria. Weaknesses included identification and handling of confounding factors (7 studies, 29.2%, failing or unclear) and appropriateness of statistical analysis (3 studies, 12.5%, not meeting). The text also makes evident that heterogeneity prevented meta-analysis, and it remains unclear whether legitimate sex-based variations were captured.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
11 Valid · 0 Not valid · 4 Not checked · 37 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Recent research has shown that commercially available LLMs can perpetuate race-based medical misconceptions [3, 5, 6].”
§ Evaluating and addressing demographic disparities in medical large language models: a systematic review > Introduction
What the sources say
- Partly supportedcites [3] Gross N. What ChatGPT tells Us about gender: A cautionary Tale about performativity and gender biases in AI. Soc Sci. 2023;12(8):435. sourceExperimental — accuracy not yet measured
Section 2 (p. 3) says LLM biases can involve race and that biased training data can be absorbed into models. It does not address race-based medical misconceptions, so the medical-specific part of the claim is unsupported. (LLM, conf=0.82)
- Not checked — no full text of the sourcecites [5] Schramowski P, Turan C, Andersen N, Rothkopf CA, Kersting K. Large pre-trained Language models contain human-like biases of what is right and wrong to do. Nat Mach Intell. 2022;4(3):258–68. source
- Supportedcites [6] Yang Y, Liu X, Jin Q, Huang F, Lu Z. Unmasking and Quantifying Racial Bias of Large Language Models in Medical Report Generation [Internet]. arXiv; 2024 [cited 2024 Jun 20]. Available from: http://arxiv.org/abs/2401.13867 source
The Main section identifies ChatGPT and GPT-4 as widely available models. It reports that GPT-3.5-turbo generated fabricated patient histories and racially skewed diagnoses, including different diagnoses for patients with identical conditions. This supports the paraphrase that such models can perpetuate race-based medical misconceptions. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“LLMs being less likely to recommend advanced imaging for patients from underrepresented racial groups [6].”
§ Evaluating and addressing demographic disparities in medical large language models: a systematic review > Introduction
What the sources say
- Partly supportedcites [6] Yang Y, Liu X, Jin Q, Huang F, Lu Z. Unmasking and Quantifying Racial Bias of Large Language Models in Medical Report Generation [Internet]. arXiv; 2024 [cited 2024 Jun 20]. Available from: http://arxiv.org/abs/2401.13867 sourceExperimental — accuracy not yet measured
The Main section reports that generated medical reports favored White patients with superior treatments, including surgery for a White patient where a Black patient received conservative care. It does not report that LLMs were less likely to recommend advanced imaging for patients in other racial groups; searches for imaging, MRI, and CT did not identify such a finding. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“a recent survey of FDA-approved AI clinical decision support tools found none included a bias evaluation, defined as an analysis to determine whether the tool’s outcomes are fair across different patient groups [10].”
§ Evaluating and addressing demographic disparities in medical large language models: a systematic review > Introduction
What the sources say
- Partly supportedcites [10] Lee JT, Moffett AT, Maliha G, Faraji Z, Kanter GP, Weissman GE. Analysis of devices authorized by the FDA for clinical decision support in critical care. JAMA Intern Med. 2023;183(12):1399–401. sourceExperimental — accuracy not yet measured
The Results report that none of the 10 FDA-authorized AI/ML clinical decision support devices identified for critical care mentioned an evaluation of performance bias. The Methods specify that the authors examined FDA decision summaries, so the finding concerns what those documents reported—not whether the tools themselves underwent any bias evaluation. The citing sentence also omits the critical-care restriction. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In a study evaluating four LLMs across multiple healthcare scenarios, all models demonstrated instances of promoting debunked racial stereotypes in medicine [8].”
§ Evaluating and addressing demographic disparities in medical large language models: a systematic review > Introduction
What the sources say
- Supportedcites [8] Omiye JA, Lester JC, Spichak S, Rotemberg V, Daneshjou R. Large Language models propagate race-based medicine. NPJ Digit Med. 2023;6:195. source
The Abstract states that four LLMs were tested across eight scenarios and that every model had instances of promoting race-based medicine or racist tropes. The Methods describe nine questions covering multiple aspects of medicine, with responses reviewed for debunked race-based content. (LLM, conf=0.94)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Older models like BERT displayed bias primarily in classification tasks, with responses differing based on patient race and gender.”
§ Evaluating and addressing demographic disparities in medical large language models: a systematic review > Results > Summary of the included studies
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
18 more claims
Create a free account to see all 52 claims and every source verdict.
- Probably not a substantial claim. 19
- It may state two separate claims as one. 13
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
Rayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
Addressing bias in big data and AI for health care: A call for open science
Norori, N., Hu, Q., Aellen, F. M., Faraci, F. D., & Tzovara, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100347 · Cited by 903
Stereotype Content: Warmth and Competence Endure
Fiske, S. T. (2018) · Current Directions in Psychological Science · doi:10.1177/0963721417738825 · Cited by 839
Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions
Abd-alrazaq, A., AlSaad, R., Alhuwail, D., Ahmed, A., Healy, P. M., Latifi, S., Aziz, S., Damseh, R., Alabed Alrazak, S., & Sheikh, J. (2023) · JMIR Medical Education · doi:10.2196/48291 · Cited by 722
PROSPERO: An International Register of Systematic Review Protocols
Schiavo, J. H. (2019) · Medical Reference Services Quarterly · doi:10.1080/02763869.2019.1588072 · Cited by 643
Bias in artificial intelligence algorithms and recommendations for mitigation
Nazer, L. H., Zatarah, R., Waldrip, S., Ke, J. X. C., Moukheiber, M., Khanna, A. K., Hicklen, R. S., Moukheiber, L., Moukheiber, D., Ma, H., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000278 · Cited by 621 · cited 1×
“Current mitigation strategies include debiasing algorithms, prompt engineering, and diverse training data [9].”
Zack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00225-x · Cited by 510 · cited 2×
“These biases appear mainly in written output text, but also affect prognosis predictions and recommendations for treatments and management protocols [18].”
Large language models propagate race-based medicine
Omiye, J. A., Lester, J. C., Spichak, S., Rotemberg, V., & Daneshjou, R. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00939-z · Cited by 417
Research works printed but not linked: 2
- Lefebvre C. Chapter 4: Searching for and selecting studies [Internet]. [cited 2024 Aug 10]. Available from: https://training.cochrane.org/handbook/current/chapter-04
- Yeh KC, Chi JA, Lian DC, Hsieh SK, Evaluating Interfaced LLM, Bias. In 2023 [cited 2024 Jun 20]. Available from: https://www.semanticscholar.org/paper/Evaluating-Interfaced-LLM-Bias-Yeh-Chi/be4adc35746c179eb4e660894f7af0ced88b6bdb
Cited by
Large Language Models and Empathy: Systematic Review
Sorin, V., Brin, D., Barash, Y., Konen, E., Charney, A., Nadkarni, G., & Klang, E. (2024) · Journal of Medical Internet Research · doi:10.2196/52597 · Cited by 240
Self-citationSociodemographic biases in medical decision making by large language models
Omar, M., Soffer, S., Agbareia, R., Bragazzi, N. L., Apakama, D. U., Horowitz, C. R., Charney, A. W., Freeman, R., Kummer, B., Glicksberg, B. S., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03626-6 · Cited by 173
Self-citationFine-Tuning Large Language Models for Specialized Use Cases
Anisuzzaman, D., Malins, J. G., Friedman, P. A., & Attia, Z. I. (2024) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2024.11.005 · Cited by 264
Large Language Models in Medical Diagnostics: Scoping Review With Bibliometric Analysis
Su, H., Sun, Y., Li, R., Zhang, A., Yang, Y., Xiao, F., Duan, Z., Chen, J., Hu, Q., Yang, T., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/72062 · Cited by 79
Omar, M., Agbareia, R., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2025) · JMIR Medical Informatics · doi:10.2196/66917 · Cited by 53
Self-citation
Metadata
- Authors
- Omar, M., Sorin, V., Agbareia, R., Apakama, D. U., Soroush, A., Sakhuja, A., Freeman, R., Horowitz, C. R., Richardson, L. D., Nadkarni, G. N., & Klang, E.
- Affiliations
- Icahn School of Medicine at Mount Sinai (US)
- Mayo Clinic (US)
- Tel Aviv Sourasky Medical Center (IL)
- Corresponding author
- Omar, Mahmud
- ORCID
- Venue
- International Journal for Equity in Health
- Published
- 2025-02-26
- Volume
- 24
- Issue
- 1
- Pages
- 57-57
- ISSN
- 1475-9276 (ISSN-L)
- PMC
- PMC11866893
- OpenAlex
- W4407965332
- Cited by
- 110
- References
- 42
- Citations by year
- 2026: 72 · 2025: 34
- FWCI
- 16.40
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- International Journal for Equity in Health (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (acceptedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Gender Studies › Diversity and Career in Medicine
- Keywords
- large language models, demographic bias, health disparities, gender bias, racial and ethnic bias, bias mitigation, prompt engineering, algorithmic fairness, artificial intelligence in healthcare, bias detection
- MeSH terms
- Large Language Models, Demography, Ethnicity, Female, Humans, Healthcare Disparities, Sexism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluating and addressing demographic disparities in medical large language models: a systematic review. https://app.cochat.ai/papers/10.1186/s12939-025-02419-0
- The paper
- Omar, M., Sorin, V., Agbareia, R., Apakama, D. U., Soroush, A., Sakhuja, A., Freeman, R., Horowitz, C. R., Richardson, L. D., Nadkarni, G. N., & Klang, E. (2025). Evaluating and addressing demographic disparities in medical large language models: a systematic review. International Journal for Equity in Health. https://doi.org/10.1186/s12939-025-02419-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.


