Inherent bias in large language models: a random sampling analysis.
Open accessAbstract
There are mounting concerns regarding inherent bias, safety, and tendency toward misinformation of large language models (LLMs), which could have significant implications in health care. This study sought to determine whether generative artificial intelligence (AI)-based simulations of physicians making life-and-death decisions in a resource-scarce environment would demonstrate bias. Thirteen questions were developed that simulated physicians treating patients in resource-limited environments. Through a random sampling of simulated physicians using OpenAI's generative pretrained transformer (GPT-4), physicians were tasked with choosing only 1 patient to save owing to limited resources. This simulation was repeated 1000 times per question, representing 1000 unique physicians and patients each. Patients and physicians spanned a variety of demographic characteristics. All patients had similar a priori likelihood of surviving the acute illness. Overall, simulated physicians consistently demonstrated racial, gender, age, political affiliation, and sexual orientation bias in clinical decision-making. Across all demographic characteristics, physicians most frequently favored patients with similar demographic characteristics as themselves, with most pairwise comparisons showing statistical significance ( P <.05). Nondescript physicians favored White, male, and young demographic characteristics. The male doctor gravitated toward the male, White, and young, whereas the female doctor typically preferred female, young, and White patients. In addition to saving patients with their own political affiliation, Democratic physicians favored Black and female patients, whereas Republicans preferred White and male demographic characteristics. Heterosexual and gay/lesbian physicians frequently saved patients of similar sexual orientation. Overall, publicly available chatbot LLMs demonstrate significant biases, which may negatively impact patient outcomes if used to support clinical care decisions without appropriate precautions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Reported biases among LLMs have included propagation of biases based on race and ethnicity, gender, educational level, social class, sexuality, and age.
“These have included the propagation of biases based on race and ethnicity, gender, educational level, social class, sexuality, and age.184,185,186”
From Large Language Models for Chatbot Health Advice Studies: A Systematic Review (2025) · cites this paper as [186]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Implicit bias in healthcare professionals: a systematic review
FitzGerald, C., & Hurst, S. (2017) · BMC Medical Ethics · doi:10.1186/s12910-017-0179-8 · Cited by 2,398 · see citing works
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738 · see citing works
What ChatGPT and generative AI mean for science
Stokel-Walker, C., & Van Noorden, R. (2023) · Nature · doi:10.1038/d41586-023-00340-6 · Cited by 828 · see citing works
Large language models propagate race-based medicine
Omiye, J. A., Lester, J. C., Spichak, S., Rotemberg, V., & Daneshjou, R. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00939-z · Cited by 417 · see citing works
Zestcott, C. A., Blair, I. V., & Stone, J. (2016) · Group Processes & Intergroup Relations · doi:10.1177/1368430216642029 · Cited by 381 · see citing works
The Interaction of Patient Race, Provider Bias, and Clinical Ambiguity on Pain Management Decisions
Hirsh, A. T., Hollingshead, N. A., Ashburn-Nardo, L., & Kroenke, K. (2015) · The Journal of Pain · doi:10.1016/j.jpain.2015.03.003 · Cited by 162 · see citing works
Large Language Models Answer Medical Questions Accurately, but Can’t Match Clinicians’ Knowledge
Harris, E. (2023) · JAMA · doi:10.1001/jama.2023.14311 · Cited by 70 · see citing works
The Impact of Cognitive and Implicit Bias on Patient Safety and Quality
Balakrishnan, K., & Arjmand, E. M. (2018) · Otolaryngologic Clinics of North America · doi:10.1016/j.otc.2018.08.016 · Cited by 55 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Large Language Models for Chatbot Health Advice Studies: A Systematic Review
Huo, B., Boyle, A., Marfo, N., Tangamornsuksan, W., Steen, J. P., McKechnie, T., Lee, Y., Mayol, J., Antoniou, S. A., Thirunavukarasu, A. J., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.57879 · Cited by 283 · see citing works
Retrieval-augmented generation for generative artificial intelligence in health care
Yang, R., Ning, Y., Keppo, E., Liu, M., Hong, C., Bitterman, D. S., Ong, J. C. L., Ting, D. S. W., & Liu, N. (2025) · npj Health Systems · doi:10.1038/s44401-024-00004-1 · Cited by 175 · see citing works
Bouguettaya, A., Stuart, E. M., & Aboujaoude, E. N. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01746-4 · Cited by 55 · see citing works
Cognitive Biases and Artificial Intelligence
Wang, J., & Redelmeier, D. A. (2024) · NEJM AI · doi:10.1056/aics2400639 · Cited by 43 · see citing works
Belkhouribchia, J., & Pen, J. J. (2025) · Frontiers in Nutrition · doi:10.3389/fnut.2025.1635682 · Cited by 43 · see citing works
Examining the responsible use of zero-shot AI approaches to scoring essays
Johnson, M., & Zhang, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-79208-2 · Cited by 33 · see citing works
Sezgin, E., Jackson, D. I., Kocaballi, A. B., Bibart, M., Zupanec, S., Landier, W., Audino, A., Ranalli, M., & Skeens, M. (2025) · Cancer Medicine · doi:10.1002/cam4.70554 · Cited by 30 · see citing works
Traylor, D., Kern, K. V., Anderson, E. E., & Henderson, R. (2025) · Cureus · doi:10.7759/cureus.76825 · Cited by 30 · see citing works
Gender Representation of Health Care Professionals in Large Language Model–Generated Stories
Menz, B. D., Kuderer, N. M., Chin-Yee, B. H., Logan, J. M., Rowland, A., Sorich, M. J., & Hopkins, A. M. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.34997 · Cited by 22 · see citing works
Foundation models for radiology—the position of the AI for Health Imaging (AI4HI) network
de Almeida, J. G., Alberich, L. C., Tsakou, G., Kostas Marias, A., Tsiknakis, M., Lekadir, K., Martí‐Bonmatí, L., & Papanikolaou, N. K. (2025) · Insights into Imaging · doi:10.1186/s13244-025-02056-9 · Cited by 21 · see citing works
Sobowale, K., & Humphrey, D. K. (2025) · JMIR Formative Research · doi:10.2196/65605 · Cited by 15 · see citing works
Alexandrou, M., Kumar, S., Mahtani, A. U., Strepkos, D., Carvalho, P. E. P., Mutlu, D., Ser, Ö. S., Rempakos, A., Mastrodemos, O. C., Rangan, B. V., et al. (2025) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2025.08.019 · Cited by 9 · see citing works
Balakrishnan, S., Thongprayoon, C., Wathanavasin, W., Miao, J., Mao, M. A., Craici, I. M., & Cheungpasitporn, W. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1525937 · Cited by 11 · see citing works
Crawford, J. L. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000757 · Cited by 11 · see citing works
Understanding Social Biases in Large Language Models
Gupta, O., Marrone, S., Gargiulo, F., Jaiswal, R. R., & Marassi, L. (2025) · AI · doi:10.3390/ai6050106 · Cited by 10 · see citing works
Technosolutionism and the empathetic medical chatbot
Sharon, T. (2025) · AI & Society · doi:10.1007/s00146-025-02441-4 · Cited by 11 · see citing works
A Survey on Bias in Machine Learning Research
Mikołajczyk, A., & Grochowski, M. (2025) · IEEE Access · doi:10.1109/access.2025.3649998 · Cited by 10 · see citing works
Sokhansanj, B. A., & ROSEN, G. L. (2025) · Human Genetics · doi:10.1007/s00439-025-02768-4 · Cited by 7 · see citing works
Šimunović, I., Rezić, K., Franić, N., Boduljak, G., Batinić, M., Jukić, I. K. D., Jelovina, I., Biočić, J., Pogorelić, Z., & Markić, J. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03258-3 · Cited by 9 · see citing works
Yang, E., Garcia, T., Williams, H., Kumar, B., Ramé, M., Rivera, E., Ma, Y., Amar, J., Catalani, C., & Jia, Y. (2025) · JMIR Formative Research · doi:10.2196/75421 · Cited by 5 · see citing works
Evaluating chatbots in psychiatry: Rasch-based insights into clinical knowledge and reasoning
Chang, Y., Huang, S., Hsu, W., & Liu, Y. (2025) · PLoS ONE · doi:10.1371/journal.pone.0330303 · Cited by 5 · see citing works
Using Generative Artificial Intelligence for Healthcare-Associated Infection Surveillance
Morgan, D. J., Goodman, K. E., Branch‐Elliman, W., Shenoy, E. S., Salinas, J. L., Jones, M., Singh, S. P., Schrank, G. M., Pineles, L. L., Alshanqeeti, S., et al. (2025) · Clinical Infectious Diseases · doi:10.1093/cid/ciaf619 · Cited by 5 · see citing works
STEM, a Non-Place for Women? Evidences and Transformative Initiatives
Cernadas, E., Lorenzo, E. A., Fernández-Delgado, M., & Iglesias, E. C. (2025) · Social Sciences · doi:10.3390/socsci14060384 · Cited by 4 · see citing works
Using labels to limit AI misuse in health
Nsoesie, E. O., & Ghassemi, M. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00676-7 · Cited by 3 · see citing works
Yang, E., Garcia, T., Williams, H., Kumar, B., Ramé, M., Rivera, E., Ma, Y., Amar, J., Catalani, C., & Jia, Y. (2025) · doi:10.2196/preprints.75421 · Cited by 3 · see citing works
Bai, T., Jiang, K., Yu, Y., Qie, S., Wang, C., Wang, B., & Zhang, W. (2026) · Future Internet · doi:10.3390/fi18020095 · Cited by 2 · see citing works
Farrag, A. N., Soliman, A., Hatef, E., Goodin, A. J., & Rouhizadeh, M. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf201 · Cited by 3 · see citing works
Promoting fairness in LLMs: detection and mitigation of gender bias
Sachdeva, T., Singhal, M., & Khetarpaul, S. (2026) · Knowledge and Information Systems · doi:10.1007/s10115-026-02731-3 · Cited by 2 · see citing works
Ahmadi, A., Baudry, J. Y., Tenhundfeld, N. L., Goff, K., & Adamek, D. (2026) · PLoS ONE · doi:10.1371/journal.pone.0342700 · Cited by 1 · see citing works
Ethical issues of large language models: a multi-level thematic synthesis of the academic literature
Pervez, H., Minkkinen, M., Jylhä, H., & Mäntymäki, M. (2026) · AI and Ethics · doi:10.1007/s43681-026-01173-5 · Cited by 3 · see citing works
Impact of Artificial Intelligence on Lingual Connection and Its Perspectives
Yahya, H. T. A., Al-Muoaeweed, O., & Bilal, D. A. (2025) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-81308-5_32 · Cited by 1 · see citing works
Farrag, A. N., Soliman, A. S. A., Hatef, E., Goodin, A. J., & Rouhizadeh, M. (2025) · medRxiv · doi:10.1101/2025.07.04.25330866 · Cited by 1 · see citing works
How AI Tools Can Mitigate Bias and Improve Health Equity
Shaw, G. (2025) · Neurology Today · doi:10.1097/01.wnt.0001110396.54249.03
Gu, W., Xu, H., Gu, T., Li, X., Zhang, P., Liu, B., Li, X., & Yu, Z. (2025) · doi:10.2196/preprints.81058
Nielsen, C. S., Soetikno, B. T., Pollreisz, A., & Ting, D. S. W. (2025) · Current Opinion in Ophthalmology · doi:10.1097/icu.0000000000001173
Grach, S., Badawi, A., Ahmad, F., & Dolatabadi, E. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105077
Clinical trials futures: reimagining health equity through a design and foresight approach
Swana, M., & Kamau, E. (2025) · Medical Humanities · doi:10.1136/medhum-2025-013349
Halagur, A. S., Polkampally, S., Chang, M. T., & Ayoub, N. (2026) · Journal of Neurological Surgery Part B Skull Base · doi:10.1055/a-2781-9088
Self-citationRawlsian Agents: An Application of Large Language Models (LLM) to Forge Fairer Bilateral Agreements
Ferro, S. M., Tai, J., Tsai, R., Verma, S., Ma, J., Skjerdal, N., & Lopatka, M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-15632-7_1
Behind the Words: Unmasking Bias Lurking in LLMs and Data
Dhingra, N., Kale, G., Chavan, H., & Arora, S. (2025) · IEEE Pune Section International Conference (PuneCon) · doi:10.1109/punecon67554.2025.11378839
Potential Applications of Artificial Intelligence in Neurology Education
Mainali, R., Deng, A., Rayala, H., & Southerland, A. M. (2026) · Seminars in Neurology · doi:10.1055/a-2832-4638
Moving beyond disease-centered care toward whole-person health
Wang, Y., & Hui, K.-K. P. (2026) · Integrative Medicine Research · doi:10.1016/j.imr.2026.101361
He, L., Do, D., Shet, V. G., Farghaly, O., Deshpande, P., Madiraju, P., Ye, J., & Beestrum, M. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag089
Wu, S. S., Shanmugam, M. V., Lee, Y., & Ayoub, N. (2026) · World Journal of Otorhinolaryngology - Head and Neck Surgery · doi:10.1002/wjo2.70130
Self-citationAdvancements in the Implementation of Artificial Intelligence for Imaging and Image Analysis
Niz, M. D., Kirchenbüechler, D., Dluhy, P., Piguel, N. H., & Arvanitis, C. (2026) · Microscopy and Microanalysis · doi:10.1093/mam/ozag051
Beyond the Black Box: A Point-Of-Care Framework for LLM Literacy and Critical Appraisal
Stephenson, C. R., Aakre, C. A., Croghan, I. T., Hurt, R. T., & West, C. P. (2026) · Journal of Primary Care & Community Health · doi:10.1177/21501319261491056
Advancing Patient‐Centred AI With Adaptive Machine Learning
Kalogeropoulos, D., Barach, P. R., Downing, A., Verhulst, S., & Lustberg, M. B. (2026) · Cancer Medicine · doi:10.1002/cam4.72371
- Authors
- Ayoub, N. F., Balakrishnan, K., Ayoub, M. S., Barrett, T. F., David, A. P., & Gray, S. T.
- Affiliations
- Massachusetts Eye and Ear Infirmary (US)
- Stanford Medicine (US)
- Northwell Health (US)
- Washington University in St. Louis (US)
- Corresponding author
- Ayoub, Noel F.
- Venue
- Mayo Clinic Proceedings Digital Health
- Publisher
- Elsevier BV
- Published
- 2024-06
- Volume
- 2
- Issue
- 2
- Pages
- 186-191
- ISSN
- 2949-7612 (ISSN-L)
- PMC
- PMC11975844
- OpenAlex
- W4394723029
- Cited by
- 47
- References
- 8
- Citations by year
- 2026: 14 · 2025: 29 · 2024: 4
- FWCI
- 19.48
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Mayo Clinic Proceedings Digital Health (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Gender Studies › Diversity and Career in Medicine
- Keywords
- large language models, algorithmic bias, GPT-4, resource allocation decisions, triage, demographic bias, racial bias, gender bias, political affiliation, sexual orientation, clinical decision support, healthcare resource shortages
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
