Algorithmic bias, generalist models, and clinical medicine
Open accessIntegrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Training models with data representative of the diverse populations being served is helpful, but new types of bias, such as semantic biases, may arise in LLMs.
“Training models with data representative of the diverse populations being served is helpful, but new types of bias, such as semantic biases, may arise in LLMs [59].”
From The Opportunities and Risks of Large Language Models in Mental Health (2024) · cites this paper as [59]
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · see citing works
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 3× · see citing works
“For example, Obermeyer et al. (2019) found that a model used in the US to enroll sufficiently sick patients into a narrow care program exhibited racial bias.”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 2× · see citing works
“…techniques for bias mitigation in large language models such as data filtration and ethical fine-tuning and to incorporate such techniques into guidance and regulatory policy on the safe and ethical development of clinical machine learning models (Bender et al., 2021; Lauren¸con et al., 2022).”
Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016) · Journal of Big Data · doi:10.1186/s40537-016-0043-6 · Cited by 6,260 · see citing works
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1× · see citing works
“A related issue is that language models may be more likely to hallucinate false information in domains of discourse that are underrep-resented in the training data (Ji et al., 2023).”
Rajkomar, A., Dean, J., & Kohane, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1814259 · Cited by 4,579 · see citing works
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
Hall, W. J., Chapman, M. V., Lee, K. M., Merino, Y. M., Thomas, T. W., Payne, B. K., Eng, E., Day, S. H., & Coyne-Beasley, T. (2015) · American Journal of Public Health · doi:10.2105/ajph.2015.302903 · Cited by 2,176 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
On the Elicitation of Preferences for Alternative Therapies
McNeil, B. J., Pauker, S. G., Sox, H. C., & Tversky, A. (1982) · New England Journal of Medicine · doi:10.1056/nejm198205273062103 · Cited by 1,668 · see citing works
Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data
Gianfrancesco, M. A., Tamang, S., Yazdany, J., & Schmajuk, G. (2018) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.3763 · Cited by 1,395 · see citing works
Ensuring fairness in machine learning to advance health equity.
Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018) · Annals of Internal Medicine · doi:10.7326/m18-1990 · Cited by 1,341 · see citing works
Artificial Intelligence and Machine Learning in Clinical Medicine, 2023
Haug, C. J., & Drazen, J. M. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2302038 · Cited by 1,215 · see citing works
The role of artificial intelligence in healthcare: a structured literature review
Secinaro, S. F., Calandra, D., Secinaro, A., Muthurangu, V., & Biancone, P. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01488-9 · Cited by 1,203 · see citing works
Preparing Medical Imaging Data for Machine Learning
Willemink, M. J., Koszek, W. A., Hardell, C., Wu, J., Fleischmann, D., Harvey, H., Folio, L. R., Summers, R. M., Rubin, D. L., & Lungren, M. P. (2020) · Radiology · doi:10.1148/radiol.2020192224 · Cited by 1,020 · see citing works
Artificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956 · see citing works
Addressing bias in big data and AI for health care: A call for open science
Norori, N., Hu, Q., Aellen, F. M., Faraci, F. D., & Tzovara, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100347 · Cited by 903 · see citing works
Peng, Y., Yan, S., & Lu, Z. (2019) · Proceedings of the 18th BioNLP Workshop and Shared Task · doi:10.18653/v1/w19-5006 · Cited by 899 · see citing works
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00455-y · Cited by 886 · see citing works
Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare
Cirillo, D., Catuara-Solarz, S., Morey, C., Guney, E., Subirats, L., Mellino, S., Gigante, A., Valencia, A., Rementeria, M. J., Chadha, A. S., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0288-5 · Cited by 553 · see citing works
Disparities in dermatology AI performance on a diverse, curated clinical image set
Daneshjou, R., Vodrahalli, K., Novoa, R. A., Jenkins, M., Liang, W., Rotemberg, V., Ko, J., Swetter, S. M., Bailey, E. E., Gevaert, O., et al. (2022) · Science Advances · doi:10.1126/sciadv.abq6147 · Cited by 544 · see citing works
Artificial intelligence and algorithmic bias: implications for health systems
Panch, T., Mattie, H., & Atun, R. (2019) · Journal of Global Health · doi:10.7189/jogh.09.020318 · Cited by 533 · see citing works
Persistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476 · see citing works
Frosch, D. L., May, S. G., Rendle, K. A. S., Tietbohl, C. K., & Elwyn, G. J. (2012) · Health Affairs · doi:10.1377/hlthaff.2011.0576 · Cited by 472 · see citing works
The Woman Worked as a Babysitter: On Biases in Language Generation
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1339 · Cited by 431 · see citing works
Hospital Readmission in General Medicine Patients: A Prediction Model
Hasan, O., Meltzer, D. O., Shaykevich, S., Bell, C. M., Kaboli, P. J., Auerbach, A. D., Wetterneck, T. B., Arora, V. M., Zhang, J., & Schnipper, J. L. (2009) · Journal of General Internal Medicine · doi:10.1007/s11606-009-1196-1 · Cited by 399 · see citing works
Red Teaming Language Models with Language Models
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363 · see citing works
Sun, M., Oliwa, T., Peek, M. E., & Tung, E. L. (2022) · Health Affairs · doi:10.1377/hlthaff.2021.01423 · Cited by 340 · see citing works
Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks
Tschandl, P., Rosendahl, C. O., Akay, B. N., Argenziano, G., Blum, A., Braun, R. P., Cabo, H., Gourhant, J.-Y., Kreusch, J. F., Lallas, A., et al. (2018) · JAMA Dermatology · doi:10.1001/jamadermatol.2018.4378 · Cited by 327 · see citing works
Pre-training of Graph Augmented Transformers for Medication Recommendation
Shang, J., Ma, T., Xiao, C., & Sun, J. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/825 · Cited by 311 · see citing works
Algorithmic bias: Senses, sources, solutions
Fazelpour, S., & Danks, D. J. (2021) · Philosophy Compass · doi:10.1111/phc3.12760 · Cited by 274 · see citing works
Clinical applications of continual learning machine learning
Lee, C. S., & Lee, A. Y. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30102-3 · Cited by 254 · see citing works
Radiology artificial intelligence: a systematic review and evaluation of methods (RAISE)
Kelly, B. S., Judge, C., Bollard, S. M., Clifford, S. M., Healy, G. M., Aziz, A., Mathur, P., Islam, S., Yeom, K. W., Lawlor, A., et al. (2022) · European Radiology · doi:10.1007/s00330-022-08784-6 · Cited by 252 · see citing works
Groh, M., Harris, C., Soenksen, L. R., Lau, F., Han, R., Kim, A., Koochek, A., & Badri, O. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00201 · Cited by 247 · see citing works
Machine Learning for Clinical Outcome Prediction
Shamout, F. E., Zhu, T.-T., & Clifton, D. (2020) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.3007816 · Cited by 210 · see citing works
Kumar, A., Gautam, A., Dey, A., Saith, R., Uttamacharya, Achyut, P., Gautam, V., Agarwal, D., Chakraverty, A., Mozumdar, A., et al. (2020) · BMC Health Services Research · doi:10.1186/s12913-019-4778-6 · Cited by 205 · see citing works
Racial disparities and temporal trends in dementia misdiagnosis risk in the United States
Gianattasio, K. Z., Prather, C., Glymour, M. M., Ciarleglio, A., & Power, M. C. (2019) · Alzheimer's & Dementia: Translational Research & Clinical Interventions · doi:10.1016/j.trci.2019.11.008 · Cited by 168 · see citing works
Default Options In Advance Directives Influence How Patients Set Goals For End-Of-Life Care
Halpern, S. D., Loewenstein, G. F., Volpp, K. G. M., Cooney, E. L., Vranas, K. C., Quill, C. M., McKenzie, M. S., Harhay, M. O., Gabler, N. B., Silva, T., et al. (2013) · Health Affairs · doi:10.1377/hlthaff.2012.0895 · Cited by 157 · see citing works
On statistical criteria of algorithmic fairness
Hedden, B. (2021) · Philosophy & Public Affairs · doi:10.1111/papa.12189 · Cited by 137 · see citing works
Relationship Between Glycated Hemoglobin and Stroke Risk: A Systematic Review and Meta‐Analysis
Mitsios, J. P., Ekinci, E. I., Mitsios, G. P., Churilov, L. P., & Thijs, V. (2018) · Journal of the American Heart Association · doi:10.1161/jaha.117.007858 · Cited by 122 · see citing works
Entertainment Chatbot for the Digital Inclusion of Elderly People Without Abstraction Capabilities
García-Méndez, S., de Arriba-Pérez, F., González-Castaño, F. J., Regueiro-Janeiro, J. A., & Gil‐Castiñeira, F. (2021) · IEEE Access · doi:10.1109/access.2021.3080837 · Cited by 96 · see citing works
Grefenstette, G. (1999) · Text, speech and language technology · doi:10.1007/978-94-015-9273-4_9 · Cited by 96 · see citing works
Life Support Withdrawal: Communication and Conflict
Norton, S. A., Tilden, V. P., Tolle, S. W., Nelson, C. A., & Eggman, S. T. (2003) · American Journal of Critical Care · doi:10.4037/ajcc2003.12.6.548 · Cited by 95 · see citing works
Gramling, R. E., Stanek, S., Ladwig, S., Gajary-Coots, E., Cimino, J. W., Anderson, W. G., Norton, S. A., Aslakson, R. A., Ast, K., Elk, R. A., et al. (2015) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2015.10.018 · Cited by 93 · see citing works
Chowdhury, A., Rosenthal, J., Waring, J., & Umeton, R. (2021) · Informatics · doi:10.3390/informatics8030059 · Cited by 85 · see citing works
Sex differences in the risk of stroke and HbA1c among diabetic patients
Zhao, W., Katzmarzyk, P. T., Horswell, R. L., Wang, Y., Johnson, J. L., & Hu, G. (2014) · Diabetologia · doi:10.1007/s00125-014-3190-3 · Cited by 59 · see citing works
Does BERT need domain adaptation for clinical negation detection?
Lin, C., Bethard, S. J., Dligach, D., Sadeque, F., Savova, G., & Miller, T. A. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa001 · Cited by 59 · see citing works
Ross, A. B., Kalia, V., Chan, B. Y., & Li, G. (2020) · BMC Health Services Research · doi:10.1186/s12913-020-05698-1 · Cited by 53 · see citing works
Enabling Fairness in Healthcare Through Machine Learning
Grote, T., & Keeling, G. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09658-7 · Cited by 44 · see citing works
Self-citation“I’m Here for You”: Can Social Chatbots Truly Support Their Users? A Literature Review
van Wezel, M. M. C., Croes, E. A. J., & Antheunis, M. L. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68288-0_7 · Cited by 36 · see citing works
On Algorithmic Fairness in Medical Practice
Grote, T., & Keeling, G. (2022) · Cambridge Quarterly of Healthcare Ethics · doi:10.1017/s0963180121000839 · Cited by 32 · see citing works
Self-citationRacial Barriers May Hamper Diagnosis, Care of Patients With Alzheimer Disease
Stephenson, J. (2001) · JAMA · doi:10.1001/jama.286.7.779-jmn0815-3-1 · Cited by 28 · see citing works
Uthoff, J., Nagpal, P., Sanchez, R., Gross, T. J., Lee, C. H., & Sieren, J. C. (2019) · Translational Lung Cancer Research · doi:10.21037/tlcr.2019.12.19 · Cited by 28 · see citing works
Sirrianni, J., Sezgın, E., Claman, D. B., & Linwood, S. L. (2022) · Methods of Information in Medicine · doi:10.1055/a-1900-7351 · Cited by 27 · see citing works
Karystianis, G., Cabral, R. C., Han, S. C., Poon, J., & Butler, T. G. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.602683 · Cited by 24 · see citing works
Machine Learning for Violence Risk Assessment Using Dutch Clinical Notes
Mosteiro, P. J., Rijcken, E., Zervanou, K., Kaymak, U., Scheepers, F. E., & Spruit, M. R. (2021) · Journal of Artificial Intelligence for Medical Sciences · doi:10.2991/jaims.d.210225.001 · Cited by 17 · see citing works
Flagged first, then by citations
6 of the 64 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
The Opportunities and Risks of Large Language Models in Mental Health
Lawrence, H. R., Schneider, R. A., Rubin, S. B., Matarić, M. J., McDuff, D. J., & Jones Bell, M. (2024) · JMIR Mental Health · doi:10.2196/59479 · Cited by 235 · see citing works
BioSignal Copilot: Leveraging the power of LLMs in drafting reports for biomedical signalsPreprint
Liu, C., Ma, Y., Kothur, K., Nikpour, A., & Kavehei, O. (2023) · medRxiv · doi:10.1101/2023.06.28.23291916 · Cited by 15 · see citing works
Application of Large Language Models in Traditional Chinese Medicine: A State-of-the-Art Review
Shataer, D., Cao, S.-X., Liu, X., Aierken, K., Bhattacharya, P., Sinha, A., & Liu, H. (2025) · The American Journal of Chinese Medicine · doi:10.1142/s0192415x25500375 · Cited by 14 · see citing works
Blinded evaluation of GPT-4 and physician responses to patient inquiries across multiple specialties
Sallabi, A., Hrvoj, H., & Štajduhar, A. (2025) · Digital Health · doi:10.1177/20552076251390001 · Cited by 1 · see citing works
Morandín-Ahuerma, F. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2565393
- Authors
- Keeling, G.
- Affiliations
- Google (United Kingdom) (GB)
- Corresponding author
- Keeling, Geoff
- ORCID
- Keeling, Geoff
- Venue
- AI and Ethics
- Published
- 2023-08-14
- Volume
- 4
- Issue
- 4
- Pages
- 1533-1544
- ISSN
- 2730-5953 (ISSN-L)
- 2730-5961
- OpenAlex
- W4385806884
- Cited by
- 5
- References
- 64
- Citations by year
- 2025: 3 · 2024: 1 · 2023: 1
- FWCI
- 1.25
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- AI and Ethics (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- algorithmic bias, machine learning in healthcare, large language models, BERT, Arecaceae, prompting, fine-tuning, biomedical data, bias mitigation, bioethics, diagnosis, treatment guidelines, training data bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
