The DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index; 54 of the 84 sources its claims cite were checked for support.
Bias in artificial intelligence algorithms and recommendations for mitigation
Open accessAbstract
The adoption of artificial intelligence (AI) algorithms is rapidly increasing in healthcare. Such algorithms may be shaped by various factors such as social determinants of health that can influence health outcomes. While AI algorithms have been proposed as a tool to expand the reach of quality healthcare to underserved communities and improve health equity, recent literature has raised concerns about the propagation of biases and healthcare disparities through implementation of these algorithms. Thus, it is critical to understand the sources of bias inherent in AI-based algorithms. This review aims to highlight the potential sources of bias within each step of developing AI algorithms in healthcare, starting from framing the problem, data collection, preprocessing, development, and validation, as well as their full implementation. For each of these steps, we also discuss strategies to mitigate the bias and disparities. A checklist was developed with recommendations for reducing bias during the development and implementation stages. It is important for developers and users of AI-based algorithms to keep these important considerations in mind to advance health equity for all populations.
Overview
CoChat Summary
TL;DR
This review maps sources of bias across the AI algorithm lifecycle in healthcare—problem framing, data collection, preprocessing, development/validation, and implementation—and proposes mitigation strategies and a checklist. It argues that bias can arise at every stage and that diverse teams, representative data, bias identification, preprocessing safeguards, validation, and monitoring are needed to advance health equity.
Key findings
- Bias can enter at each stage: framing (e.g., cystic fibrosis with ~40,000 US patients receives over 3 times more NIH funding per affected individual than sickle cell disease with ~100,000 patients; sex funding favored males in about three-quarters of gender-dominant diseases), data collection (an AKI model used data from over 700,000 patients but was predominantly older non-Black men, average age 62, over 90% male), preprocessing (SOFA score shows suboptimal performance in Black, female, and disabled patients), development/validation (only half of 152 machine-learning prediction studies examined potential overfitting), and implementation (Obermeyer algorithm would reduce Black patients needing additional care by more than half; remediation increased the percentage receiving additional care from 17.7% to 46.5%).
- Mitigation recommendations include inclusive diverse teams, combining datasets, open code sharing, identifying bias sources, preprocessing techniques such as re-weighing, suppression, massaging, and multiple imputation, de-biasing/oversampling, external validation, and monitoring for data and concept drift.
- An AHRQ systematic review identified 6 major strategies to mitigate racial and ethnic bias; the most common was removing race, used in 15 of 33 studies, but no preferred specific strategy could be recommended.
- A proposed checklist covers framing the problem, data sources, data preprocessing, model development, model validation, and model implementation.
Methods
- Narrative review, not a systematic or comprehensive review; it aims to highlight main sources of bias and mitigation strategies within each step of developing AI algorithms in healthcare.
- Draws on published literature and examples; no primary data collection, sample, or empirical analysis is reported.
- Synthesizes mitigation strategies and presents Table 1, a checklist to aid in mitigating bias during development and implementation.
Limitations
- Stated in the paper: the review does not aim to provide a comprehensive description of published sources of bias and mitigation strategies; current understanding of bias in AI reflects “no more than the tip of the iceberg”; most identified bias sources reflect developed countries, mostly North America and Europe, limiting understanding of global healthcare bias; de-biasing techniques need more research; and AHRQ could not recommend a preferred mitigation strategy.
- Evident from the text: no systematic search strategy or formal quality appraisal is described, and the proposed checklist is presented without validation data.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
44 Valid · 1 Not valid · 23 Not checked · 19 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“the authors demonstrated that utilizing the algorithm would reduce the number of Black patients needing additional care by more than half.”
§ Bias in artificial intelligence algorithms and recommendations for mitigation > Introduction
What the sources say
- Disputedcites [17] Z Obermeyer, 2019, Dissecting racial bias in an algorithm used to manage the health of populations, Science sourceExperimental — accuracy not yet measured
The summary on p. 9 says the algorithm’s racial bias reduces the number of Black patients *identified* for extra care by more than half; it does not reduce the number who need care. The paper’s opening summary (p. 2) says remedying that disparity would increase the percentage of Black patients receiving additional help from 17.7% to 46.5%. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Literature has demonstrated how AI can revolutionize healthcare through its ability to perform various clinical tasks with a comparable or arguably even faster and with greater precision performance to that of humans [10–12].”
§ Bias in artificial intelligence algorithms and recommendations for mitigation > Introduction
What the sources say
- Partly supportedcites [10] F Homayounieh, 2021, An artificial intelligence-based chest x-ray model on human nodule detection accuracy from a multicenter study, JAMA Netw Open sourceExperimental — accuracy not yet measured
The Results (Sensitivity, Specificity, and Accuracy; page not provided) report greater detection accuracy when radiologists used AI assistance, and the Discussion reports higher standalone AI sensitivity than the readers for pulmonary-nodule detection. The paper does not establish faster performance or performance across various clinical tasks; it studies nodule detection on chest radiographs. (LLM, conf=0.88)
- Partly supportedcites [11] RM Wehbe, 2021, DeepCOVID-XR: An artificial intelligence algorithm to detect COVID-19 on chest radiographs trained and tested on a large U.S. clinical data set, Radiology sourceExperimental — accuracy not yet measured
The Results (Comparison with Expert Thoracic Radiologists) report similar performance and faster analysis for DeepCOVID-XR on one task: detecting COVID-19 on chest radiographs. They do not establish greater precision than the radiologists or performance across various clinical tasks. (LLM, conf=0.88)
- Supportedcites [12] J Bajwa, 2021, Artificial intelligence in healthcare: transforming the practice of medicine, Future Healthc J source
The “Diagnostic imaging” section reports AI meeting or exceeding human experts’ performance in image-based diagnoses across several specialties. “Improving the precision and reducing waiting timings for radiotherapy planning” reports that AI can cut image-preparation time by up to 90%. These examples support the claim about various clinical tasks, though not every clinical task. (LLM, conf=0.87)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In a study that evaluated the methodological quality of 152 studies on prediction models developed using machine learning techniques, only half of the included studies examined potential overfitting of the models using appropriate strategies [43].”
§ Bias in artificial intelligence algorithms and recommendations for mitigation > Introduction > 4. Development and validation of AI-based algorithms
What the sources say
- Supportedcites [43] CLA Navarro, 2021, Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review, BMJ source
The Discussion states that only half of the included studies examined potential overfitting using split data, bootstrapping, or cross validation. The Results identify 152 included studies and report that 76 of 152 developed models accounted for overfitting and optimism. No page numbers are supplied in our copy. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The lack of such data limits the ability of researchers to understand the impact of disability on outcomes”
§ Bias in artificial intelligence algorithms and recommendations for mitigation > Introduction > 2. Data collection
What the sources say
- Not checked — no full text of the sourcecites [24] GL Krahn, 2015, Persons with disabilities as an unrecognized health disparity population, Am J Public Health source
- Partly supportedcites [32] C El Morr, 2021, A Virtual Community for Disability Advocacy: Development of a Searchable Artificial Intelligence-Supported Platform, JMIR Form Res sourceExperimental — accuracy not yet measured
The Introduction says a lack of disability data hampers critical health research and limits knowledge about the experiences of people with disabilities. It does not establish that disability data are absent from most datasets or specifically discuss measuring disability’s impact on outcomes. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“Though the dataset used to develop the algorithm was large and diverse, containing data from over 700,000 patients from multiple centers”
§ Bias in artificial intelligence algorithms and recommendations for mitigation > Introduction > 2. Data collection
What the sources say
- Supportedcites [27] N Tomašev, 2019, A clinically applicable approach to continuous prediction of future acute kidney injury, Nature source
The paper states that the model was trained on a multisite dataset of 703,782 adult patients from the US Department of Veterans Affairs. The Methods (Data Description) also describes data aggregated across VA facilities, supporting the reference to multiple centers. (LLM, conf=0.94)
plainn = 700,000- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
45 more claims · 23 with their cited sources checked
Create a free account to see all 87 claims and to verify the 5 references still open.
- It may state two separate claims as one. 20
- Probably not a substantial claim. 23
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current mitigation strategies include debiasing algorithms, prompt engineering, and diverse training data [9].”
From Evaluating and addressing demographic disparities in medical large language models: a systematic review (2025) · cites this paper as [9]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Such biases can result in inaccurate findings that might influence clinical guidelines or medical interventions, and recent studies have underscored these concerns, suggesting that these biases can lead to significant health disparities.8”
From Balancing innovation and integrity: the role of AI in research and scientific writing (2023) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“Obermeyer et al. (2019) [36] and Nazerid et al., (2023) [8] discuss how biases in training data can lead to biased AI systems, which can perpetuate healthcare disparities.”
From Leading with AI in critical care nursing: challenges, opportunities, and the human factor (2024) · cites this paper as Nazerid et al., (2023) [8]
In the citing paper
“A type of algorithmic bias strongly impacting model generalizability is aggregation bias, which occurs during the data preprocessing phase10.”
From Bias recognition and mitigation strategies in artificial intelligence healthcare applications (2025) · cites this paper as [10]
In the citing paper
“For example, patients who are sicker, have higher comorbidity, or are unable to participate in longitudinal research protocols for geographic or economic reasons may not be represented10.”
From Bias recognition and mitigation strategies in artificial intelligence healthcare applications (2025) · cites this paper as [10]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 2×
“A study by Obermeyer and colleagues further emphasized this important point by demonstrating racial bias in an algorithm widely utilized in the United States healthcare system [17].”
BackgroundPROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984
Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock 2021.
Evans, L., Rhodes, A., Alhazzani, W., Antonelli, M., Coopersmith, C. M., French, C., Machado, F. R., Mcintyre, L., Ostermann, M., Prescott, H. C., et al. (2021) · Critical Care Medicine · doi:10.1097/ccm.0000000000005337 · Cited by 2,935 · cited 1×
“The Surviving Sepsis Campaign guidelines recommend the use of machine learning algorithms that utilize scoring systems for the early screening of patients [36].”
MethodArtificial intelligence in healthcare: transforming the practice of medicine
Bajwa, J., Munir, U., Nori, A., & Williams, B. (2021) · Future Healthcare Journal · doi:10.7861/fhj.2021-0095 · Cited by 1,856
Precision medicine, AI, and the future of personalized health care
Johnson, K. B., Wei, W., Weeraratne, D., Frisse, M. E., Misulis, K., Rhee, K., Zhao, J., & Snowdon, J. L. (2021) · Clinical and Translational Science · doi:10.1111/cts.12884 · Cited by 1,854
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers
Mongan, J., Moy, L., & Kahn, C. E. (2020) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2020200029 · Cited by 1,443 · cited 1×
“A number of comprehensive frameworks and checklists have been developed, such as the Translational Evaluation of Healthcare AI (TEHAI) [46], the DECIDE-AI [47], the Consolidated Standards of Reporting Trials-Artificial Intelligence (CONSORT-AI) [48], the prediction model risk of bias assessment tool (PROBAST) [49], an…”
MethodLiu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421
Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data
Gianfrancesco, M. A., Tamang, S., Yazdany, J., & Schmajuk, G. (2018) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.3763 · Cited by 1,395 · cited 1×
“This can occur when patients receive different care or are incorrectly diagnosed based on sociodemographic factors reflecting practitioner bias [28].”
BackgroundData preprocessing techniques for classification without discrimination
Kamiran, F., & Calders, T. (2012) · Knowledge and Information Systems · doi:10.1007/s10115-011-0463-8 · Cited by 1,305
Research works printed but not linked: 1
- D Cirillo, 2020, Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare, NPJ Digit Med, 3, 81, 10.1038/s41746-020-0288-5 · DOI · Paper page
Cited by
Enhancing mental health with Artificial Intelligence: Current trends and future prospects
Olawade, D. B., Wada, O. Z., Odetayo, A., David-Olawade, A. C., Asaolu, F., & Eberhardt, J. (2024) · Journal of Medicine, Surgery, and Public Health · doi:10.1016/j.glmedi.2024.100099 · Cited by 514
“Consequently, underrepresented communities may experience misdiagnosis, insufficient treatment recommendations, or even worsening of their mental health conditions [122,123].”
BackgroundBias recognition and mitigation strategies in artificial intelligence healthcare applications
Hasanzadeh, F., Josephson, C. B., Waters, G., Adedinsewo, D., Azizi, Z., & White, J. A. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01503-7 · Cited by 320
“Inthecontextof healthcare AI, biascanbe de fi ned asany systematicand/or unfair difference in how predictions are generated for different patient populations that could lead to disparate care delivery 10 .”
InfluentialA multinational study on the factors influencing university students’ attitudes and usage of ChatGPT
Abdaljaleel, M., Barakat, M., Alsanafi, M., Salim, N. A., Abazid, H., Malaeb, D., Mohammed, A. H., Hassan, B. A. R., Wayyes, A. M., Farhan, S. S., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-52549-8 · Cited by 291
Ethical and legal considerations in healthcare AI: innovation and policy for safe and fair use
Pham, T. (2025) · Royal Society Open Science · doi:10.1098/rsos.241873 · Cited by 261
A Scoping Review of Reporting Gaps in FDA-Approved AI Medical Devices
Muralidharan, V., Adewale, B. A., Huang, C. J., Nta, M. T., Ademiju, P. O., Pathmarajah, P., Hang, M. K., Adesanya, O., Abdullateef, R. O., Babatunde, A. O., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01270-x · Cited by 201
Metadata
- Authors
- Nazer, L. H., Zatarah, R., Waldrip, S., Ke, J. X. C., Moukheiber, M., Khanna, A. K., Hicklen, R. S., Moukheiber, L., Moukheiber, D., Ma, H., & Mathur, P.
- Affiliations
- King Hussein Cancer Center (JO)
- Morehouse School of Medicine (US)
- Dalhousie University (CA)
- Massachusetts Institute of Technology (US)
- Atrium Health Wake Forest Baptist (US)
- The University of Texas MD Anderson Cancer Center (US)
- Beth Israel Deaconess Medical Center (US)
- Cleveland Clinic (US)
- Corresponding author
- Nazer, Lama H.
- ORCID
- Venue
- PLOS Digital Health
- Publisher
- Public Library of Science (PLoS)
- Published
- 2023-06-22
- Volume
- 2
- Issue
- 6
- Pages
- e0000278
- ISSN
- 2767-3170 (ISSN-L)
- PMC
- PMC10287014
- OpenAlex
- W4381716616
- Cited by
- 621
- References
- 59
- Citations by year
- 2026: 180 · 2025: 285 · 2024: 144 · 2023: 11
- FWCI
- 24.01
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLOS Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Greater South Information System
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- algorithmic bias, artificial intelligence in healthcare, health equity, health care disparities, bias mitigation, social determinants of health, underserved populations, data collection bias, data preprocessing, algorithm validation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
