Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index; 54 of the 84 sources its claims cite were checked for support.

Bias in artificial intelligence algorithms and recommendations for mitigation

Open access

Abstract

The adoption of artificial intelligence (AI) algorithms is rapidly increasing in healthcare. Such algorithms may be shaped by various factors such as social determinants of health that can influence health outcomes. While AI algorithms have been proposed as a tool to expand the reach of quality healthcare to underserved communities and improve health equity, recent literature has raised concerns about the propagation of biases and healthcare disparities through implementation of these algorithms. Thus, it is critical to understand the sources of bias inherent in AI-based algorithms. This review aims to highlight the potential sources of bias within each step of developing AI algorithms in healthcare, starting from framing the problem, data collection, preprocessing, development, and validation, as well as their full implementation. For each of these steps, we also discuss strategies to mitigate the bias and disparities. A checklist was developed with recommendations for reducing bias during the development and implementation stages. It is important for developers and users of AI-based algorithms to keep these important considerations in mind to advance health equity for all populations.

Overview

CoChat Summary

TL;DR

This review maps sources of bias across the AI algorithm lifecycle in healthcare—problem framing, data collection, preprocessing, development/validation, and implementation—and proposes mitigation strategies and a checklist. It argues that bias can arise at every stage and that diverse teams, representative data, bias identification, preprocessing safeguards, validation, and monitoring are needed to advance health equity.

Key findings

  • Bias can enter at each stage: framing (e.g., cystic fibrosis with ~40,000 US patients receives over 3 times more NIH funding per affected individual than sickle cell disease with ~100,000 patients; sex funding favored males in about three-quarters of gender-dominant diseases), data collection (an AKI model used data from over 700,000 patients but was predominantly older non-Black men, average age 62, over 90% male), preprocessing (SOFA score shows suboptimal performance in Black, female, and disabled patients), development/validation (only half of 152 machine-learning prediction studies examined potential overfitting), and implementation (Obermeyer algorithm would reduce Black patients needing additional care by more than half; remediation increased the percentage receiving additional care from 17.7% to 46.5%).
  • Mitigation recommendations include inclusive diverse teams, combining datasets, open code sharing, identifying bias sources, preprocessing techniques such as re-weighing, suppression, massaging, and multiple imputation, de-biasing/oversampling, external validation, and monitoring for data and concept drift.
  • An AHRQ systematic review identified 6 major strategies to mitigate racial and ethnic bias; the most common was removing race, used in 15 of 33 studies, but no preferred specific strategy could be recommended.
  • A proposed checklist covers framing the problem, data sources, data preprocessing, model development, model validation, and model implementation.

Methods

  • Narrative review, not a systematic or comprehensive review; it aims to highlight main sources of bias and mitigation strategies within each step of developing AI algorithms in healthcare.
  • Draws on published literature and examples; no primary data collection, sample, or empirical analysis is reported.
  • Synthesizes mitigation strategies and presents Table 1, a checklist to aid in mitigating bias during development and implementation.

Limitations

  • Stated in the paper: the review does not aim to provide a comprehensive description of published sources of bias and mitigation strategies; current understanding of bias in AI reflects “no more than the tip of the iceberg”; most identified bias sources reflect developed countries, mostly North America and Europe, limiting understanding of global healthcare bias; de-biasing techniques need more research; and AHRQ could not recommend a preferred mitigation strategy.
  • Evident from the text: no systematic search strategy or formal quality appraisal is described, and the proposed checklist is presented without validation data.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pdig.0000278

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig 1 Sources of bias that may contribute to health disparities within each step of developing an AI-based algorithm.
Fig 1 Sources of bias that may contribute to health disparities within each step of developing an AI-based algorithm.CC BY · PubMed Central

Verification

Claims the paper makes

44 Valid · 1 Not valid · 23 Not checked · 19 Own results and interpretations

45 more claims · 23 with their cited sources checked

Create a free account to see all 87 claims and to verify the 5 references still open.

See all claims — free account
  • It may state two separate claims as one. 20
  • Probably not a substantial claim. 23

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 59 references

Research works printed but not linked: 1

  1. D Cirillo, 2020, Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare, NPJ Digit Med, 3, 81, 10.1038/s41746-020-0288-5 · DOI · Paper page

Cited by

Metadata

Authors
Nazer, L. H., Zatarah, R., Waldrip, S., Ke, J. X. C., Moukheiber, M., Khanna, A. K., Hicklen, R. S., Moukheiber, L., Moukheiber, D., Ma, H., & Mathur, P.
Affiliations
  • King Hussein Cancer Center (JO)
  • Morehouse School of Medicine (US)
  • Dalhousie University (CA)
  • Massachusetts Institute of Technology (US)
  • Atrium Health Wake Forest Baptist (US)
  • The University of Texas MD Anderson Cancer Center (US)
  • Beth Israel Deaconess Medical Center (US)
  • Cleveland Clinic (US)
Corresponding author
Nazer, Lama H.
Venue
PLOS Digital Health
Published
2023-06-22
Volume
2
Issue
6
Pages
e0000278
ISSN
2767-3170 (ISSN-L)
OpenAlex
W4381716616
Cited by
621
References
59
Citations by year
2026: 180 · 2025: 285 · 2024: 144 · 2023: 11
FWCI
24.01
Type
journal article
Language
en
Open access
Locations
  • PLOS Digital Health (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Greater South Information System
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
algorithmic bias, artificial intelligence in healthcare, health equity, health care disparities, bias mitigation, social determinants of health, underserved populations, data collection bias, data preprocessing, algorithm validation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.