Cited by 376Ethics and Social Impacts of AIArtificial Intelligence in Healthcare and EducationExplainable Artificial Intelligence (XAI)

The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index; 1 of the 29 sources its claims cite were checked for support.

Bias in AI-based models for medical applications: challenges and mitigation strategies

Open access

Abstract

Artificial intelligence systems are increasingly being applied to healthcare. In surgery, AI applications hold promise as tools to predict surgical outcomes, assess technical skills, or guide surgeons intraoperatively via computer vision. On the other hand, AI systems can also suffer from bias, compounding existing inequities in socioeconomic status, race, ethnicity, religion, gender, disability, or sexual orientation. Bias particularly impacts disadvantaged populations, which can be subject to algorithmic predictions that are less accurate or underestimate the need for care. Thus, strategies for detecting and mitigating bias are pivotal for creating AI technology that is generalizable and fair. Here, we discuss a recent study that developed a new strategy to mitigate bias in surgical AI systems.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This commentary discusses bias in AI-based medical applications, focusing on surgical AI systems (SAIS) that assess surgeon skill from robotic surgery videos. It highlights the TWIX add-on strategy developed by Kiyasseh et al., which mitigated SAIS bias, and surveys future challenges and regulatory efforts around bias detection and mitigation.

Key findings

  • Kiyasseh et al. deployed SAIS on videos of robotic surgeries from three hospitals to assess surgeon skill (e.g., needle handling, needle driving); the model reliably assessed performance but exhibited bias, showing underskilling (erroneously downgrading performance) and overskilling (erroneously upgrading performance) at different rates across surgeon sub-cohorts, measured via negative and positive predictive values respectively.
  • The TWIX add-on taught SAIS to predict the importance of video clips used for skill assessment, hypothesizing that bias arose from the system latching onto unreliable frames; TWIX mitigated SAIS bias, improving performance both for disadvantaged surgeon sub-cohorts and for surgical skill assessments overall.
  • Bias is generally defined as a difference in performance between subgroups for a predictive task, and can arise across data collection/preparation, model development, evaluation, and clinical deployment.
  • The paper notes open questions: what level of bias is acceptable, whether models should be more generalizable (larger, more diverse datasets) or more localized, and the risk that continuously learning/adaptive models may increase or add new bias if new data are biased.

Methods

  • This is a commentary/perspective article, not a primary study; it discusses a recent study (Kiyasseh et al.) that developed TWIX to mitigate bias in surgical AI systems.
  • It reviews bias mitigation categories (pre-processing via sampling, in-processing via mathematical approaches, post-processing, and "keeping the human in the loop") and regulatory developments, including the STANDING Together initiative (launched September 2022) and the FDA's AI/ML-Based SaMD Action Plan (January 2021) and draft guidance on predetermined change control plans.
  • No primary data, sample, or statistical analysis are reported by this paper.

Limitations

  • The paper does not state formal limitations for itself; it is a commentary without primary data or quantitative results of its own.
  • Evident from the text: all empirical claims about SAIS and TWIX are attributed to the Kiyasseh et al. study rather than independently tested here, and the discussion of acceptable bias thresholds, generalizability versus localization, and adaptive-model risks is framed as unresolved questions rather than settled findings.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41746-023-00858-z

Cites

17 of 17 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

19 Not checked yet · 3 Own interpretation · 1 Supported

Verify all 28 references — free account

Show all 11 claims

  • Probably not a substantial claim. 6
  • It may state two separate claims as one. 6
12 of 12
  • The paper relies on this without support.

    AI systems are increasingly being applied to healthcare.

    “Artificial intelligence systems are increasingly being applied to healthcare.”
    “Artificial intelligence (AI) technology is increasingly applied to healthcare, from AI-augmented clinical research to algorithms for image analysis or disease prediction.”
  • The paper relies on this without support.

    The integration of AI into medical technology and healthcare systems is only going to increase in the coming years.

    “The integration of AI into medical technology and healthcare systems is only going to increase in the coming years.”
  • The paper relies on this without support.

    Performance discrepancy may differ based on the data and population an AI algorithm is trained on and then subsequently applied to.

    “Performance discrepancy may also differ based on the data and population an AI algorithm is trained on and then subsequently applied to.”
  • The paper relies on this without support.

    The issue of bias being exhibited, perpetuated, or even amplified by AI algorithms is an increasing concern within healthcare.

    “The issue of bias being exhibited, perpetuated, or even amplified by AI algorithms is an increasing concern within healthcare.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 17 references

Cited by

Sorted by citations

Items 101–200 of 376

Metadata

Authors
Mittermaier, M., Raza, M. M., & Kvedar, J. C.
Affiliations
  • Humboldt-Universität zu Berlin (DE)
  • Harvard University (US)
Corresponding author
Mittermaier, Mirja
Venue
npj Digital Medicine
Published
2023-06-14
Volume
6
Issue
1
Pages
113-113
ISSN
2398-6352 (ISSN-L)
OpenAlex
W4380538374
Cited by
372
References
17
Citations by year
2026: 99 · 2025: 172 · 2024: 87 · 2023: 14
Type
journal article
Language
en
Open access
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Deutsche Forschungsgemeinschaft
  • Berlin Institute of Health
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
algorithmic bias, bias mitigation, health care equity, algorithmic fairness, surgical outcome prediction, generalizability, disadvantaged groups

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review