Cited by 124Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AIArtificial Intelligence in Healthcare

Ensuring that biomedical AI benefits diverse populations

Open access
Zou, James, Schiebinger, Londa·2021·EBioMedicine

Abstract

Artificial Intelligence (AI) can potentially impact many aspects of human health, from basic research discovery to individual health assessment. It is critical that these advances in technology broadly benefit diverse populations from around the world. This can be challenging because AI algorithms are often developed on non-representative samples and evaluated based on narrow metrics. Here we outline key challenges to biomedical AI in outcome design, data collection and technology evaluation, and use examples from precision health to illustrate how bias and health disparity may arise in each stage. We then suggest both short term approaches-more diverse data collection and AI monitoring-and longer term structural changes in funding, publications, and education to address these challenges.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.ebiom.2021.103358

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Diversity of biomedical datasets is paramount as it constitutes the first step to ensure generalizability to the broader population.

    “However, diversity of biomedical datasets is paramount as it constitutes the first step to ensure generalizability to the broader population 145”

    From Multimodal biomedical AI (2022) · cites this paper as [145]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 8

  1. [9]Smith Taylor J Women's health research: progress, pitfalls, and promise: institute of medicine of the national academies 2011 National Academies Press Washington, DC
  2. [11]National Center for Health Statistics Health, United States, 2018 2019 Hyattsville, MD 31944640
  3. [26]Moran-Thomas Amy How a popular medical device encodes racial bias Boston Review 2020 157 172 http://bostonreview.net/science-nature-race/amy-moran-thomas-how-popular-medical-device-encodes-racial-bias Accessed 5 August 2020
  4. [44]American Medical Assocation, Aubmented Intelligence in Health Care H-480.940. https://policysearch.ama-assn.org/policyfinder/detail/augmented%20intelligence?uri=%2FAMADoc%2FHOD.xml-H-480.940.xml. Accessed 16 March 2021.
  5. [58]Both authors participated in this process. A review is in progress.
  6. [60]Gibney E. The battle to embed ethics in AI research Nature 577 7792 2020 609 31992885
  7. [62]NeurIPS, Thirty-fourth conference on neural information processing systems, Call for papers, 2020 https://nips.cc/Conferences/2020/CallForPapers. Accessed 23 October 2020.
  8. [64]The Lancet. Advancing racial equality. 2020. https://www.thelancet.com/racial-equality. Accessed 16 Mar 2021.

Cited by

Sorted by citations

Authors
Zou, J., & Schiebinger, L.
Affiliations
Stanford University (US)
Corresponding author
Schiebinger, Londa
Venue
EBioMedicine
Publisher
Elsevier BV
Published
2021-05
Volume
67
Pages
103358
ISSN
2352-3964 (ISSN-L)
OpenAlex
W3158393586
Cited by
124
References
60
Citations by year
2026: 16 · 2025: 43 · 2024: 23 · 2023: 17 · 2022: 22 · 2021: 3
FWCI
1.32
Type
journal article
Language
en
Open access
Locations
  • EBioMedicine (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • National Science Foundation (1942926)
  • National Institutes of Health (P30AG059307, U01MH098953)
  • National Science Foundation Directorate for Engineering
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
algorithmic bias, health disparities, underrepresented groups, precision health, data collection, AI evaluation metrics, AI monitoring, research funding
MeSH terms
Artificial Intelligence, Health Policy, Humans, Medical Informatics, Health Status Disparities

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review