Cited by 26Artificial Intelligence in Healthcare and EducationEthics and Social Impacts of AIExplainable Artificial Intelligence (XAI)

Building an ethical and trustworthy biomedical AI ecosystem for the translational and clinical integration of foundation models

Open access
Sankar, Baradwaj Simha, Gilliland, Destiny, Rincon, Jack, Hermjakob, Henning, Yan, Yu, Adam, Irsyad, Lemaster, Gwyneth, Wang, Dean +4 more·2024·Bioengineering

Abstract

Foundation Models (FMs) are gaining increasing attention in the biomedical artificial intelligence (AI) ecosystem due to their ability to represent and contextualize multimodal biomedical data. These capabilities make FMs a valuable tool for a variety of tasks, including biomedical reasoning, hypothesis generation, and interpreting complex imaging data. In this review paper, we address the unique challenges associated with establishing an ethical and trustworthy biomedical AI ecosystem, with a particular focus on the development of FMs and their downstream applications. We explore strategies that can be implemented throughout the biomedical AI pipeline to effectively tackle these challenges, ensuring that these FMs are translated responsibly into clinical and translational settings. Additionally, we emphasize the importance of key stewardship and co-design principles that not only ensure robust regulation but also guarantee that the interests of all stakeholders-especially those involved in or affected by these clinical and translational applications-are adequately represented. We aim to empower the biomedical AI community to harness these models responsibly and effectively. As we navigate this exciting frontier, our collective commitment to ethical stewardship, co-design, and responsible translation will be instrumental in ensuring that the evolution of FMs truly enhances patient care and medical decision-making, ultimately leading to a more equitable and trustworthy biomedical AI ecosystem.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/bioengineering11100984

Cites

117 of 117 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Although the number of studies comparing different AI methods in clinical research is growing, comprehensive evaluations that employ a patient-centered approach remain limited and are needed to fully assess their relative utility.

    “Although the number of studies comparing different AI methods in clinical research is growing, comprehensive evaluations that employ a patient-centered approach remain limited and are needed to fully assess their relative utility.11,12”

    From Advancing methodological development of artificial intelligence in patient-centered comparative clinical effectiveness research: Patient-Centered Outcomes Research Institute’s unique contribution to research done differently (2025) · cites this paper as [11]

    Section 5.2 (page unavailable) calls for standardized metrics and evaluation processes, and the concluding section emphasizes engaging stakeholders in clinical AI development. The paper does not establish that studies comparing AI methods are growing or that comprehensive patient-centered comparisons remain limited; it also does not assess the methods’ relative utility. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 16

  1. [18]Li X. Cui Z. Wu Y. Gu L. Harada T. Estimating and Improving Fairness with Adversarial Learning arXiv 2021 2103.04243
  2. [22]Webster K. Wang X. Tenney I. Beutel A. Pitler E. Pavlick E. Chen J. Chi E. Petrov S. Measuring and Reducing Gendered Correlations in Pre-Trained Models arXiv 2021 2010.06032
  3. [33]Cheng P. Hao W. Yuan S. Si S. Carin L. Fairfil: Contrastive neural debiasing method for pretrained text encoders arXiv 2021 2103.06413
  4. [36]Shen T. Jin R. Huang Y. Liu C. Dong W. Guo Z. Wu X. Liu Y. Xiong D. Large Language Model Alignment: A Survey arXiv 2023 2309.15025
  5. [37]Kaufmann T. Weng P. Bengs V. Hüllermeier E. A Survey of Reinforcement Learning from Human Feedback arXiv 2024 2312.14925
  6. [39]Rafailov R. Sharma A. Mitchell E. Ermon S. Manning C.D. Finn C. Direct Preference Optimization: Your Language Model Is Secretly a Reward Model arXiv 2024 2305.18290
  7. [52]Hart S. Shapley Value Game Theory Eatwell J. Milgate M. Newman P. Palgrave Macmillan London, UK 1989 210 216 978-1-349-20181-5
  8. [54]Saravanan A.P. Kocielnik R. Jiang R. Han P. Anandkumar A. Exploring Social Bias in Downstream Applications of Text-to-Image Foundation Models arXiv 2023 2312.10065
  9. [57]Sun H. Supervised Fine-Tuning as Inverse Reinforcement Learning arXiv 2024 2403.12017
  10. [61]Bai Y. Kadavath S. Kundu S. Askell A. Kernion J. Jones A. Chen A. Goldie A. Mirhoseini A. McKinnon C. Constitutional AI: Harmlessness from AI Feedback arXiv 2022 2212.08073
  11. [68]Sevillano-García I. Luengo J. Herrera F. SHIELD: A Regularization Technique for eXplainable Artificial Intelligence arXiv 2024 2404.02611
  12. [98]Dar S.U.H. Seyfarth M. Kahmann J. Ayx I. Papavassiliu T. Schoenberg S.O. Frey N. Baeßler B. Foersch S. Truhn D. Unconditional Latent Diffusion Models Memorize Patient Imaging Data: Implications for Openly Sharing Synthetic Data arXiv 2024 2402.01054
  13. [99]El-Mhamdi E.-M. Farhadkhani S. Guerraoui R. Gupta N. Hoang L.-N. Pinot R. Rouault S. Stephan J. On the Impossible Safety of Large AI Models arXiv 2023 2209.15259
  14. [101]Carlini N. Ippolito D. Jagielski M. Lee K. Tramer F. Zhang C. Quantifying Memorization Across Neural Language Models arXiv 2023 2202.07646
  15. [109]Sucholutsky I. Griffiths T.L. Alignment with Human Representations Supports Robust Few-Shot Learning arXiv 2023 2301.11990
  16. [122]Broder A.Z. On the Resemblance and Containment of Documents Proceedings of the Compression and Complexity of SEQUENCES 1997 (Cat. No.97TB100171) Salerno, Italy 13 June 1997 21 29

Cited by

Sorted by citations

Authors
Sankar, B. S., Gilliland, D., Rincon, J., Hermjakob, H., Yan, Y., Adam, I., Lemaster, G., Wang, D., Watson, K., Bui, A., Wang, W., & Ping, P.
Affiliations
  • University of California, Los Angeles (US)
  • European Bioinformatics Institute (GB)
Corresponding author
  • Wang, Wei
  • Ping, Peipei
Venue
Bioengineering
Publisher
MDPI AG
Published
2024-09-29
Volume
11
Issue
10
Pages
984
ISSN
2306-5354 (ISSN-L)
OpenAlex
W4402987506
Cited by
26
References
111
Citations by year
2026: 9 · 2025: 15 · 2024: 2
FWCI
0.59
Type
journal article
Language
en
Open access
Locations
  • Bioengineering (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • eScholarship (California Digital Library) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
NIH awards
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
foundation models, AI ethics, trustworthy AI, clinical translation, translational research, AI stewardship, co-design, stakeholder engagement, AI regulation, clinical decision-making

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review