Cited by 65Explainable Artificial Intelligence (XAI)Artificial Intelligence in Healthcare and EducationMachine Learning in Healthcare

Translating AI to Clinical Practice: Overcoming Data Shift with Explainability

Open access
Choi, Youngwon, Yu, Wenxi, Nagarajan, Mahesh B., Teng, Pangyu, Goldin, Jonathan G., Raman, Steven S., Enzmann, Dieter R., Kim, Grace Hyun J. +1 more·2023·RadioGraphics

Abstract

To translate artificial intelligence (AI) algorithms into clinical practice requires generalizability of models to real-world data. One of the main obstacles to generalizability is data shift, a data distribution mismatch between model training and real environments. Explainable AI techniques offer tools to detect and mitigate the data shift problem and develop reliable AI for clinical practice. Most medical AI is trained with datasets gathered from limited environments, such as restricted disease populations and center-dependent acquisition conditions. The data shift that commonly exists in the limited training set often causes a significant performance decrease in the deployment environment. To develop a medical application, it is important to detect potential data shift and its impact on clinical translation. During AI training stages, from premodel analysis to in-model and post hoc explanations, explainability can play a key role in detecting model susceptibility to data shift, which is otherwise hidden because the test data have the same biased distribution as the training data. Performance-based model assessments cannot effectively distinguish the model overfitting to training data bias without enriched test sets from external environments. In the absence of such external data, explainability techniques can aid in translating AI to clinical practice as a tool to detect and mitigate potential failures due to data shift. ©RSNA, 2023 Quiz questions for this article are available in the supplemental material.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1148/rg.220105

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Data bias can cause AI systems to make inaccurate or unfair predictions or decisions.

    “Data bias can cause AI systems to make inaccurate or unfair predictions or decisions.”

    From Ethical Considerations of Using ChatGPT in Health Care (2023) · cites this paper as [69]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 6

  1. Bevan PJ, Atapour-Abarghouei A. Skin Deep Unlearning: Artefact and Instrument Debiasing in the Context of Melanoma Classification. Proceedings of the 39th International Conference on Machine Learning. Proc Mach Learn Res 2022;162:1874-1892.
  2. Hastie T, Tibshirani R, Friedman JH. The elements of statistical learn- ing: data mining, inference, and prediction. New York, NY: Springer, 2009; 219-260, 389-416.
  3. Goodfellow I, Bengio Y, Courville A. Deep learning. Cambridge, Mass: MIT Press, 2016; 98-168.
  4. James G, Witten D, Hastie T, Tibshirani R. An introduction to statistical learning. New York, NY: Springer, 2013; 15-58, 175-202.
  5. Ye H, Xie C, Cai T, Li R, Li Z, Wang L. Towards a theoretical framework of out-of-distribution generalization. Adv Neural Inf Process Syst 2021; 34:23519-23531.
  6. Dou Q, Coelho de Castro D, Kamnitsas K, Glocker B. Domain gen- eralization via model-agnostic learning of semantic features. Adv Neural Inf Process Syst 2019; 32. https://papers.nips.cc/paper/2019/ hash/2974788b53f73e7950e8aa49f3a306db-Abstract.html.

Cited by

Sorted by citations

Authors
Choi, Y., Yu, W., Nagarajan, M. B., Teng, P., Goldin, J. G., Raman, S. S., Enzmann, D. R., Kim, G. H. J., & Brown, M. S.
Affiliations
  • University of California, Los Angeles (US)
  • From the Center for Computer Vision and Imaging Biomarkers, 924 Westwood Blvd, Los Angeles, CA 90024 (Y.C., W.Y., M.B.N., P.T., J.G.G., G.H.J.K., M.S.B.); and Department of Radiology, University of California–Los Angeles, Los Angeles, Calif (Y.C., W.Y., M.B.N., P.T., J.G.G., S.S.R., D.R.E., G.H.J.K., M.S.B.).
Venue
RadioGraphics
Published
2023-05-01
Volume
43
Issue
5
Pages
e220105-e220105
ISSN
  • 0271-5333 (ISSN-L)
  • 1527-1323
OpenAlex
W4367174392
Cited by
65
References
34
Citations by year
2026: 23 · 2025: 28 · 2024: 11 · 2023: 3
FWCI
6.13
Type
journal article
Language
en
Open access
Locations
  • Radiographics (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • eScholarship (California Digital Library) (submittedVersion)
Funders
David Geffen School of Medicine, University of California, Los Angeles
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
distribution shift, explainable AI, clinical translation of AI, model generalization, artificial intelligence in healthcare, training data bias, external validation, post-hoc explanation, overfitting, artificial intelligence in radiology
MeSH terms
Algorithms, Artificial Intelligence, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review