CoChat

Survey of Explainable AI Techniques in Healthcare

Open access

The DOI matches its Crossref record; 119 of the 120 references it cites were found in a registry or an index, and 1 are not checked yet; 8 of the 50 sources its claims cite were checked for support.

Abstract

Artificial intelligence (AI) with deep learning models has been widely applied in numerous domains, including medical imaging and healthcare tasks. In the medical field, any judgment or decision is fraught with risk. A doctor will carefully judge whether a patient is sick before forming a reasonable explanation based on the patient’s symptoms and/or an examination. Therefore, to be a viable and accepted tool, AI needs to mimic human judgment and interpretation skills. Specifically, explainable AI (XAI) aims to explain the information behind the black-box model of deep learning that reveals how the decisions are made. This paper provides a survey of the most recent XAI techniques used in healthcare and related medical imaging applications. We summarize and categorize the XAI types, and highlight the algorithms used to increase interpretability in medical imaging topics. In addition, we focus on the challenging XAI problems in medical applications and provide guidelines to develop better interpretations of deep learning models using XAI concepts in medical image and text analysis. Furthermore, this survey provides future directions to guide developers and researchers for future prospective investigations on clinical topics, particularly on applications with medical imaging.

Verified 2026-10-05 · 1 reference not checked yet

Overview

CoChat Summary

TL;DR

This survey reviews recent explainable AI (XAI) techniques for healthcare and medical imaging, categorizing them by interpretation type, model specificity, explanation scope, and explanation form. It highlights commonly used methods such as Grad-CAM, LIME, SHAP, attention, prototypes, and radiomics, and discusses human-centered, deployment, quality, and future challenges.

Key findings

  • XAI methods are categorized as intrinsic vs. post hoc, model-specific vs. model-agnostic, local vs. global, and feature-based/textual/example-based; post hoc methods such as CAM/Grad-CAM and LIME are most commonly used in medical imaging.
  • The paper reports that clinician satisfaction with LIME chart interpretation reached 78%, though many low-satisfaction clinicians gave low scores despite recognizing the explanations.
  • In an evaluation of saliency maps, eight interpretable methods (+ Grad-CAM, guided backpropagation, and guided Grad-CAM) were assessed, but performance on the testing datasets was not competitive.
  • For Trojan attack detection, backpropagation had a high success rate, CAM detected the entire trigger region but not always accurate localization, and LIME had high success for small triggers; LIME and SHAP showed unstable explanations with limited examples.

Methods

  • Design: narrative survey of recent XAI techniques used in healthcare and related medical imaging, summarizing and categorizing XAI types and algorithms.
  • Data/sample: literature and tables (Table 1, Table 2, Table 3) covering XAI methods, modalities, and explanations; no primary dataset or systematic review protocol is reported.
  • Analysis: qualitative synthesis of principles, deployment, performance, and challenges, including discussion of radiomics and future directions.

Limitations

  • The paper states that the definition of XAI is not clear enough and that explainability depends on the prediction task and target users rather than uniform standards.
  • The paper states that post hoc application in the medical field requires more effort; TCAV needs more investigation; trainable attention needs more work; SHAP applications are limited to specific problems; LIME’s low repeatability limits interpretability; and evaluating XAI explanations remains a challenge.
  • Evident from the text: the survey does not report a systematic search strategy, inclusion/exclusion criteria, or quantitative meta-analysis; it is a narrative overview.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/s23020634

Cites

119 of 120 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Flowchart of visual comparison between black-box and explainable artificial intelligence, and how the results affect the user. The top branch shows the process of a black-box model. Typically, it provides only results such as classes (e.g., COVID or non-COVID). The other two branches (middle and bottom) represent two XAI methods, referred to in Section 3.1. Specifically, the XAI model (middle) shows the example of saliency map, and the second one (bottom) is the prototype method, as explained in Section 4.1 and Section 4.6, respectively. An example of a CT image obtained from the COVID-19 CT scan dataset [10].
Figure 1 Flowchart of visual comparison between black-box and explainable artificial intelligence, and how the results affect the user. The top branch shows the process of a black-box model. Typically, it provides only results such as classes (e.g., COVID or non-COVID). The other two branches (middle and bottom) represent two XAI methods, referred to in Section 3.1. Specifically, the XAI model (middle) shows the example of saliency map, and the second one (bottom) is the prototype method, as explained in Section 4.1 and Section 4.6, respectively. An example of a CT image obtained from the COVID-19 CT scan dataset [10].CC BY · PubMed Central
Figure 2 Number of XAI publications added per year from 1976 to 2021. The x axis represents the publishing year, and the y axis shows the number of publications added in a certain year. The number of publications indexed on PubMed (accessed on 1 July 2022: https://pubmed.ncbi.nlm.nih.gov) that matched the search queries related to explainable AI topics in this survey with a term searched of (explainable AI OR explainable artificial intelligence) AND (medicine OR healthcare).
Figure 2 Number of XAI publications added per year from 1976 to 2021. The x axis represents the publishing year, and the y axis shows the number of publications added in a certain year. The number of publications indexed on PubMed (accessed on 1 July 2022: https://pubmed.ncbi.nlm.nih.gov) that matched the search queries related to explainable AI topics in this survey with a term searched of (explainable AI OR explainable artificial intelligence) AND (medicine OR healthcare).CC BY · PubMed Central
Figure 3 Categorization of explainable AI methods used in this paper. These criteria and categories are summarized from [25,26,27]. The orange and blue grids represent the criteria and categories, respectively.
Figure 3 Categorization of explainable AI methods used in this paper. These criteria and categories are summarized from [25,26,27]. The orange and blue grids represent the criteria and categories, respectively.CC BY · PubMed Central

Verification

Claims the paper makes

5 Valid · 2 Not valid · 36 Not checked · 19 Own results and interpretations

18 more claims

Create a free account to see all 62 claims and to verify the 42 references still open.

See all claims — free account
  • Probably not a substantial claim. 30
  • It may state two separate claims as one. 13

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Research works printed but not linked: 6

  1. Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv.
  2. Yang, X., He, X., Zhao, J., Zhang, Y., Zhang, S., and Xie, P. (2020). COVID-CT-dataset: A CT scan dataset about COVID-19. arXiv.
  3. Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv.
  4. Jiang, C., Chen, Y., Chang, J., Feng, M., Wang, R., and Yao, J. (2021). Fusion of medical imaging and electronic health records with attention and multi-head machanisms. arXiv.
  5. Shapley, L.S. (2016). 17. A value for n-person games. Contributions to the Theory of Games (AM-28), Volume II, Princeton University Press.
  6. Nguyen, H.T.T., Cao, H.Q., Nguyen, K.V.T., and Pham, N.D.K. (2021, January 6–7). Evaluation of Explainable Artificial Intelligence: SHAP, LIME, and CAM. Proceedings of the FPT AI Conference 2021, Ha Noi, Viet Nam.

Cited by

See all 631

Metadata

Authors
Chaddad, A., Peng, J., Xu, J., & Bouridane, A.
Affiliations
  • Guilin University of Electronic Technology (CN)
  • University of Sharjah (AE)
Corresponding author
Chaddad, Ahmad
Venue
Sensors
Publisher
MDPI AG
Published
2023-01-05
Volume
23
Issue
2
Pages
634
ISSN
1424-8220 (ISSN-L)
OpenAlex
W4313648826
Cited by
591
References
111
Citations by year
2026: 144 · 2025: 211 · 2024: 159 · 2023: 76
FWCI
12.33
Type
journal article
Language
en
Open access
Locations
  • Sensors (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • NATIONAL NATURAL SCIENCE FOUNDATION (82260360, QN2021033002L)
  • FOREIGN YOUNG TALENTS PROGRAM (82260360, QN2021033002L)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
Keywords
explainable AI, health care, diagnostic imaging, deep learning interpretability, black-box models, medical image analysis, clinical decision support
MeSH terms
Algorithms, Artificial Intelligence, Humans, Judgment, Physicians, Research Personnel

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.