CoChat
Sign Up
Cited by 233Explainable Artificial Intelligence (XAI)Artificial Intelligence in Healthcare and EducationCOVID-19 diagnosis using AI

Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches

Open access
Borys, Katarzyna, Schmitt, Yasmin Alyssa, Nauta, Meike, Seifert, Christin, Krämer, Nicole, Friedrich, Christoph M., Nensa, Felix·2023·European Journal of Radiology
DOI 10.1016/j.ejrad.2023.110786
Open this paper in CoChat — free accountOpen PDF

Abstract

Driven by recent advances in Artificial Intelligence (AI) and Computer Vision (CV), the implementation of AI systems in the medical domain increased correspondingly. This is especially true for the domain of medical imaging, in which the incorporation of AI aids several imaging-based tasks such as classification, segmentation, and registration. Moreover, AI reshapes medical research and contributes to the development of personalized clinical care. Consequently, alongside its extended implementation arises the need for an extensive understanding of AI systems and their inner workings, potentials, and limitations which the field of eXplainable AI (XAI) aims at. Because medical imaging is mainly associated with visual tasks, most explainability approaches incorporate saliency-based XAI methods. In contrast to that, in this article we would like to investigate the full potential of XAI methods in the field of medical imaging by specifically focusing on XAI techniques not relying on saliency, and providing diversified examples. We dedicate our investigation to a broad audience, but particularly healthcare professionals. Moreover, this work aims at establishing a common ground for cross-disciplinary understanding and exchange across disciplines between Deep Learning (DL) builders and healthcare professionals, which is why we aimed for a non-technical overview. Presented XAI methods are divided by a method's output representation into the following categories: Case-based explanations, textual explanations, and auxiliary explanations.

Verified 2026-10-09 · 1 reference not checked yet

AI snapshot

Written from the paper’s full text.

TL;DR

This review provides a non-technical overview of non-saliency XAI methods for medical imaging, categorizing them into case-based, textual, and auxiliary explanations. It argues these methods diversify XAI beyond visual saliency maps and can help uncover model flaws, data outliers, distributional discrepancies, and biases, though clinical integration and empirical assessment remain under-researched.

Key findings

  • Non-visual XAI methods are grouped by output representation into case-based explanations (e.g., TCAV, ACE, Network Dissection, influence functions, GANterfactuals), textual explanations (e.g., VQA, Visual Dialog, image captioning), and auxiliary explanations (e.g., t-SNE, UMAP, ProtoTree, TreeView, uncertainty quantification with BNNs and Deep Ensembles).
  • The paper cites limitations of saliency-based methods: Adebayo et al. found saliency methods can be insensitive; Ghorbani et al. demonstrated vulnerability to image perturbations and adversarial attacks; Tomsett et al. noted inconsistent evaluation metrics in sanity-check studies.
  • Textual explanation datasets include VQA with ~0.25 million images, ~0.76 million questions, and ~10 million answers, and VisDial with ten question-answer pairs on ~140k COCO images, totaling ~1.4M question-answer pairs.
  • Uncertainty quantification is highlighted: BNNs were applied to BraTS 2017 with 285 subjects with high- or low-grade gliomas; Deep Ensembles can distinguish aleatoric from epistemic uncertainty, but low diversity between ensemble members can negatively affect uncertainty estimates and out-of-distribution detection.

Methods

  • Design: narrative, non-technical review aimed at healthcare professionals and DL practitioners, focusing on XAI techniques not relying on saliency in medical imaging.
  • Data/sample: literature and method examples rather than a primary patient cohort; cited datasets include MedNIST, ImageCLEF 2019 VQA-Med, BraTS 2017 (285 subjects), COCO/VisDial, ROCO, and IU X-Ray.
  • Main analysis: methods are categorized by expected output into case-based, textual, and auxiliary explanations; Table 1 summarizes strengths, weaknesses, explanation target (data, internals, predictions), and explanation perspective (global, local).

Limitations

  • Stated: there are no recommendations or restrictions for selecting appropriate XAI methods; assessment guidelines are relatively sparse; clinical integration and empirical assessment of XAI impact are little researched.
  • Stated: non-visual XAI techniques may seem more specific and in some cases rely on method-specific knowledge for correct interpretation.
  • Evident: this is a narrative review without a stated systematic search strategy, inclusion criteria, or quantitative synthesis; many examples rely on non-medical or limited medical datasets, so clinical generalizability is not established.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.ejrad.2023.110786

Cites

68 of 69 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 90 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    Influence Functions approximate influence by upweighting a specific sample by an infinitesimally small amount and measuring the change in loss and parameters.

    “A prominent example is Influence Functions (IF) [30] proposed by Koh and Liang which approximate this during upweighting a specific sample by an infinitesimally small amount and measuring the change in loss and parameters.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30]

      Not checked in the free run.

  • Premisereasoning

    Textual explanations depend on sophisticated datasets that include images alongside descriptive semantic information.

    “However, this leads to a challenging requirement of textual explanations, namely the dependence on sophisticated datasets, including images alongside descriptive semantic information.”
    Quote in the paperNot a substantial claim
  • Interpretationcitation

    Selecting an appropriate XAI method for a given task is challenging because of subjective end-user perceptions, individual needs, and sparse assessment guidelines.

    “Because of that, selecting an appropriate XAI method for the task at hand remains a challenge, especially considering the subjective perceptions of end-users alongside their individual needs [18,19] and relatively sparse assessment guidelines.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [18] source

      Not checked in the free run.

    • cites [19] source

      Not checked in the free run.

  • Interpretationcitation

    Non-visual XAI methods can be divided into three subgroups based on the type of result a user expects: auxiliary explanations, case-based explanations, and textual explanations.

    “Consequently, apart from visual explanations, three subgroups of non-visual XAI methods were identified according to the type of result a user can expect when applying an XAI technique [17]: auxiliary explanations, case-based explanations, and textual explanations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [17]

      Not checked in the free run.

  • Premisecitation

    Sanity-check studies of visual explanations lack consistency in evaluation metrics, making full evaluation of visual explanations challenging.

    “Tomsett et al. [16] pointed out a lack of consistency concerning evaluation metrics in sanity-check studies concluding that despite the increasing effort, it remains challenging to evaluate visual explanations fully.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16] source

      Not checked in the free run.

  • The quote only partly states this claim. 1
  • May combine two claims. 33
  • Classified as probably not a substantial claim. 63

  • The quote only partly states this claim.

    Premisecitation

    In a user study, counterfactual explanations led to significantly better results regarding mental models, explanation satisfaction, and trust compared with Layerwise Relevance Propagation and Local Interpretable Model-agnostic Explanations.

    “The authors stated that counterfactual explanations led to significantly better results regarding mental models, explanation satisfaction, and trust.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [32] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Influence Functions allow understanding model behavior and detecting outliers in training data.

    “Conclusively, IF allow for understanding model behavior and detecting outliers in training data.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    ACE mitigates the need for human supervision because manual labeling is not required.

    “A significant advantage of ACE, compared to TCAV, is that it mitigates the need for human supervision, as manual labeling is not required.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [24]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Humans tend to devalue explanations that contradict their prior beliefs.

    “because humans tend to devalue explanations that contradict their prior beliefs.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Network Dissection is a framework for quantifying the interpretability of individual units within a neural network.

    “A somewhat different approach called Network Dissection [26] is a general framework designed to quantify the interpretability of individual units (e.g., neurons or channels) within a neural network.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Visual XAI methods are generally regarded as intuitive and easy to interpret, but studies have identified significant limitations.

    “Even though visual XAI methods are considered easy to interpret and intuitive, some studies pointed out significant limitations.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [14]

      Not checked in the free run.

    • cites [15] source

      Not checked in the free run.

    • cites [16] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    TCAV determines a model's sensitivity to a high-level concept for a given class by training a linear classifier to separate images containing the concept from those that do not.

    “TCAV determines a model's sensitivity to an underlying high-level concept for a given class by training a linear classifier to separate the images containing the defined concept from those that do not.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Saliency maps are vulnerable to image perturbations and fragile to adversarial attacks.

    “Ghorbani et al. [15] demonstrated saliency maps' vulnerability to image perturbations and fragility to adversarial attacks”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [15] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Medical imaging is mainly associated with visual tasks, and therefore most explainability approaches in this field use visual or saliency-based XAI methods.

    “Since medical imaging is mainly associated with visual tasks, most explainability approaches incorporate visual XAI methods, including attribution and heat maps [9].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [9] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    According to Miller, a good explanation is contrastive, selective, and truthful.

    “Miller argues that a good explanation is contrastive, selective, and truthful.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] source

      Not checked in the free run.

  • May combine two claims.

    Premisenone given

    Channels in neural networks are often not fully disentangled and cannot be interpreted in isolation, so the interpretability of units activating to certain concepts should not be over-interpreted.

    “However, channels are often not fully disentangled and can not be interpreted in isolation, so the interpretation ability of units activating to certain concepts should not be over-interpreted.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Counterfactual explanations aim to enable contrastive reasoning by altering the input image so that a given classifier would make a different prediction than its actual label.

    “For example, so-called counterfactual explanations aim to enable contrastive reasoning by altering the input image so that a given classifier would make a different prediction than its actual label.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Some research asserts that case-based explanations are the most intuitive for human users to comprehend.

    “However, some research asserts that this approach is the most intuitive for human users to comprehend [7].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Counterfactual explanation methods such as GANterfactual lack flexibility and are strongly model-specific because a slight change in task would require a new dataset and retraining of the CycleGAN and binary classifier.

    “However, a limitation of this method is its lack of flexibility and strong model-specificity since a slight change in the task would require the acquisition of a new dataset alongside another training of the CycleGAN and the binary classifier.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [32] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    ACE relies on manual parameter setting, which can affect outcomes, for example by producing duplicated or mixed concepts.

    “However, the method relies on the manual setting of parameters, which can affect the outcome, e.g., resulting in duplicated or mixed concepts [25].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [25] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Network Dissection authors confirmed that representations at different layers disentangle distinct categories of meaning and that different training techniques can significantly impact the representation learned by hidden units.

    “With this technique, the authors confirmed that representations at different layers disentangle distinct categories of meaning and that different training techniques can significantly impact the representation learned by hidden units.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Applying VQA or Image Captioning methods does not require an in-depth understanding or method-specific interpretation.

    “Moreover, applying these methods does not require an in-depth understanding or method-specific interpretation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Uncertainty modeling in deep learning systems makes those models safer for clinical use.

    “Moreover, as the authors of [75] pointed out, uncertainty modeling in DL systems makes such models safer for clinical use.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [75]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    This dependence on sophisticated datasets is especially challenging in medical imaging because several imaging modalities, anatomical areas, and disease entities must be considered.

    “This challenge is specifically present in medical imaging, where several imaging modalities, anatomical areas, and disease entities must be considered.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Diagnostic report generation is time-consuming and knowledge-intensive.

    “In general, diagnostic report generation is a time-consuming and knowledge-intensive task [45].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [45] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Human classification of images or objects strongly relies on subconscious comparison with prototypical parts or characteristics associated with certain objects.

    “Human classification of images or objects strongly relies on a subconscious comparison with prototypical parts or characteristics associated with certain objects.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Model uncertainty comprises two forms: aleatoric uncertainty, arising from noise in data or labels, and epistemic uncertainty, arising from data sparsity or out-of-distribution inputs.

    “Uncertainty may appear in two forms, aleatoric uncertainty and epistemic uncertainty [67], with the former being caused by data or label noise and the latter being caused by sparsity or out-of-data distribution.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [67] source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning

    Alternative uncertainty estimation methods exist because Bayesian approaches typically require complex training modifications and can be computationally expensive.

    “In contrast, there are also other methods for uncertainty estimation, justified by the fact that Bayesian approaches usually require complex modifications of the training process and can be computationally expensive.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Bayesian approaches typically require complex modifications to the training process and can be computationally expensive.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    t-SNE builds upon conditional probabilities to express distances between data points and find similarities.

    “It builds upon conditional probabilities to express the distances between data points and find similarities.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    When combined with visualizations, image captioning systems can serve an educational role by projecting described findings onto the original image and indicating which image region produced a specific caption part.

    “Coupled with visualizations, systems encompassing image captioning can even pose as an educative, as findings described within generated captions can be projected directly onto the initial image, justifying which image area is responsible for the generation of a specific caption part.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Although uncertainty quantification is often treated separately from explainability, its contribution argues against separation and supports incorporating uncertainty estimation as an intrinsic part of XAI research.

    “Even though uncertainty quantification is often treated as a distinct topic separated from explainability, its contribution to this area speaks against separation and, on the contrary, promotes the incorporation of uncertainty estimation as an intrinsic part of XAI research [17].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [17]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Interpretation of TreeView may be challenging without a thorough understanding of how the method works.

    “However, an interpretation may be challenging without a thorough understanding of how this method works.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Explanation methods that use uncertainty can identify model limitations and inform possible optimization approaches.

    “Explanation methods involving this uncertainty category enable the identification of model limitations and provide information about possible optimization approaches.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Generated diagnostic reports must focus on normal and abnormal findings related to medical characteristics rather than general descriptions, and high precision is mandatory.

    “Moreover, high precision is mandatory, and generated reports must focus on normal and abnormal findings related to medical characteristics rather than general descriptions [45,46].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [45] source

      Not checked in the free run.

    • cites [46]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Counterfactual explanations arise from deceiving the model by applying minimally required changes to the input, thereby revealing models' susceptibility to attacks.

    “as such explanations arise from deceiving the model by applying minimally required changes to the input, simultaneously revealing models' susceptibility to attacks [82].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [82] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    RadVisDial is the first publicly available dataset for visual dialog in radiology.

    “For example, in [41], Visual Dialog was deployed in radiology specific to chest X-ray images bringing forth RadVisDial, the first publicly available dataset for visual dialog in radiology.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [41] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    ProtoTree's hierarchical structure is intended to increase interpretability and lead to more insights regarding clusters in the data by exploiting positive and negative reasoning.

    “Its hierarchical structure is intended to increase interpretability and lead to more insights regarding clusters in the data by exploiting the positive and the negative reasoning process.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [62] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    UMAP is a successor of t-SNE that is superior in run time performance, scalability for larger datasets, and preservation of data structure.

    “Uniform Manifold Approximation and Projection (UMAP) [58] is a successor of t-SNE, superior in run time performance, scalability regarding larger datasets, and perseverance of data structure.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Because of these requirements, systems such as VQA or VisualDialog are clinically limited and need further optimization research that accounts for them.

    “For these reasons, systems such as VQA or VisualDialog are clinically limited and rely on further optimization research that takes the presented requirements into account.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    t-SNE provides a depiction or intuition of how data is arranged in a high-dimensional space.

    “In simpler terms, t-SNE provides a depiction or intuition of how the data is arranged in a highdimensional space.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Uncertainty quantification is attributed great potential.

    “Great potential is attributed to the quantification of uncertainty.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Interpreting textual explanations usually does not require a thorough understanding of the model's internal functioning, because generated texts are easily interpretable.

    “Also, their interpretation usually does not rely on a thorough understanding of the underlying internal functioning, as generated texts are easily interpretable.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    X-rays are only one of many data points available for a patient's diagnosis, and including additional parameters such as medications, lab data, and clinical data in AI agents is important to overcome diagnostic limitations.

    “However, they also pointed out that X-rays are only one of the many data points available (e.g., medications, lab data, clinical data) for a patient's diagnosis and emphasize the importance of including these additional parameters in AI agents to overcome diagnostic limitations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [41] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    These categories often differ greatly in their contribution to explainability and their underlying methodology.

    “As has been shown, these categories are often very different regarding their individual contribution to explainability and the underlying methodology.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    In terms of generated output, textual explanations are less diversified than case-based explanations.

    “In relation to the generated output of the methods, textual explanations are less diversified than case-based explanations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Distinguishing epistemic from aleatoric uncertainty can help deep learning engineers decide whether model or training adjustments are necessary.

    “The differentiation between epistemic and aleatoric uncertainty can be used by DL engineers, in particular, to determine whether, and if so, model or training adjustments are necessary.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    These networks predict each voxel's class probability and produce calibrated confidence intervals for downstream biomarkers.

    “The networks predict a voxel's probability of belonging to each segmentation class and generate calibrated confidence intervals of downstream biomarkers.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [75]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Automated report generation can be particularly helpful for inexperienced or in-training healthcare professionals seeking to confirm diagnostic findings with a support system.

    “which can be particularly helpful for inexperienced or in-training healthcare professionals who want to confirm their diagnostic findings with the help of a support system.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    These considerations are also important for distinguishing whether a model operates under an open-world or closed-world assumption.

    “Among other things, these considerations are also important for differentiating whether a model acts according to an open-world or closed-world assumption.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The form of the distribution determines the range of a system's behavior.

    “Ultimately, the distribution form characterizes the range of a system's behavior [69].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [69] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Bayesian Neural Networks are stochastic artificial neural networks trained with Bayesian inference.

    “Bayesian Neural Networks (BNNs) [68,71,72] are stochastic ANNs trained using Bayesian inference.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [68]

      Not checked in the free run.

    • cites [71] source

      Not checked in the free run.

    • cites [72] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Eaton-Rosene et al. proposed a generalizable Bayesian Neural Network technique for uncertainty quantification in semantic segmentation, evaluated on the BraTS 2017 dataset of 285 subjects with high- or low-grade gliomas.

    “Eaton-Rosene et al. [75] proposed a generalizable technique for quantifying uncertainty with Bayesian Neural Networks for semantic segmentation on the BraTS 2017 dataset [76], including 285 subjects with high-or low-grade gliomas.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [75]

      Not checked in the free run.

    • cites [76] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Visualization using TreeView enables a convenient transition between factors, class labels, and an input data space.

    “Overall, a visualization using TreeView enables a convenient transition between factors, class labels, and an input data space.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [65]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A major disadvantage of UMAP is its lack of maturity as a relatively new technique, with best practices not yet established.

    “A major disadvantage of UMAP is its lack of maturity, as it is a relatively new technique, and best practices are not yet established [60].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [60]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    t-SNE plots cannot always visualize the relative sizes of clusters appropriately, so cluster size and distance should not be over-interpreted.

    “Secondly, t-SNE plots cannot always visualize the relative sizes of clusters appropriately; hence, cluster size and distance should not be over-interpreted.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given

    A key advantage of Deep Ensembles is that they can distinguish aleatoric from epistemic uncertainty: the ensemble mean can be interpreted as aleatoric uncertainty, while the standard deviation of the mean estimates represents epistemic uncertainty, which is low when estimates are similar and grows when they differ.

    “An essential advantage of Deep Ensembles is the possibility to distinguish between aleatoric and epistemic uncertainty: The arrangements' mean can be interpreted as aleatoric uncertainty, while the agreements' standard deviation represents epistemic uncertainty, which is low if all mean estimates yield a similar value and grows if the mean estimates differ strongly.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Epistemic uncertainty is reducible through additional data or improved training, whereas aleatoric uncertainty is not reducible because of a task's inherent randomness and the fact that a model only approximates it.

    “While epistemic uncertainty is reducible by acquiring more data or optimizing the training process, aleatoric uncertainty is not because of a task's intrinsic randomness and the fact that a model is only an approximation of it [13,17].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [13] source

      Not checked in the free run.

    • cites [17]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Image interpretation and the generation of diagnostic reports or documentation can be automated.

    “A significant advantage is that the interpretation of images, and thus, the generation of diagnostic reports or documentation, can also be automated”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Bayesian Neural Networks can be designed with different choices for stochastic components, distribution approximation, and inference methods.

    “BNNs can be designed in different ways regarding the selection of stochastic components, distribution approximation, and inference approaches [74].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [74] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Systems such as VQA and Image Captioning are not XAI methods as such but illustrate how semantic and visual elements can interact to support the diagnostic process.

    “Overall, it is noteworthy that systems like VQA or Image Captioning are not XAI methods as such but rather serve as an example of how an interplay between semantic and visual elements could serve as a rectification within the diagnostic process.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Altering t-SNE hyperparameters such as perplexity can drastically change the plot.

    “Altering tunable hyperparameters, such as the perplexity (balancing the attention t-SNE gives to local and global aspects of the data), can drastically change the plot.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Image Captioning depends on complex datasets including images and textual annotations.

    “Dependence on complex datasets including images and textual annotations”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [43] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    ProtoTree has possibility of ambiguous result.

    “Possibility of ambiguous result”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [62] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Image Captioning's semantic relationship between images and textual descriptions is intuitive, even for non-experts.

    “Semantic relationship between images and textual descriptions is intuitive, even for nonexperts”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [43] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    In medical contexts, it would be incorrect to state that a patient does not have a specific disease merely because there is no record of it, unless more information confirms that assumption.

    “Especially with medical questions, it would be incorrect to state that the patient does not suffer from a specific disease if there is no record reporting this unless more information is given to confirm this assumption.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Image Captioning with Visual Explanations shows an interplay between an image's semantic content description and a visual explanation.

    “Interplay between an image's semantic content description and a visual explanation”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [47]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Visual Dialog is interactive and context-related.

    “Interactive and context-related dialog”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Clinical integration and empirical assessment of the impact of such XAI methods have so far been little researched.

    “An aspect that has so far been little researched is the clinical integration and empirical assessment of the impact of such XAI methods.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    BNNs require complex modifications to training process.

    “Requires complex modifications to training process”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [71] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    UMAP lacks maturity.

    “Lack of maturity”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Automated concept-based explanations require no manual labeling.

    “No manual labeling required”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [24]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Network Dissection detects concepts beyond classes in classification tasks.

    “Detection of concepts beyond classes in classification task”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Influence functions are only applicable to models with a second-order differentiable loss.

    “Only applicable to models with a 2ndorder differentiable loss”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30]

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Although non-visual XAI methods may appear less intuitive than saliency-based approaches such as GradCAM, they significantly contribute to diversifying XAI by enabling systematic uncovering of model flaws, data outliers, distributional discrepancies, and biases.

    “Conclusively, even though non-visual XAI methods may appear less intuitive compared to prominent saliency-based approaches such as GradCAM [11], they significantly contribute to the diversification of XAI, allowing for the systematic uncovering of model flaws, data outliers, distributional discrepancies, and biases.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [11] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Network Dissection considers only positive activations.

    “Only positive activations are considered”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Distinguishing open-world from closed-world assumptions is crucial because open-world systems can indicate low confidence for unknown answers, while closed-world systems would simply return the most likely class from a given subset when the opposite cannot be confirmed.

    “Differentiation of these assumptions can be crucial as open-world systems have the ability to denote low confidence when answering questions to which the answer is unknown, whereas closed-world systems would simply answer with the most likely answer (class) of a given subset as the opposite cannot be confirmed.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Cluster size and distance in t-SNE should not be over-interpreted.

    “Cluster size and distance should not be over-interpreted”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    t-SNE allows for explorative interpretation of activations within a neural network but also investigation.

    “Allows for explorative interpretation of activations within a neural network but also investigation”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    GANterfactuals depend on a binary classifier.

    “Dependence on binary classifier”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [32] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    VQA is free-form and open-ended.

    “Free-form and open-ended”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [37] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    TreeView interpretation requires a thorough understanding of the method.

    “Interpretation requires a thorough understanding of the method”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [65]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    TreeView provides transition between factors, class labels, and input data space.

    “Transition between factors, class labels, and input data space”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [65]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    ProtoTree requires no manual formulation of prototypes/concepts.

    “No manual formulation of prototypes/ concepts is required”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [62] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Network Dissection communicates inner workings in a nontechnical way.

    “Communication of inner workings in a nontechnical way”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    UMAP has improved time performance and better perseverance of data structure.

    “Improved time performance, Better perseverance of data structure”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    TCAV provides explorative explanations beyond feature attribution.

    “Explorative explanations beyond feature attribution”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23]

      Not checked in the free run.

81 of 81

  • The paper relies on this without support.

    A gap exists in non-technical overviews of non-visual XAI methods for medical imaging aimed at healthcare professionals.

    “This review aims to fill that gap by providing a non-technical overview of common non-visual XAI methods that apply to medical imaging, along with their advantages, pitfalls, and limitations.”

  • The authors state this as an assumption.

    The main need of the intended end-users, namely radiologists, clinicians, and doctors, is linking knowledge and confirming diagnosis.

    “In this paper, we primarily refer to end-users as radiologists, clinicians, and doctors whose main need presumably might be the linking of knowledge and confirmation of diagnosis.”

  • The paper relies on this without support.

    There is no mathematical or standardized definition of explainability and interpretability.

    “Since there is neither a mathematical nor a standardized definition of explainability and interpretability, in this paper, a non-mathematical definition from the social sciences perspective will be used as given by Miller [7]”

  • The paper relies on this without support.

    Models are typically not equipped to reject an input during training or inference if it is poorly represented by the underlying data or falls out of distribution.

    “as models are usually not equipped with the ability to reject an input during training or inference if it is not represented well by the underlying data or falls out of distribution.”

  • The paper relies on this without support.

    Bayesian approaches typically require complex modifications to the training process and can be computationally expensive.

    “justified by the fact that Bayesian approaches usually require complex modifications of the training process and can be computationally expensive.”

  • The paper relies on this without support.

    The agreement among models in an ensemble represents confidence in the overall ensemble.

    “The models' agreement represents the confidence in the overall model ensemble.”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Recent developments include interactive dashboards that allow clinicians to explore how individual variables contribute to patient-specific risk scores.

    “Recent developments include interactive dashboards that allow clinicians to explore how individual variables contribute to patient-specific risk scores364,424.”

    From Artificial intelligence-driven multi-omics approaches in Alzheimer's disease: Progress, challenges, and future directions (2025) · cites this paper as [364]

    Section 2.3 describes importance scores and graphical or tabular explanations for individual predictions. Section 2 also gives an example of exploring how changes to clinical parameters might affect a patient’s predicted outcome. Neither passage describes an interactive dashboard for clinicians or variable contributions to patient-specific risk scores; searches for “dashboard” and “risk scores” found no such passage. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • XAI methods help discern if confounders or biases are present in the model, allowing for their control or removal.

    “These methods help us discern if confounders or biases are present in the model, allowing for their control or removal.107”

    From Bias in artificial intelligence for medical imaging: fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects (2024) · cites this paper as [107]

Cites

Flagged first, then by citations

8 of the 77 references couldn’t be matched to a known paper.

Research works printed but not linked: 11

  1. Van der Maaten, Laurens, Hinton, Geoffrey, Visualizing data using t-SNE., ., vol. 9, no. 86, pp. 2579–2605, 2008.
  2. J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, B. Kim, Sanity checks for saliency maps, in: Proceedings of the 32nd International Conference on Neural Information Processing Systems.
  3. B. Kim et al., Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV), in: Proceedings of the 35th International Conference on Machine Learning, Jul. 2018, pp. 2668–2677. Accessed: Feb. 14, 2022. [Online]. Available: https://proceedings.mlr.press/v80/kim18d.html.
  4. A. Ghorbani, J. Wexler, J.Y. Zou, B. Kim, Towards Automatic Concept-based Explanations, in: Advances in Neural Information Processing Systems, 2019, vol. 32. Accessed: Jun. 03, 2022. [Online]. Available: https://proceedings.neurips.cc/paper/2019/hash/77d2afcb31f6493e350fca61764efb9a-Abstract.html.
  5. P.W. Koh, P. Liang, Understanding Black-box Predictions via Influence Functions, in: Proceedings of the 34th International Conference on Machine Learning, Jul. 2017, pp. 1885–1894. Accessed: Sep. 20, 2022. [Online]. Available: https://proceedings.mlr.press/v70/koh17a.html.
  6. K. Xu et al., Show, attend and tell: neural image caption generation with visual attention, in: Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, Lille, France, Jul. 2015, pp. 2048–2057.
  7. T. Pearce, A. Brintrup, M. Zaki, A. Neely, High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach, in: Proceedings of the 35th International Conference on Machine Learning, Jul. 2018, pp. 4075–4084. Accessed: Dec. 12, 2022. [Online]. Available: https://proceedings.mlr.press/v80/pearce18a.html.
  8. M.S. Ayhan, P. Berens, Test-time Data Augmentation for Estimation of Heteroscedastic Aleatoric Uncertainty in Deep Neural Networks, presented at the Medical Imaging with Deep Learning, Apr. 2018. Accessed: Apr. 07, 2022. [Online]. Available: https://openreview.net/forum?id=rJZz-knjz.
  9. Y. Gal, Z. Ghahramani, Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning, in: Proceedings of The 33rd International Conference on Machine Learning, Jun. 2016, pp. 1050–1059. Accessed: Jul. 08, 2022. [Online]. Available: https://proceedings.mlr.press/v48/gal16.html.
  10. M.E.E. Khan, A. Immer, E. Abedi, M. Korzepa, Approximate Inference Turns Deep Networks into Gaussian Processes, in: Advances in Neural Information Processing Systems, 2019, vol. 32. Accessed: Jun. 08, 2022. [Online]. Available: https://proceedings.neurips.cc/paper/2019/hash/b3bbccd6c008e727785cb81b1aa08ac5-Abstract.html.
  11. F. D’Angelo, V. Fortuin, “Repulsive Deep Ensembles are Bayesian,” presented at the Neural Information Processing Systems, Jun. 2021. Accessed: Dec. 18, 2022. [Online]. Available: https://www.semanticscholar.org/paper/Repulsive-Deep-Ensembles-are-Bayesian-D'Angelo-Fortuin/be5491660a61d60606aaec8dc0e7e046fb930110.

Cited by

Sorted by citations

Related

Authors
Borys, K., Schmitt, Y. A., Nauta, M., Seifert, C., Krämer, N., Friedrich, C. M., & Nensa, F.
Affiliations
  • Essen University Hospital (DE)
  • University of Duisburg-Essen (DE)
  • Dortmund University of Applied Sciences and Arts (DE)
Corresponding author
Borys, Katarzyna
ORCID
Venue
European Journal of Radiology
Publisher
Elsevier BV
Published
2023-05
Volume
162
Pages
110786
ISSN
0720-048X (ISSN-L)
DOI
10.1016/j.ejrad.2023.110786
OpenAlex
W4327919044
Semantic Scholar
b08ce42005c672f681c6ab4cff96f830ce0bf9fc
Cited by
233
References
77
Citations by year
2026: 77 · 2025: 101 · 2024: 46 · 2023: 9
FWCI
4.87
Type
journal article
Language
en
Open access
Licence
Locations
  • European Journal of Radiology (publishedVersion, other-oa)
  • PubMed (publishedVersion)
  • University of Twente Research Information (submittedVersion, other-oa)
  • University of Twente Research Information (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
Keywords
explainable AI, diagnostic imaging, case-based explanation, textual explanations, deep learning interpretability, clinicians, computer-aided diagnosis, image classification, image segmentation, image registration
MeSH terms
Artificial Intelligence, Health Personnel, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review