CoChat
Sign Up
Cited by 119Explainable Artificial Intelligence (XAI)Artificial Intelligence in Healthcare and EducationSoftware Testing and Debugging Techniques

The DOI matches its Crossref record; 52 of the 53 references it cites were found in a registry or an index.

GANterfactual—Counterfactual Explanations for Medical Non-experts Using Generative Adversarial Learning

Open access
Mertes, Silvan, Huber, Tobias, Weitz, Katharina, Heimerl, Alexander, André, Elisabeth·2022·Frontiers in Artificial Intelligence
DOI 10.3389/frai.2022.825565PMC PMC9024220
Open this paper in CoChat — free accountOpen PDF

Abstract

With the ongoing rise of machine learning, the need for methods for explaining decisions made by artificial intelligence systems is becoming a more and more important topic. Especially for image classification tasks, many state-of-the-art tools to explain such classifiers rely on visual highlighting of important areas of the input data. Contrary, counterfactual explanation systems try to enable a counterfactual reasoning by modifying the input image in a way such that the classifier would have made a different prediction. By doing so, the users of counterfactual explanation systems are equipped with a completely different kind of explanatory information. However, methods for generating realistic counterfactual explanations for image classifiers are still rare. Especially in medical contexts, where relevant information often consists of textural and structural information, high-quality counterfactual images have the potential to give meaningful insights into decision processes. In this work, we present GANterfactual , an approach to generate such counterfactual image explanations based on adversarial image-to-image translation techniques. Additionally, we conduct a user study to evaluate our approach in an exemplary medical use case. Our results show that, in the chosen medical use-case, counterfactual explanations lead to significantly better results regarding mental models, explanation satisfaction, trust, emotions, and self-efficacy than two state-of-the art systems that work with saliency maps, namely LIME and LRP.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/frai.2022.825565

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Schematic overview of our approach. A CycleGAN architecture is extended with the classifier that shall be explained. Both the generators of the CycleGAN include the classifier's decisions for the generated data into their loss function.
Figure 1 Schematic overview of our approach. A CycleGAN architecture is extended with the classifier that shall be explained. Both the generators of the CycleGAN include the classifier's decisions for the generated data into their loss function.CC BY · PubMed Central
Figure 2 Example images of the used dataset. The top row shows images that are labeled as Normal, while the bottom row shows images labeled as Lung Opacity, indicating lungs that are suffering from pneumonia.
Figure 2 Example images of the used dataset. The top row shows images that are labeled as Normal, while the bottom row shows images labeled as Lung Opacity, indicating lungs that are suffering from pneumonia.CC BY · PubMed Central
Figure 3 Examples of counterfactual images produced with our proposed approach. In each pair, the left image shows the original image, while the right image shows the corresponding counterfactual explanation. The red boxes were added manually to point the reader to the regions that were altered the most. The original images in the top row were classified as normal, while the original images in the bottom row were classified as pneumonia. The shown counterfactual images were all classified as the opposite as their respective counterpart.
Figure 3 Examples of counterfactual images produced with our proposed approach. In each pair, the left image shows the original image, while the right image shows the corresponding counterfactual explanation. The red boxes were added manually to point the reader to the regions that were altered the most. The original images in the top row were classified as normal, while the original images in the bottom row were classified as pneumonia. The shown counterfactual images were all classified as the opposite as their respective counterpart.CC BY · PubMed Central
Figure 4 Computational evaluation results of the counterfactual image generation performance. The confusion matrices show the number of samples out of each subset (Normal, Pneumonia, Total) of the rsna dataset that the classifier predicted to be the respective class before (y-axis) and after (x-axis) the samples had been translated by either the original CycleGAN or by our approach.
Figure 4 Computational evaluation results of the counterfactual image generation performance. The confusion matrices show the number of samples out of each subset (Normal, Pneumonia, Total) of the rsna dataset that the classifier predicted to be the respective class before (y-axis) and after (x-axis) the samples had been translated by either the original CycleGAN or by our approach.CC BY · PubMed Central
Figure 5 An example x-ray image classified as Pneumonia, as well as the different XAI visualizations used in our study when the slider is fully on the right side. Best viewed in color.
Figure 5 An example x-ray image classified as Pneumonia, as well as the different XAI visualizations used in our study when the slider is fully on the right side. Best viewed in color.CC BY · PubMed Central
Figure 6 A simplified schematic of our prediction task.
Figure 6 A simplified schematic of our prediction task.CC BY · PubMed Central
Figure 7 Results of the explanation satisfaction and trust questionnaires. Error bars represent the 95% Confidence Interval (CI).
Figure 7 Results of the explanation satisfaction and trust questionnaires. Error bars represent the 95% Confidence Interval (CI).CC BY · PubMed Central
Figure 8 Results of the prediction task, and the task reflection questions. Error bars represent the 95% Confidence Interval (CI).
Figure 8 Results of the prediction task, and the task reflection questions. Error bars represent the 95% Confidence Interval (CI).CC BY · PubMed Central
Figure 9 Results of the emotion questionnaires. Participants in the counterfactual condition felt significantly less angry and more relaxed compared to the LRP saliency map condition. For LIME, no significant differences were found. Error bars represent the 95% CI.
Figure 9 Results of the emotion questionnaires. Participants in the counterfactual condition felt significantly less angry and more relaxed compared to the LRP saliency map condition. For LIME, no significant differences were found. Error bars represent the 95% CI.CC BY · PubMed Central
Figure 10 Significant differences regarding self-efficacy and general confidence of the participants in their predictions of the AI between the counterfactual condition and the saliency map conditions (LRP and LIME). Error bars represent the 95% CI.
Figure 10 Significant differences regarding self-efficacy and general confidence of the participants in their predictions of the AI between the counterfactual condition and the saliency map conditions (LRP and LIME). Error bars represent the 95% CI.CC BY · PubMed Central
Figure 11 Confidence of the participants in correct and false predictions. The significant difference between the counterfactual condition and the saliency map conditions is based on the confidence in correct predictions, not in the incorrect ones. Error bars represent the 95% CI.
Figure 11 Confidence of the participants in correct and false predictions. The significant difference between the counterfactual condition and the saliency map conditions is based on the confidence in correct predictions, not in the incorrect ones. Error bars represent the 95% CI.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 2

  1. Hall, 2018, Introduction to Machine Learning Interpretability
  2. Stone, 2016, Artificial Intelligence and Life in 2030. One Hundred Year Study on Artificial Intelligence: Report of the 2015-2016 Study Panel

Cited by

Sorted by citations

See all 119

Related

See all

Metadata

Authors
Mertes, S., Huber, T., Weitz, K., Heimerl, A., & André, E.
Affiliations
University of Augsburg (DE)
Corresponding author
Mertes, Silvan
ORCID
Venue
Frontiers in Artificial Intelligence
Publisher
Frontiers Media SA
Published
2022-04-08
Volume
5
Pages
825565-825565
ISSN
2624-8212 (ISSN-L)
DOI
10.3389/frai.2022.825565
PMC
PMC9024220
OpenAlex
W3187709298
Cited by
119
References
53
Citations by year
2026: 32 · 2025: 34 · 2024: 31 · 2023: 16 · 2022: 6
FWCI
7.13
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Artificial Intelligence (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • OPUS (Augsburg University) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
  • Deutsche Forschungsgemeinschaft
  • Bayerisches Staatsministerium für Wissenschaft, Forschung und Kunst
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords
counterfactual explanations, generative adversarial networks, image-to-image translation, medical image classification, explainable AI, adversarial perturbation generation, non-expert users, user study, mental models, trust in AI, self-efficacy, saliency maps, lime, LRP

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review