CoChat

Explainable Artificial Intelligence for Bias Detection in COVID CT-Scan Classifiers

Open access
Palatnik de Sousa, Iam, Vellasco, Marley M. B. R., Costa da Silva, Eduardo · 2021 · Sensors

The DOI matches its Crossref record; 22 of the 24 references it cites were found in a registry or an index.

Abstract

Problem: An application of Explainable Artificial Intelligence Methods for COVID CT-Scan classifiers is presented. Motivation: It is possible that classifiers are using spurious artifacts in dataset images to achieve high performances, and such explainable techniques can help identify this issue. Aim: For this purpose, several approaches were used in tandem, in order to create a complete overview of the classificatios. Methodology: The techniques used included GradCAM, LIME, RISE, Squaregrid, and direct Gradient approaches (Vanilla, Smooth, Integrated). Main results: Among the deep neural networks architectures evaluated for this image classification task, VGG16 was shown to be most affected by biases towards spurious artifacts, while DenseNet was notably more robust against them. Further impacts: Results further show that small differences in validation accuracies can cause drastic changes in explanation heatmaps for DenseNet architectures, indicating that small changes in validation accuracy may have large impacts on the biases learned by the networks. Notably, it is important to notice that the strong performance metrics achieved by all these networks (Accuracy, F1 score, AUC all in the 80 to 90% range) could give users the erroneous impression that there is no bias. However, the analysis of the explanation heatmaps highlights the bias.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/s21165657

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Samples from the two classes (NonCOVID and COVID).
Figure 1 Samples from the two classes (NonCOVID and COVID).CC BY · PubMed Central
Figure 2 Visual diagram of XAI techniques used in this study. (a) GradCAM: gradients to the last convolution are computed and weighted sum is calculated. (b) Integrated Gradients: Sum/integral based on interpolations of the image. (c) Vanilla Gradients: Direct gradient computation. (d) Smooth Gradients: gradient computation on noisy versions of the image. (e) RISE: weighted sum of perturbation masks. (f) Squaregrid: perturbations using square grid divisions. (g) LIME: perturbations using superpixels computed via SLIC.
Figure 2 Visual diagram of XAI techniques used in this study. (a) GradCAM: gradients to the last convolution are computed and weighted sum is calculated. (b) Integrated Gradients: Sum/integral based on interpolations of the image. (c) Vanilla Gradients: Direct gradient computation. (d) Smooth Gradients: gradient computation on noisy versions of the image. (e) RISE: weighted sum of perturbation masks. (f) Squaregrid: perturbations using square grid divisions. (g) LIME: perturbations using superpixels computed via SLIC.CC BY · PubMed Central
Figure 3 Ensemble accuracies: (a) Validation and (b) test accuracies, as well as the number of models per ensemble. Each point in these plots might have overlapping ensembles, so the markers have transparency to aid in visualization. A total of 16,383 different ensemble combinations were tested in this way. Notably, the validation results are used for ensemble selection, and the test accuracies are just included here for visualization.
Figure 3 Ensemble accuracies: (a) Validation and (b) test accuracies, as well as the number of models per ensemble. Each point in these plots might have overlapping ensembles, so the markers have transparency to aid in visualization. A total of 16,383 different ensemble combinations were tested in this way. Notably, the validation results are used for ensemble selection, and the test accuracies are just included here for visualization.CC BY · PubMed Central
Figure 4 Comparison of XAI heatmaps for image with colored artifact: (a) VGGval831. (b) Effval856. (c) Denseval898. (d) Denseval907. (e) Ensemble. Top row for each panel shows the heatmaps, while bottom row highlights the areas of highest heatmap weights. Note that GradCAM is not defined for ensembles of models. The scales for the heatmaps go from 0 to 1 for GradCAM, minimum to maximum for RISE and -maximum to maximum for Squaregrid and Gradcam.
Figure 4 Comparison of XAI heatmaps for image with colored artifact: (a) VGGval831. (b) Effval856. (c) Denseval898. (d) Denseval907. (e) Ensemble. Top row for each panel shows the heatmaps, while bottom row highlights the areas of highest heatmap weights. Note that GradCAM is not defined for ensembles of models. The scales for the heatmaps go from 0 to 1 for GradCAM, minimum to maximum for RISE and -maximum to maximum for Squaregrid and Gradcam.CC BY · PubMed Central
Figure 5 Comparison of XAI heatmaps for image without colored artifacts: (a) VGGval831. (b) Effval856. (c) Denseval898. (d) Denseval907. (e) Ensemble. See the caption of Figure 4 for details on heatmap scales.
Figure 5 Comparison of XAI heatmaps for image without colored artifacts: (a) VGGval831. (b) Effval856. (c) Denseval898. (d) Denseval907. (e) Ensemble. See the caption of Figure 4 for details on heatmap scales.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 5

  1. Zhao, J., Zhang, Y., He, X., and Xie, P. (2020). COVID-CT-Dataset: A CT scan dataset about COVID-19. arXiv.
  2. Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv.
  3. Tan, M., and Le, Q.V. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv.
  4. Petsiuk, V., Das, A., and Saenko, K. (2018). Rise: Randomized input sampling for explanation of black-box models. arXiv.
  5. Sundararajan, M., Taly, A., and Yan, Q. (2017). Axiomatic attribution for deep networks. arXiv.

Cited by

See all 46

Metadata

Authors
Palatnik de Sousa, I., Vellasco, M. M. B. R., & Costa da Silva, E.
Affiliations
  • Department of Electrical Engineering, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro 22453-900, Brazil
  • Pontifícia Universidade Católica do Rio de Janeiro (BR)
Venue
Sensors
Publisher
MDPI AG
Published
2021-08-23
Volume
21
Issue
16
Pages
5657
ISSN
1424-8220 (ISSN-L)
OpenAlex
W3194757962
Cited by
43
References
24
Citations by year
2025: 11 · 2024: 9 · 2023: 14 · 2022: 9
FWCI
2.82
Type
journal article
Language
en
Open access
Locations
  • Sensors (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (001)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
explainable AI, bias detection, Grad-CAM, lime, RISE, gradient-based explanation, VGG16, DenseNet, model robustness
MeSH terms
COVID-19, SARS-CoV-2, Artificial Intelligence, Humans, Tomography, X-Ray Computed, Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.