CoChat
Sign Up
PreprintCited by 23Explainable Artificial Intelligence (XAI)Adversarial Robustness in Machine LearningSoftware Testing and Debugging Techniques

Saliency Methods for Explaining Adversarial Attacks

Open access
Gu, Jindong, Tresp, Volker·2019·arXiv
DOI 10.48550/arxiv.1908.08413arXiv 1908.08413
Open this paper in CoChat — free accountOpen PDF

Abstract

The classification decisions of neural networks can be misled by small imperceptible perturbations. This work aims to explain the misled classifications using saliency methods. The idea behind saliency methods is to explain the classification decisions of neural networks by creating so-called saliency maps. Unfortunately, a number of recent publications have shown that many of the proposed saliency methods do not provide insightful explanations. A prominent example is Guided Backpropagation (GuidedBP), which simply performs (partial) image recovery. However, our numerical analysis shows the saliency maps created by GuidedBP do indeed contain class-discriminative information. We propose a simple and efficient way to enhance the saliency maps. The proposed enhanced GuidedBP shows the state-of-the-art performance to explain adversary classifications.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1908.08413

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Common visual explanations remain unchanged even when precise modifications are made to the input that substantially alter the model's predictions, even when those attacks lead to incorrect model predictions.

    “showed that common visual explanations remain unchanged even when precise modifications are made to the input that substantially alter the model's predictions (a process known as an adversarial attack), even when those attacks lead to incorrect model predictions (figure 2).”

    From The false hope of current approaches to explainable artificial intelligence in health care (2021) · cites this paper as [23]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 23

  1. [1][1] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In ICLR , 2013. -
  2. [2][2] Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In ECCV , pages 818-833. Springer, 2014. -
  3. [3][3] Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin A. Riedmiller. Striving for simplicity: The all convolutional net. In ICLR , 2014. -
  4. [4][4] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one , 10(7):e0130140, 2015. -
  5. [5][5] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD , pages 1135-1144. ACM, 2016. -
  6. [6][6] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In ICML , 2017. -
  7. [7][7] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 , 2017. - · arXiv
  8. [8][8] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In ICML , 2017. -
  9. [9][9] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra, et al. Grad-cam: Visual explanations from deep networks via gradientbased localization. In ICCV , pages 618-626, 2017. -
  10. [10][10] Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. A unified view of gradientbased attribution methods for deep neural networks. In NIPS 2017-Workshop on Interpreting, Explaining and Visualizing Deep Learning . ETH Zurich, 2017. -
  11. [11][11] Luisa M. Zintgraf, Taco Cohen, Tameem Adel, and Max Welling. Visualizing deep neural network decisions: Prediction difference analysis. In ICLR , 2017. -
  12. [12][12] Piotr Dabkowski and Yarin Gal. Real time image saliency for black box classifiers. In NeuIPS , pages 6967-6976, 2017. -
  13. [13][13] Ruth C. Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful perturbation. ICCV , pages 3449-3457, 2017. -
  14. [15][15] Aravindh Mahendran and Andrea Vedaldi. Salient deconvolutional networks. In ECCV , 2016. -
  15. [16][16] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In NeurIPS , pages 9525-9536, 2018. -
  16. [17][17] Weili Nie, Yang Zhang, and Ankit Patel. A theoretical explanation for perplexing behaviors of backpropagation-based visualizations. In 2018 Workshop on Human Interpretability in Machine Learning (WHI) , 2018. -
  17. [18][18] David Alvarez-Melis and Tommi S Jaakkola. On the robustness of interpretability methods. In Workshop on Human Interpretability in Machine Learning (WHI) , 2018. -
  18. [19][19] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR , 2014. -
  19. [20][20] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR , 2015. -
  20. [21][21] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014. - · arXiv
  21. [22][22] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. IJCV , 88(2):303-338, 2010. -
  22. [23][23] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533 , 2016. - · arXiv
  23. [24][24] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP) , pages 39-57. IEEE, 2017. -

Cited by

Sorted by citations

Authors
Gu, J., & Tresp, V.
Affiliations
Siemens (Germany) (DE)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1908.08413
arXiv
1908.08413
OpenAlex
W2969599988
Cited by
23
References
2
Citations by year
2024: 7 · 2023: 1 · 2022: 4 · 2021: 7 · 2020: 2 · 2019: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords
saliency maps, adversarial attacks, explainable AI, guided backpropagation, neural network interpretability, robustness to disturbances, image classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review