CoChat

Adversarial explanations for understanding image classification decisions and improved neural network robustness

Open access
Woods, Walt, Chen, Jack, Teuscher, Christof · 2019 · Nature Machine Intelligence
Cited by 48

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Abstract

For sensitive problems, such as medical imaging or fraud detection, neural network (NN) adoption has been slow due to concerns about their reliability, leading to a number of algorithms for explaining their decisions. NNs have also been found to be vulnerable to a class of imperceptible attacks, called adversarial examples, which arbitrarily alter the output of the network. Here we demonstrate both that these attacks can invalidate previous attempts to explain the decisions of NNs, and that with very robust networks, the attacks themselves may be leveraged as explanations with greater fidelity to the model. We also show that the introduction of a novel regularization technique inspired by the Lipschitz constraint, alongside other proposed improvements including a half-Huber activation function, greatly improves the resistance of NNs to adversarial examples. On the ImageNet classification task, we demonstrate a network with an accuracy-robustness area (ARA) of 0.0053, an ARA 2.4 times greater than the previous state-of-the-art value. Improving the mechanisms by which NN decisions are understood is an important direction for both establishing trust in sensitive domains and learning more about the stimuli to which NNs respond. Deep neural networks can be led to misclassify an image when minute changes that are imperceptible to humans are introduced. While for some networks this ability can cast doubt on the reliability of the model, it also offers explainability for networks that use more robust regularization.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s42256-019-0104-6

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 15 references

Research works printed but not linked: 37

  1. Tsao, H.-Y., Chan, P.-Y. & Su, E. C.-Y. Predicting diabetic retinopathy and identifying interpretable biomedical features using machine learning algorithms. BMC Bioinform. 19, 283 (2018). · DOI · Paper page
  2. Szegedy, C. et al. Intriguing properties of neural networks. Preprint at https://arxiv.org/abs/1312.6199 (2013). · arXiv
  3. Selvaraju, R. R. et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision 618–626 (IEEE, 2017). · DOI · Paper page
  4. Ribeiro, M. T., Singh, S. & Guestrin, C. Why should I trust you? Explaining the predictions of any classifier. In Proc. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 1135–1144 (ACM, 2016). · DOI · Paper page
  5. Simonyan, K., Vedaldi, A. & Zisserman, A. Deep inside convolutional networks: visualising image classification models and saliency maps. Preprint at https://arxiv.org/abs/1312.6034 (2013). · arXiv
  6. Landecker, W. Interpretable Machine Learning and Sparse Coding for Computer Vision. PhD thesis, Portland State Univ. (2014).
  7. Bau, D. et al. Visualizing and understanding generative adversarial networks. In International Conference on Learning Representations (ICLR, 2019).
  8. Hong, S., You, T., Kwak, S. & Han, B. Online tracking by learning discriminative saliency map with convolutional neural network. In International Conference on Machine Learning 597–606 (ICML, 2015).
  9. Zeiler, M. D. & Fergus, R. Visualizing and understanding convolutional networks. In European Conference on Computer Vision 818–833 (Springer, 2014). · DOI · Paper page
  10. Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996). · DOI · Paper page
  11. Li, K., Wu, Z., Peng, K.-C., Ernst, J. & Fu, Y. Tell me where to look: guided attention inference network. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 9215–9223 (IEEE, 2018). · DOI · Paper page
  12. Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems 5998–6008 (NIPS, 2017).
  13. Cui, Y. et al. Attention-over-attention neural networks for reading comprehension. In Proc. 55th Annual Meeting of the Association for Computational Linguistics Vol. 1, 593–602 (ACL, 2017). · DOI · Paper page
  14. Athalye, A., Engstrom, L., Ilyas, A. & Kwok, K. Synthesizing robust adversarial examples. In Proc. 35th International Conference on Machine Learning (eds Dy, J. & Krause, A.) 284–293 (PMLR, 2018).
  15. Goodfellow, I., Shlens, J. & Szegedy, C. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR, 2015).
  16. Chakraborty, A., Alam, M., Dey, V., Chattopadhyay, A. & Mukhopadhyay, D. Adversarial attacks and defences: a survey. Preprint at https://arxiv.org/abs/1810.00069 (2018). · arXiv
  17. Khoury, M. & Hadfield-Menell, D. On the geometry of adversarial examples. Preprint at https://arxiv.org/abs/1811.00525 (2018). · arXiv
  18. Tsipras, D., Santurkar, S., Engstrom, L., Turner, A. & Madry, A. Robustness may be at odds with accuracy. In International Conference on Learning Representations (ICLR, 2019).
  19. Weng, T.-W. et al. Evaluating the robustness of neural networks: an extreme value theory approach. In International Conference on Learning Representations (ICLR, 2018).
  20. Cisse, M., Bojanowski, P., Grave, E., Dauphin, Y. & Usunier, N. Parseval networks: improving robustness to adversarial examples. In Proc. 34th International Conference on Machine Learning 854–863 (JMLR, 2017).
  21. Behrmann, J., Grathwohl, W., Chen, R. T. Q., Duvenaud, D. & Jacobsen, J.-H. Invertible residual networks. In Proc. 36th International Conference on Machine Learning (eds Chaudhuri, K. & Salakhutdinov, R.) 573–582 (PMLR, 2019).
  22. Madry, A., Makelov, A., Schmidt, L., Tsipras, D. & Vladu, A. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR, 2018).
  23. Russakovsky, O. et al. ImageNet large scale visual recognition challenge. Int. J. Comput. Vision (IJCV) 115, 211–252 (2015). · DOI · Paper page
  24. Carlini, N. & Wagner, D. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy 39–57 (IEEE, 2017). · DOI · Paper page
  25. Pei, K., Cao, Y., Yang, J. & Jana, S. Deepxplore: Automated whitebox testing of deep learning systems. In Proc. 26th Symposium on Operating Systems Principles 1–18 (ACM, 2017). · DOI · Paper page
  26. Cohen, J., Rosenfeld, E. & Kolter, Z. Certified adversarial robustness via randomized smoothing. In Proc. 36th International Conference on Machine Learning (eds Chaudhuri, K. & Salakhutdinov, R.) 1310–1320 (PMLR, 2019).
  27. Liao, F. et al. Defense against adversarial attacks using high-level representation guided denoiser. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 1778–1787 (IEEE, 2018). · DOI · Paper page
  28. Kurakin, A. et al. Adversarial attacks and defences competition. In The NIPS’17 Competition: Building Intelligent Systems 195–231 (Springer, 2018). · DOI · Paper page
  29. Wong, E., Schmidt, F., Metzen, J. H. & Kolter, J. Z. Scaling provable adversarial defenses. In Advances in Neural Information Processing Systems 8400–8409 (NIPS, 2018).
  30. Su, D. et al. Is robustness the cost of accuracy? A comprehensive study on the robustness of 18 deep image classification models. In European Conference on Computer Vision 631–648 (Springer, 2018). · DOI · Paper page
  31. Rony, J. et al. Decoupling direction and norm for efficient gradient-based L2 adversarial attacks and defenses. In The IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2019). · DOI · Paper page
  32. Shiraishi, J. et al. Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. Am. J. Roentgenol. 174, 71–74 (2000). · DOI · Paper page
  33. Lin, T.-Y. et al. Microsoft COCO: common objects in context. In European Conference on Computer Vision 740–755 (Springer, 2014). · DOI · Paper page
  34. Carlini, N. et al. On evaluating adversarial robustness. Preprint at https://arxiv.org/abs/1902.06705 (2019). · arXiv
  35. He, K., Zhang, X., Ren, S. & Sun, J. Identity mappings in deep residual networks. In European Conference on Computer Vision 630–645 (Springer, 2016). · DOI · Paper page
  36. Paszke, A. et al. Automatic differentiation in PyTorch. In Proc. 31st Conference on Neural Information Processing Systems (NIPS, 2017).
  37. Yamada, Y., Iwamura, M., Akiba, T. & Kise, K. Shakedrop regularization for deep residual learning. Preprint at https://arxiv.org/abs/1802.02375 (2018). · arXiv

Cited by

See all 48

Metadata

Authors
Woods, W., Chen, J., & Teuscher, C.
Venue
Nature Machine Intelligence
Published
2019-11-04
Volume
1
Issue
11
Pages
508-516
ISSN
2522-5839 (ISSN-L)
OpenAlex
W2964693503
Cited by
48
References
15
FWCI
3.11
Type
journal article
Language
en
Open access
green
Licence
Funders
  • United States Department of Defense | Defense Threat Reduction Agency (HDTRA1-18-1-0009)
  • Others are listed in the acknowledgments of the manuscript, and did not have reference numbers.

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Adversarial explanations for understanding image classification decisions and improved neural network robustness. https://app.cochat.ai/papers/10.1038/s42256-019-0104-6
The paper
Woods, W., Chen, J., & Teuscher, C. (2019). Adversarial explanations for understanding image classification decisions and improved neural network robustness. Nature Machine Intelligence. https://doi.org/10.1038/s42256-019-0104-6

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.