CoChat
Sign Up
Cited by 255Explainable Artificial Intelligence (XAI)Software Testing and Debugging TechniquesEthics and Social Impacts of AI

Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges

Open access
Ras, Gabriëlle, van Gerven, Marcel, Haselager, Pim·2018·The Springer Series on Challenges in Machine Learning
DOI 10.1007/978-3-319-98131-4_2
Open this paper in CoChat — free accountOpen PDF

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/978-3-319-98131-4_2

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 51

  1. Philip Adler, Casey Falk, Sorelle A. Friedler, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, and Suresh Venkatasubramanian. Auditing black-box models for indirect influence. In 2016 IEEE 16th International Conference on Data Mining (ICDM) . IEEE, 2016.
  2. Marco Ancona, Enea Ceolini, Cengiz Oztireli, and Markus Gross. Towards better understanding of gradient-based attribution methods for deep neural networks. In 6th International Conference on Learning Representations (ICLR 2018) , 2018.
  3. Robert Andrews, Joachim Diederich, and Alan B. Tickle. Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems , 8(6): 373-389, 1995.
  4. A. Duygu Arbatli and H. Levent Akin. Rule extraction from trained neural networks using genetic algorithms. Nonlinear Analysis: Theory, Methods & Applications , 30(3):1639-1648, 1997.
  5. Sebastian Bach, Alexander Binder, Gr´ egoire Montavon, Frederick Klauschen, Klaus-Robert M¨ uller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE , 10(7), 2015.
  6. David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert M¨ uller. How to explain individual classification decisions. Journal of Machine Learning Research (JMLR) , 11:1803-1831, 2010.
  7. Alexander Binder, Sebastian Bach, Gr´ egoire Montavon, Klaus-Robert M¨ uller, and Wojciech Samek. Layer-wise relevance propagation for deep neural network architectures. In Information Science and Applications (ICISA) 2016 , pages 913-922. Springer, 2016.
  8. Mariusz Bojarski, Philip Yeres, Anna Choromanska, Krzysztof Choromanski, Bernhard Firner, Lawrence Jackel, and Urs Muller. Explaining how a deep neural network trained with endto-end learning steers a car. arXiv preprint arXiv:1704.07911 , 2017. · arXiv
  9. Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science , 356(6334):183-186, 2017.
  10. Nicholas Carlini and David Wagner. Audio adversarial examples: Targeted attacks on speechto-text. arXiv preprint arXiv:1801.01944 , 2018. · arXiv
  11. Mark W. Craven and Jude W. Shavlik. Using sampling and queries to extract rules from trained neural networks. In Machine Learning Proceedings 1994 , pages 37-45. Elsevier, 1994.
  12. Ekin D. Cubuk, Barret Zoph, Samuel S. Schoenholz, and Quoc V. Le. Intriguing properties of adversarial examples. arXiv preprint arXiv:1711.02846 , 2017. · arXiv
  13. Yinpeng Dong, Hang Su, Jun Zhu, and Fan Bao. Towards interpretable deep neural networks by leveraging adversarial examples. arXiv preprint arXiv:1708.05493 , 2017a. · arXiv
  14. Yinpeng Dong, Hang Su, Jun Zhu, and Bo Zhang. Improving interpretability of deep neural networks with semantic information. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, 2017b.
  15. Derek Doran, Sarah Schulz, and Tarek R. Besold. What does explainable AI really mean? a new conceptualization of perspectives. arXiv preprint arXiv:1710.00794 , 2017. · arXiv
  16. Luciano Floridi, Nir Fresco, and Giuseppe Primiero. On malfunctioning software. Synthese , 192 (4):1199-1220, 2015.
  17. Ruth C. Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful perturbation. In 2017 IEEE International Conference on Computer Vision (ICCV) . IEEE, 2017.
  18. Yaˇ gmur Gu¸ cl¨ ut¨ urk, Umut G¨ u¸ cl¨ u, Marc Perez, Hugo Jair Escalante, Xavier Baro, Isabelle Guyon, Carlos Andujar, Julio Jacques Junior, Meysam Madadi, Sergio Escalera, Marcel A. J. van Gerven, and Rob van Lier. Visualizing apparent personality analysis with deep residual networks. In 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) , pages 3101-3109, 2017.
  19. Andreas Holzinger, Markus Plass, Katharina Holzinger, Gloria Cerasela Cri¸ san, CameliaMihaela Pintea, and Vasile Palade. A glass-box interactive machine learning approach for solving NP-hard problems with the human-in-the-loop. arXiv preprint arXiv:1708.01104 , 2017b. · arXiv
  20. Pieter-Jan Kindermans, Kristof T. Sch¨ utt, Klaus-Robert M¨ uller, and Sven D¨ ahne. Investigating the influence of noise and distractors on the interpretation of neural networks. arXiv preprint arXiv:1611.07270 , 2016. · arXiv
  21. Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Sch¨ utt, Sven D¨ ahne, Dumitru Erhan, and Been Kim. The (un) reliability of saliency methods. arXiv preprint arXiv:1711.00867 , 2017. · arXiv
  22. Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In Proceedings of the 34th International Conference on Machine Learning (ICML) , volume 70 of Proceedings of Machine Learning Research (PMLR) , pages 1885-1894, 2017.
  23. Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovec. Interpretable & explorable approximations of black box models. arXiv preprint arXiv:1707.01154 , 2017. · arXiv
  24. Jiwei Li, Will Monroe, and Dan Jurafsky. Understanding neural networks through representation erasure. arXiv preprint arXiv:1612.08220 , 2016. · arXiv
  25. Xilai Li, Tianfu Wu, Xi Song, and Hamid Krim. AOGNets: Deep AND-OR grammar networks for visual recognition. arXiv preprint arXiv:1711.05847 , 2017. · arXiv
  26. Yen-Chen Lin, Ming-Yu Liu, Min Sun, and Jia-Bin Huang. Detecting adversarial attacks on neural network policies with visual foresight. arXiv preprint arXiv:1710.00814 , 2017. · arXiv
  27. Christos Louizos, Uri Shalit, Joris M. Mooij, David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. In Advances in Neural Information Processing Systems 30 (NIPS) , pages 6446-6456. 2017.
  28. Jianjun Lu, Shozo Tokinaga, and Yoshikazu Ikeda. Explanatory rule extraction based on the trained neural network and the genetic programming. Journal of the Operations Research Society of Japan (JORSJ) , 49(1):66-82, 2006.
  29. Urszula Markowska-Kaczmar and Paweglyph[suppress] l Wnuk-Lipi´ nski. Rule extraction from neural network by genetic algorithm with pareto optimization. Artificial Intelligence and Soft ComputingICAISC 2004 , pages 450-455, 2004.
  30. W. James Murdoch and Arthur Szlam. Automatic rule extraction from long short term memory networks. In International Conference on Learning Representations (ICLR) , 2017.
  31. W. James Murdoch, Peter J. Liu, and Bin Yu. Beyond word importance: Contextual decomposition to extract interactions from LSTMs. In International Conference on Learning Representations (ICLR) , 2018.
  32. Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill , 2017. Available online at: https://distill.pub/2017/feature-visualization.
  33. Chris Olah, Arvind Satyanarayan, Ian Johnson, Shan Carter, Ludwig Schubert, Katherine Ye, and Alexander Mordvintsev. The building blocks of interpretability. Distill , 2018. Available online at: https://distill.pub/2018/building-blocks.
  34. Rasmus Berg Palm, Ulrich Paquet, and Ole Winther. Recurrent relational networks for complex relational reasoning. arXiv preprint arXiv:1711.08028 , 2017. · arXiv
  35. Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security (ASIA CCS '17) , pages 506-519, 2017.
  36. Ambrish Rawat, Martin Wistuba, and Maria-Irina Nicolae. Adversarial phenomenon in the eyes of bayesian deep learning. arXiv preprint arXiv:1711.08244 , 2017. · arXiv
  37. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Nothing else matters: Model-agnostic explanations by identifying prediction invariance. In NIPS Workshop on Interpretable Machine Learning in Complex Systems , 2016a.
  38. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 'Why should I trust you?' Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '16) , pages 1135-1144, 2016b.
  39. Marko Robnik- ˇ Sikonja and Igor Kononenko. Explaining classifications for individual instances. IEEE Transactions on Knowledge and Data Engineering , 20(5):589-600, 2008.
  40. Adam Santoro, David Raposo, David G.T. Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. A simple neural network module for relational reasoning. arXiv preprint arXiv:1706.01427 , 2017. · arXiv
  41. Christin Seifert, Aisha Aamir, Aparna Balagopalan, Dhruv Jain, Abhinav Sharma, Sebastian Grottel, and Stefan Gumhold. Visualizations of deep neural networks in computer vision: A survey. In Transparent Data Mining for Big and Small Data , pages 123-144. Springer, 2017.
  42. Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In Proceedings of the 34th International Conference on Machine Learning (ICML) , volume 70 of Proceedings of Machine Learning Research (PMLR) , 2017.
  43. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 , 2013. · arXiv
  44. John von Neumann and Oskar Morgenstern. Theory of Games and Economic Behavior . Princeton University Press, Princeton, NJ, 3rd edition, 1953.
  45. Sandra Wachter, Brent Mittelstadt, and Luciano Floridi. Transparent, explainable, and accountable AI for robotics. Science Robotics , 2(6), 2017.
  46. Adrian Weller. Challenges for transparency. Workshop on Human Interpretability in Machine Learning - ICML 2017 , 2017.
  47. Tianfu Wu, Xilai Li, Xi Song, Wei Sun, Liang Dong, and Bo Li. Interpretable R-CNN. arXiv preprint arXiv:1711.05226 , 2017. · arXiv
  48. Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European Conference on Computer Vision (ECCV) , pages 818-833. Springer, 2014.
  49. Matthew D. Zeiler, Dilip Krishnan, Graham W. Taylor, and Rob Fergus. Deconvolutional networks. In 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2528-2535. IEEE, 2010.
  50. Jan Ruben Zilke, Eneldo Loza Menc´ ıa, and Frederik Janssen. DeepRED - Rule extraction from deep neural networks. In International Conference on Discovery Science (ICDS) , pages 457-473. Springer, 2016.
  51. Luisa M. Zintgraf, Taco S. Cohen, Tameem Adel, and Max Welling. Visualizing deep neural network decisions: Prediction difference analysis. In International Conference on Learning Representations (ICLR) , 2017.

Cited by

Sorted by citations

Authors
Ras, G., van Gerven, M., & Haselager, P.
Affiliations
Radboud University Nijmegen (NL)
Corresponding author
Ras, Gabriëlle
ORCID
Haselager, Pim
Venue
The Springer Series on Challenges in Machine Learning
Publisher
Springer International Publishing
Published
2018
Pages
19-36
ISSN
  • 2520-131X (ISSN-L)
  • 2520-1328
DOI
10.1007/978-3-319-98131-4_2
OpenAlex
W2791535658
Cited by
255
References
13
Citations by year
2026: 5 · 2025: 33 · 2024: 24 · 2023: 51 · 2022: 37 · 2021: 46 · 2020: 28 · 2019: 21 · 2018: 9
FWCI
38.92
Type
book chapter
Language
en
Open access
Licence
www.springer.com/tdm (tdm)
Locations
  • The Springer series on challenges in machine learning (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Radboud Repository (Radboud University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
explainable AI, explanation methods, deep neural network, user concerns, non-expert users, expert users, General Data Protection Regulation, visual explanations, algorithmic bias, fairness, data protection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review