The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Explainable AI Methods - A Brief Overview
Open accessAbstract
Explainable Artificial Intelligence (xAI) is an established field with a vibrant community that has developed a variety of very successful approaches to explain and interpret predictions of complex machine learning models such as deep neural networks. In this article, we briefly introduce a few selected methods and discuss them in a short, clear and concise way. The goal of this article is to give beginners, especially application engineers and data scientists, a quick overview of the state of the art in this current topic. The following 17 methods are covered in this chapter: LIME, Anchors, GraphLIME, LRP, DTD, PDA, TCAV, XGNN, SHAP, ASV, Break-Down, Shapley Flow, Textual Explanations of Visual Models, Integrated Gradients, Causal Models, Meaningful Perturbations, and X-NeSyL.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · cited 1×
“However, model-specific implementations exist for tree-based methods (random forest, xgboost etc.) or additive models [49].”
MethodReinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,414
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
Self-citationMethods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665
Self-citationAnchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1×
“Idea: The basic idea is that individual predictions of any black-box classification model are explained by finding a decision rule that sufficiently ”anchors” the prediction - hence the name ”anchors” [68].”
BackgroundPrinciples alone cannot guarantee ethical AI
Mittelstadt, B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0114-4 · Cited by 1,886 · cited 1×
“Increasing legal and data protection aspects, e.g., due to the new European General Data Protection Regulation (GDPR, in force since May 2018), complicate the use of black-box approaches, particularly in domains that affect human life, such as the medical field [56], [63], [73], [76].”
BackgroundExplaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477 · cited 2×
“Therefore, LIME is also often referred to as surrogate-based explanation technique [70].”
BackgroundSelf-citationStudent (1908) · Biometrika · doi:10.1093/biomet/6.1.1 · Cited by 1,363
Explaining nonlinear classification decisions with deep taylor decomposition
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357 · cited 9×
“Idea: The Deep Taylor Decomposition (DTD) method [60] is a propagationbased explanation technique, which explains decisions of a neural network by decomposition.”
BackgroundMethodInfluentialSelf-citationKim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2017) · arXiv · doi:10.48550/arxiv.1711.11279 · Cited by 1,240
Interpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097
Unmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K.-R. (2019) · Nature Communications · doi:10.1038/s41467-019-08987-4 · Cited by 1,047
Self-citationLayer-Wise Relevance Propagation: An Overview
Montavon, G., Binder, A., Lapuschkin, S., Samek, W., & Müller, K.-R. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_10 · Cited by 750
Self-citationA survey and critique of multiagent deep reinforcement learning
Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 671
Bengio, Y., Lecun, Y., & Hinton, G. (2021) · Communications of the ACM · doi:10.1145/3448250 · Cited by 661
The seven tools of causal inference, with reflections on machine learning
Pearl, J. (2019) · Communications of the ACM · doi:10.1145/3241036 · Cited by 659 · cited 1×
“Therefore, it will be important to measure the quality of explanations so that an xAI method achieves a certain level of causal understanding [65] for a user with effectiveness, efficiency and satisfaction in a given context of use”
MethodGenerating Visual Explanations
Hendricks, L. A., Akata, Z., Rohrbach, M., Donahue, J., Schiele, B., & Darrell, T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_1 · Cited by 534
O'Sullivan, S., Nevejans, N., Allen, C., Blyth, A., Leonard, S., Pagallo, U., Holzinger, K., Holzinger, A., Sajid, M. I., & Ashrafian, H. (2019) · The International Journal of Medical Robotics and Computer Assisted Surgery · doi:10.1002/rcs.1968 · Cited by 437 · cited 1×
“, due to the new European General Data Protection Regulation (GDPR, in force since May 2018), complicate the use of black-box approaches, particularly in domains that affect human life, such as the medical field [56], [63], [73], [76].”
BackgroundSelf-citationGraphLIME: Local Interpretable Model Explanations for Graph Neural Networks
Huang, Q., Yamada, M., Tian, Y., Singh, D., & Chang, Y. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3187455 · Cited by 424
Layer-Wise Relevance Propagation for Neural Networks with Local Renormalization Layers
Binder, A., Montavon, G., Lapuschkin, S., Müller, K.-R., & Samek, W. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-44781-0_8 · Cited by 418
Self-citationHolzinger, A., Carrington, A., & Müller, H. (2020) · KI - Künstliche Intelligenz · doi:10.1007/s13218-020-00636-z · Cited by 406
Self-citationInterpretable deep neural networks for single-trial EEG classification
Sturm, I., Lapuschkin, S., Samek, W., & Müller, K.-R. (2016) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2016.10.008 · Cited by 374
Self-citationExplaining Recurrent Neural Network Predictions in Sentiment Analysis
Arras, L., Montavon, G., Müller, K.-R., & Samek, W. (2017) · Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis · doi:10.18653/v1/w17-5221 · Cited by 361
Self-citationParameterized Explainer for Graph Neural Network
Luo, D., Cheng, W., Xu, D., Yu, W., Zong, B., Chen, H., & Zhang, X. (2020) · arXiv · doi:10.48550/arxiv.2011.04573 · Cited by 355
Holzinger, A., Malle, B., Saranti, A., & Pfeifer, B. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.01.008 · Cited by 352
Self-citationExplanatory Model Analysis: Explore, Explain and Examine Predictive Models
Biecek, P., & Burzykowski, T. (2021) · Chapman & Hall/CRC eBooks · doi:10.1201/9780429027192 · Cited by 344
Self-citationXGNN: Towards Model-Level Explanations of Graph Neural Networks
Yuan, H., Tang, J., Hu, X., & Ji, S. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403085 · Cited by 272
Pruning by explaining: A novel criterion for deep neural network pruning
Yeom, S.-K., Seegerer, P., Lapuschkin, S., Binder, A., Wiedemann, S., Müller, K.-R., & Samek, W. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.107899 · Cited by 263
Self-citationExplaining Classifications For Individual Instances
Robnik-Sikonja, M., & Kononenko, I. (2008) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.190734 · Cited by 254
Causes and Explanations: A Structural-Model Approach. Part II: Explanations
Halpern, J. Y., & Pearl, J. (2005) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axi148 · Cited by 241
Holzinger, A., Plass, M., Kickmeier-Rust, M., Holzinger, K., Crişan, G. C., Pintea, C.-M., & Palade, V. (2019) · Applied Intelligence · doi:10.1007/s10489-018-1361-5 · Cited by 228
Self-citationTowards Best Practice in Explaining Neural Network Decisions with LRP
Kohlbrenner, M., Bauer, A., Nakajima, S., Binder, A., Samek, W., & Lapuschkin, S. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206975 · Cited by 228
Self-citationGraph Neural Network: A Comprehensive Review on Non-Euclidean Space
Asif, N. A., Sarker, Y., Chakrabortty, R. K., Ryan, M. J., Ahamed, M. H., Saha, D. K., Badal, F. R., Das, S. K., Ali, M. F., Moyeen, S. I., et al. (2021) · IEEE Access · doi:10.1109/access.2021.3071274 · Cited by 228
General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models
Molnar, C., König, G., Herbinger, J., Freiesleben, T., Dandl, S., Scholbeck, C. A., Casalicchio, G., Grosse-Wentrup, M., & Bischl, B. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-04083-2_4 · Cited by 211
Self-citationFinding and removing Clever Hans: Using explanation methods to debug and improve deep models
Anders, C. J., Weber, L., Neumann, D., Samek, W., Müller, K., & Lapuschkin, S. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.07.015 · Cited by 189
Self-citationCLEVR-XAI: A benchmark dataset for the ground truth evaluation of neural network explanations
Arras, L., Osman, A., & Samek, W. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.008 · Cited by 179
Self-citationExplaining and Interpreting LSTMs
Arras, L., Arjona-Medina, J., Widrich, M., Montavon, G., Gillhofer, M., Müller, K.-R., Hochreiter, S., & Samek, W. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_11 · Cited by 135
Self-citationNeural-Symbolic Learning Systems
Garcez, A. D., Broda, K. B., & Gabbay, D. M. (2002) · Perspectives in neural computing · doi:10.1007/978-1-4471-0211-3 · Cited by 121
Explainability-based Backdoor Attacks Against Graph Neural Networks
Xu, J., Xue, M., & Picek, S. (2021) · ACM Workshop on Wireless Security and Machine Learning (WiseML) · doi:10.1145/3468218.3469046 · Cited by 120
Hedström, A., Weber, L., Bareeva, D., Krakowczyk, D., Motzkus, F., Samek, W., Lapuschkin, S., & Höhne, M. M. C. (2022) · arXiv · doi:10.48550/arxiv.2202.06861 · Cited by 116
Self-citationTowards explaining anomalies: A deep Taylor decomposition of one-class models
Kauffmann, J., Müller, K., & Montavon, G. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107198 · Cited by 113
Toward Human–AI Interfaces to Support Explainability and Causability in Medical AI
Holzinger, A., & Müller, H. (2021) · Computer · doi:10.1109/mc.2021.3092610 · Cited by 104
Self-citationMamalakis, A., Ebert‐Uphoff, I., & Barnes, E. A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-04083-2_16 · Cited by 89
The European Legal Framework for Medical AI
Schneeberger, D., Stöger, K., & Holzinger, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-57321-8_12 · Cited by 85
Self-citationGlobal and Local Interpretability for Cardiac MRI Classification
Clough, J. R., Oksuz, I., Puyol-Antón, E., Ruijsink, B., King, A. P., & Schnabel, J. A. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-32251-9_72 · Cited by 73
Explaining Image Classifiers by Removing Input Features Using Generative Models
Agarwal, C., & Nguyen, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69544-6_7 · Cited by 70
Visualization of neural networks using saliency maps
Mørch, N. J. S., Kjems, U., Hansen, L. K., Svarer, C., Law, I. G., Lautrup, B. E., Strother, S. C., & Rehm, K. (2002) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/icnn.1995.488997 · Cited by 58
Explainable Reinforcement Learning through a Causal Lens
Madumal, P., Miller, T., Sonenberg, L., & Vetere, F. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5631 · Cited by 58
Explain and improve: LRP-inference fine-tuning for image captioning models
Sun, J., Lapuschkin, S., Samek, W., & Binder, A. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.07.008 · Cited by 57
Self-citationLegal aspects of data cleansing in medical AI
Stöger, K., Schneeberger, D. M., Kieseberg, P., & Holzinger, A. (2021) · Computer law & security review · doi:10.1016/j.clsr.2021.105587 · Cited by 51
Self-citationExplaining the Predictions of Unsupervised Learning Models
Montavon, G., Kauffmann, J., Samek, W., & Müller, K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-04083-2_7 · Cited by 41
Self-citationDo Not Trust Additive Explanations
Gosiewska, A., & Biecek, P. (2019) · arXiv · doi:10.48550/arxiv.1903.11420 · Cited by 34
Self-citationA Rate-Distortion Framework for Explaining Black-Box Model Decisions
Kolek, S., Nguyễn, Đ. A., Levie, R., Bruna, J., & Gitta Kutyniok, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-04083-2_6 · Cited by 20
Insights into Learning Competence Through Probabilistic Graphical Models
Saranti, A., Taraghi, B., Ebner, M., & Holzinger, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29726-8_16 · Cited by 19
Self-citationECQ$$^{\text {x}}$$: Explainability-Driven Quantization for Low-Bit and Sparse DNNs
Becking, D., Dreyer, M., Samek, W., Müller, K., & Lapuschkin, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-04083-2_14 · Cited by 17
Self-citationDíaz-Rodríguez, N., Lamas, A., Sanchez, J., Franchi, G., Donadello, I., Tabik, S., Filliat, D., Cruz, P., Montes, R., & Herrera, F. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.09.022 · Cited by 2
Cited by
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME
Salih, A. M., Raisi‐Estabragh, Z., Galazzo, I. B., Radeva, P., Petersen, S. E., Lekadir, K., & Menegaz, G. (2025) · Advanced Intelligent Systems · doi:10.1002/aisy.202400304 · Cited by 897
Albahri, A. S., Duhaim, A. M., Fadhel, M. A., Alnoor, A., Baqer, N. S., Alzubaidi, L., Albahri, O. S., Alamoodi, A. H., Bai, J., Salhi, A., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.03.008 · Cited by 716
AI for life: Trends in artificial intelligence for biotechnology
Holzinger, A., Keiblinger, K., Holub, P., Zatloukal, K., & Müller, H. (2023) · New Biotechnology · doi:10.1016/j.nbt.2023.02.001 · Cited by 438
Self-citationApplying large language models and chain-of-thought for automatic scoring
Lee, G.-G., Latif, E., Wu, X., Liu, N., & Zhai, X. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100213 · Cited by 186
Metadata
- Authors
- Holzinger, A., Saranti, A., Molnar, C., Biecek, P., & Samek, W.
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2022
- Pages
- 13-38
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W4226108798
- Cited by
- 379
- References
- 63
- FWCI
- 47.93
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.