Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 223 of the 230 references it cites were found in a registry or an index.

Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications

Open access
Samek, Wojciech, Montavon, Gregoire, Lapuschkin, Sebastian, Anders, Christopher J., Muller, Klaus-Robert·2021·Proceedings of the IEEE

Abstract

With the broader and highly successful usage of machine learning (ML) in industry and the sciences, there has been a growing demand for explainable artificial intelligence (XAI). Interpretability and explanation methods for gaining a better understanding of the problem-solving abilities and strategies of nonlinear ML, in particular, deep neural networks, are, therefore, receiving increased attention. In this work, we aim to: 1) provide a timely overview of this active emerging field, with a focus on “post hoc” explanations, and explain its theoretical foundations; 2) put interpretability algorithms to a test both from a theory and comparative evaluation perspective using extensive simulations; 3) outline best practice aspects, i.e., how to best include interpretation methods into the standard usage of ML; and 4) demonstrate successful usage of XAI in a representative selection of application scenarios. Finally, we discuss challenges and possible future directions of this exciting foundational field of ML.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/jproc.2021.3060483

Cites

223 of 223 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 230 references

Research works printed but not linked: 13

  1. Arras, 2020, Ground truth evaluation of neural network explanations with CLEVR-XAI, arXiv:2003.07258 · arXiv
  2. 10.1146/annurev.neuro.26.041002.131047 · DOI · Paper page
  3. Chollet, 2015, Keras
  4. 10.1109/ICCV.2019.00304 · DOI
  5. Kokhlikyan, 2019, Pytorch Captum
  6. Lapuschkin, 2016, The layer-wise relevance propagation toolbox for artificial neural networks, J. Mach. Learn. Res., 17, 1
  7. 10.48550/arXiv.1312.4400 · DOI · Paper page
  8. 10.5555/2969033.2969153 · doi:10.5555/2969033.2969153
  9. 10.5555/3157382.3157477 · doi:10.5555/3157382.3157477
  10. Okuta, CuPy: A numpy-compatible library for NVIDIA GPU calculations, 151
  11. Oliphant, 2006, A Guide to NumPy, 1
  12. 10.48550/ARXIV.1404.7828 · DOI
  13. 10.5555/3045118.3045336 · doi:10.5555/3045118.3045336

Cited by

See all 1,477

Metadata

Authors
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R.
Affiliations
  • Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
  • BIFOLD✔Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
  • Berlin Institute for the Foundations of Learning and Data (DE)
Venue
Proceedings of the IEEE
Published
2021-03
Volume
109
Issue
3
Pages
247-278
ISSN
  • 0018-9219 (ISSN-L)
  • 1558-2256
OpenAlex
W3132191748
Cited by
1,477
References
230
Citations by year
2026: 305 · 2025: 343 · 2024: 326 · 2023: 273 · 2022: 152 · 2021: 74
FWCI
72.25
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the IEEE (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion, other-oa)
  • Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Institute of Information & Communications Technology Planning & Evaluation (IITP) grants funded by the Korea Government (2017-0-00451, 2019-0-00079)
  • German Ministry for Education and Research (01IS14013A-E, 01GQ1115, 01GQ0850, 01IS18025A, 01IS18037A)
  • German Research Foundation (Math+, EXC 2046/1, 390685689)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
explainable AI, interpretability, deep neural network, post-hoc explanation, interpretability evaluation, nonlinear machine learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications. https://app.cochat.ai/papers/10.1109/jproc.2021.3060483
The paper
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021). Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications. Proceedings of the IEEE. https://doi.org/10.1109/jproc.2021.3060483

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.