The DOI matches its Crossref record; 47 of the 51 references it cites were found in a registry or an index.
Evaluating the Visualization of What a Deep Neural Network Has Learned
Abstract
Deep neural networks (DNNs) have demonstrated impressive performance in complex machine learning tasks such as image classification or speech recognition. However, due to their multilayer nonlinear structure, they are not transparent, i.e., it is hard to grasp what makes them arrive at a particular classification or recognition decision, given a new unseen data sample. Recently, several approaches have been proposed enabling one to understand and interpret the reasoning embodied in a DNN for a single test image. These methods quantify the “importance” of individual pixels with respect to the classification decision and allow a visualization in terms of a heatmap in pixel/input space. While the usefulness of heatmaps can be judged subjectively by a human, an objective quality measure is missing. In this paper, we present a general methodology based on region perturbation for evaluating ordered collections of pixels such as heatmaps. We compare heatmaps computed by three different methods on the SUN397, ILSVRC2012, and MIT Places data sets. Our main result is that the recently proposed layer-wise relevance propagation algorithm qualitatively and quantitatively provides a better explanation of what made a DNN arrive at a particular classification decision than the sensitivity-based approach or the deconvolution method. We provide theoretical arguments to explain this result and discuss its practical implications. Finally, we investigate the use of heatmaps for unsupervised assessment of the neural network performance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
An introduction to variable and feature selection
GuyonIsabelle, & ElisseeffAndré (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944968 · Cited by 8,354
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Research works printed but not linked: 1
- jia, 2013, Caffe An Open Source Convolutional Architecture for Fast Feature Embedding
Cited by
Sorted by citations
Items 501–600 of 1,000
Discovering Molecular Functional Groups Using Graph Convolutional Neural NetworksPreprint
Pope, P., Kolouri, S., Rostrami, M., Martin, C. E., & Hoffmann, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00265 · Cited by 23
Revisiting Sanity Checks for Saliency MapsPreprint
Yona, G., & Greenfeld, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14297 · Cited by 14
CoughLIME: Sonified Explanations for the Predictions of COVID-19 Cough Classifiers
Wullenweber, A., Akman, A., & Schuller, B. W. (2022) · 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) · doi:10.1109/embc48229.2022.9871291 · Cited by 13
Interpretability of Machine Learning Methods Applied to Neuroimaging
Thibeau–Sutre, E., Collin, S., Burgos, N., & Colliot, O. (2023) · Neuromethods · doi:10.1007/978-1-0716-3195-9_22 · Cited by 13
Importance-Aware Information Bottleneck Learning Paradigm for Lip Reading
Sheng, C., Liu, L., Deng, W., Bai, L., Liu, Z., Lao, S., Kuang, G., & Pietikäinen, M. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3210761 · Cited by 13
Towards Explainable Occupational Fraud Detection
Tritscher, J., Schlör, D., Gwinner, F., Krause, A., & Andreas Hotho, A. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-23633-4_7 · Cited by 13
Global geometry of chemical graph neural network representations in terms of chemical moieties
El-Samman, A. M., Incé A. Husain, Huynh, M., Castro, S. D., Morton, B., Baerdemacker, S. D., & Baerdemacker, S. D. (2024) · Digital Discovery · doi:10.1039/d3dd00200d · Cited by 14
Lamprou, V., Kallipolitis, A., & Maglogiannis, I. G. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108238 · Cited by 15
A Fresh Look at Sanity Checks for Saliency Maps
Hedström, A., Weber, L., Lapuschkin, S., & Höhne, M. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-63787-2_21 · Cited by 13
Self-citationStructuring Neural Networks for More Explainable Predictions
Rieger, L., Chormai, P., Montavon, G., Hansen, L. K., & Müller, K. (2018) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-98131-4_5 · Cited by 13
Self-citationEnhanced Rotational Invariant Convolutional Neural Network for Supernovae Detection
Reyes, E., Estévez, P. A., Reyes, I., Cabrera-Vives, G., Huijse, P., Carrasco-Davis, R., & Förster, F. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489627 · Cited by 13
Enhancing Representation of Deep Features for Sensor-Based Activity Recognition
Li, X., Nie, L., Si, X., Ding, R., & Zhan, D. (2020) · Mobile Networks and Applications · doi:10.1007/s11036-020-01689-y · Cited by 13
Xu, D., Zhou, H., Jiang, X., Li, S., Zhang, Q., Baker, J. S., & Gu, Y. (2022) · Frontiers in Veterinary Science · doi:10.3389/fvets.2022.836043 · Cited by 13
ExSum: From Local Explanations to Model Understanding
Zhou, Y., Ribeiro, M. T., & Shah, J. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.392 · Cited by 18
Logic Traps in Evaluating Attribution Scores
Ju, Y., Zhang, Y., Zhao, Y., Jiang, Z., Liu, K., & Zhao, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.407 · Cited by 23
Iterative deblending using unsupervised learning with double-deep neural networks
Wang, K., Hu, T., & Wang, S. (2023) · Geophysics · doi:10.1190/geo2022-0299.1 · Cited by 13
Explaining What a Neural Network has Learned: Toward Transparent Classification
Amarasinghe, K., & Manic, M. (2019) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2019.8858899 · Cited by 15
Approximate Inverse Reinforcement Learning from Vision-based Imitation Learning
Lee, K., Vlahov, B., Gibson, J., Rehg, J. M., & Theodorou, E. A. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560916 · Cited by 13
Fast multi-resolution occlusion: a method for explaining and understanding deep neural networks
Behzadi-Khormouji, H., & Rostami, H. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01946-3 · Cited by 13
Measuring and Improving Faithfulness of Attention in Neural Machine Translation
Moradi, P., Kambhatla, N., & Sarkar, A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.243 · Cited by 14
Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks
Chen, H., Feng, S., Ganhotra, J., Wan, H., Gunasekara, C., Joshi, S., & Ji, Y. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.306 · Cited by 15
Towards Ground Truth Evaluation of Visual ExplanationsPreprint
Osman, A., Arras, L., & Samek, W. (2020) · arXiv (Cornell University) · Cited by 13
Self-citationUsing Neural Networks to Predict Micro-Spatial Economic Growth
Khachiyan, A., Thomas, A. H., Zhou, H., Hanson, G., Cloninger, A., Rosing, T., & Khandelwal, A. K. (2021) · National Bureau of Economic Research · doi:10.3386/w29569 · Cited by 13
Jin, W., Li, X., & Hamarneh, G. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.06487 · Cited by 13
Eisbach, S., Mai, O., & Hertel, G. (2024) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2024.101077 · Cited by 12
Towards Explainable Deep Learning in Computational Neuroscience: Visual and Clinical Applications
Mehmood, A., Mehmood, F., & Kim, J. (2025) · Mathematics · doi:10.3390/math13203286 · Cited by 14
Caffe2Unity: Immersive Visualization and Interpretation of Deep Neural Networks
Aamir, A., Tamošiūnaitė, M., & Wörgötter, F. (2021) · Electronics · doi:10.3390/electronics11010083 · Cited by 12
Explaining COVID-19 diagnosis with Taylor decompositions
Hassan, M. M., AlQahtani, S. A., Alelaiwi, A. A., & Papa, J. P. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-08021-7 · Cited by 13
Ellis, C. A., Miller, R. L., & Calhoun, V. D. (2023) · Neuroimage Reports · doi:10.1016/j.ynirp.2023.100186 · Cited by 12
Schnake, T., Jafari, F. R., Lederer, J., Xiong, P., Nakajima, S., Gugler, S., Montavon, G., & Müller, K. (2025) · Information Fusion · doi:10.1016/j.inffus.2024.102923 · Cited by 18
Self-citationDeep Neural Model Inspection and Comparison via Functional Neuron Pathways
Fiacco, J., Choudhary, S., & Rosé, C. P. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1575 · Cited by 12
Rieckmann, A., Dworzyński, P., Arras, L., Lapuschkin, S., Samek, W., Arah, O. A., Rod, N. H., & Ekstrøm, C. T. (2022) · International Journal of Epidemiology · doi:10.1093/ije/dyac078 · Cited by 12
Self-citationBenchmarking explanation methods for mental state decoding with deep learning models
Thomas, A. W., Ré, C., & Poldrack, R. A. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120109 · Cited by 17
Strategies to Exploit XAI to Improve Classification Systems
Apicella, A., Lorenzo, L. D., Isgrò, F., Pollastro, A., & Prevete, R. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_9 · Cited by 15
Nonlinear VW-SAE Based Deep Learning for Quality-Related Feature Learning and Soft Sensor Modeling
Yuan, X., Ou, C., Wang, Y., & Yang, C. (2018) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon.2018.8591240 · Cited by 12
Yue, X., Wang, J., & Huang, W. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.076 · Cited by 12
Chang, A. C. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-823337-5.00005-6 · Cited by 12
Semantic Bottlenecks: Quantifying and Improving Inspectability of Deep Representations
Losch, M., Fritz, M., & Schiele, B. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01498-0 · Cited by 13
Flexible Instance-Specific Rationalization of NLP Models
Chrysostomou, G., & Αλέτρας, Ν. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21298 · Cited by 14
Clothing and People - A Social Signal Processing Perspective
Aghaei, M., Parezzan, F., Dimiccoli, M., Radeva, P. I., & Cristani, M. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.71 · Cited by 12
Gradient-based feature-attribution explainability methods for spiking neural networks
Bitar, A., Rosales, R., & Paulitsch, M. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1153999 · Cited by 17
Toward Learning Model-Agnostic Explanations for Deep Learning-Based Signal Modulation Classifiers
Tian, Y., Xu, D., Tong, E., Sun, R., Chen, K., Li, Y., Baker, T., Niu, W., & Liu, J. (2024) · IEEE Transactions on Reliability · doi:10.1109/tr.2024.3367780 · Cited by 11
Explaining predictive uncertainty by exposing second-order effects
Bley, F., Lapuschkin, S., Samek, W., & Montavon, G. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111171 · Cited by 11
Self-citationGao, Y., Zhu, Y., & Yu, Q. (2022) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3524842.3528472 · Cited by 11
Vardhan, A., Makhnevich, A., Omprakash, P., Hirschorn, D. S., Barish, M. A., Cohen, S. L., & Zanos, T. P. (2023) · Bioelectronic Medicine · doi:10.1186/s42234-022-00103-0 · Cited by 12
Georgievskaya, A., Tlyachev, T., Kiselev, K., Hillebrand, G. G., Chekanov, K., Danko, D., Golodyayev, A., & Majmudar, G. (2024) · Experimental Dermatology · doi:10.1111/exd.15045 · Cited by 12
Evaluating Input Perturbation Methods for Interpreting CNNs and Saliency Map Comparison
Brunke, L., Agrawal, P., & George, N. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-66415-2_8 · Cited by 10
Improving Feature Attribution through Input-specific Network PruningPreprint
Khakzar, A., Baselizadeh, S., Khanduja, S., Rupprecht, C., Kim, S. T., & Navab, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.11081 · Cited by 11
An Explainable Deep Learning Approach for Multimodal Electrophysiology ClassificationPreprint
Ellis, C. A., Carbajal, D. A., Zhang, R., Miller, R. L., Calhoun, V. D., & Wang, M. D. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.05.12.443594 · Cited by 11
Waefler, T., & Schmid, U. (2021) · doi:10.20378/irb-49775 · Cited by 11
Kanda, D., Kawai, S., & Nobuhara, H. (2020) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2020.p0676 · Cited by 10
Interpretability of deep neural networks used for the diagnosis of Alzheimer's disease
Pohl, T., Jakab, M., & Benešová, W. (2021) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.22657 · Cited by 11
Xu, D., Zhou, H., Quan, W., Ugbolue, U. C., Fekete, G., & Gu, Y. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e26052 · Cited by 11
Coarse-to-Fine: A hierarchical DNN inference framework for edge computing
Zhang, Z., Zhang, Y., Bao, W., Li, C., & Yuan, D. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.03.009 · Cited by 10
Artificial Intelligence in Neuro Degenerative Diseases: Opportunities and Challenges
Rajaraman, S. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-53148-4_8 · Cited by 12
Forensic research of satellite images forgery: a comprehensive survey
Ding, X., Nie, Y., Yao, J., Tang, J., & Lang, Y. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10909-w · Cited by 10
Jeon, D., Han, S., Jung, S. K., & Moon, J. (2025) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2025.141959 · Cited by 11
Explaining 3D Semantic Segmentation Through Generative AI ‐Based Counterfactuals
Rozajac, D., Lukač, N., Schweng, S., Gollob, C., Nothdurft, A., Stampfer, K., Ser, J. D., & Holzinger, A. (2025) · Expert Systems · doi:10.1111/exsy.70163 · Cited by 11
Spatio-Temporal Perturbations for Video Attribution
Li, Z., Wang, W., Li, Z., Huang, Y., & Sato, Y. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3081761 · Cited by 11
Explainable gait recognition with prototyping encoder–decoder
Moon, J., Shin, Y.-M., Park, J.-D., Minaya, N. H., Shin, W.-Y., & Choi, S. (2022) · PLoS ONE · doi:10.1371/journal.pone.0264783 · Cited by 11
Gandomkar, Z., Khong, P. L., Punch, A., & Lewis, S. J. (2022) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-022-00631-w · Cited by 13
LRP2A: Layer-wise Relevance Propagation based Adversarial attacking for Graph Neural Networks
Liu, L., Du, Y. P., Wang, Y., Cheung, W. K., Zhang, Y., Liu, Q., & Wang, G. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109830 · Cited by 10
Pixel-Level Explanation of Multiple Instance Learning Models in Biomedical Single Cell Images
Sadafi, A., Adonkina, O., Khakzar, A., Lienemann, P., Hehr, M., Rueckert, D., Navab, N., & Marr, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-34048-2_14 · Cited by 10
HEAR4Health: a blueprint for making computer audition a staple of modern healthcare
Triantafyllopoulos, A., Kathan, A., Baird, A., Christ, L., Gebhard, A., Gerczuk, M., Karas, V., Hübner, T., Jing, X., Liu, S., et al. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2023.1196079 · Cited by 12
A Multi-Phased Co-design of an Interactive Analytics System for MOBA Game Occurrences
Li, Q., Wu, Z., Xu, P. F., Qu, H., & Ma, X. (2018) · Designing Interactive Systems Conference · doi:10.1145/3196709.3196771 · Cited by 10
A Life Cycle for Modeling Biology at Different Scales
Harline, K., Martínez‐Gómez, J., Specht, C. D., & Roeder, A. (2021) · Frontiers in Plant Science · doi:10.3389/fpls.2021.710590 · Cited by 10
Yoshimura, N., Maekawa, T., & Hara, T. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533888 · Cited by 10
Gong, K., Dai, Q., Wang, J., Zheng, Y., Shi, T., Yu, J., Chen, J., Huang, S., & Wang, Z. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1118340 · Cited by 10
Ensembling Visual Explanations
Rajani, N. F., & Mooney, R. J. (2018) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-98131-4_7 · Cited by 10
Software and Application Patterns for Explanation Methods
Alber, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_22 · Cited by 12
Khakzar, A., Zhang, Y., Mansour, W., Cai, Y., Li, Y., Zhang, Y., Kim, S. T., & Navab, N. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87199-4_37 · Cited by 10
Practical Machine Learning Safety: A Survey and PrimerPreprint
Mohseni, S., Wang, H., Yu, Z., Xiao, C., Wang, Z., & Yadawa, J. (2021) · arXiv (Cornell University) · Cited by 10
Robust Models Are More Interpretable Because Attributions Look NormalPreprint
Wang, Z., Fredrikson, M., & Datta, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11257 · Cited by 14
Ellis, C. A., Sattiraju, A., Miller, R. L., & Calhoun, V. D. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.03.20.533467 · Cited by 10
Local Path Integration for Attribution
Yang, P., Akhtar, N., Wen, Z., & Mian, A. S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25422 · Cited by 9
Deep Attentive Model for Knowledge Tracing
Wang, X., Chen, L., & Zhang, M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26214 · Cited by 11
Role of locality, fidelity and symmetry regularization in learning explainable representations
Ronco, M., & Camps‐Valls, G. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126884 · Cited by 10
Explainable AI-Based Interface System for Weather Forecasting Model
Kim, S., Choi, Y., Choi, Y., Lee, S., Stitsyuk, A., Park, M.-K., Jeong, S., Baek, Y.-S., & Choi, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-48057-7_7 · Cited by 9
Deep reinforcement learning imbalanced credit risk of SMEs in supply chain finance
Zhang, W., Yan, S., Li, J., Peng, R., & Tian, X. (2024) · Annals of Operations Research · doi:10.1007/s10479-024-05921-w · Cited by 9
Ribeiro, E., Cárdenas, D. A. C., Dias, F. M., Krieger, J. E., & Gutierrez, M. A. (2024) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztae045 · Cited by 9
Awal, M. A., & Roy, C. K. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112159 · Cited by 9
Propagating Transparency: A Deep Dive into the Interpretability of Neural Networks
Somani, A., Horsch, A., Bopardikar, A. S., & Prasad, D. K. (2024) · Nordic Machine Intelligence · doi:10.5617/nmi.10755 · Cited by 9
Suh, J., Kim, J., Kwon, S., Jung, E., Ahn, H., Lee, K.-Y., Choi, E., & Rhee, W. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109088 · Cited by 9
Ganguly, B., Dasgupta, R., & Dey, D. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110481 · Cited by 9
Learning Propagation Rules for Attribution Map Generation
Yang, Y., Qiu, J., Song, M., Tao, D., & Wang, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_40 · Cited by 9
An Interpretable Deep Learning Approach for Biomarker Detection in LC-MS Proteomics Data
Iravani, S., & Conrad, T. (2022) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2022.3141656 · Cited by 10
Optimisation of Deep Learning Small-Object Detectors with Novel Explainable Verification
Mohamed, E., Sirlantzis, K., Howells, G., & Hoque, S. (2022) · Sensors · doi:10.3390/s22155596 · Cited by 9
Using Neural Networks to Predict Micro-Spatial Economic Growth
Khachiyan, A., Thomas, A. H., Zhou, H., Hanson, G., Cloninger, A., Rosing, T., & Khandelwal, A. K. (2021) · American Economic Review Insights · doi:10.1257/aeri.20210422 · Cited by 23
RETRACTED ARTICLE: Robust adversarial uncertainty quantification for deep learning fine-tuning
Ahmed, U., & Lin, J. C. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-023-05087-5 · Cited by 10
RetractedLocal Interpretation of Transformer Based on Linear Decomposition
Yang, S., Huang, S., Zou, W., Zhang, J., Dai, X., & Chen, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.572 · Cited by 11
Reliability of Deep Neural Networks for an End-to-End Imitation Learning-Based Lane Keeping
Liu, S., & Müller, S. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3299229 · Cited by 9
Qi, R., Zheng, Y., Yang, Y., Cao, C. C., & Hsiao, J. H. (2024) · British Journal of Psychology · doi:10.1111/bjop.12714 · Cited by 19
Explainable data-driven Q-learning control for a class of discrete-time linear autonomous systems
Perrusquía, A., Zou, M., & Guo, W. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121283 · Cited by 9
What Sketch Explainability Really Means for Downstream Tasks?
Bandyopadhyay, H., Chowdhury, P. N., Bhunia, A. K., Sain, A., Xiang, T., & Song, Y.-Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01046 · Cited by 9
Eberle, O., Büttner, J., El-Hajj, H., Montavon, G., Müller, K., & Valleriani, M. (2024) · Science Advances · doi:10.1126/sciadv.adj1719 · Cited by 20
Self-citationERIC: Extracting Relations Inferred from Convolutions
Townsend, J. A., Kasioumis, T., & Inakoshi, H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69535-4_13 · Cited by 9
Enhancing Deep Neural Network Saliency Visualizations With Gradual Extrapolation
Szandała, T. (2021) · IEEE Access · doi:10.1109/access.2021.3093824 · Cited by 9
Cross-model consensus of explanations and beyond for image classification models: an empirical study
Li, X., Xiong, H., Huang, S., Ji, S., & Dou, D. (2023) · Machine Learning · doi:10.1007/s10994-023-06312-1 · Cited by 9
Objective criteria for explanations of machine learning models
Yeh, C., & Ravikumar, P. (2021) · Applied AI Letters · doi:10.1002/ail2.57 · Cited by 11
A Gradient-based Approach for Explaining Multimodal Deep Learning Classifiers
Ellis, C. A., Zhang, R., Calhoun, V. D., Carbajal, D. A., Miller, R. L., & Wang, M. D. (2021) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe52308.2021.9635460 · Cited by 9
Metadata
- Authors
- Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R.
- Affiliations
- Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
- Singapore University of Technology and Design (SG)
- Technische Universität Berlin, Berlin, Germany
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2017-11
- Volume
- 28
- Issue
- 11
- Pages
- 2660-2673
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2240067561
- Cited by
- 1,164
- References
- 51
- Citations by year
- 2026: 120 · 2025: 105 · 2024: 139 · 2023: 163 · 2022: 147 · 2021: 171 · 2020: 123 · 2019: 92 · 2018: 66 · 2017: 26
- FWCI
- 53.43
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- Funders
- Brain Korea 21 Plus Program through the National Research Foundation of Korea Funded by the Ministry of Education
- DFG (MU 987/17-1)
- German Ministry for Education and Research as Berlin Big Data Center (01IS14013A)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- deep neural network interpretability, layer-wise relevance propagation, deconvolution, explainable AI, image classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review