The DOI matches its Crossref record; 47 of the 51 references it cites were found in a registry or an index.
Evaluating the Visualization of What a Deep Neural Network Has Learned
Abstract
Deep neural networks (DNNs) have demonstrated impressive performance in complex machine learning tasks such as image classification or speech recognition. However, due to their multilayer nonlinear structure, they are not transparent, i.e., it is hard to grasp what makes them arrive at a particular classification or recognition decision, given a new unseen data sample. Recently, several approaches have been proposed enabling one to understand and interpret the reasoning embodied in a DNN for a single test image. These methods quantify the “importance” of individual pixels with respect to the classification decision and allow a visualization in terms of a heatmap in pixel/input space. While the usefulness of heatmaps can be judged subjectively by a human, an objective quality measure is missing. In this paper, we present a general methodology based on region perturbation for evaluating ordered collections of pixels such as heatmaps. We compare heatmaps computed by three different methods on the SUN397, ILSVRC2012, and MIT Places data sets. Our main result is that the recently proposed layer-wise relevance propagation algorithm qualitatively and quantitatively provides a better explanation of what made a DNN arrive at a particular classification decision than the sensitivity-based approach or the deconvolution method. We provide theoretical arguments to explain this result and discuss its practical implications. Finally, we investigate the use of heatmaps for unsupervised assessment of the neural network performance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
An introduction to variable and feature selection
GuyonIsabelle, & ElisseeffAndré (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944968 · Cited by 8,354
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Flagged first, then by citations
4 of the 51 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- jia, 2013, Caffe An Open Source Convolutional Architecture for Fast Feature Embedding
Cited by
Sorted by citations
Items 101–200 of 1,000
On Completeness-aware Concept-Based Explanations in Deep Neural NetworksPreprint
Yeh, C., Kim, B., Arık, S. Ö., Li, C., Pfister, T., & Ravikumar, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.07969 · Cited by 144
Xu, D., Quan, W., Zhou, H., Sun, D., Baker, J. S., & Gu, Y. (2022) · Scientific Reports · doi:10.1038/s41598-022-07054-1 · Cited by 146
Explainable Artificial Intelligence Methods in Combating Pandemics: A Systematic Review
Giuste, F., Shi, W., Zhu, Y., Naren, T., Isgut, M., Sha, Y., Li, T., Gupte, M., & Wang, M. D. (2022) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2022.3185953 · Cited by 145
Evaluation of post-hoc interpretability methods in time-series classification
Turbé, H., Bjelogrlic, M., Lovis, C., & Mengaldo, G. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00620-w · Cited by 130
Yuan, X., Ou, C., Wang, Y., Yang, C., & Gui, W. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.11.107 · Cited by 128
Towards Faithful Model Explanation in NLP: A Survey
Lyu, Q., Apidianaki, M., & Callison-Burch, C. (2024) · Computational Linguistics · doi:10.1162/coli_a_00511 · Cited by 196
Fooling Neural Network Interpretations via Adversarial Model ManipulationPreprint
Heo, J., Joo, S., & Moon, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02041 · Cited by 137
Interpreting and Explaining Deep Neural Networks for Classification of Audio SignalsPreprint
Becker, S., Ackermann, M. R., Lapuschkin, S., Müller, K., & Samek, W. (2018) · arXiv (Cornell University) · Cited by 124
Self-citationToward Explainable Artificial Intelligence for Regression Models: A methodological perspective
Letzgus, S., Wagner, P., Lederer, J., Samek, W., Müller, K., & Montavon, G. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2022.3153277 · Cited by 139
Self-citationDetecting Linguistic Characteristics of Alzheimer’s Dementia by Interpreting Neural Models
Karlekar, S., Niu, T., & Bansal, M. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2110 · Cited by 125
Finding and removing Clever Hans: Using explanation methods to debug and improve deep models
Anders, C. J., Weber, L., Neumann, D., Samek, W., Müller, K., & Lapuschkin, S. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.07.015 · Cited by 145
Self-citationDeepClue: Visual Interpretation of Text-Based Deep Stock Prediction
Shi, L., Teng, Z., Wang, L., Zhang, Y., & Binder, A. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2854193 · Cited by 124
Self-citationRestricting the Flow: Information Bottlenecks for AttributionPreprint
Schulz, K., Sixt, L., Tombari, F., & Landgraf, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.00396 · Cited by 154
Multi-scale deep feature fusion for automated classification of macular pathologies from OCT images
Das, V., Dandapat, S., & Bora, P. K. (2019) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2019.101605 · Cited by 117
Robust Explainability: A tutorial on gradient-based attribution methods for deep neural networks
Nielsen, I. E., Dera, D., Rasool, G., Ramachandran, R. P., & Bouaynaya, N. C. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2022.3142719 · Cited by 135
Explainable Artificial Intelligence and Cardiac Imaging: Toward More Interpretable Models
Salih, A. M., Galazzo, I. B., Gkontra, P., Lee, A. M., Lekadir, K., Raisi‐Estabragh, Z., & Petersen, S. E. (2023) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.122.014519 · Cited by 124
A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based VisualizationsPreprint
Nie, W., Zhang, Y., & Patel, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07039 · Cited by 115
From Clustering to Cluster Explanations via Neural Networks
Kauffmann, J., Esders, M., Ruff, L., Montavon, G., Samek, W., & Müller, K. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3185901 · Cited by 143
Self-citationMachine learning in cardiac CT: Basic concepts and contemporary data
Singh, G., Alaref, S. J., van Assen, M., Kim, T. S., van Rosendael, A., Kolli, K. K., Dwivedi, A., Maliakal, G., Pandey, M. K., Wang, J., et al. (2018) · Journal of cardiovascular computed tomography · doi:10.1016/j.jcct.2018.04.010 · Cited by 113
An analysis of explainability methods for convolutional neural networks
Vonderhaar, L., Elvira, T., & Ochoa, O. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105606 · Cited by 123
XAI Systems Evaluation: A Review of Human and Computer-Centred Methods
Lopes, P., Silva, E., Braga, C., Oliveira, T., & Rosado, L. (2022) · Applied Sciences · doi:10.3390/app12199423 · Cited by 112
Transferred Deep Learning-Based Change Detection in Remote Sensing Images
Yang, M., Jiao, L., Liu, F., Hou, B., & Yang, S. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2909781 · Cited by 111
Do Feature Attribution Methods Correctly Attribute Features?
Zhou, Y., Booth, S., Ribeiro, M. T., & Shah, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i9.21196 · Cited by 132
Explainable AI for Medical Data: Current Methods, Limitations, and Future Directions
Hossain, M. I., Zamzmi, G., Mouton, P. R., Salekin, M. S., Sun, Y., & Goldgof, D. B. (2023) · ACM Computing Surveys · doi:10.1145/3637487 · Cited by 135
Benchmarking Deep Learning Interpretability in Time Series PredictionsPreprint
Ismail, A. A., Gunady, M. K., Bravo, H. C., & Feizi, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13924 · Cited by 132
Nourani, M., Kabir, S., Mohseni, S., & Ragan, E. D. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5284 · Cited by 110
Explaining the Black-box Smoothly- A Counterfactual Approach
Singla, S., Eslami, M., Pollack, B., Wallace, S., & Batmanghelich, K. N. (2021) · Medical Image Analysis · doi:10.1016/j.media.2022.102721 · Cited by 126
Anik, A. I., & Bunt, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445736 · Cited by 103
Neural network attribution methods for problems in geoscience: A novel synthetic benchmark dataset
Mamalakis, A., Ebert‐Uphoff, I., & Barnes, E. A. (2022) · Environmental Data Science · doi:10.1017/eds.2022.7 · Cited by 109
Towards explaining anomalies: A deep Taylor decomposition of one-class models
Kauffmann, J., Müller, K., & Montavon, G. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107198 · Cited by 103
Self-citationUnderstanding and Comparing Deep Neural Networks for Age and Gender Classification
Samek, W., Binder, A., Lapuschkin, S., & Müller, K. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.191 · Cited by 136
Self-citationExplaining and Interpreting LSTMs
Arras, L., Arjona-Medina, J., Widrich, M., Montavon, G., Gillhofer, M., Müller, K.-R., Hochreiter, S., & Samek, W. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_11 · Cited by 111
Self-citationComputer aided diagnosis system using deep convolutional neural networks for ADHD subtypes
Ahmadi, A., Kashefi, M., Shahrokhi, H., & Nazari, M. A. (2020) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2020.102227 · Cited by 101
Nam, W.-J., Gur, S., Choi, J., Wolf, L., & Lee, S.-W. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5632 · Cited by 98
Qin, N., Liang, K., Huang, D., Ma, L., & Kemp, A. H. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2966744 · Cited by 98
Explaining Predictions of Non-Linear Classifiers in NLP
Arras, L., Horn, F., Montavon, G., Müller, K., & Samek, W. (2016) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w16-1601 · Cited by 118
Self-citationMachine learning applications in nanomaterials: Recent advances and future perspectives
Yang, L., Wang, H., Leng, D., Fang, S., Yang, Y., & Du, Y. (2024) · Chemical Engineering Journal · doi:10.1016/j.cej.2024.156687 · Cited by 110
Fully automated carbonate petrography using deep convolutional neural networks
Koeshidayatullah, A., Morsilli, M., Lehrmann, D. J., Al-Ramadan, K. A., & Payne, J. L. (2020) · Marine and Petroleum Geology · doi:10.1016/j.marpetgeo.2020.104687 · Cited by 100
Hadjiiski, L. M., Cha, K., Chan, H., Drukker, K., Morra, L., Näppi, J. J., Sahiner, B., Yoshida, H., Chen, Q., Deserno, T. M., et al. (2022) · Medical Physics · doi:10.1002/mp.16188 · Cited by 107
Deep Neural Network Based Inertial Odometry Using Low-Cost Inertial Measurement Units
Chen, C., Lu, C. X., Wahlström, J., Markham, A., & Trigoni, N. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2960780 · Cited by 98
Towards Improving Adversarial Training of NLP Models
Yoo, J. Y., & Qi, Y. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.81 · Cited by 128
To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods
Amparore, E. G., Perotti, A., & Bajardi, P. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.479 · Cited by 103
Explainable deep learning in plant phenotyping
Mostafa, S., Mondal, D., Panjvani, K., Kochian, L. V., & Stavness, I. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1203546 · Cited by 99
VisualBackProp: Efficient Visualization of CNNs for Autonomous Driving
Bojarski, M., Choromanska, A., Choromański, K., Firner, B., Ackel, L. J., Müller, U., Yeres, P., & Zieba, K. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461053 · Cited by 91
Stenzinger, A., Alber, M., Allgäuer, M., Jurmeister, P. S., Bockmayr, M. L., Budczies, J., Lennerz, J. K., Eschrich, J., Kazdal, D. N., Schirmacher, P., et al. (2021) · Seminars in Cancer Biology · doi:10.1016/j.semcancer.2021.02.011 · Cited by 94
Gazda, M., Hireš, M., & Drotár, P. (2021) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.3048892 · Cited by 95
Explainability in AI-based behavioral malware detection systems
Galli, A., Gatta, V. L., Moscato, V., Postiglione, M., & Sperlí, G. (2024) · Computers & Security · doi:10.1016/j.cose.2024.103842 · Cited by 117
Visualizing convolutional neural network protein-ligand scoring
Hochuli, J. E., Helbling, A., Skaist, T., Ragoza, M., & Koes, D. R. (2018) · Journal of Molecular Graphics and Modelling · doi:10.1016/j.jmgm.2018.06.005 · Cited by 94
Wu, M., Su, W., Chen, L., Liu, Z., Cao, W., & Hirota, K. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2897330 · Cited by 89
CNN-Based Super-Resolution of Hyperspectral Images
Arun, P. V., Buddhiraju, K. M., Porwal, A. K., & Chanussot, J. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.2973370 · Cited by 90
Image domain dual material decomposition for dual‐energy CT using butterfly network
Zhang, W., Zhang, H., Wang, L., Wang, X., Hu, X., Cai, A., Li, L., Niu, T., & Yan, B. (2019) · Medical Physics · doi:10.1002/mp.13489 · Cited by 87
Nomura, Y., Xu, Q., Shirato, H., Shimizu, S., & Xing, L. (2019) · Medical Physics · doi:10.1002/mp.13583 · Cited by 90
Advances of AI in image-based computer-aided diagnosis: A review
Yeasmin, M. N., Amin, M. A., Joti, T. J., Aung, Z., & Azim, M. A. (2024) · Array · doi:10.1016/j.array.2024.100357 · Cited by 105
Benchmarking Attribution Methods with Relative Feature ImportancePreprint
Yang, M., & Kim, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.09701 · Cited by 108
Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection
Chen, H., Zheng, G., & Ji, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.494 · Cited by 101
Lee, S., Kim, J., Park, S. W., Jin, S., & Park, S. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3002022 · Cited by 89
ExplAIn: Explanatory artificial intelligence for diabetic retinopathy diagnosis
Quellec, G., Hajj, H. A., Lamard, M., Conze, P.-H., Massin, P., & Cochener, B. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102118 · Cited by 91
Explaining Anomalies Detected by Autoencoders Using SHAPPreprint
Antwarg, L., Miller, R. M., Shapira, B., & Rokach, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.02407 · Cited by 83
Explaining Machine Learning Models for Clinical Gait Analysis
Slijepcevic, D., Horst, F., Lapuschkin, S., Horsak, B., Raberger, A.-M., Kranzl, A., Samek, W., Breiteneder, C., Schöllhorn, W. I., & Zeppelzauer, M. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3474121 · Cited by 82
Self-citationQuantifying Explainers of Graph Neural Networks in Computational Pathology
Jaume, G., Pati, P., Bozorgtabar, B., Foncubierta–Rodríguez, A., Anniciello, A. M., Feroce, F., Rau, T. T., Thiran, J., Gabrani, M., & Göksel, O. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00801 · Cited by 89
Explainable AI: A review of applications to neuroimaging data
Farahani, F. V., Fiok, K., Lahijanian, B., Karwowski, W., & Douglas, P. K. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.906290 · Cited by 87
Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement
Poerner, N., Schütze, H., & Roth, B. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1032 · Cited by 86
Bommer, P. L., Kretschmer, M., Hedström, A., Bareeva, D., & Höhne, M. (2024) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-23-0074.1 · Cited by 99
Deep Semantic-Preserving Ordinal Hashing for Cross-Modal Similarity Search
Jin, L., Li, K., Li, Z., Xiao, F., Qi, G.-J., & Tang, J. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2869601 · Cited by 77
Barriers and pitfalls for artificial intelligence in gastroenterology: Ethical and regulatory issues
Ahmad, O. F., Stoyanov, D., & Lovat, L. B. (2020) · Techniques and Innovations in Gastrointestinal Endoscopy · doi:10.1016/j.tgie.2019.150636 · Cited by 78
Evaluating Recurrent Neural Network Explanations
Arras, L., Osman, A., Müller, K., & Samek, W. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4813 · Cited by 88
Self-citationTownsend, J. A., Chaton, T., & Monteiro, J. M. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2944672 · Cited by 80
Detecting Climate Signals Using Explainable AI With Single‐Forcing Large Ensembles
Labe, Z. M., & Barnes, E. A. (2021) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2021ms002464 · Cited by 77
Deep learning of pharmacogenomics resources: moving towards precision oncology
Chiu, Y., Chen, H.-I. H., Gorthi, A., Mostavi, M., Zheng, S., Huang, Y., & Chen, Y. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz144 · Cited by 80
A survey on artificial intelligence in histopathology image analysis
Abdelsamea, M. M., Zidan, U., Senousy, Z., Gaber, M. M., Rakha, E. A., & Ilyas, M. (2022) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1474 · Cited by 84
AudioMNIST: Exploring Explainable Artificial Intelligence for audio analysis on a simple benchmark
Becker, S., Vielhaben, J., Ackermann, M., Müller, K., Lapuschkin, S., & Samek, W. (2023) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2023.11.038 · Cited by 181
Self-citationWilliford, J. R., May, B. B., & Byrne, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_15 · Cited by 76
Gait Spatiotemporal Signal Analysis for Parkinson’s Disease Detection and Severity Rating
Alharthi, A. S., Casson, A. J., & Ozanyan, K. B. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3018262 · Cited by 83
Interpretability in the medical field: A systematic mapping and review study
Hakkoum, H., Abnane, I., & Idri, A. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.108391 · Cited by 82
Quantifying Interpretability and Trust in Machine Learning SystemsPreprint
Schmidt, P. S., & Bießmann, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08558 · Cited by 74
Small perturbations are enough: Adversarial attacks on time series prediction
Wu, T., Wang, X., Qiao, S., Xian, X., Liu, Y., & Zhang, L. (2022) · Information Sciences · doi:10.1016/j.ins.2021.11.007 · Cited by 72
Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps
Kim, B., Seo, J., Jeon, S., Koo, J., Choe, J., & Jeon, T. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00510 · Cited by 80
IMPROVED EXPLAINABILITY OF CAPSULE NETWORKS: RELEVANCE PATH BY AGREEMENT
Shahroudnejad, A., Afshar, P., Plataniotis, K. N., & Mohammadi, A. (2018) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2018.8646474 · Cited by 75
Toward Interpretable Machine Learning: Transparent Deep Neural Networks and BeyondPreprint
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Müller, K. (2020) · arXiv (Cornell University) · Cited by 70
Self-citationVisualizations of Deep Neural Networks in Computer Vision: A Survey
Seifert, C., Aamir, A., Balagopalan, A., Jain, D., Sharma, A., Grottel, S., & Gumhold, S. (2017) · Studies in big data · doi:10.1007/978-3-319-54024-5_6 · Cited by 75
When Explanations Lie: Why Many Modified BP Attributions FailPreprint
Sixt, L., Granz, M., & Landgraf, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.09818 · Cited by 74
Mohamed, E., Sirlantzis, K., & Howells, G. (2022) · Displays · doi:10.1016/j.displa.2022.102239 · Cited by 67
Kiryu, S., Yasaka, K., Akai, H., Nakata, Y., Sugomori, Y., Hara, S., Seo, M., Abe, O., & Ohtomo, K. (2019) · European Radiology · doi:10.1007/s00330-019-06327-0 · Cited by 71
Pawlicki, M., Pawlicka, A., Uccello, F., Szelest, S., D’Antonio, S., Kozik, R., & Choraś, M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128282 · Cited by 78
Dyrba, M., Hanzig, M., Altenstein, S., Bader, S., Ballarini, T., Brosseron, F., Büerger, K., Cantré, D., Dechent, P., Dobisch, L., et al. (2021) · Alzheimer s Research & Therapy · doi:10.1186/s13195-021-00924-2 · Cited by 67
Rajaraman, S., Silamut, K., Hossain, M. A., Ersoy, I., Maude, R. J., Jaeger, S., Thoma, G. R., & Antani, S. (2018) · Journal of medical imaging · doi:10.1117/1.jmi.5.3.034501 · Cited by 70
Weakly Supervised Object Detection via Object-Specific Pixel Gradient
Shen, Y., Ji, R., Wang, C., Li, X., & Li, X. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2816021 · Cited by 65
Wang, D., Honnorat, N., Fox, P. T., Ritter, K., Eickhoff, S. B., Seshadri, S. S., & Habes, M. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.119929 · Cited by 70
Yuan, H., Bi, J., & Zhou, M. (2019) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2019.2892480 · Cited by 63
A graph neural network framework for causal inference in brain networks
Wein, S., Malloni, W. M., Tomé, A. M., Frank, S., Henze, G., Wüst, S., Greenlee, M. W., & Lang, E. W. (2021) · Scientific Reports · doi:10.1038/s41598-021-87411-8 · Cited by 66
Supervised Contrastive Learning for Generalizable and Explainable DeepFakes Detection
Xu, Y., Raja, K., & Pedersen, M. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw54805.2022.00044 · Cited by 62
Advanced Design of Soft Robots with Artificial Intelligence
Cao, Y., Xu, B., Li, B., & Fu, H. C. (2024) · Nano-Micro Letters · doi:10.1007/s40820-024-01423-3 · Cited by 67
Deep learning in histopathology: A review
Banerji, S., & Mitra, S. (2021) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1439 · Cited by 69
Evaluating Feature Importance EstimatesPreprint
Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2018) · arXiv (Cornell University) · Cited by 60
Survey of Explainable Machine Learning with Visual and Granular Methods Beyond Quasi-Explanations
Kovalerchuk, B., Ahmad, M. A., & Teredesai, A. M. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-64949-4_8 · Cited by 63
Bearing Fault Diagnosis Based on Multiscale Convolutional Neural Network Using Data Augmentation
Han, S., Oh, S., & Jeong, J. (2021) · Journal of Sensors · doi:10.1155/2021/6699637 · Cited by 59
Solving the online batching problem using deep reinforcement learning
Cals, B., Zhang, Y., Dijkman, R., & van Dorst, C. (2021) · Computers & Industrial Engineering · doi:10.1016/j.cie.2021.107221 · Cited by 62
Influence-Directed Explanations for Deep Convolutional Networks
Leino, K. R. M., Sen, S., Datta, A., Fredrikson, M., & Li, L. (2018) · IEEE International Test Conference (ITC) · doi:10.1109/test.2018.8624792 · Cited by 70
Explainable Sentiment Analysis with Applications in Medicine
Zucco, C., Liang, H., Fatta, G. D., & Cannataro, M. (2018) · 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2018.8621359 · Cited by 59
Metadata
- Authors
- Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R.
- Affiliations
- Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
- Singapore University of Technology and Design (SG)
- Technische Universität Berlin, Berlin, Germany
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2017-11
- Volume
- 28
- Issue
- 11
- Pages
- 2660-2673
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2240067561
- Cited by
- 1,164
- References
- 51
- Citations by year
- 2026: 120 · 2025: 105 · 2024: 139 · 2023: 163 · 2022: 147 · 2021: 171 · 2020: 123 · 2019: 92 · 2018: 66 · 2017: 26
- FWCI
- 53.43
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- Funders
- Brain Korea 21 Plus Program through the National Research Foundation of Korea Funded by the Ministry of Education
- DFG (MU 987/17-1)
- German Ministry for Education and Research as Berlin Big Data Center (01IS14013A)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- deep neural network interpretability, layer-wise relevance propagation, deconvolution, explainable AI, image classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
