Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Open accessThe DOI matches its Crossref record; 224 of the 230 references it cites were found in a registry or an index.
Abstract
With the broader and highly successful usage of machine learning (ML) in industry and the sciences, there has been a growing demand for explainable artificial intelligence (XAI). Interpretability and explanation methods for gaining a better understanding of the problem-solving abilities and strategies of nonlinear ML, in particular, deep neural networks, are, therefore, receiving increased attention. In this work, we aim to: 1) provide a timely overview of this active emerging field, with a focus on “post hoc” explanations, and explain its theoretical foundations; 2) put interpretability algorithms to a test both from a theory and comparative evaluation perspective using extensive simulations; 3) outline best practice aspects, i.e., how to best include interpretation methods into the standard usage of ML; and 4) demonstrate successful usage of XAI in a representative selection of application scenarios. Finally, we discuss challenges and possible future directions of this exciting foundational field of ML.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“While XAI techniques exist [47], their application in complex, real-time energy systems, and providing meaningful explanations to non-experts, remain challenging”
From Addressing Challenges for the Effective Adoption of Artificial Intelligence in the Energy Sector (2025) · cites this paper as [47]
In the citing paper
“Despite previous attempts to address the challenges surrounding model interpretability, most solutions have been implemented post hoc (such as the attention mechanism) focusing on explanations in certain domains, such as vision [184].”
From Human-centred artificial intelligence for mobile health sensing: challenges and opportunities (2023) · cites this paper as [184]
In the citing paper
“LIME produces an explanation list that shows the contribution of individual features to the prediction.”
From Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2022) · cites this paper as [64]
In the citing paper
“This local explanation allows the end-user to determine which feature is important for the precise prediction and how the perturbation in feature affects the prediction results.”
From Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2022) · cites this paper as [64]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 12×
“This modified gradient view on LRP can also be leveraged to achieve a simpler and more general implementation of LRP based on ‘forward hooks’, which we describe in the second part of Appendix B, and which we use to apply LRP on VGG-16 [175] and ResNet-50 [65] in Section IV.”
BackgroundMethodInfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 6×
“This modified gradient view on LRP can also be leveraged to achieve a simpler and more general implementation of LRP based on ‘forward hooks’, which we describe in the second part of Appendix B, and which we use to apply LRP on VGG-16 [175] and ResNet-50 [65] in Section IV.”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 4×
“Consequently, we also exchange the neural network architecture and use an AlexNet [99] model, which is able to process the transformed input data using 2-D-convolution operators.”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“, [23], [52], [70], [107], [108], and [161]) have gained highest popularity.”
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 9×
“A comparison of performance on the Adience benchmark of the original model (pretrained on ImageNet only) and the improved model is given in the table below.”
MethodInfluentialRegression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 8×
“A comparison of performance on the Adience benchmark of the original model (pretrained on ImageNet only) and the improved model is given in the table below.”
BackgroundInfluentialThe Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“In games such as the arcade game Atari Breakout, the computer player would progressively learn strategies commonly employed by human players, such as “tunneldigging” [124], [200].”
Background
Research works printed but not linked: 13
- Arras, 2020, Ground truth evaluation of neural network explanations with CLEVR-XAI, arXiv:2003.07258 · arXiv
- 10.1146/annurev.neuro.26.041002.131047 · DOI · Paper page
- Chollet, 2015, Keras
- 10.1109/ICCV.2019.00304 · DOI
- Kokhlikyan, 2019, Pytorch Captum
- Lapuschkin, 2016, The layer-wise relevance propagation toolbox for artificial neural networks, J. Mach. Learn. Res., 17, 1
- 10.48550/arXiv.1312.4400 · DOI · Paper page
- 10.5555/2969033.2969153 · doi:10.5555/2969033.2969153
- 10.5555/3157382.3157477 · doi:10.5555/3157382.3157477
- Okuta, CuPy: A numpy-compatible library for NVIDIA GPU calculations, 151
- Oliphant, 2006, A Guide to NumPy, 1
- 10.48550/ARXIV.1404.7828 · DOI
- 10.5555/3045118.3045336 · doi:10.5555/3045118.3045336
Cited by
Items 801–900 of 1,002
Artificial Intelligence Techniques for Short-Term Load Forecasting in Microgrids
Khanum, N. U. M., & Bansal, R. C. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-95540-2_29 · Cited by 4
Fuzzy Kernel Stochastic Configuration Networks for Industrial Data Modeling
Chen, Y., & Wang, D. (2025) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2025.3580614 · Cited by 2
Sarkar, S., Bora, R. P., Kaushal, B., George, S. N., & Raja, K. B. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105717 · Cited by 2
Özen, F., Kabaoğlu, R. O., & Mumcu, T. V. (2025) · Electronics · doi:10.3390/electronics14193876 · Cited by 2
Meta: Error Detection in Data Quality for AI-Powered Business Intelligence
Nethala, S. (2025) · IEEE data descriptions. · doi:10.1109/ieeedata.2025.3619493 · Cited by 2
“Samek et al. [25] emphasized that while these models may detect patterns, their lack of transparency limits adoption in use cases where traceability and auditability are critical.”
Quantitative effects of soil organic matter on thermal conductivity modeling
Wang, X., Zhao, T., Zhao, C., Zvomuya, F., & He, H. (2025) · Agricultural and Forest Meteorology · doi:10.1016/j.agrformet.2025.110887 · Cited by 2
Waqas, M., Humphries, U. W., & Wangwongchai, A. (2025) · Environmental Monitoring and Assessment · doi:10.1007/s10661-025-14712-0 · Cited by 2
“Notably, these DL models can now be interpreted using explainable AI (XAI) methods, which allow one to understand what physical processes drive the changes predicted by the model and enable it to be more realistic in decision-making (Samek et al., 2017, 2021).”
Explainable AI in High-Stakes Decision Making: Beyond Accuracy
Kalathoti, R. (2025) · Scientific Journal of Artificial Intelligence and Blockchain Technologies · doi:10.63345/sjaibt.v2.i3.103 · Cited by 6
Xin, N., Su, J., & Hasan, M. M. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.114090 · Cited by 6
“However, three key challenges persist: (1) standard DL models can be computationally intensive, less interpretable, and lack uncertainty analysis [14]; (2) achieving the adaptability and robustness for reliable uncertainty quantification across diverse real-world conditions remains challenging…”
Directed Vectors for Generation of Independent Subspaces in the Bio-inpired Networks
Ishii, N., Iwata, K., Odagiri, K., & Matsuo, T. (2025) · International Journal of Neural Systems · doi:10.1142/s0129065725500790 · Cited by 2
Adaptive Clustering-Enabled Large-Scale Decentralized Federated Learning
Tang, J., Liang, X., Siew, M., & Quek, T. Q. S. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3630332 · Cited by 3
“Meanwhile, by mining these data, deep neural networks and even advanced large language models have shown remarkable performance in a variety of tasks and domains, such as image classification, natural language processing, and multi-object recognition [5], [6].”
Explaining Neural Networks: Hierarchical Backpropagated Ensemble Learning
He, G., Wang, Z., Tang, L., Zhang, R., Wang, R., Li, X., & Nie, F. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3625434 · Cited by 4
“However, despite their impressive performance, deep models face a key challenge of interpretability [6], [7].”
Zhang, M., Liu, X. S., Chu, W., Zhang, H., & Wang, Y. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c01397 · Cited by 2
Liu, Y., Chen, X., & Xi, C. (2025) · Big Data and Earth System · doi:10.1016/j.bdes.2025.100039 · Cited by 2
Enhancing randomized recurrent neural networks with explainable attribution methods
Spinnato, F., Ceni, A., Cossu, A., Guidotti, R., Gallicchio, C., & Bacciu, D. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.132318 · Cited by 2
Kang, Y. J., Lee, Y. G., Chung, M. J., Kim, J. H., & Choi, N. (2025) · Scientific Reports · doi:10.1038/s41598-025-27428-5 · Cited by 4
Towards desiderata-driven design of visual counterfactual explainers
Bender, S., Herrmann, J., Müller, K. R., & Montavon, G. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112811 · Cited by 14
“Spurious local variations are commonplace in image-based classifiers (e.g. [67, 68, 3, 69]), and also occur along the data manifold M [66].”
Self-citationPrecision compensation technology of industrial robot based on OOA-DNN and transfer learning
Wang, S., Huang, Y., Mo, Y., & Guo, J. (2025) · Mechanics & Industry · doi:10.1051/meca/2025031 · Cited by 2
Real-Time Acoustic Anomaly Detection in Vehicles Using AI Processor and Machine Learning
Aradi, A., & Varga, A. K. (2025) · International Journal of Automotive Science And Technology · doi:10.30939/ijastech..1769036 · Cited by 2
Gu, Y., Shao, C., Li, J., & Fan, Y. (2026) · Scientific Reports · doi:10.1038/s41598-025-34238-2 · Cited by 5
“However, the power of these deep learning models is accompanied by a significant challenge for scientific inquiry: their “black box” nature, a well-recognized challenge in the field of Explainable Artificial Intelligence (XAI) that hinders trust and scientific progress 17–19 .”
Carlini, L., Stefano, L. D., Lena, C., Massimi, D., Rizkala, T., Hassan, C., & Momi, E. D. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2026.109260 · Cited by 2
Ali, M. B. G., & Smiti, A. (2026) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2026.102730 · Cited by 4
Attribution Explanations for Deep Neural Networks: A Theoretical Perspective
Deng, H., Pei, H., Zhang, Q., & Du, M. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3667600 · Cited by 6
“Most attribution methods are developed heuristically, with limited or no formal theoretical justification [39].”
InfluentialA Visually Explainable Dynamic Similarity Network for Few-Shot Classification
Pei, Z., Meng, Z., Diao, T., Miao, P., Meng, Y., & Tan, C. (2026) · Journal of Artificial Intelligence and Soft Computing Research · doi:10.2478/jaiscr-2026-0012 · Cited by 2
“Some approaches use human-readable structures like decision trees [44] or attention mechanisms [32], while others adopt concept-based methods, such as Concept Bot-tleneck Models [45, 55], or visualize the regions the model focuses on through activation maps and attri-bution techniques [46, 43].”
Enhanced cervical cancer classification using convolutional tsetlin machines with transfer learning
Ahishakiye, E., Nkalubo, L., Kanobe, F., Taremwa, D., Nantongo, B. A., & Ahimbisibwe, S. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01057-x · Cited by 2
Drug repurposing in oncology: Bridging computational discovery to clinical application
Baskar, S., & Prasad, N. R. (2026) · International Journal of Cancer · doi:10.1002/ijc.70429 · Cited by 2
“66 Reproducibility concerns further complicate AI-driven drug repurposing, as many published models lack adequate documentation of hyperpara-meters, training procedures, and validation strategies necessary for independent replication.”
Deep Neural Network–based Retail Business Analysis using AI for Smart Environments and IoT
Eswaran, B., K.Vengatesan, K., Samee, S., Kharade, K. G., Rajesh, D., & Bhagawati, R. (2026) · International Conference on AI-Driven Smart Systems and Ubiquitous Computing (ICAUC) · doi:10.1109/icauc68182.2026.11441062 · Cited by 2
“Using image-based and sequential data, CNNs and RNNs enhance feature extraction [8].”
Li, B., Zheng, H., & Zhang, X. (2026) · AI in Civil Engineering · doi:10.1007/s43503-026-00091-z · Cited by 2
AI for Lithium-ion battery research under data-scarce scenarios
Zhang, J., Zhai, X., Zhang, Q. (., Li, P., Li, M., Wu, K., Yang, L. I. U., & Wan, J. (2026) · The Innovation Materials · doi:10.59717/j.xinn-mater.2026.100209 · Cited by 2
Deep Learning‐Based Data Hiding Techniques for Secure Communication: A Comprehensive Review
Awasthi, D., Tiwari, A., Khare, P., & Srivastava, V. K. (2026) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.70706 · Cited by 4
Wu, Y., Zhu, J., Yuan, J., & Huang, K. (2026) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2026.108991 · Cited by 2
Liu, Y., Fu, Y., & Wu, J. (2026) · Applied Energy · doi:10.1016/j.apenergy.2026.128100 · Cited by 2
Saeed, M., Arham, M., Zafar, I., Jamal, A., Hussian, M., Usman, M., Bahwerth, F. S., Noman, M., & Hossain, M. B. (2026) · Biotechnology Journal · doi:10.1002/biot.70255 · Cited by 2
“…AI note that the quality of explanation, faithfulness, and robustness are crucial, particularly when the aim of the explanation is to make a decision rather than tell a story, and, in this regard, the importance of seeking careful validation of interpretability assertions in sensitive areas [17].”
Dovedi, T., Upadhyay, R., & Kumar, V. (2026) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2026.111318 · Cited by 2
Redundancy as the Shadow of Explainability: A Trade-Off Principle for AI-Intensive Systems
Liu, Y., Huang, J., Hamou‐Lhadj, A., & Li, Z. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3786582.3786829 · Cited by 2
“Samek et al. [20] argue for formalizing the utility of explanations.”
Jan, M. B., Wu, Z., & Chai, M. (2026) · Metals · doi:10.3390/met16070795 · Cited by 3
“Fully connected deep network learning nonlinear mappings [59].”
Symbolic–KAN: Kolmogorov-Arnold networks with discrete symbolic structure for interpretable learning
Faroughi, S. A., Mostajeran, F., Arzani, A., & Faroughi, S. (2026) · Journal of Computational Physics · doi:10.1016/j.jcp.2026.115223 · Cited by 9
MR.DRP: A Parameter Adaptive Machine Learning Framework for Drug Response Prediction
Cai, Z., He, J., Yu, B., & Wei, L. (2022) · International Conference on Applied Machine Learning (ICAML) · doi:10.1109/icaml57167.2022.00019 · Cited by 2
“Fortunately, with the remarkable development of high-throughput sequencing technologies [3,4] and the emergence of deep neural networks [5,6], large-scale genomic data and new solutions for cancer drug response prediction have been provided.”
BackgroundTowards Early Prediction of Human iPSC Reprogramming Success
Singh, A. K., Jasra, I. T., Mouhammed, O., Dadheech, N., Ray, N., & Shapiro, A. M. J. (2023) · The Journal of Machine Learning for Biomedical Imaging · doi:10.59275/j.melba.2023-3d9d · Cited by 4
Nguyen, Q., Nguyen, N., Dang, T., & Tran, V. (2023) · doi:10.1145/3638584.3638634 · Cited by 2
“In the 2010s, deep neural networks [14] and recurrent neural networks [12] improved recognition accuracy and have become fundamental to modern ASR systems [8].”
BackgroundZhao, Y., Xiao, L., & Wong, S. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3390623 · Cited by 6
“However, we would like to point out that AI techniques are notorious for not being explainable in some cases [56], [57].”
BackgroundRahman, J. U., Makhdoom, F., & Lu, D. (2024) · Annals of computational physics and material science. · doi:10.33140/acpms.01.01.04 · Cited by 16
A Comparative Study of Deep Leaning-Aided Relay Selection Protocols in Cooperative Communication
Akdemir, B., Karabulut, M. A., & İlhan, H. (2024) · Wireless Personal Communications · doi:10.1007/s11277-024-11269-y · Cited by 4
Agiollo, A., Siebert, L. C., Murukannaiah, P. K., & Omicini, A. (2024) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-024-09663-8 · Cited by 11
“However, a detailed analysis of these LPEs is out of the scope of this paper and we refer interested readers to [33, 39, 41].”
BackgroundCase Study: Using Data Mining to Predict Student Performance Based on Demographic Attributes
Zahruddin, N. A. B. M., Kamarudin, N. D., Jusoh, R. M., Fataf, N. A. A., & Hidayat, R. S. A. (2023) · JOIV International Journal on Informatics Visualization · doi:10.62527/joiv.7.4.2454 · Cited by 2
(2024) · International Research Journal of Modernization in Engineering Technology and Science · doi:10.56726/irjmets60972 · Cited by 4
A global perspective on AI innovation and effective use in the research lab
Jeng, A. C., Sibley, I. J., & Bale, T. L. (2024) · Neuroscience · doi:10.1016/j.neuroscience.2024.09.034 · Cited by 3
Chen, H., Chen, J., Wu, Y., Cai, S., Ahmad, B., Huang, R., Wang, S., & Zhang, C. (2024) · Software Testing Verification and Reliability · doi:10.1002/stvr.1897 · Cited by 2
A New Framework for Integrating DNN-Based Geographic Simulation Models within GISystems
Zhang, P., Wu, W., Xue, C., Shi, S., & Su, F. (2024) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi13100361 · Cited by 2
Deep neural networks in low energy algorithms for wireless sensor networks
Jia, L. (2024) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-024-04874-z · Cited by 2
“This is an important beginning and foundation for the development of cluster routing protocols (Samek et al. 2021).”
Conditional Potential User Mining framework via explainable surrogate models
Zhao, Y., Xu, Y., Liu, Y. H., Yang, L., Jiang, W., Ning, W., Sun, X., & Cui, L. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125587 · Cited by 3
A Methodological Approach to Securing Cyber-Physical Systems for Critical Infrastructures
Calabrò, A., Cambiaso, E., Cheminod, M., Bertolotti, I. C., Durante, L., Forestiero, A., Lombardi, F., Manco, G., Marchetti, E., Orlando, A., et al. (2024) · Future Internet · doi:10.3390/fi16110418 · Cited by 2
Improving the Efficiency of Production Processes by Reducing Human Errors Using Intelligent Methods
Musiał, K., Balashov, A., Burduk, A., Sulowski, R., & Pihnastyi, O. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-75013-7_3 · Cited by 2
Attention on Sleep Stage Specific Characteristics
Huijben, I. A. M., Overeem, S., van Gilst, M. M., & van Sloun, R. J. G. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10782554 · Cited by 2
“A multitude of post-hoc explanation methods for neural networks has been proposed [1], of which some found their way to sleep analyses as well [2], [3].”
Xiao, N., Zhao, K., & Zhang, H. (2024) · IEEE Access · doi:10.1109/access.2024.3519741 · Cited by 2
Ye, F., Jin, Z., Zhang, P., Xu, D., Lin, L., & Sun, J. Q. (2025) · International Journal of Intelligent Systems · doi:10.1155/int/8899669 · Cited by 3
“Terefore, in the absence of sufcient conditions for constructing a physics-informed neural network [27, 28], it is a practical approach to comprehensively refer to the results calculated from diferent models.”
Rieger, I. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-81542-3_10 · Cited by 3
Artificial Intelligence Techniques for Early Prediction of Neonatal Jaundice
Ashwarya, M., Sharma, R., & Raju, B. R. S. S. (2024) · International Conference on Sustainable Computing and Integrated Communication in Changing Landscape of AI (ICSCAI) · doi:10.1109/icscai61790.2024.10866866 · Cited by 2
Takahashi, K. (2024) · IEEE International Conference on Future Machine Learning and Data Science (FMLDS) · doi:10.1109/fmlds63805.2024.00010 · Cited by 2
Potable Water Quality Assessment Through an Intelligent Classification Model
Timiraos, M., Díaz-Longueira, A., Michelena, Á., Gato, F. Z., Casteleiro‐Roca, J., & ROLLE, J. L. C. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-83117-1_28 · Cited by 2
Interpretable bearing fault diagnosis based on ensemble learning with improved ResNet
Li, J., Liu, S., & Wang, H. (2025) · Engineering Research Express · doi:10.1088/2631-8695/adc8ff · Cited by 3
Chen, Z., Liu, D., Wang, Y., Han, D., & Gong, H. (2025) · Nuclear Instruments and Methods in Physics Research Section A Accelerators Spectrometers Detectors and Associated Equipment · doi:10.1016/j.nima.2025.170557 · Cited by 2
The state of hybrid artificial intelligence for interstellar missions
Ellery, A. A. (2025) · Progress in Aerospace Sciences · doi:10.1016/j.paerosci.2025.101100 · Cited by 3
Optimizing Local Explainability in Robotic Grasp Failure Prediction
Acun, C., Ashary, A., Popa, D. O., & Nasraoui, O. (2025) · Electronics · doi:10.3390/electronics14122363 · Cited by 3
Wei, R., Deng, S., Yan, X., Peng, M., Ke, K., Wang, L., Hu, Z., Yang, K., Huo, B., & Cao, L. (2025) · Processes · doi:10.3390/pr13061873 · Cited by 4
The advent of the deep learning evolutionary algorithm EvoDN2 and its recent applications
Chakraborti, N. (2025) · Computer Methods in Materials Science. · doi:10.7494/cmms.2025.2.1021 · Cited by 4
AI-powered deep ultraviolet laser diode design for resource-efficient optimization
Sarwar, A., Usman, M., Hussain, M., Jadoon, K. K., Manzoor, T., & Ali, S. (2025) · Journal of Computational Electronics · doi:10.1007/s10825-025-02352-z · Cited by 2
“Among the research focused on explaining individual predictions, one of the most active areas is feature attribution [67], which aims to identify which parts of an input contribute to a model’s output decision.”
ECG Statement Classification and Lead Reconstruction Using CNN-Based Models
Rajotte, K. J., Islam, B., Huang, X., McManus, D. D., & Clancy, E. A. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3592852 · Cited by 6
“Techniques such as layer-wise relevance propagation(LRP)orgradient-weightedclassactivationmapping(Grad-CAM)maybesuitableforexplaininghowtheCNNmodelis makingdecisions[32],[33],[34],[35].”
Shan, P., Zhang, L., Yan, C., Xu, H., Zheng, M., Xi, B., & Xu, G. (2025) · Applied Sciences · doi:10.3390/app152212070 · Cited by 2
Explainable deep learning for decrypting disease signatures in multiple sclerosis
Cruciani, F., Brusini, L., Zucchelli, M., Pinheiro, G. R., Setti, F., Rachid Deriche, B., Rittner, L., Calabrese, M., Galazzo, I. B., & Menegaz, G. (2023) · Elsevier eBooks · doi:10.1016/b978-0-32-396098-4.00012-0 · Cited by 3
Interpreting Deep Learning Models for Multi-modal Neuroimaging
Müller, K. R., & Hofmann, S. (2023) · International Winter Conference on Brain-Computer Interface (BCI) · doi:10.1109/bci57258.2023.10078502 · Cited by 2
“Recent developments in explainable AI (XAI; e.g., Montavon et al. 2018, Lapuschkin et al. 2019, Lundberg et al. 2020, Samek et al. 2021, Achtibat et al., 2022), however, have opened possibilities for whole brain regressions to, e.g., biological markers, health status or age with complex neural…”
BackgroundModeling mental peculiarities of a decision maker by a Fourier-holography technique
Pavlov, A. V., & Гаугель, А. О. (2023) · Computer Optics · doi:10.18287/2412-6179-co-1189 · Cited by 1
Knowledge Distillation in Granular Fuzzy Models by Solving Fuzzy Relation Equations
Rakytyanska, H. B. (2023) · Studies in computational intelligence · doi:10.1007/978-3-031-32095-8_4 · Cited by 5
Aligned deep neural network for integrative analysis with high-dimensional input
Zhang, S., Zhang, S., Yi, H., & Ma, S. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104434 · Cited by 3
Study of Distractors in Neural Models of Code
Rabin, M. R. I., Hussain, A., Suneja, S., & Alipour, M. A. (2023) · doi:10.1109/intense59150.2023.00005 · Cited by 4
“There has been a lot of work in the area of transparent or interpretable machine learning for computer vision, text, and natural language processing, that focuses on understanding the underlying reasoning of neural models [38, 39].”
BackgroundSanity Checks for Saliency Methods Explaining Object Detectors
Padmanabhan, D. C., Plöger, P. G., Arriaga, O., & Valdenegro-Toro, M. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_23 · Cited by 5
Predicting peak deviatoric stress of gravels with a novel segmented feature transformation approach
Li, D., Zou, D., Liu, J., Xu, K., Ning, F., Zhan, Z., & Jin, W. (2023) · Computers and Geotechnics · doi:10.1016/j.compgeo.2023.105935 · Cited by 2
Automated Extraction of Security Profile Information from XAI Outcomes
Jahan, S., Alqahtani, S., Gamble, R., & Bayesh, M. (2023) · IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C) · doi:10.1109/acsos-c58168.2023.00049 · Cited by 3
“It contains at least three layers in the neural network [23], [24].”
BackgroundWu, S., Zhou, Z., Bruce, N. D. B., Bardelcik, A., Chiriac, C., & Shi, C. (2024) · Materials Today Communications · doi:10.1016/j.mtcomm.2024.108492 · Cited by 2
Chowdhury, T. A., Gratz‐Kelly, S., Moretti, G., Wagner, E., Motzki, P., & Lehser, M. (2024) · Soft Mechatronics and Wearable Systems · doi:10.1117/12.3009859 · Cited by 2
“A deep neural network [4] model is developed to further analyze the manually generated thermal images and to distinguish between human and object from the low pixel generated images.”
MethodNegro, P., & Pons, C. (2024) · International Journal of Artificial Intelligence and Machine Learning · doi:10.4018/ijaiml.347988 · Cited by 3
Hardening Active Directory Graphs via Evolutionary Diversity Optimization-based Policies
Goel, D., Ward, M., Neumann, A., Neumann, F., Nguyen, H., & Guo, M. (2025) · ACM Transactions on Evolutionary Learning and Optimization · doi:10.1145/3688401 · Cited by 6
“Section 10 presents the discussion, and Section 11 concludes the paper.”
Blockchain technology and neural networks for the Internet of Medical Things
Falola, P. B., Adeniyi, A. E., & Awotunde, J. B. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-34226-4.00006-x · Cited by 2
Deep learning for tilted-wave interferometry
Hoffmann, L., Fortmeier, I., & Elster, C. (2021) · tm - Technisches Messen · doi:10.1515/teme-2021-0103 · Cited by 2
“The other applies explainability methods to gain a better understanding of how the network output relates to the input signal [27].”
IMLD: A Python-Based Interactive Machine Learning Demonstration
Cap, T., Kreitzer, A., Miranda, M., Vadimsky, D., & Picone, J. (2021) · IEEE Signal Processing in Medicine and Biology Symposium · doi:10.1109/spmb52430.2021.9672265 · Cited by 2
Xu, H., Sang, S., Yao, H., Herghelegiu, A. I., Lu, H., Yurkovich, J. T., & Yang, L. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.07.02.450937 · Cited by 4
“Explainable AI (XAI) seeks to enhance the model training, representation learning and decisionmaking processes with humanunderstandable explanations 9 .”
BackgroundDiscriminative Attribution from CounterfactualsPreprint
Eckstein, N., Bates, A. S., Jefferis, G. S. X. E., & Funke, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.13412 · Cited by 6
“A large number of approaches for feature attribution have been proposed in recent years (for a recent review see Samek et al. (2021) and related work below).”
BackgroundREM: An Integrative Rule Extraction Methodology for Explainable Data Analysis in HealthcarePreprint
Shams, Z., Dimanov, B., Kola, S., Simidjievski, N., Andrés-Terré, H., Scherer, P., Matjašec, U., Abraham, J., Jamnik, M., & Lió, P. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.01.22.427799 · Cited by 6
Editorial: The cognitive neuroscience of visual working memory, Volume II
Sigala, N., Káldy, Z., & Reynolds, G. D. (2022) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2022.1017754 · Cited by 2
Performance of Deep Learning Methods for Different Relay Selection ProtocolsPreprint
Akdemir, B., Karabulut, M. A., & İlhan, H. (2022) · Research Square · doi:10.21203/rs.3.rs-2176465/v1 · Cited by 2
Mohan, V. G., Ameedeen, M. A., & Ali, A.-F. M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4788544 · Cited by 3
Quantitative Attributions with CounterfactualsPreprint
Adjavon, D.-Y., Eckstein, N., Bates, A. S., Jefferis, G. S. X. E., & Funke, J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.11.26.625505 · Cited by 4
Ikromov, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5361525 · Cited by 3
Aichroth, P., & Liebetrau, J. (2023) · Springer eBooks · doi:10.1007/978-3-658-42060-4_4 · Cited by 1
Explainable ECG Beat Classification On The Edge for Smart, Trustworthy and Low-Power Wearables
Bhat, A., & Raychowdhury, A. (2023) · IEEE Biomedical Circuits and Systems Conference (BioCAS) · doi:10.1109/biocas58349.2023.10389065 · Cited by 1
“Explainable AI (XAI) methods aim to interpret these black-box models [8].”
MethodGuo, C., Cai, Q., & Liu, K. (2024) · International Journal of Surgery · doi:10.1097/js9.0000000000001262 · Cited by 1
Dias, J. (2024) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem29433 · Cited by 1
Dangerous Behavior Image Recognition Algorithm of Smart Port Based on Deep Neural Network
Cui, D., Sun, G., & Zhan, X. (2023) · International Conference on Network and Information Systems for Computers (ICNISC) · doi:10.1109/icnisc60562.2023.00102 · Cited by 1
A Trajectory Prediction Method for Drifting Buoy Based on GA-DNN Model
Hu, W., Song, M., Hu, T., Yan, X., Song, G., Zheng, S., Zhang, J., Jiao, Z., Song, H., & Gao, S. (2023) · International Conference on Robotics, Intelligent Control and Artificial Intelligence (RICAI) · doi:10.1109/ricai60863.2023.10489575 · Cited by 1
“Deep Neural Networks (DNN) is an advanced machine learning technology based on artificial neural networks (ANN), which has achieved remarkable achievements in scientific research and industrial applications[9].”
BackgroundEfficiency-optimized Diels-Alder reactions based on random forest
Zhang, L., Dong, H., Lin, A., Lu, R., Zhu, X., Zhang, M., & Lü, F. (2024) · Molecular Catalysis · doi:10.1016/j.mcat.2024.114236 · Cited by 1
Metadata
- Authors
- Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R.
- Affiliations
- Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
- BIFOLD✔Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
- Berlin Institute for the Foundations of Learning and Data (DE)
- ORCID
- Venue
- Proceedings of the IEEE
- Published
- 2021-03
- Volume
- 109
- Issue
- 3
- Pages
- 247-278
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W3132191748
- Cited by
- 1,477
- References
- 230
- Citations by year
- 2026: 305 · 2025: 343 · 2024: 326 · 2023: 273 · 2022: 152 · 2021: 74
- FWCI
- 72.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the IEEE (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion, other-oa)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Institute of Information & Communications Technology Planning & Evaluation (IITP) grants funded by the Korea Government (2017-0-00451, 2019-0-00079)
- German Ministry for Education and Research (01IS14013A-E, 01GQ1115, 01GQ0850, 01IS18025A, 01IS18037A)
- German Research Foundation (Math+, EXC 2046/1, 390685689)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- explainable AI, interpretability, deep neural network, post-hoc explanation, interpretability evaluation, nonlinear machine learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.