Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Open accessThe DOI matches its Crossref record; 224 of the 230 references it cites were found in a registry or an index.
Abstract
With the broader and highly successful usage of machine learning (ML) in industry and the sciences, there has been a growing demand for explainable artificial intelligence (XAI). Interpretability and explanation methods for gaining a better understanding of the problem-solving abilities and strategies of nonlinear ML, in particular, deep neural networks, are, therefore, receiving increased attention. In this work, we aim to: 1) provide a timely overview of this active emerging field, with a focus on “post hoc” explanations, and explain its theoretical foundations; 2) put interpretability algorithms to a test both from a theory and comparative evaluation perspective using extensive simulations; 3) outline best practice aspects, i.e., how to best include interpretation methods into the standard usage of ML; and 4) demonstrate successful usage of XAI in a representative selection of application scenarios. Finally, we discuss challenges and possible future directions of this exciting foundational field of ML.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“While XAI techniques exist [47], their application in complex, real-time energy systems, and providing meaningful explanations to non-experts, remain challenging”
From Addressing Challenges for the Effective Adoption of Artificial Intelligence in the Energy Sector (2025) · cites this paper as [47]
In the citing paper
“Despite previous attempts to address the challenges surrounding model interpretability, most solutions have been implemented post hoc (such as the attention mechanism) focusing on explanations in certain domains, such as vision [184].”
From Human-centred artificial intelligence for mobile health sensing: challenges and opportunities (2023) · cites this paper as [184]
In the citing paper
“LIME produces an explanation list that shows the contribution of individual features to the prediction.”
From Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2022) · cites this paper as [64]
In the citing paper
“This local explanation allows the end-user to determine which feature is important for the precise prediction and how the perturbation in feature affects the prediction results.”
From Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2022) · cites this paper as [64]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 12×
“This modified gradient view on LRP can also be leveraged to achieve a simpler and more general implementation of LRP based on ‘forward hooks’, which we describe in the second part of Appendix B, and which we use to apply LRP on VGG-16 [175] and ResNet-50 [65] in Section IV.”
BackgroundMethodInfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 6×
“This modified gradient view on LRP can also be leveraged to achieve a simpler and more general implementation of LRP based on ‘forward hooks’, which we describe in the second part of Appendix B, and which we use to apply LRP on VGG-16 [175] and ResNet-50 [65] in Section IV.”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 4×
“Consequently, we also exchange the neural network architecture and use an AlexNet [99] model, which is able to process the transformed input data using 2-D-convolution operators.”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“, [23], [52], [70], [107], [108], and [161]) have gained highest popularity.”
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 9×
“A comparison of performance on the Adience benchmark of the original model (pretrained on ImageNet only) and the improved model is given in the table below.”
MethodInfluentialRegression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 8×
“A comparison of performance on the Adience benchmark of the original model (pretrained on ImageNet only) and the improved model is given in the table below.”
BackgroundInfluentialThe Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“In games such as the arcade game Atari Breakout, the computer player would progressively learn strategies commonly employed by human players, such as “tunneldigging” [124], [200].”
Background
Research works printed but not linked: 13
- Arras, 2020, Ground truth evaluation of neural network explanations with CLEVR-XAI, arXiv:2003.07258 · arXiv
- 10.1146/annurev.neuro.26.041002.131047 · DOI · Paper page
- Chollet, 2015, Keras
- 10.1109/ICCV.2019.00304 · DOI
- Kokhlikyan, 2019, Pytorch Captum
- Lapuschkin, 2016, The layer-wise relevance propagation toolbox for artificial neural networks, J. Mach. Learn. Res., 17, 1
- 10.48550/arXiv.1312.4400 · DOI · Paper page
- 10.5555/2969033.2969153 · doi:10.5555/2969033.2969153
- 10.5555/3157382.3157477 · doi:10.5555/3157382.3157477
- Okuta, CuPy: A numpy-compatible library for NVIDIA GPU calculations, 151
- Oliphant, 2006, A Guide to NumPy, 1
- 10.48550/ARXIV.1404.7828 · DOI
- 10.5555/3045118.3045336 · doi:10.5555/3045118.3045336
Cited by
Items 901–1,000 of 1,002
SW-CAM: Class Activation Map with Feature Map Selection and Weight Modification
Qu, S., Hedjam, R., Wang, Z., Gu, Y., Yao, L., Li, H., & Zhong, G. (2024) · Asia Conference on Computer Vision, Image Processing and Pattern Recognition (CVIPPR) · doi:10.1145/3663976.3664232 · Cited by 1
“Explainable methods can be divided into self-explanatory models [1, 16] and post explanatory methods [10, 14, 22].”
Audio Detection and Fault Diagnosis of Ultra-High Voltage Transformers
Dong, W., Chi, R., Yang, H., & Wang, Z. (2024) · International Conference on Sentiment Analysis and Deep Learning (ICSADL) · doi:10.1109/icsadl61749.2024.00124 · Cited by 1
“Following the acquisition of the aforementioned features, the fault detection process involves the use of a deep neural network [19]-[20].”
MethodErvianto, E., Marpaung, N. L., Raisal, A. Y., Hutabarat, S., Hassan, R., Siagian, R. C., Nurhalim, N., & Amri, R. (2023) · Indonesian Review of Physics · doi:10.12928/irip.v6i2.8216 · Cited by 3
“Research by [71] shows that artificial neural networks, especially deep learning, have the ability to perform highly accurate predictions in various domains, including the prediction of environmental values such as UV Index.”
BackgroundGuo, Y., Zhou, D., Chen, H., Yue, X., & Cheng, Y. (2024) · Processes · doi:10.3390/pr12091999 · Cited by 2
An efficient deep neural network with amplifying sine unit for nonlinear oscillatory systems
Makhdoom, F., Jamshaid, U. R., & Naveed, A. (2024) · i-manager’s Journal on Mechanical Engineering · doi:10.26634/jme.14.2.21191 · Cited by 1
Chen, L., Fang, Z., Jin, X., & Cai, X. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.104185 · Cited by 2
City Recommender System: A Comparative Study of AI-Driven Approaches
Soleymani, D., Diva, M. M., Ozougwu, L. U., Mahbub, R., Sadeghi, Z., Samir, A. M., & Matwin, S. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-73125-9_17 · Cited by 1
Deep Networks for Medical Images Classification, a Comparative Study
Ibrahim, H. K., Rokbani, N., Wali, A., & Alimi, A. M. (2024) · IEEE International Conference on Artificial Intelligence & Green Energy (ICAIGE) · doi:10.1109/icaige62696.2024.10776632 · Cited by 1
“They have a very large panel of applications ranging such as speech recognition, medical image, medical data clustering, medical images classification, industrial applications, control systems and robotics [1].”
Liu, W., Guan, Y., Yan, X. H., He, S., Jiang, M., Gong, J., Chang, C., Shen, X., Liu, Y., & Zhang, G. (2025) · Physics of Fluids · doi:10.1063/5.0246235 · Cited by 4
Improvement of Weakly Supervised Building Region Extraction Using Graph‐Based Segmentation
Okajima, R., Futagami, T., & Hayasaka, N. (2025) · IEEJ Transactions on Electrical and Electronic Engineering · doi:10.1002/tee.70014 · Cited by 1
“In recent years, image recognition using deep neural networks (DNNs) [1] has been applied to automatic facade inspection systems [2,3].”
Revealing the Unseen: An Insightful Survey on Explainable AI for Deepfake Face Detection
Alotaibi, S. (2024) · International Conference on Developments in eSystems Engineering (DeSE) · doi:10.1109/dese63988.2024.10911942 · Cited by 2
“Despite existing surveys on explainability in AI such as [6] and [7], there is a lack of specific focus on XAI for deepfake detection models.”
Improving forecasting of concrete strength using advanced machine learning methods
Ellappan, P., Keshav, L., Raja, K. C. P., & Sanijya, G. (2025) · Matéria (Rio de Janeiro) · doi:10.1590/1517-7076-rmat-2024-0789 · Cited by 2
“Machine learning (ML) models have revolutionized the field of materials science and civil engineering by offering precise and reliable predictions of complex material behaviours [30].”
Filter differentiation: An effective approach to interpret convolutional neural networks
Fan, Y., Bao, H., & Xia, L. (2025) · Information Sciences · doi:10.1016/j.ins.2025.122253 · Cited by 2
Evolutionary Computation for Explainable Deep Learning
Zhou, R., & Hu, T. (2025) · Natural computing series · doi:10.1007/978-981-96-2540-6_4 · Cited by 1
Dutta, P., & Muppalaneni, N. B. (2025) · Evolving Systems · doi:10.1007/s12530-025-09693-x · Cited by 1
“…XAI in the domain of OCR has never been explored in the literature to the best of the author’s knowledge (Antoniadi et al. 2021; Islam et al. 2022; Samek et al. 2021; Ding et al. 2022) except for (Melnyk et al. 2020) that used Class Activation Mapping for visualizing the Chinese handwritten…”
Localization-Guided Supervision for Robust Medical Image Classification by Vision Transformers
Itzhak, S. B., Kiryati, N., Portnoy, O., & Mayer, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92648-8_8 · Cited by 3
“This sparked a surge of interest in explainable AI (XAI) that provides human-understandable justifications for model behavior [4,9,15,17,22,28,29].”
BackgroundPinto, W. N., & Shin, J. (2025) · Journal of Educational Measurement · doi:10.1111/jedm.12438 · Cited by 3
Khalique, V., Wang, Z., Horie, K., Kawana, F., Kokubo, T., Yanagisawa, M., & Kitagawa, H. (2025) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi64645.2025.00028 · Cited by 1
“Deep neural networks (DNN) offer a promising solution to this problem due to their ability to learn complex data patterns [20].”
Diffraction-informed deep learning for molecular-specific holograms of breast cancer cells
Song, T., Cao, M., Min, J., Im, H., Lee, H., & Lee, K. (2025) · APL Bioengineering · doi:10.1063/5.0246495 · Cited by 2
“42,43 However, because diffraction images are fundamentally different from typical images, conventional convolutional neural networks (CNNs) may not provide accurate interpretability for holograms, which is a reliable quantification of each input region ’ s contribution to the model ’ s…”
Gong, Y., Zhang, Y.-P., Li, J., Wu, S., Mahyuddin, M. N., & Seman, M. T. A. (2025) · Ironmaking & Steelmaking Processes Products and Applications · doi:10.1177/03019233251356174 · Cited by 2
UCFN: Uncertainty-aware cross-granularity fusion network for visual intention understanding
Xu, W., Li, X., Hu, C., & Liu, H. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131117 · Cited by 1
Lu, W., Zhao, Y. H., Yu, S., & Yang, J. (2025) · Ocean Engineering · doi:10.1016/j.oceaneng.2025.122384 · Cited by 1
Qian, S., Lu, K. J., Yang, P., & Zhu, K. (2025) · International Conference on Machine Learning and Intelligent Systems Engineering (MLISE) · doi:10.1109/mlise66443.2025.11100199 · Cited by 1
“It aims to create a suite of techniques that produce more explainable models whilst maintaining high performance levels [13].”
Neural Network-assisted Analysis of Tube Vocal Tract Models
Song, R., Sjons, J., & Ekström, A. G. (2026) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3qp8kma6ohov · Cited by 1
Omics landscapes in molecular mechanisms with Amomum tsaoko as an example
Fu, D., Wang, Y., & Zhang, J. (2025) · Food Chemistry Molecular Sciences · doi:10.1016/j.fochms.2025.100294 · Cited by 1
“(Ayesha et al., 2020; J. Bao et al., 2024; Bergenstråhle et al., 2022; Cai et al., 2023; Reel et al., 2021; Samek et al., 2021; Siblini et al., 2019; Yan & Wang, 2023 (GWAS) as five classical models, DNN can be applied to various histological data to predict phenotypes; it utilizes multilayered…”
Exploring Underlying Features in Hidden Layers of Neural Network
Tirumala, S. S., & Whalley, J. L. (2025) · Sensors · doi:10.3390/s25185755 · Cited by 1
“One of the most prominent areas of research since the discovery of the success of deep learning has been that of knowledge discovery [10,11].”
Missing Data Inference in Spatio-Temporal Mobile Crowd Sensing Using Graph Neural Network
Deuri, P., Nongmaithem, L., & Marchang, N. (2025) · IEEE Guwahati Subsection Conference (GCON) · doi:10.1109/gcon65540.2025.11173307 · Cited by 2
“Research in this field has significantly increased since the development of neural networks [9].”
Reconfigurable scattering potential for enhanced optical neuromorphic operations
Xia, T., Wu, C., Guo, D.-Y., Liu, B., Zhang, L., Lin, T., Ni, X., Khoo, I. C., & Liu, Z. (2025) · Optics Letters · doi:10.1364/ol.573330 · Cited by 1
Explaining anomalies through semi-supervised Autoencoders
Angiulli, F., Fassetti, F., Ferragina, L., & Nisticò, S. (2025) · Array · doi:10.1016/j.array.2025.100537 · Cited by 3
A visual language model enabling intelligent nanomaterial scanning electron micrograph annotation
Cai, Y.-Z., & Wang, H. (2025) · Nanoscale · doi:10.1039/d5nr03027g · Cited by 4
Zeng, L., & Chen, Z. (2025) · Intelligent Decision Technologies · doi:10.1177/18724981251387992 · Cited by 1
Zhan, Q., Wang, Z., Cao, Z., & Liu, X. (2025) · European Journal of Mechanics - B/Fluids · doi:10.1016/j.euromechflu.2025.204400 · Cited by 1
Explainable AI (XAI) for visual data: A survey of methods and applications
Kaur, G., Mehra, M., Chopra, R. K., Saini, S., Bansal, J., Kaur, P., & Singh, S. (2025) · AIP conference proceedings · doi:10.1063/5.0292577 · Cited by 1
An IPNN-Based Parameter Identification Method for a Vibration Sensor Sensitivity Model
Li, H., Liu, Z., Cai, C., Yao, K., Pan, J., & Yang, M. B. (2025) · Mathematics · doi:10.3390/math13223609 · Cited by 1
Genetic Algorithm to Understand Image Classification
Barreto, M. H. L., Oliveira, C. L. P., Santos, F., Nováis, P., Matos, L. N., & Britto, A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-97119-8_41 · Cited by 1
Bui, Q. D., Ha, H., Hoang, T., Nguyen, V., & Luu, C. (2025) · Innovative Infrastructure Solutions · doi:10.1007/s41062-025-02378-z · Cited by 1
“Using spatial analysis tools, these [45].”
Complex network dismantling with node diffusion and node similarity
Shi, J., Tang, J.-Y., Tan, Z., Hu, Y.-L., & Fang, Y. (2026) · Expert systems with applications · doi:10.1016/j.eswa.2025.130433 · Cited by 2
Qian, W., & Tian, L. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3635823 · Cited by 1
“A fundamental limitation stems from the inherent lack of interpretability, which severely constrains technological advancement in this domain [17], [18].”
KANs Layer Integration: Benchmarking Deep Learning Architectures for Tornado Prediction
Yang, S. (., Vilataj, E., Raza, M. F., & Srinivasan, S. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9120324 · Cited by 1
“This is particularly useful in scientific and safety-critical applications where understanding model behavior is essential [45,46].”
Detection of Wheat Powdery Mildew by Combined MVO_RF and Polarized Remote Sensing
Qian, Q., Tianquan, 梁. L., Wu, Z., Chen, X., Tang, Q., & Yu, Q. (2025) · Agriculture · doi:10.3390/agriculture15212268 · Cited by 1
Cengil, A. B., Ekşioğlu, B., Ekşioğlu, S. D., Hayes, C. J., BOGULSKI, C. A., & Ali, M. M. (2026) · JMIR Medical Informatics · doi:10.2196/78030 · Cited by 1
“These models were selected because they are widely used and complementary—random forests provide robustness and interpretability, XGBoost is highly efficient for structured tabular data, and DNNs are well-suited for capturing complex nonlinear relationships in large-scale datasets [36,37].”
Sulistianingsih, N., & Martono, G. H. (2025) · International Conference on Cybernetics and Intelligent System (ICORIS) · doi:10.1109/icoris67789.2025.11296036 · Cited by 1
“This not only strengthens trust in the model’s decisions but also supports its use in real-world marine conservation efforts [8].”
LSDA-YOLO: Enhanced SAR Target Detection with Large Kernel and SimAM Dual Attention
Yang, J., & Zhu, L. (2025) · Symmetry · doi:10.3390/sym18010023 · Cited by 2
Precision Strike: Targeted Misclassification of Accelerated CNNs with a Single Clock Glitch
Malik, A. A., Aydın, F., & Aysu, A. (2025) · IEEE Physical Assurance and Inspection of Electronics (PAINE) · doi:10.1109/paine66113.2025.11320182 · Cited by 1
“DNNs are a type of machine learning algorithm made up of multiple layers of interconnected nodes, or neurons [17].”
Richards, J., & Huser, R. (2026) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2025.2609641 · Cited by 6
“Recent work has been done on interpreting the outputs of machine learning algorithms, see, e.g., Bau et al. (2017); Zhang and Zhu (2018); Samek et al. (2021), but not in the context of performing regression.”
Raghupathi, W. (2026) · Algorithms · doi:10.3390/a19010063 · Cited by 2
PEER: Towards reliable and efficient inference via Patience-Based Early Exiting with Rejection
Zhan, Z., Zhou, S., & Zhang, R. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.104988 · Cited by 4
Large Language Models in the Assessment and Care of Internet Gaming Disorder
Kranas, A., & Verykios, V. S. (2026) · AI in Medicine · doi:10.3390/aimed1010006 · Cited by 1
“The opaque nature of contemporary LLMs complicates clinical validation and oversight, making explainable AI techniques essential to clarify how outputs are produced and to support auditability in practice [58].”
Novel deep learning method for detecting nucleon-nucleon correlations
Huang, Y.-J., Meng, Z., Pang, L.-G., & Wang, X.-N. (2026) · Physical Review C · doi:10.1103/hymw-tknk · Cited by 6
Niu, J., Liu, L., & zhou, W. (2026) · IEEE Signal Processing Letters · doi:10.1109/lsp.2026.3667452 · Cited by 1
“Deep neural networks (DNNs) [3], [4], [5], [6] have emerged as a core technology for remote sensing image understanding, supporting tasks including scene classification [7], [8], object recognition [9], [10], [11], [12], [13], and semantic segmentation [14].”
Relevance-Guided Activation Sparsification for Bandwidth-Efficient Collaborative Inference
Hoefler, M. A., Becking, D., Müller, K., Marpe, D., & Samek, W. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00798 · Cited by 3
“…of the ε -rule adds numerical stability—preventing relevance scores from blowing up when activations are near zero—and its layer-by-layer propagation naturally captures hierarchical, structure-level contributions (e.g. full contours and textures) that gradient-based saliencies often miss [3, 19].”
Self-citationHe, A., Wang, Q., Zhao, G., Fan, Y., Yu, K., Ren, Z., Yang, B., & Luo, S. (2026) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2026.141655 · Cited by 2
Li, C., Wang, H., Liu, J., Liu, J., Yue, Z., Li, X., Jiang, S., Deng, J., Hu, Y., & Yin, S. (2026) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca68181.2026.11408476 · Cited by 3
“It also integrates a deep neural network (DNN) [55], [61]-accelerated wafer-scale thermal simulator that efficiently maps power profiles to temperature fields, serving as a surrogate for computationally expensive ANSYS simulations.”
Cluster-aware deep learning-based vessel trajectory prediction for maritime traffic management
Li, Y., Song, L., Yang, Z., & Li, H. (2026) · Ocean & Coastal Management · doi:10.1016/j.ocecoaman.2026.108160 · Cited by 1
Suchithra, K. P., Mohan, N., Acharya, U. R., & Sachin Kumar, S. (2026) · Array · doi:10.1016/j.array.2026.100792 · Cited by 1
Yuan, X., Gu, W., Wang, M., Zhao, G., Li, P., & Atroshchenko, E. (2026) · Wave Motion · doi:10.1016/j.wavemoti.2026.103748 · Cited by 2
Hybrid and Physics-Informed AI Models in Earth System Analysis
Kaif, M., Halder, P., Mittu, S., Sarangi, A., & Pandey, N. K. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-0778-5.ch003 · Cited by 1
Martelli, M. E., Colella, S., Meloni, R., Gigliotti, F., Rosato, A., Panella, M., & Sogos, C. (2026) · NeuroSci · doi:10.3390/neurosci7020041 · Cited by 1
Chen, Z., LIANG, G., Liu, Y., Meng, X., Wang, B., Chen, Y., & Zhang, Y. (2026) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2026.103609 · Cited by 1
Pei, X., Yang, Z., Wang, Y., Zhang, D., Xu, P., Gu, T., & Fei, Q. (2026) · International Journal of Pressure Vessels and Piping · doi:10.1016/j.ijpvp.2026.105834 · Cited by 1
Artificial Intelligence Powers Protein Functional Annotation
Wang, W., Yang, Q., Zeng, M., Zheng, R., & Li, M. (2026) · Advanced Science · doi:10.1002/advs.202524373 · Cited by 1
“For example, Deep Neural Networks (DNNs) [ 30 ] often serve as embedding layers to project sparse one-hot representations of amino acids and domains into dense vectors, facilitating the capture of deep non-linear relationships.”
Li, W. H., & Zhou, Y. (2026) · GeoHazards · doi:10.3390/geohazards7020040 · Cited by 2
Abdelkafy, A. Y. (2026) · Computational Discovery and Intelligent Systems (CDIS) · doi:10.66279/z86vqm30 · Cited by 1
“These methods provide a direct correlation between the identified salient regions and the model’s predictive performance [31].”
Interpretable attention framework for materials property prediction and mechanistic discovery
Xu, P., Bai, B., Jia, L., Yan, J., Tian, M., He, X., & Yang, W. (2026) · Materials & Design · doi:10.1016/j.matdes.2026.116049 · Cited by 2
Bley, F., Kauffmann, J., Krug, S. L., Müller, K. R., & Montavon, G. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.113682 · Cited by 1
“Consequently, they face a trade-o ff , where performing more evaluations improves explanation accuracy but increases computational cost (cf. [22, 23]).”
Self-citationSun, X., Zhang, W., Li, Y., Zhang, J., Wang, X., Pan, J., & Wang, X. (2026) · Mathematics · doi:10.3390/math14091502 · Cited by 2
Абрамов, Д. В., Курочкина, Л. С., & Moreido, V. M. (2026) · Russian Meteorology and Hydrology · doi:10.3103/s1068373926030052 · Cited by 1
“Using the historical information that can be complemented with data of hydrological yearbooks, it is possible to look at the methods for assessing runoff changes at a new angle and to adopt modern statistical methods working with big data, for example, deep neural networks [46].”
Teingua, A. P. W., Saddem, R., & Lamiroy, B. (2026) · Control Engineering Practice · doi:10.1016/j.conengprac.2026.107067 · Cited by 2
Empowering healthcare 5.0 with deep learning: techniques, trends, and future directions
Maji, P. K., Chakraborty, S., Sadiq, A., Basu, S., & Ghosh, K. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11593-8 · Cited by 1
Idaji, M. J., Hense, J., Neuhauser, T., Krause, A., Luo, Y., Eberle, O., Schnake, T., Ciernik, L., Jafari, F. R., Vahidimajd, R., et al. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104148 · Cited by 10
“Layerwise Relevance Propagation (LRP) LRP [89, 36, 87, 90, 91] explains neural network predictions by redistributing the output’s relevance back through the network to the input features.”
Pradhan, K., & Anoop, S. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01534-3 · Cited by 1
“Nevertheless, issues of interpretability and clinician trust remained identified as hurdles to adoption in high-stakes settings [3, 26].”
InfluentialÖztaş, D., Salehi, H., DEMİR, Ö., Açıcı, K., Güzel, M. S., Eraslan, E. C., Sevindik, М., Akata, I., & Ekinci, F. (2026) · Plant Systematics and Evolution · doi:10.1007/s00606-026-02005-z · Cited by 2
“However, recent work has demonstrated that saliency and attribution methods can be sensitive to implementation details and model randomization, raising concerns regarding attribution artifacts and interpretability reliability (Adebayo et al. 2018; Samek et al. 2021).”
Tiwari, A. K., & Bhandari, A. K. (2026) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2026.100404 · Cited by 2
A partially linear model for longitudinal data using deep neural networks
Zhang, B., Xie, M., Zhou, J., & Wei, Z. (2026) · Science China Mathematics · doi:10.1007/s11425-025-2582-8 · Cited by 1
A conceptual link to long-wave and tsunami-like events using neural networks
Hemnath, N., Saha, S., & Raut, S. (2026) · Indian Journal of Physics · doi:10.1007/s12648-026-04116-2 · Cited by 1
Explainable AI for Early Detection of Neurological Disorders Using Multimodal Biomedical Data
Gautum, R., Tanwar, S., & Singh, R. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.06.574 · Cited by 1
Waseem, M., Ahmad, M., & Lakshmi, G. S. (2026) · Next Energy · doi:10.1016/j.nxener.2026.100759 · Cited by 2
Monirvaghefi, Z., & Asgarian, B. (2026) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2026.113149 · Cited by 1
Autonomous laboratories for sustainable nanomaterials discovery
Jathar, A., Nawghare, B. R., & Ganure, K. A. (2026) · Next Materials · doi:10.1016/j.nxmate.2026.102883 · Cited by 2
Olmo, P. D.-D., Arévalo, J. D., Villalaín, C., Herraìz, I., Galindo, A., Cernada, M., Kuligowski, J., Iglesia, I., Rodríguez, G., Larqué, E., et al. (2026) · Women s Health · doi:10.1177/17455057261475065 · Cited by 2
Artificial Intelligence in Biomaterials: Revolutionizing Design and Manufacturing
Liu, Y., Wang, Z., Wei, H., Ding, X., & Fan, Y. (2026) · Journal of Biomedical Materials Research Part A · doi:10.1002/jbm.a.70144 · Cited by 1
Shen, X., Shu, Z., & Bao, Z. (2026) · Technologies · doi:10.3390/technologies14090585 · Cited by 1
“Interpretability should be tested rather than illustrated: channel perturbation, temporal masking, frequency occlusion, and cross-participant stability can reveal whether a model depends on reproducible neural information [59,60].”
Deep Interpretation with Sign Separated and Contribution Recognized Decomposition
Hui, L. Y. W., & Soh, D. W. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85030-2_33 · Cited by 1
Langevin Cooling for Unsupervised Domain Translation
Srinivasan, V., Müller, K.-R., Samek, W., & Nakajima, S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3145812 · Cited by 2
“[93, 94, 95]), might help identify the reasons for successes and failures [96] of DT, suggesting possible ways to improve the performance.”
BackgroundSelf-citationAdversarial Defense Mechanisms for Supervised Learning
Chivukula, A. S., Yang, X., Liu, B., Liu, W., & Zhou, W. (2022) · Springer eBooks · doi:10.1007/978-3-030-99772-4_5 · Cited by 4
Yao, S. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.01406141 · Cited by 2
“And its application fields, application effects, and application prospects were analyzed and discussed [17].”
BackgroundAnalysis of Handwritten Texts to Detect Selected Psychological Characteristics of a Person
Woda, M., & Oliwa, G. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-37720-4_30 · Cited by 1
Understanding Artificial Intelligence Through Its Applications and Concerns
Maki, H. A., Mubarak, M. A., & Bakır, A. (2023) · Internet of things · doi:10.1007/978-3-031-35525-7_9 · Cited by 3
PMG-Net: Electronic Music Genre Classification using Deep Neural Networks
Tang, Y. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.0140877 · Cited by 1
“DNN is a technology in the field of machine learning [17].”
BackgroundFloreale, G., Baraldi, P., Zio, E., & Fink, O. (2023) · European Safety and Reliability Conference (ESREL) · doi:10.3850/978-981-18-8071-1_p662-cd · Cited by 1
“Different types of explainability techniques have been proposed to identify the elements of the input that contribute the most to the output of the DL model(Samek et al. 2021).”
Buffer ladder feature fusion architecture for semantic segmentation improvement
Liu, Z., & Lei, Z. (2023) · Signal Image and Video Processing · doi:10.1007/s11760-023-02754-1 · Cited by 1
“Based on the current analysis of the performance of deep neural networks [21], the robustness of networks has become an important basis for measuring network performance, which inspires to select the backbone network.”
Comparison of Fourier Bases and Asymmetric Network Bases in the Bio-Inspired Networks
Ishii, N., Iwata, K., Iwahori, Y., & Matsuo, T. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43085-5_16 · Cited by 1
Deep Learning-Based Stress Identification in Children: Unraveling Physiological Patterns
P, H. V. V., Shahkutty, A., & Akshay, S. (2023) · IJIREEICE · doi:10.17148/ijireeice.2024.12103 · Cited by 1
Case Study: Using Data Mining to Predict Student Performance Based on Demographic Attributes
Zahruddin, N. A. B. M., Kamarudin, N. D., Jusoh, R. M., Fataf, N. A. A., & Hidayat, R. S. A. (2023) · JOIV International Journal on Informatics Visualization · doi:10.30630/joiv.7.4.2454 · Cited by 1
Zhou, C., Deng, J., & Tang, H. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-9299-7_8 · Cited by 1
Towards explaining deep neural network-based heart age estimation
Hempel, P., Bender, T., Gandhi, K., & Spicher, N. (2023) · IEEE EMBS Special Topic Conference on Data Science and Engineering in Healthcare, Medicine and Biology · doi:10.1109/ieeeconf58974.2023.10405038 · Cited by 1
“Hence, methods of explainable artificial intelligence (XAI) are researched intensively [1] and especially in time series analysis [2], [3].”
MethodUnsupervised Traffic Sign Classification Relying on Explanatory Visible Factors
Wöber, W., Waikat, J., Mehnen, L., & Olaverri-Monreal, C. (2023) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc57777.2023.10422705 · Cited by 1
“However, a common critical point of CNNs relates to the fact that the training processes involved are hard to explain [4], [5], [6].”
BackgroundRoot Cause Explanation of Outliers under Noisy Mechanisms
Nguyen, P., Tran, T., Gupta, S., Nguyen, T., & Venkatesh, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.30035 · Cited by 6
Datasets Collection Framework for Low-Resourced Languages in South Africa
Magangane, N. M., Zwane, S. G., & Adigun, M. O. (2024) · Conference on Information Communications Technology and Society (ICTAS) · doi:10.1109/ictas59620.2024.10507140 · Cited by 1
“A large number of such layers are combined to give the model a high prediction power [18].”
BackgroundRemarks on an Optimal Predictive Control Using a Quaternion Neural Network-based Identifier
Takahashi, K., Aoki, R., Kagamiishi, E., & Hashimoto, M. (2024) · IEEE International Workshop on Advanced Motion Control (AMC) · doi:10.1109/amc58169.2024.10505702 · Cited by 2
“Neural networks (NNs) trained with deep learning techniques have proven successful in addressing intractable problems across various fields [1]–[4].”
Background
Metadata
- Authors
- Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R.
- Affiliations
- Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
- BIFOLD✔Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
- Berlin Institute for the Foundations of Learning and Data (DE)
- ORCID
- Venue
- Proceedings of the IEEE
- Published
- 2021-03
- Volume
- 109
- Issue
- 3
- Pages
- 247-278
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W3132191748
- Cited by
- 1,477
- References
- 230
- Citations by year
- 2026: 305 · 2025: 343 · 2024: 326 · 2023: 273 · 2022: 152 · 2021: 74
- FWCI
- 72.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the IEEE (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion, other-oa)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Institute of Information & Communications Technology Planning & Evaluation (IITP) grants funded by the Korea Government (2017-0-00451, 2019-0-00079)
- German Ministry for Education and Research (01IS14013A-E, 01GQ1115, 01GQ0850, 01IS18025A, 01IS18037A)
- German Research Foundation (Math+, EXC 2046/1, 390685689)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- explainable AI, interpretability, deep neural network, post-hoc explanation, interpretability evaluation, nonlinear machine learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.