Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Open accessThe DOI matches its Crossref record; 224 of the 230 references it cites were found in a registry or an index.
Abstract
With the broader and highly successful usage of machine learning (ML) in industry and the sciences, there has been a growing demand for explainable artificial intelligence (XAI). Interpretability and explanation methods for gaining a better understanding of the problem-solving abilities and strategies of nonlinear ML, in particular, deep neural networks, are, therefore, receiving increased attention. In this work, we aim to: 1) provide a timely overview of this active emerging field, with a focus on “post hoc” explanations, and explain its theoretical foundations; 2) put interpretability algorithms to a test both from a theory and comparative evaluation perspective using extensive simulations; 3) outline best practice aspects, i.e., how to best include interpretation methods into the standard usage of ML; and 4) demonstrate successful usage of XAI in a representative selection of application scenarios. Finally, we discuss challenges and possible future directions of this exciting foundational field of ML.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“While XAI techniques exist [47], their application in complex, real-time energy systems, and providing meaningful explanations to non-experts, remain challenging”
From Addressing Challenges for the Effective Adoption of Artificial Intelligence in the Energy Sector (2025) · cites this paper as [47]
In the citing paper
“Despite previous attempts to address the challenges surrounding model interpretability, most solutions have been implemented post hoc (such as the attention mechanism) focusing on explanations in certain domains, such as vision [184].”
From Human-centred artificial intelligence for mobile health sensing: challenges and opportunities (2023) · cites this paper as [184]
In the citing paper
“LIME produces an explanation list that shows the contribution of individual features to the prediction.”
From Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2022) · cites this paper as [64]
In the citing paper
“This local explanation allows the end-user to determine which feature is important for the precise prediction and how the perturbation in feature affects the prediction results.”
From Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2022) · cites this paper as [64]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 12×
“This modified gradient view on LRP can also be leveraged to achieve a simpler and more general implementation of LRP based on ‘forward hooks’, which we describe in the second part of Appendix B, and which we use to apply LRP on VGG-16 [175] and ResNet-50 [65] in Section IV.”
BackgroundMethodInfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 6×
“This modified gradient view on LRP can also be leveraged to achieve a simpler and more general implementation of LRP based on ‘forward hooks’, which we describe in the second part of Appendix B, and which we use to apply LRP on VGG-16 [175] and ResNet-50 [65] in Section IV.”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 4×
“Consequently, we also exchange the neural network architecture and use an AlexNet [99] model, which is able to process the transformed input data using 2-D-convolution operators.”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“, [23], [52], [70], [107], [108], and [161]) have gained highest popularity.”
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 9×
“A comparison of performance on the Adience benchmark of the original model (pretrained on ImageNet only) and the improved model is given in the table below.”
MethodInfluentialRegression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 8×
“A comparison of performance on the Adience benchmark of the original model (pretrained on ImageNet only) and the improved model is given in the table below.”
BackgroundInfluentialThe Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“In games such as the arcade game Atari Breakout, the computer player would progressively learn strategies commonly employed by human players, such as “tunneldigging” [124], [200].”
Background
Research works printed but not linked: 13
- Arras, 2020, Ground truth evaluation of neural network explanations with CLEVR-XAI, arXiv:2003.07258 · arXiv
- 10.1146/annurev.neuro.26.041002.131047 · DOI · Paper page
- Chollet, 2015, Keras
- 10.1109/ICCV.2019.00304 · DOI
- Kokhlikyan, 2019, Pytorch Captum
- Lapuschkin, 2016, The layer-wise relevance propagation toolbox for artificial neural networks, J. Mach. Learn. Res., 17, 1
- 10.48550/arXiv.1312.4400 · DOI · Paper page
- 10.5555/2969033.2969153 · doi:10.5555/2969033.2969153
- 10.5555/3157382.3157477 · doi:10.5555/3157382.3157477
- Okuta, CuPy: A numpy-compatible library for NVIDIA GPU calculations, 151
- Oliphant, 2006, A Guide to NumPy, 1
- 10.48550/ARXIV.1404.7828 · DOI
- 10.5555/3045118.3045336 · doi:10.5555/3045118.3045336
Cited by
Items 701–800 of 1,002
Investigating Data Hierarchies in Multifidelity Machine Learning for Excitation Energies
Vinod, V., & Zaspel, P. (2025) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.4c01491 · Cited by 6
“Both have seen rapid development in tandem allowing for quick prediction of QC properties in place of the costly conventional calculations [1, 2, 9, 10, 15, 23].”
Canha, D., Kubler, S., Främling, K., & Fagherazzi, G. (2025) · ACM Computing Surveys · doi:10.1145/3737445 · Cited by 10
Safety and Trust in Artificial Intelligence with Abstract Interpretation
Singh, G., Laurel, J., Misailovíc, S., Banerjee, D., Singh, A., Xu, C., Ugare, S., & Zhang, H. (2025) · Foundations and Trends® in Programming Languages · doi:10.1561/2500000062 · Cited by 11
[Retracted] Research on Information Visualization Graphic Design Teaching Based on DBN Algorithm
Man-jun, X. (2021) · Computational Intelligence and Neuroscience · doi:10.1155/2021/3355030 · Cited by 4
“In the teaching design, we need to study the visual graphic display of data information acquisition, capture, screening, logical structure, and so on [5, 6].”
BackgroundRetractedInvestigating Shape Variation Using Generalized Procrustes Analysis and Machine Learning
Wöber, W., Mehnen, L., Curto, M., Tibihika, P. D., Tesfaye, G., & Meimberg, H. (2022) · Applied Sciences · doi:10.3390/app12063158 · Cited by 3
Feng, Y., Huang, P. C., Chen, Y., Zhang, Y., Li, R., Zhou, Z., Liu, L., Liu, X., & Kang, J. (2023) · IEEE Electron Device Letters · doi:10.1109/led.2023.3239608 · Cited by 4
“O VER the last decade, the deep neural networks (DNNs) has demonstrated spectacular capabilities for artificial intelligence [1], [2], [3].”
BackgroundChallenges for cognitive decoding using deep learning methodsPreprint
Thomas, A. W., Ré, C., & Poldrack, R. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06896 · Cited by 7
ECQ x : Explainability-Driven Quantization for Low-Bit and Sparse DNNs.Preprint
Becking, D., Dreyer, M., Samek, W., Müller, K., & Lapuschkin, S. (2021) · arXiv (Cornell University) · Cited by 4
Self-citationRobustMap: Visual Exploration of DNN Adversarial Robustness in Generative Latent Space
Li, J., & Kuang, J. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3471551 · Cited by 5
“A well-trained DNN with high accuracy may be less robust [2], [3], [4], which is vulnerable and likely to output biased results confronted with perturbed data.”
Flow time history deep learning for feature decomposition and disentanglement
Zhan, Q., Liu, X., Bai, C., Chao, Y., Bao, D., Wang, Z., & Sun, X. (2024) · Physica D Nonlinear Phenomena · doi:10.1016/j.physd.2024.134470 · Cited by 3
AlGaiar, M. M., Bano, S., Lashin, A., Hossain, M., Faisal, N. H., & Salem, H. S. A. (2025) · Renewable Energy · doi:10.1016/j.renene.2025.123154 · Cited by 5
Apriyanti, D. H., Spreeuwers, L., & Lucas, P. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111961 · Cited by 4
Kernel Stochastic Configuration Networks for Industrial Data Regression
Chen, Y., & Wang, D. (2025) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2025.3598447 · Cited by 4
“N EURAL networks have been widely applied for nonlinear regression analysis, aiming at establishing a feasible representation between observed predictors and responses through learning from a collection of samples [1], [2], [3].”
Lu, Y.-S., Yamanaka, K., Ishii, H., Bui, T.-X., Hsieh, Y.-J., & Fang, T. (2025) · Materials Today Physics · doi:10.1016/j.mtphys.2025.101900 · Cited by 3
Lai, S., Liao, M., Hu, Z., Yang, J., Chen, W., Xiao, H., Tang, J., Liao, H., & Yue, Y. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3758157 · Cited by 5
“Deep learning has profoundly transformed numerous fields, achieving unprecedented performance across a diverse range of tasks, particularly in multimodal data processing such as computer vision, natural language processing, and audio analysis [4, 10].”
Newen, C., Müller, E., & Newen, A. (2025) · Topoi · doi:10.1007/s11245-025-10287-0 · Cited by 7
O’Shea, K., Yan, S., Yu, M., Chen, X., Mauceri, S., Dhariyal, B., Xu, L., O'Connor, N. E., & Liu, M. (2025) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2025.3634737 · Cited by 5
“Furthermore, attention weights serve as a means of improving the explainability for attention-based models [58], [74].”
Wang, Y., Wu, T., Li, X., Yu, H., Yu, H., Cen, S., Wáng, Y., & Li, Z. (2025) · Chinese Journal of Natural Medicines · doi:10.1016/s1875-5364(25)60967-8 · Cited by 8
Self-powered flexible wireless sensing for smart farming machinery
Xiang, P., Xue, M., Chen, X., Liu, F., Wu, X., Wang, S., Zhang, W., & Xiao, X. (2025) · Nano Energy · doi:10.1016/j.nanoen.2025.111663 · Cited by 3
Ribeiro, E., Dias, F. M., Soares, Q. B., Krieger, J. E., & Gutierrez, M. A. (2025) · Research on Biomedical Engineering · doi:10.1007/s42600-025-00432-4 · Cited by 5
“Specifically, we progressively masked (set to zero) image segments in descending order of relevance, as determined by XAI method, and observed the corresponding drop in the prediction score (Samek et al. 2021).”
Uncovering spatial process heterogeneity from graph-based deep spatial regression
Zhu, D., Wang, S., & Luo, P. (2026) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.12.008 · Cited by 3
Jin, Z., Ćetenović, D., Mitrovic, M., Levi, V. A., Bečejac, V., Taylor, P., & Terzija, V. V. (2026) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2025.116560 · Cited by 3
Artificial intelligence in dermatology: Clinical promise and environmental impact
Shen, C. Z., Zhao, A. T., Rotemberg, V. M., Daneshjou, R., Parker, E. R., & Rosenbach, M. A. (2026) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2026.03.039 · Cited by 3
Transparency in Medical Artificial Intelligence Systems
Quakulinski, L., Koumpis, A., & Beyan, O. (2023) · International Journal of Semantic Computing · doi:10.1142/s1793351x23630011 · Cited by 3
On an Intelligent Hybrid Intuitionistic Fuzzy Method for Breast Cancer Classification
Sotirova, E., Bozov, H., Stewart, R., Sotirov, S., & Dobrev, P. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-39777-6_9 · Cited by 3
Mamalakis, M., Macfarlane, S. C., Notley, S. V., Gad, A. K. B., & Panoutsos, G. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109052 · Cited by 6
Verifying Neural Networks with Non-Linear SMT Solvers: a Short Status Report
Guidotti, D., Pandolfo, L., & Pulina, L. (2023) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai59109.2023.00068 · Cited by 3
“On one hand, the intrinsic complexity of understanding why a neural network behaves in a certain way has proven to be quite hard to overcome, even if in this regard a great deal of research has been done (see, e.g., [2]).”
BackgroundCollision Classification for Unmanned Surface Vehicle using Inertial Measurement Unit Data
Nguyen, Q. M., Luu, T., Tran, T., Nguyen, C., Tran, M., & Nguyen, H. D. (2024) · International Conference on Mechatronics, Control and Robotics (ICMCR) · doi:10.1109/icmcr60777.2024.10481927 · Cited by 3
“In literature, there are several AI techniques successfully employed to classify images, including Artificial Neural Networks (ANN) [9], [10], Deep Neural Networks (DNN) [11]–[13], and Reinforcement Learning (RL) [14], [15].”
MethodInversion of Aerosol Optical Depth: Incorporating Multimodel Approach
Sun, X., Sun, L., Sun, Y., Zhang, J., Fan, X., & Ma, C. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3397315 · Cited by 5
“Machine learning (ML), such as decision trees, with limited model depth for a specific dataset, cannot fully capturing the complexity and diversity of the data, unlike deep neural networks that can arbitrarily increase complexity [36].”
BackgroundFeasibility Analysis of Applying Deep Neural Network on Driving Distance Estimation
Lee, S., & Moon, J. (2024) · IEEE Access · doi:10.1109/access.2024.3410316 · Cited by 3
“It is well known that DNN can be applied to a very diverse range of problems, but sometimes it is not easy to interpret why it works [18], [19], [20], [21], [22].”
BackgroundYang, Z., Su, B., Ding, H., Qiu, Y., & Zhong, D. (2024) · Road Materials and Pavement Design · doi:10.1080/14680629.2024.2383915 · Cited by 4
The impact of feature representation on the accuracy of photonic neural networks
de Queiroz, M. G., Jimenez, P., Cardoso, R., Costa, M. V., Abdalla, M., O’Connor, I., Bosio, A., & Pavanello, F. (2024) · APL Machine Learning · doi:10.1063/5.0226172 · Cited by 4
Jiang, W., Yan, F., Ren, K., Zhang, X., Wei, B., & Zhang, M. (2024) · Electronics · doi:10.3390/electronics13183757 · Cited by 4
“After fully pre-training the BERT model and GCN model by using rich comment data on source tasks, various rich knowledge such as user information, message content, and message comments of rumor text is obtained, and some parameters of the GCN layer are migrated.”
Fisher regularized discriminative broad learning system for visual classification
Li, X., Wei, J., Jin, J., Xu, T., & Yu, D. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112341 · Cited by 3
Advancing Aerial Imagery Analysis: A Comparative Study of YOLOv3 and YOLOv8 on the VisDrone Dataset
Chandrasekaran, N., Alamsyah, N., Haq, Q. M. U., Fan, H.-W., Chou, C.-Y., & Lu, K. (2024) · International Conference on Electronic and Electrical Engineering and Intelligent System (ICE3IS) · doi:10.1109/ice3is62977.2024.10775447 · Cited by 3
“Research shows that YOLOv4's enhancements include adaptive attention mechanisms and high-resolution processing, which are crucial for handling aerial images with varying object scales and complex backgrounds [24], [25].”
Hoffman, L., Mazloff, M. R., Gille, S. T., Giglio, D., & Heimbach, P. (2025) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-24-0027.1 · Cited by 7
A new protection scheme for loss of excitation based on CNN
Hosseini, M., Moravej, Z., & Pazoki, M. (2025) · Electrical Engineering · doi:10.1007/s00202-025-02966-3 · Cited by 3
Yang, W., Tang, J., Tian, H., & Wang, T. (2025) · Sustainability · doi:10.3390/su17073048 · Cited by 3
Learning-based prediction and mix proportion analysis of basic magnesium sulfate cement concrete
Wu, Y., Feng, T., Yu, H., & Tan, Y. (2025) · Materials Today Communications · doi:10.1016/j.mtcomm.2025.112567 · Cited by 3
Yang, J., Guo, L., Li, Y., Wang, W., Li, Z., Li, M., Huang, Z., Lin, Y., Yin, Y., & Xu, H. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10993212 · Cited by 3
“However, deep neural networks (DNNs) possess powerful nonlinear fitting capabilities [24].”
Metric tools for sensitivity analysis with applications to neural networks
Gonzalo, J. P., Alfaya, D., Portela, J., & Muñoz, A. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113300 · Cited by 5
Nalbantoğlu, Ö. U., Ermiş, B. H., & Gündoğdu, A. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108146 · Cited by 4
Abi, R. (2025) · International Journal of Research Publication and Reviews · doi:10.55248/gengpi.6.0625.2126 · Cited by 22
Barhoon, M., & Mahdiuni, H. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c01029 · Cited by 3
Don’t get me wrong: How to apply deep visual interpretations to time series
Löffler, C., Lai, W.-C., Zanca, D., Schmidt, L., Eskofier, B. M., & Mutschler, C. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06798-3 · Cited by 13
Moges, W. K., Tegegne, A. S., Mitku, A. A., Tesfahun, E., & Hailemeskel, S. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1484299 · Cited by 3
“This approach helps to detect targeted interventions to prevent LBW, leading to better health outcomes for mothers and babies, and it also helps to connect causal inference and DL within quasi-experimental settings (Zhang et al., 2022; Samek et al., 2021).”
Zhou, R., & Paesani, F. (2025) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.5c01335 · Cited by 7
Some game theoretic marketing attribution models
Molina, E., Tejada, J., & Weiss, T. (2022) · Annals of Operations Research · doi:10.1007/s10479-022-04944-5 · Cited by 5
“…(ii) its simple expression in this case, which is also computationally tractable; and (iii) its widespread use in real-world applications, which has increased substantially in recent years (see Moretti and Patrone 2008; Thomson, 2019b; Sánchez-Soriano, 2019; Samek et al.,2021, among many others).”
BackgroundKünstliche Intelligenz als Lösung des PathologInnenmangels?
Jurmeister, P. S., Müller, K., & Klauschen, F. (2022) · Der Pathologe · doi:10.1007/s00292-022-01071-7 · Cited by 4
A Whale’s Tail - Finding the Right Whale in an Uncertain World
Marcos, D., Kierdorf, J., Cheeseman, T., Tuia, D., & Roscher, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-04083-2_15 · Cited by 5
Szczepański, M., Pawlicki, M., Kozik, R., & Choraś, M. (2022) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa54385.2022.10032372 · Cited by 3
“As shown in [44] and [45], the size of the explanation can be directly linked to the interpretability.”
BackgroundToward Better Analysis of Breast Cancer Diagnosis: Interpretable AI for Breast Cancer Classification
Prananda, A. R., & Frannita, E. L. (2023) · IT JOURNAL RESEARCH AND DEVELOPMENT · doi:10.25299/itjrd.2023.11563 · Cited by 3
Italian sentiment analysis on climate change: Emerging patterns from 2016 to today
Bruno, M., Scannapieco, M., Catanese, E., & Valentino, L. (2023) · Statistical Journal of the IAOS · doi:10.3233/sji-220064 · Cited by 3
“A detailed description of such models is out of the scope of the paper, nevertheless a brief description of the methods and main applications of DNNs can be found in [37,38].”
SIGNed explanations: Unveiling relevant features by reducing bias
Gumpfer, N., Prim, J., Keller, T., Seeger, B., Guckert, M., & Hannig, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101883 · Cited by 9
Gotovtsev, P. M., Gorin, K. V., Sergeeva, Y. E., Parunova, Y. M., Vishnevskaya, M. V., Sukhinov, D. V., Petrova, M. G., Migalev, A. S., Pozhidaev, V. M., Gaeva, T. N., et al. (2023) · Nanobiotechnology Reports · doi:10.1134/s2635167623010032 · Cited by 4
Bohnsack, K. S., Voigt, J., Kaden, M., Heinke, F., & Villmann, T. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126632 · Cited by 3
Agiollo, A., Siebert, L. C., Murukannaiah, P. K., & Omicini, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40878-6_6 · Cited by 7
Diabetes Detection Using Gait Analysis and Machine Learning
Chee, L. Z., Hwong, H. H., & Sivakumar, S. (2023) · International Conference on Digital Applications, Transformation & Economy (ICDATE) · doi:10.1109/icdate58146.2023.10248507 · Cited by 3
“DNN is applied in disease diagnosis, stock price detection and image generation.”
BackgroundInfluentialImproving Environmental Water Supply in Wetlands through Optimal Cropping Patterns
Sedighkia, M., & Datta, B. (2023) · Agriculture · doi:10.3390/agriculture13101942 · Cited by 3
DExT: Detector Explanation Toolkit
Padmanabhan, D. C., Plöger, P. G., Arriaga, O., & Valdenegro-Toro, M. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44067-0_23 · Cited by 5
Longo, L. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44070-0 · Cited by 7
“One prominent way to address these growing concerns and new objectives is for modern (blackbox) AI methods to be able to explain their outputs [1], leading to a surge in the development of explainable AI (XAI) over the last years for a host of different applications, domains, and data types [1, 16, 50].”
Quantitative Prediction of Rock Pore-Throat Radius Based on Deep Neural Network
Hong, Y., Li, S., Wang, H., Liu, P., & Cao, Y. (2023) · Energies · doi:10.3390/en16217277 · Cited by 3
Recent advances in longitudinal data analysis
Fu, L., Wang, Y., & Wu, J. (2024) · Handbook of statistics · doi:10.1016/bs.host.2023.10.007 · Cited by 4
Case Study: Using Data Mining to Predict Student Performance Based on Demographic Attributes
Zahruddin, N. A. B. M., Kamarudin, N. D., Jusoh, R. M., Fataf, N. A. A., & Hidayat, R. S. A. (2023) · JOIV International Journal on Informatics Visualization · doi:10.30630/joiv.7.4.02454 · Cited by 3
Automatic Detection of Ballast Unevenness Using Deep Neural Network
Bojarczak, P., Lesiak, P., & Nowakowski, W. (2024) · Applied Sciences · doi:10.3390/app14072811 · Cited by 3
Weakly supervised anomaly detection based on sparsity prior
Wang, K., Zhang, S., Cao, Y., & Yang, L. (2024) · Electronic Research Archive · doi:10.3934/era.2024169 · Cited by 4
“In industry [1], it is employed to address safety and security concerns.”
BackgroundA Review of Semantic Annotation in the Context of the Linked Open Data Cloud
Kadhim, K. J., & Hadi, A. S. (2024) · Ingénierie des systèmes d information · doi:10.18280/isi.290330 · Cited by 3
“In NLP, deep neural networks, such as transformer models and recurrent neural networks (RNNs), have the ability to be trained on a huge number of textual datasets for the purpose of learning semantic representations of words or sentences [38].”
BackgroundEyharabide, V. (2024) · Digital Medievalist · doi:10.16995/dm.15119 · Cited by 3
Thy-DAMP: deep artificial neural network model for prediction of thyroid cancer mortality
Barfejani, A. H., Rahimi, M., Safdari, H., Gholizadeh, S., Borzooei, S., Roshanaei, G., Golparian, M., & Tarokhian, A. (2024) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-024-08918-0 · Cited by 4
“Furthermore, they can enhance cur - rent rule-based models by incorporating their variables [ 7 ].”
Electromyography-based Hand Gesture Classification Using IGOA and DNN
Agarwal, S., Kaur, B., & Bhati, B. S. (2024) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt61001.2024.10724945 · Cited by 3
“The proposed work uses a DNN [28] toward the improvement of EMG-based gesture classification.”
MethodWieczorek, O., Steinhardt, I., Schmidt, R., Mauermeister, S., & Schneijderberg, C. (2024) · Futures · doi:10.1016/j.futures.2024.103537 · Cited by 10
An effective image annotation using self-attention based stacked bidirectional capsule network
Palekar, V. R., & Sathish Kumar, L. (2025) · Computer Standards & Interfaces · doi:10.1016/j.csi.2025.103973 · Cited by 3
A Dual Approach with Grad-CAM and Layer-Wise Relevance Propagation for CNN Models Explainability
Mishra, A., & Malhotra, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-80842-5_10 · Cited by 4
Generalizing neural network verification to the family of piece-wise linear activation functions
Antal, L., Ábrahám, E., & Masara, H. (2025) · Science of Computer Programming · doi:10.1016/j.scico.2025.103269 · Cited by 6
Vafaie, S., Bal, D., Thorne, M. A. S., & Forgoston, E. (2025) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0240788 · Cited by 3
Albeladi, K., Zafar, B. A., & Mueen, A. (2024) · Revista Gestão & Tecnologia · doi:10.20397/2177-6652/2024.v24ispecial.2818 · Cited by 3
“In real-world problem-solving, shallow neural networks may require significant complexity to achieve optimal predictions, whereas DNNs have been suggested as a means of developing more effective predictive models (Samek et al., 2021).”
Short video content and recommendation algorithm based on deep learning
Yang, Y. (2025) · Journal of Computational Methods in Sciences and Engineering · doi:10.1177/14727978251337878 · Cited by 4
“Using the weighted summation method, the fi nal fusion feature expression of the behavior data of user u is: Based on obtaining the fusion features of user behavior, a user interest model is established using DNN. DNN can extract complex interest patterns from high-dimensional and dense data, and is a current research…”
XpertAI: Uncovering Regression Model Strategies for Sub-manifolds
Letzgus, S., Müller, K., & Montavon, G. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08327-2_19 · Cited by 4
“Explainable artificial intelligence (XAI) has emerged as a step towards enhancing transparency and allows for insights into the inner workings of these highly complex AI models [5,40].”
Self-citationZhang, X., & Luo, G. (2021) · JMIR Medical Informatics · doi:10.2196/28287 · Cited by 3
“As surveyed in the book written by Molnar [29] and the previous papers written by several research groups [30,46-48], other researchers have proposed many automated methods to explain machine learning predictions.”
BackgroundMethodExplanation of Anomalies in Business Process Event Logs with Linguistic Summaries
Chouhan, S., Wilbik, A., & Dijkman, R. (2022) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee55066.2022.9882673 · Cited by 4
Viaña, J., Ralescu, S., Крейнович, В., Ralescu, A., & Cohen, K. D. O. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-16038-7_30 · Cited by 3
Measurably Stronger Explanation Reliability Via Model Canonization
Motzkus, F., Weber, L., & Lapuschkin, S. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897282 · Cited by 7
“Current work from the field has highlighted the usefulness of contemporary XAI approaches for understanding and improving models and datasets in various application scenarios [1].”
BackgroundSelf-citationA self-explanatory contrastive logical knowledge learning method for sentiment analysis
Chen, Y., Yuan, B., Liao, B., & Gabbay, D. M. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110863 · Cited by 5
AIM and Explainable Methods in Medical Imaging and Diagnostics
Anwar, S. M. (2021) · Springer eBooks · doi:10.1007/978-3-030-58080-3_293-1 · Cited by 3
Fahmy, H. M., Pastore, F., & Briand, L. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510454.3516858 · Cited by 3
“A number of tools supporting DNN explanation are available nowadays [4, 16].”
MethodDhaubhadel, S., Kolade, B., Ribeiro, R. M., Ganguly, K., Hengartner, N., Bhattacharya, T., Cohn, J. D., Agarwal, K., Cho, K., Costa, L., et al. (2022) · medRxiv · doi:10.1101/2022.10.31.22281742 · Cited by 3
“While techniques exist to compare predictor variables to mechanisms for non-linear models [66], these methods themselves are complex and we chose not to explore them further here.”
MethodMetric Tools for Sensitivity Analysis with Applications to Neural NetworksPreprint
Gonzalo, J. P., Alfaya, D., Portela, J., & Roque, A. M. S. (2023) · doi:10.36227/techrxiv.22679269.v1 · Cited by 4
Dai, X., Ke, Z., Li, M., Dong, L., & Atluri, S. N. (2024) · International Journal for Numerical Methods in Engineering · doi:10.1002/nme.7455 · Cited by 2
Pednekar, M. S., & Bhattacharyya, K. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19490-4 · Cited by 2
A Guided Tour of Post-hoc XAI Techniques in Image Segmentation
Hasany, S. N., Mériaudeau, F., & Petitjean, C. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-63797-1_9 · Cited by 3
Vilela, L. F. C., Cabral, N. O., Destêfani, A. C., & Destefani, V. C. (2024) · Revista Ibero-Americana de Humanidades, Ciências e Educação · doi:10.51891/rease.v10i8.15395 · Cited by 8
“Efforts to develop explainable AI models and visualize their decision-making process are crucial to build trust and facilitate clinical adoption (33).”
BackgroundA Hybrid ODE-NN Framework for Modeling Incomplete Physiological Systems
Demirkaya, A., Lockwood, K., Stratis, G., Imbiriba, T., Ilieş, I., Rampersad, S. M., Alhajjar, E., Guidoboni, G., Danziger, Z. C., & Erdogmus, D. (2024) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2024.3505796 · Cited by 8
“The field of explainable AI attempts to address this issue, and recent efforts to make data-driven methods more interpretable are described by Samek et al. [42].”
Lasota, E., Houben, T., Polz, J., Schmidt, L., Glawion, L., Schäfer, D., Bumberger, J., & Chwala, C. (2024) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-24-0032.1 · Cited by 4
Comprehensive Study on Reinforcement Learning and Deep Reinforcement Learning Schemes
Azhar, M. H., Khuhro, M. A., Waqas, M., Saeed, U., & Niazi, M. K. (2024) · Sir Syed University Research Journal of Engineering & Technology · doi:10.33317/ssurj.638 · Cited by 2
Deep evidential reasoning rule learning
Liu, H., & Zhou, Z. (2025) · Signal Processing · doi:10.1016/j.sigpro.2025.109984 · Cited by 2
Li, Y., Yang, J., & Zhang, J. (2025) · Water · doi:10.3390/w17060842 · Cited by 2
A theory of fine-grained lineage for functions on structured objects
Hallé, S., & Tremblay, H. (2025) · Theoretical Computer Science · doi:10.1016/j.tcs.2025.115192 · Cited by 3
A method for explaining individual predictions in neural networks
Oh, S. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2802 · Cited by 4
“Although they can explain image classi fi cation results to some extent ( Samek et al., 2021 ; Lin, Lee & Celik, 2021 ; Rguibi et al., 2022 ), they still struggle to explain the classi fi cation and regression results for tabular data.”
Speech Recognition with Cochlea‐Inspired In‐Sensor Computing
Beoletto, P. H., Milano, G., Ricciardi, C., Bosia, F., & Gliozzi, A. S. (2025) · Advanced Intelligent Systems · doi:10.1002/aisy.202500526 · Cited by 4
Chen, Y., Yuan, B., Liao, B., Gabbay, D. M., & Cheng, L. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129337 · Cited by 2
Metadata
- Authors
- Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R.
- Affiliations
- Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
- BIFOLD✔Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
- Berlin Institute for the Foundations of Learning and Data (DE)
- ORCID
- Venue
- Proceedings of the IEEE
- Published
- 2021-03
- Volume
- 109
- Issue
- 3
- Pages
- 247-278
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W3132191748
- Cited by
- 1,477
- References
- 230
- Citations by year
- 2026: 305 · 2025: 343 · 2024: 326 · 2023: 273 · 2022: 152 · 2021: 74
- FWCI
- 72.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the IEEE (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion, other-oa)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Institute of Information & Communications Technology Planning & Evaluation (IITP) grants funded by the Korea Government (2017-0-00451, 2019-0-00079)
- German Ministry for Education and Research (01IS14013A-E, 01GQ1115, 01GQ0850, 01IS18025A, 01IS18037A)
- German Research Foundation (Math+, EXC 2046/1, 390685689)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- explainable AI, interpretability, deep neural network, post-hoc explanation, interpretability evaluation, nonlinear machine learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.