Explainable AI: A Review of Machine Learning Interpretability Methods
Open accessThe DOI matches its Crossref record; 170 of the 170 references it cites were found in a registry or an index.
Abstract
Recent advances in artificial intelligence (AI) have led to its widespread industrial adoption, with machine learning systems demonstrating superhuman performance in a significant number of tasks. However, this surge in performance, has often been achieved through increased model complexity, turning such systems into “black box” approaches and causing uncertainty regarding the way they operate and, ultimately, the way that they come to decisions. This ambiguity has made it problematic for machine learning systems to be adopted in sensitive yet critical domains, where their value could be immense, such as healthcare. As a result, scientific interest in the field of Explainable Artificial Intelligence (XAI), a field that is concerned with the development of new methods that explain and interpret machine learning models, has been tremendously reignited over recent years. This study focuses on machine learning interpretability methods; more specifically, a literature review and taxonomy of these methods are presented, as well as links to their programming implementations, in the hope that this survey would serve as a reference point for both theorists and practitioners.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While valuable, these techniques can sometimes be inconsistent or fail to capture the true decision-making process of highly complex models, potentially leading to a false sense of interpretability66.”
From Bias recognition and mitigation strategies in artificial intelligence healthcare applications (2025) · cites this paper as [66]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These methods facilitate a deeper understanding of the model’s internal logic, support debugging and optimization processes, and help identify key variables that most strongly influence students’ behavioral intentions toward GenAI adoption (Linardatos et al., 2020).”
From A two-step structural equation modeling and explainable machine learning framework for understanding university students’ adoption of generative AI: balancing intrinsic motivations and perceived risks (2026) · cites this paper as Linardatos et al., 2020
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Yet, there is no common taxonomy neither within one field (such as data science) nor between different fields (law and data science).”
From Transparency of AI in Healthcare as a Multilayered System of Accountabilities: Between Legal Requirements and Technical Limitations (2022) · cites this paper as Linardatos et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Despite the substantial amount of research devoted to solving the black-box issue of AI, many scholars in the field agree that a clear relevant taxonomy is missing (Doshi-Velez and Kim, 2017; Adadi and Berrada, 2018; Linardatos et al., 2021).”
From Transparency of AI in Healthcare as a Multilayered System of Accountabilities: Between Legal Requirements and Technical Limitations (2022) · cites this paper as Linardatos et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prior research on XAI across industries, has also found that the perceived quality of explanations is closely tied to their actionability and practical relevance for end users [8].”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [8]
Experimental — accuracy not yet measured
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“An improvement over the initial method was proposed in [24], where the well-known Krizhevsky network [25] was utilised in order to outperform state-of-the-art saliency models by a large margin, increasing the amount of explained information by 67% when compared to state-of-the art.”
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · cited 1×
“On the one hand, there are the so called black-box models, which include deep learning [2] and ensembles [8–10].”
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · cited 3×
“[59] PDPbox InterpretML Skater PH G Agnostic tab 589.”
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 3×
“The breakDown method, as proposed in [62], is similar to SHAP [48], as both, based on the conditioned responses of a black-box model, attempt to attribute them proportionally to the input features.”
Classification and Regression by randomForest
Liaw, A. I., & Wiener, M. C. (2007) · Cited by 18,594
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 8×
“[44] showed that, while the Deconvolution [32], Guided BackPropagation [31], and LRP [37] methods help in interpreting deep neural networks, they do not produce the theoretically correct interpretation, even in the simplest neural network setting; a linear model developed while using data that were produced by a linea…”
InfluentialLearning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · cited 2×
“In many cases, this is achieved through exploiting the gradient information flowing through the layers of the network, Grad-CAM [35], a direct extension of [34], being a prime and most influential example in terms of citations per year.”
Research works printed but not linked: 3
- Luss, R., Chen, P.Y., Dhurandhar, A., Sattigeri, P., Zhang, Y., Shanmugam, K., and Tu, C.C. (2019). Generating contrastive explanations with monotonic attribute functions. arXiv.
- Van Looveren, A., and Klaise, J. (2019). Interpretable counterfactual explanations guided by prototypes. arXiv.
- Wang, X., Jin, H., and He, K. (2019). Natural language adversarial attacks and defenses in word level. arXiv.
Cited by
Items 201–300 of 1,016
Enabling explainable artificial intelligence capabilities in supply chain decision support making
Olan, F., Spanaki, K., Ahmed, W., & Zhao, G. (2024) · Production Planning & Control · doi:10.1080/09537287.2024.2313514 · Cited by 111
“…is utilized in high-performance computing systems to facilitate cognitive processes such as learning from behaviour, recalling information, acquiring knowledge, making inferences, and interpreting codes within a given context (Linardatos, Papastefanopoulos, and Kotsiantis 2020; Meske et al. 2022).”
A review of evaluation approaches for explainable AI with applications in cardiology
Salih, A. M., Galazzo, I. B., Gkontra, P., Rauseo, E., Lee, A. M., Lekadir, K., Radeva, P. I., Petersen, S. E., & Menegaz, G. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10852-w · Cited by 104
“…for each category will be discussed, although for more details on each metric and method we refer the interested readers to specific reviews on this topic as it is out of the scope of the current review (Mohseni et al. 2021; Kumarakulasin-ghe et al. 2020; Linardatos et al. 2020; Lopes et al. 2022).”
BackgroundMethodInfluentialKaya, C. (2025) · Food and Energy Security · doi:10.1002/fes3.70050 · Cited by 84
Machine learning-based prediction for settling velocity of microplastics with various shapes
Qian, S., Qiao, X., Zhang, W., Yu, Z., Dong, S., & Feng, J. (2024) · Water Research · doi:10.1016/j.watres.2023.121001 · Cited by 76
Nagahisarchoghaei, M., Nur, N., Cummins, L., Nur, N., Karimi, M. M., Nandanwar, S., Bhattacharyya, S., & Rahimi, S. (2023) · Electronics · doi:10.3390/electronics12051092 · Cited by 103
Ferro, M., Falagario, U. G., Barone, B., Maggi, M., Crocetto, F., Busetto, G. M., Giudice, F. D., Terracciano, D., Lucarelli, G. T., Lasorsa, F. M., et al. (2023) · Diagnostics · doi:10.3390/diagnostics13132308 · Cited by 89
Photocatalytic degradation of drugs and dyes using a maching learning approach
Anandhi, G., & Iyapparaja, M. (2024) · RSC Advances · doi:10.1039/d4ra00711e · Cited by 80
Yeong, D. J., Panduru, K., & Walsh, J. (2025) · Sensors · doi:10.3390/s25030856 · Cited by 110
Manthena, H., Shajarian, S., Kimmell, J. C., Abdelsalam, M., Khorsandroo, S., & Gupta, M. (2025) · IEEE Access · doi:10.1109/access.2025.3555926 · Cited by 100
“However, this increased complexity often obscures the internal decision-making process, which can lead to questions about their decision logic [40], [41].”
Ahmed, F., Naz, N. S., Khan, S., Rehman, A. U., Ismael, W. M., & Khan, M. A. (2026) · BMC Medical Imaging · doi:10.1186/s12880-025-02118-w · Cited by 101
“Then, such a decision-making process could be explained or predicted by using techniques like Gradient-weighted Class Activation Mapping (Grad-CAM), Attention mechanisms, Local Interpretable Model-Agnostic Explanations (LIME) and Shapley Additive Explanations (SHAP) [13].”
A systematic review on machine learning models for online learning and examination systems
Kaddoura, S., Popescu, D. E., & Hemanth, J. D. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.986 · Cited by 75
“ML requires basic skills and primitive knowledge for its operation that can be implemented in education ( Linardatos, Papastefanopoulos & Kotsiantis, 2020 ).”
BackgroundFarah, L., Murris, J. M., Borget, I., Guilloux, A., Martelli, N. M., & Katsahian, S. I. M. (2023) · Mayo Clinic Proceedings: Digital Health · doi:10.1016/j.mcpdig.2023.02.004 · Cited by 72
Stef, N., Başağaoğlu, H., Chakraborty, D., & Jabeur, S. B. (2023) · Energy Economics · doi:10.1016/j.eneco.2023.106822 · Cited by 81
Explainable spatially explicit geospatial artificial intelligence in urban analytics
Liu, P., Zhang, Y., & Biljecki, F. (2023) · Environment and Planning B Urban Analytics and City Science · doi:10.1177/23998083231204689 · Cited by 120
“Plausible XAI methods, which are now commonly seen in Computer Vision (Kenny and Keane, 2021; Linardatos et al., 2020) where the generated coun-terfactual can be manipulated by humans, could be a solution towards solving this dilemma; however, it is rare to see such developments in graph-based deep…”
BackgroundMachine Learning‐Enhanced Nanoparticle Design for Precision Cancer Drug Delivery
Wang, Q., Liu, Y., Li, C., Xu, S., Xu, S., & Liu, B. (2025) · Advanced Science · doi:10.1002/advs.202503138 · Cited by 102
Gomez-Cabello, C. A., Borna, S., Pressman, S., Haider, S. A., Haider, C. R., & Forte, A. J. (2024) · European Journal of Investigation in Health, Psychology and Education · doi:10.3390/ejihpe14030045 · Cited by 74
Towards artificial intelligence at scale in the chemical industry
Chiang, L. H., Braun, B., Wang, Z., & Castillo, I. (2022) · AIChE Journal · doi:10.1002/aic.17644 · Cited by 91
A review of mechanistic learning in mathematical oncology
Metzcar, J., Jutzeler, C. R., Macklin, P., Köhn‐Luque, A., & Brüningk, S. C. (2024) · Frontiers in Immunology · doi:10.3389/fimmu.2024.1363144 · Cited by 98
“Post hoc processing is required to uncover the nature of the approximated relationship through interpretability and explainability analysis (31).”
BackgroundGender biases within Artificial Intelligence and ChatGPT: Evidence, Sources of Biases and Solutions
Ho, J. Q. H., Hartanto, A., Koh, A., & Majeed, N. M. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100145 · Cited by 109
Kothadiya, D., Bhatt, C., Rehman, A. U., Alamri, F. S., & Saba, T. (2023) · IEEE Access · doi:10.1109/access.2023.3274851 · Cited by 82
“This complex learning structure generally considers as a ‘‘Black-Box’’ [5].”
Agriculture 5.0 and Explainable AI for Smart Agriculture: A Scoping Review
Razak, A., Yogarayan, S., Sayeed, M. S., & Derafi, M. I. F. M. (2024) · Emerging Science Journal · doi:10.28991/esj-2024-08-02-024 · Cited by 91
“The utilization of models, frequently employing decision trees, rule-based systems, or local explanations, has been seen as crucial in establishing confidence in the suggestions generated by AI [82, 83].”
MethodThe convergence of AI and synthetic biology: the looming deluge
Groff-Vindman, C., Trump, B. D., Cummings, C. L., Smith, M., Titus, A. J., Oye, K., Prado, V., Turmus, E. K., & Linkov, I. (2025) · npj Biomedical Innovations. · doi:10.1038/s44385-025-00021-1 · Cited by 83
“Moreover, there may be inherent trade-offs between model performance and interpretability, as the most accurate models often rely on intricate architectures that resist simple explanations 42 .”
Du, Y., Mukherjee, T., Li, R., Hou, Z., Dutta, S., Arnold, C. B., Elwany, A. H., Kung, S., Tang, J., & DebRoy, T. (2025) · Progress in Materials Science · doi:10.1016/j.pmatsci.2025.101587 · Cited by 99
Large language models for life cycle assessments: Opportunities, challenges, and risks
Preuss, N., Alshehri, A. S., & You, F. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.142824 · Cited by 71
Fair and equitable AI in biomedical research and healthcare: Social science perspectives
Baumgartner, R. R., Arora, P., Bath, C., Burljaev, D., Ciereszko, K., Custers, B. H. M., Ding, J., Ernst, W., Fosch‐Villaronga, E., Galanos, V., et al. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102658 · Cited by 90
Lorenzo, G., Ahmed, S. R., Hormuth, D. A., Vaughn, B., Kalpathy–Cramer, J., Solorio, L., Yankeelov, T. E., & Gómez, H. (2024) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-081623-025834 · Cited by 98
Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning
Gao, Y., Gu, S., Jiang, J., Hong, S.-S., Yu, D., & Zhao, L. (2024) · ACM Computing Surveys · doi:10.1145/3644073 · Cited by 121
“Several related surveys provide an in-depth classification of machine learning interpretability methods in general [9, 58, 89, 120], while others focus on more specific fields of study.”
BackgroundExplainable Anomaly Detection Framework for Maritime Main Engine Sensor Data
Kim, D., Antariksa, G., Handayani, M. P., Lee, S.-B., & Lee, J. (2021) · Sensors · doi:10.3390/s21155200 · Cited by 84
“For more literature review about explainable AI methods, please refer to [27].”
BackgroundToward Data-Driven Optimal Control: A Systematic Review of the Landscape
Prag, K., Woolway, M., & Çelik, T. (2022) · IEEE Access · doi:10.1109/access.2022.3160709 · Cited by 82
Explainable Artificial Intelligence (XAI) for Deep Learning Based Medical Imaging Classification
Ghnemat, R., Alodibat, S., & Al-Haija, Q. S. A. (2023) · Journal of Imaging · doi:10.3390/jimaging9090177 · Cited by 88
“In response to the need for quicker interpretation of radiography images, a range of deep learning-based artificial intelligence (AI) systems have been created [28].”
Martins, T., de Almeida, A., Cardoso, E., & Nunes, L. (2023) · IEEE Access · doi:10.1109/access.2023.3347028 · Cited by 98
“This category is largely accepted within the reviewed literature, where it is found as a main category for classifying XAI methods [6], [8], [10], [14], [15], [22].”
BackgroundMethodInfluentialHamdan, A., Ibekwe, K. I., Etukudoh, E. A., Umoh, A. A., & Ilojianya, V. I. (2024) · World Journal of Advanced Research and Reviews · doi:10.30574/wjarr.2024.21.1.0257 · Cited by 99
“The interpretability of AI and ML models remains a critical concern in climate change research (Linardatos et al ., 2020).”
MethodElectrospinning Technology, Machine Learning, and Control Approaches: A Review
Shabani, A., Al, G. A., Berri, N., Dominguez, B. C., Leese, H. S., & Martínez-Hernández, U. (2025) · Advanced Engineering Materials · doi:10.1002/adem.202401353 · Cited by 76
Interpretable AI for bio-medical applications
Sathyan, A., Weinberg, A. I., & Cohen, K. D. O. (2022) · Complex Engineering Systems · doi:10.20517/ces.2022.41 · Cited by 81
“However, the ability of humans to understand it is negatively correlated to model complexity[5].”
BackgroundAncajas, C. M. F., Oyedele, A. S., Butt, C. M., & Walker, A. S. (2024) · Natural Product Reports · doi:10.1039/d4np00009a · Cited by 83
Quantifying and alleviating political bias in language models
Liu, R., Jia, C., Wei, J., Xu, G., & Vosoughi, S. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2021.103654 · Cited by 68
Neubauer, A., Brandt, S., & Kriegel, M. (2025) · Energy and AI · doi:10.1016/j.egyai.2025.100480 · Cited by 86
Grad-CAM (Gradient-weighted Class Activation Mapping): A systematic literature review
Fayyaz, A. M., Abdulkadir, S. J., Talpur, N., Al-Selwi, S. M., Hassan, S. U., & Sumiea, E. H. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.111200 · Cited by 94
Human and Technological Infrastructures of Fact-checking
Juneja, P., & Mitra, T. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555143 · Cited by 99
“Past studies have also developed human-in the loop AI systems [155, 178] and tried to make them more explainable in order to foster trust in them [24, 58, 118, 125, 149].”
BackgroundA Transparency Index Framework for AI in Education
Chaudhry, M. A., Cukurova, M., & Luckin, R. (2022) · doi:10.35542/osf.io/bstcf · Cited by 34
Machine learning-based clinical decision support using laboratory data
Çubukçu, H. C., Topcu, D. İ., & Yenice, S. (2024) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2023-1037 · Cited by 66
Cui, Z., Zhang, N., Zhou, T., Zhou, X., Meng, H., Yu, Y., Zhang, Z., Zhang, Y., Wang, W., & Liu, Y. (2023) · Journal of Agricultural and Food Chemistry · doi:10.1021/acs.jafc.3c00591 · Cited by 70
A Review of Application of Machine Learning in Storm Surge Problems
Qin, Y., Su, C., Chu, D., Zhang, J., & Song, J. (2023) · Journal of Marine Science and Engineering · doi:10.3390/jmse11091729 · Cited by 74
BIG DATA ANALYTICS: A REVIEW OF ITS TRANSFORMATIVE ROLE IN MODERN BUSINESS INTELLIGENCE
Udeh, C. A., Orieno, O. H., Daraojimba, O. D., Ndubuisi, N. L., & Oriekhoe, O. I. (2024) · Computer Science & IT Research Journal · doi:10.51594/csitrj.v5i1.718 · Cited by 100
Haque, R., Sakib, A. A., Hossain, M. F., Islam, F., Aziz, F. I., Ahmed, M. R., Kannan, S., Rohan, A., & Hasan, M. J. (2024) · BioMedInformatics · doi:10.3390/biomedinformatics4020054 · Cited by 71
“Nevertheless, there are challenges such as the requirement for large datasets, complexities in interpretation, and the risk of overfitting [20,21].”
BackgroundExplainability and Interpretability in Concept and Data Drift: A Systematic Literature Review
Pelosi, D., Cacciagrano, D., & Piangerelli, M. (2025) · Algorithms · doi:10.3390/a18070443 · Cited by 60
Top ten intelligent algorithms towards smart manufacturing
Zhang, M., Tao, F., Zuo, Y., Xiang, F., Wang, L., & Nee, A. Y. C. (2023) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2023.09.006 · Cited by 71
Joseph, L. P., Deo, R. C., Casillas-Pérez, D., Prasad, R., Raj, N., & Salcedo‐Sanz, S. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.122624 · Cited by 69
A Comprehensive Study of Explainable Artificial Intelligence in Healthcare
Mohanty, A., & Mishra, S. (2022) · Studies in computational intelligence · doi:10.1007/978-981-19-1076-0_25 · Cited by 69
Malik, N., & Bilal, M. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2203 · Cited by 76
“Existing studies offer several effective methods to develop explainable and interpretable models ( Linardatos, Papastefanopoulos & Kotsiantis, 2020 ).”
MethodÇifçi, A. (2025) · IEEE Access · doi:10.1109/access.2025.3543759 · Cited by 76
“The analysis of ML models is often limited by the inability to interpret their outputs [17], [18], [19].”
It is Not “Accuracy vs. Explainability”—We Need Both for Trustworthy AI Systems
Petkovic, D. (2023) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2023.3239921 · Cited by 63
“XAI techniques then aim to explain how AI systems make their decisions, and aim to do so in easily understandable way to humans (17, 18, 19, 23, 25, 26, 27).”
BackgroundInfluentialMachine learning-based approaches for predicting the dynamic response of RC slabs under blast loads
Zhao, C., Zhu, Y., & Zhou, Z.-H. (2022) · Engineering Structures · doi:10.1016/j.engstruct.2022.115104 · Cited by 65
eXplainable AI Allows Predicting Upper Limb Rehabilitation Outcomes in Sub-Acute Stroke Patients
Gandolfi, M., Galazzo, I. B., Pavan, R., Cruciani, F., Valè, N., Picelli, A., Storti, S. F., Smania, N., & Menegaz, G. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3220179 · Cited by 71
“after post–stroke rehabilitation [19], and Local Interpretable Model-agnostic Explanations (LIME) [38] that are, by far, the most comprehensive and dominant across literature methods for visualising feature interactions and importance [39].”
BackgroundTrust and stakeholder perspectives on the implementation of AI tools in clinical radiology
Bergquist, M., Rolandsson, B., Gryska, E. A., Laesser, M., Hoefling, N., Heckemann, R. A., Schneiderman, J. F., & Björkman‐Burtscher, I. M. (2023) · European Radiology · doi:10.1007/s00330-023-09967-5 · Cited by 80
“The previous studies, while providing technical grounds for improving the trustworthiness of an algorithm, do not encompass medical reasoning in the explanations [11].”
A brief history of artificial intelligence embryo selection: from black-box to glass-box
Lee, T., Natalwala, J., Chapple, V., & Liu, Y. (2023) · Human Reproduction · doi:10.1093/humrep/dead254 · Cited by 71
“In other words, the more interpretable a model is, the easier it is to identify cause-and-effect relationships within its inputs and outputs (Linardatos et al. , 2020).”
REVIEWING THE ROLE OF BIG DATA ANALYTICS IN FINANCIAL FRAUD DETECTION
Shoetan, P. O., Oyewole, A. T., Okoye, C. C., & Ofodile, O. C. (2024) · Finance & Accounting Research Journal · doi:10.51594/farj.v6i3.899 · Cited by 93
A review and benchmark of feature importance methods for neural networks
Mandler, H., & Weigand, B. (2024) · ACM Computing Surveys · doi:10.1145/3679012 · Cited by 77
Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations
Dai, J. C., Upadhyay, S., Aïvodji, U., Bach, S. H., & Lakkaraju, H. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534159 · Cited by 127
Anomaly and intrusion detection using deep learning for software-defined networks: A survey
Ruffo, V. G. D. S., Lent, D. M. B., Komarchesqui, M., Schiavon, V. F., de Assis, M. V. O., Carvalho, L. F., & Proença, M. L. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124982 · Cited by 78
Nor, A. K. M., Pedapati, S. R., Muhammad, M. B., & Leiva, V. (2022) · Mathematics · doi:10.3390/math10040554 · Cited by 76
Multistage transfer learning for medical images
Ayana, G., Dese, K., Abagaro, A. M., Jeong, K. C., Yoon, S.-D., & Choe, S.-W. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10855-7 · Cited by 57
“These high-level representations serve as a bridge between the source and target tasks, encapsulating the model’s understanding of the most pertinent information (Linardatos et al. 2020).”
An explainable artificial intelligence model for predictive maintenance and spare parts optimization
Dereci, U., & Tuzkaya, G. (2024) · Supply Chain Analytics · doi:10.1016/j.sca.2024.100078 · Cited by 82
Koç, K., Ekmekcioğlu, Ö., & Gürgün, A. P. (2023) · Journal of Construction Engineering and Management · doi:10.1061/jcemd4.coeng-12848 · Cited by 67
Explainable Misinformation Detection Across Multiple Social Media Platforms
Joshi, G., Srivastava, A., Yagnik, B., Hasan, M. M., Saiyed, Z., Gabralla, L. A., Abraham, A., Walambe, R. A., & Kotecha, K. V. (2023) · IEEE Access · doi:10.1109/access.2023.3251892 · Cited by 74
“9 of the model's internal logic, adversely affecting these models' trust, usability, and adoption in real-world applications[57].”
BackgroundCondition-Based Maintenance in Aviation: Challenges and Opportunities
Verhagen, W. J. C., Santos, B. F., Freeman, F. C., Kessel, P., Zarouchas, D. S., Λούτας, Θ., Yeun, R., & Heiets, I. (2023) · Aerospace · doi:10.3390/aerospace10090762 · Cited by 65
“There is a growing interest and literature on Explainable Artificial Intelligence (XAI) that aims to address this challenge (e.g., [36]); 5.”
Tamagusko, T., & Ferreira, A. (2023) · Infrastructures · doi:10.3390/infrastructures8120170 · Cited by 72
Relationship between feature importance and building characteristics for heating load predictions
Neubauer, A., Brandt, S., & Kriegel, M. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.122668 · Cited by 62
Landauer, M., Skopik, F., Stojanović, B., Flatscher, A., & Ullrich, T. (2024) · International Journal of Information Security · doi:10.1007/s10207-024-00921-0 · Cited by 88
Wong, A. N. N., He, Z., Leung, K. L., To, C. C. K., Wong, C. Y., Wong, S. C. C., Yoo, J. S., Chan, C. K. R., Chan, A. Z., Lacambra, M. D., et al. (2022) · Cancers · doi:10.3390/cancers14153780 · Cited by 68
Jones, M. A., Islam, W., Faiz, R., Chen, X., & Zheng, B. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.980793 · Cited by 63
“, “heat maps”) contribute the most to the decision made by AI models (151, 152).”
BackgroundShamji, M. H., Ollert, M. W., Adcock, I. M., Bennett, O., Favaro, A., Sarama, R., Riggioni, C., Annesi‐Maesano, I., Čustović, A., Fontanella, S., et al. (2023) · Allergy · doi:10.1111/all.15667 · Cited by 65
“44 Intrinsic explainability is achieved by using a type of model that is sufficiently simple, or intrinsically understandable, that merely unpacking the internal workings of the model will provide a user with insight into how a model arrives at a decision.”
BackgroundMethodInfluentialSurvey on AI Applications for Product Quality Control and Predictive Maintenance in Industry 4.0
Johanesa, T. V. A., Equeter, L., & Mahmoudi, S. A. (2024) · Electronics · doi:10.3390/electronics13050976 · Cited by 80
“However, more complex models like DL models require the use of explainability techniques [89].”
Khan, N., Nauman, M. A., Almadhor, A. S., Akhtar, N., Alghuried, A., & Alhudhaif, A. (2024) · IEEE Access · doi:10.1109/access.2024.3420415 · Cited by 73
Explainable Artificial Intelligence in Radiological Cardiovascular Imaging—A Systematic Review
Haupt, M., Maurer, M. H., & Thomas, R. P. (2025) · Diagnostics · doi:10.3390/diagnostics15111399 · Cited by 58
“The general XAI literature underscores why this is the case: building inherently inter-pretable yet accurate models for high-dimensional image data is exceedingly difficult, so the community has gravitated toward complex models plus XAI as a compromise [5].”
InfluentialLekadir, K., Osuala, R., Gallin, C., Lazrak, N., Kushibar, K., Tsakou, G., Aussó, S., Alberich, L. C., Marias, K., Tsiknakis, M., et al. (2021) · arXiv · doi:10.48550/arxiv.2109.09658 · Cited by 58
“Towards a definition of explainability, the need for a general agreement on the term explainability of AI algorithms is to be pointed out, as it is often used interchangeably with interpretability [104, 98].”
Using Kernel SHAP XAI Method to Optimize the Network Anomaly Detection Model
Roshan, K., & Zafar, A. (2022) · International Conference on Computing for Sustainable Global Development · doi:10.23919/indiacom54597.2022.9763241 · Cited by 75
“We would highly recommend some of the review and survey papers by the authors of [5] [9] [10] [16] for detailed analysis of various methods and application areas related to XAI.”
BackgroundMethodPapacharalampous, G., & Tyralis, H. (2022) · Frontiers in Water · doi:10.3389/frwa.2022.961954 · Cited by 64
“The variable importance scores can be classified into two main categories, which are known under the terms (Linardatos et al. 2020; Kuhn 2021): (a)”
BackgroundKhanna, V. V., Chadaga, K., Sampathila, N., Chadaga, R., Prabhu, S., Swathi, K. S., Jagdale, A. S., & Bhat, D. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e22456 · Cited by 82
“Feature importance assigns a calculated value to each independent input feature of a model based on its relevance in predicting a target (Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. [49] ) Feature importance in this study is used to obtain the reliability of the XAI techniques.”
MethodAlert Prioritisation in Security Operations Centres: A Systematic Survey on Criteria and Methods
Jalalvand, F., Chhetri, M. B., Nepal, ., & Paris, C. L. (2024) · ACM Computing Surveys · doi:10.1145/3695462 · Cited by 105
“Further, researchers can explore a variety of augmentation enablers for ML methods [80], like local inter-pretable model-agnostic explanations (LIMEs) [111] and AIX360 [35], for application in AP.”
Tran, M., Balasooriya, C. D., Jonnagaddala, J., Leung, G., Mahboobani, N. R., Ramani, S., Rhee, J., Schuwirth, L., Najafzadeh-Tabrizi, N. S., Semmler, C., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01721-z · Cited by 72
Explainable AI for industrial fault diagnosis: A systematic review
Cação, J., Santos, J., & Antunes, M. (2025) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2025.100905 · Cited by 86
An Interpretable Artificial Intelligence Based Smart Agriculture System
Sabrina, F., Sohail, S., Farid, F., Jahan, S., Ahamed, F., & Gordon, S. (2022) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.026363 · Cited by 58
Mikdadi, D., O’Connell, K. A., Meacham, P. J., Dugan, M. A., Ojiere, M. O., Carlson, T. B., & Klenk, J. A. (2022) · Cancer Biomarkers · doi:10.3233/cbm-210301 · Cited by 63
“However explainability can come at the expense of accuracy [114].”
Gongane, V. U., Munot, M. V., & Anuse, A. D. (2024) · Journal of Computational Social Science · doi:10.1007/s42001-024-00248-9 · Cited by 91
“The methods focus on type of data to be handled, exploring the local or global properties, and providing model-specific or model-independent explanations [24].”
InfluentialKhude, H., & Shende, P. K. (2025) · Advances in Integrative Medicine · doi:10.1016/j.aimed.2025.100529 · Cited by 72
Brain age as a surrogate marker for cognitive performance in multiple sclerosis
Denissen, S., Engemann, D. A., Cock, A. D., Costers, L., Baijot, J., Laton, J., Penner, I., Grothe, M., Kirsch, M., D'hooghe, M. B., et al. (2022) · European Journal of Neurology · doi:10.1111/ene.15473 · Cited by 58
“Linear models are therefore both interpretable (obvious causal relationship between input and output) [36] and explainable (good understanding of the model's internal mechanisms) [36].”
BackgroundAn Explainable AI Approach for Breast Cancer Metastasis Prediction Based on Clinicopathological Data
Maouche, I., Terrissa, L. S., Benmohammed, K., & Zerhouni, N. (2023) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2023.3282840 · Cited by 60
“XAI approaches, in particular post-hoc, focus on delivering interpretability to decisions made by black-box models [30].”
Biswas, A., Kumari, A., Gaikwad, D. S., & Pandey, D. K. (2023) · OMICS A Journal of Integrative Biology · doi:10.1089/omi.2023.0197 · Cited by 67
“Developing explainable AI models and algorithms that provide transparent reasoning and interpretable results is a significant challenge in the field (Linardatos et al., 2020).”
Amri, E., Gulzar, Y., Yeafi, A., Jendoubi, S., Dhawi, F., & Mir, M. S. (2024) · Modeling Earth Systems and Environment · doi:10.1007/s40808-023-01918-9 · Cited by 63
“Ker-nelSHAP and DeepSHAP are some methods proposed for estimating SHAP values, outperforming other techniques such as LRP and LIME (Linardatos et al. 2021; Van et al. 2004).”
Federated Learning Approach for Breast Cancer Detection Based on DCNN
AlSalman, H., Al-Rakhami, M. S., Alfakih, T. M., & Hassan, M. M. (2024) · IEEE Access · doi:10.1109/access.2024.3374650 · Cited by 59
“This method provides a visual explanation of the model’s decision-making process by producing heatmaps that highlight critical areas influencing the model’s predictions [67].”
MethodCorn leaf disease: insightful diagnosis using VGG16 empowered by explainable AI
Tariq, M., Ali, U., Abbas, S., Hassan, S. G., Naqvi, R. A., Khan, M. A., & Jeong, D. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1402835 · Cited by 68
“Machine learning and interoperability mean presenting machine learning models in a way understandable to humans (Linardatos et al., 2020); Bridgelall, 2024).”
BackgroundProcedural fairness in algorithmic decision-making: the role of public engagement
Decker, M. C., Wegner, L., & Leicht-Scholten, C. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09811-4 · Cited by 78
“…trying to make comprehensible the rationale behind decisions for models whose model struc - ture is not directly understandable (contrary to, for example, linear regression models or decision trees) (Baró et al., 2018; Guidotti et al., 2019 ; Linardatos et al., 2020 ; Mittelstadt et al., 2019 ).”
Parakh, A., Awate, A., Barman, S. M., Kadu, R. K., Tulaskar, D. P., Kulkarni, M. B., & Bhaiyya, M. L. (2025) · Trends in Environmental Analytical Chemistry · doi:10.1016/j.teac.2025.e00280 · Cited by 63
Zhang, Y., Sun, Q., Liu, J., & Petrosian, O. (2023) · Sustainability · doi:10.3390/su16010019 · Cited by 63
“Addressing the challenges posed by black-box models, the explainable AI (XAI) technology [37–39], specifically SHapley Additive exPlanations (SHAP) [40–42], can be utilized.”
MethodHu, Q., Lu, J., Hui, J., Rao, Z., Ren, Y., & Wang, H. (2025) · Advanced Functional Materials · doi:10.1002/adfm.202508438 · Cited by 73
Qiu, Y., Niu, J., Zhang, C., Chen, L., Su, B., & Zhou, S. (2025) · Water Research · doi:10.1016/j.watres.2025.123129 · Cited by 54
Zhou, Z., Yu, X., Zhu, Z., Zhou, D., & Qi, H. (2022) · Safety Science · doi:10.1016/j.ssci.2022.105942 · Cited by 55
Boneh-Shitrit, T., Feighelstein, M., Bremhorst, A., Amir, S., Distelfeld, T., Dassa, Y., Yaroshetsky, S., Riemer, S., Shimshoni, I., Mills, D. S., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-27079-w · Cited by 68
Images and CNN applications in smart agriculture
Sakka, M. E., Mothe, J., & Ivanovici, M. (2024) · European Journal of Remote Sensing · doi:10.1080/22797254.2024.2352386 · Cited by 63
“Interpretable deep learning aims to solve this problem by providing a set of tools that help interpreting deep learning models, for example, by visualizing the regions of importance of an image in computer vision (Linardatos et al., 2021).”
Metadata
- Authors
- Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S.
- Affiliations
- University of Patras (GR)
- Corresponding author
- Linardatos, Pantelis
- Venue
- Entropy
- Publisher
- MDPI AG
- Published
- 2020-12-25
- Volume
- 23
- Issue
- 1
- Pages
- 18
- ISSN
- 1099-4300 (ISSN-L)
- PMC
- PMC7824368
- OpenAlex
- W3116286104
- Cited by
- 2,922
- References
- 162
- Citations by year
- 2026: 485 · 2025: 702 · 2024: 701 · 2023: 565 · 2022: 370 · 2021: 97 · 2020: 1
- FWCI
- 8.89
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Entropy (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- explainable AI, model interpretability, black-box models, medical applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.