Machine Learning Interpretability: A Survey on Methods and Metrics
Open accessThe DOI matches its Crossref record; 105 of the 106 references it cites were found in a registry or an index.
Abstract
Machine learning systems are becoming increasingly ubiquitous. These systems’s adoption has been expanding, accelerating the shift towards a more algorithmic society, meaning that algorithmically informed decisions have greater potential for significant social impact. However, most of these accurate decision support systems remain complex black boxes, meaning their internal logic and inner workings are hidden to the user and even experts cannot fully understand the rationale behind their predictions. Moreover, new regulations and highly regulated domains have made the audit and verifiability of decisions mandatory, increasing the demand for the ability to question, understand, and trust machine learning systems, for which interpretability is indispensable. The research community has recognized this interpretability problem and focused on developing both interpretable models and explanation methods over the past few years. However, the emergence of these methods shows there is no consensus on how to assess the explanation quality. Which are the most suitable metrics to assess the quality of an explanation? The aim of this article is to provide a review of the current state of the research field on machine learning interpretability while focusing on the societal impact and on the developed methods and metrics. Furthermore, a complete literature review is presented in order to identify future directions of work on this field.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Transparency enables AI systems to be tested, audited, and debugged, which is a path toward increasing their safety (Carvalho et al., 2019)15.”
From Transparency of AI in Healthcare as a Multilayered System of Accountabilities: Between Legal Requirements and Technical Limitations (2022) · cites this paper as Carvalho et al., 2019
In the citing paper
“In this paper, we follow scholars who view “interpretability” as the broader concept that, inter alia, includes “explainability.” Specifically, we agree with the statement of M. Honneger that “interpretability is the end-goal we want to achieve, and explanations are the tools to reach interpretability” (Honegger, 2018, citation in Carvalho et al., 2019).”
From Transparency of AI in Healthcare as a Multilayered System of Accountabilities: Between Legal Requirements and Technical Limitations (2022) · cites this paper as Carvalho et al., 2019
In the citing paper
“These methods can capture complex, non-linear interactions among multiple factors that traditional models may miss [29,30].”
From Machine learning-driven risk prediction for post-hospitalization diabetes case management: Integrating clinical and social determinants of health (2026) · cites this paper as [29]
In the citing paper
“The required auditability property of Trustworthy AI systems demands the development of practical tools [109] that are capable of verifying desirable properties of neural networks such as stability, sensitivity, relevance or reachability [110], as well as metrics beyond explainability [111-115], such as on traceability, data quality and integrity.”
From Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation (2023) · cites this paper as [111]
In the citing paper
“Model performance metrics (e.g., model accuracy) do not always exhibit true prediction decisions [9].”
From Explainable AI (XAI): Core Ideas, Techniques, and Solutions (2023) · cites this paper as [9]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“For example, introduced in 2016, deep residual networks (ResNets) [1] are over 200-layers deep and have been shown to beat human-level performance on object recognition tasks.”
Background(2008) · Journal of Machine Learning Research · Cited by 33,051
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · cited 1×
“Partial Dependence Plot [142] Global Feature summary Individual Condition Expectation [143] Global/Local Feature summary Accumulated Local Effects Plot [144] Global Feature summary Feature Interaction [145] Global Feature summary Feature Importance [146] Global/Local Feature summary Local Surrogate Model [98] Local Su…”
BackgroundGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 3×
“Shapley Values is sometimes called by SHAP.”
Background"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Algorithm AS 136: A K-Means Clustering Algorithm
Hartigan, J. A., & Wong, M. A. (1979) · Applied Statistics · doi:10.2307/2346830 · Cited by 14,523 · cited 1×
“These techniques range from classic descriptive statistics to data visualization methods, including Principal Component Analysis (PCA) [101] and t-SNE [102] (t-Distributed Stochastic Neighbor Embedding), and clustering methods, such as k-means [103] and MMD-critic [96] (Maximum Mean Discrepancy).”
MethodDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“Research in this type of methods exists for some years [134] but recently increased along with the ML interpretability field [135–137].”
MethodDavid, F. N., & Tukey, J. W. (1977) · Biometrics · doi:10.2307/2529486 · Cited by 12,708
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
Research works printed but not linked: 24
- Tractica (2019, January 22). Artificial Intelligence Software Market to Reach $105.8 Billion in Annual Worldwide Revenue by 2025. Available online: https://www.tractica.com/newsroom/press-releases/artificial-intelligence-software-market-to-reach-105-8-billion-in-annual-worldwide-revenue-by-2025/.
- Du, M., Liu, N., and Hu, X. (2018). Techniques for Interpretable Machine Learning. arXiv.
- Committee on Technology National Science and Technology Council and Penny Hill Press (2016). Preparing for the Future of Artificial Intelligence, CreateSpace Independent Publishing Platform.
- ACM US Public Council (2019, January 22). Statement on Algorithmic Transparency and Accountability. Available online: https://www.acm.org/binaries/content/assets/public-policy/2017_usacm_statement_algorithms.pdf.
- Kim, B., Malioutov, D.M., and Varshney, K.R. (2016). Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016). arXiv.
- Kim, B., Malioutov, D.M., Varshney, K.R., and Weller, A. (2017). Proceedings of the 2017 ICML Workshop on Human Interpretability in Machine Learning (WHI 2017). arXiv.
- Kim, B., Varshney, K.R., and Weller, A. (2018). Proceedings of the 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018). arXiv.
- Wilson, A.G., Kim, B., and Herlands, W. (2016). Proceedings of NIPS 2016 Workshop on Interpretable Machine Learning for Complex Systems. arXiv.
- Caruana, R., Herlands, W., Simard, P., Wilson, A.G., and Yosinski, J. (2017). Proceedings of NIPS 2017 Symposium on Interpretable Machine Learning. arXiv.
- Pereira-Fariña, M., and Reed, C. (2017). Proceedings of the 1st Workshop on Explainable Computational Intelligence (XCI 2017), Association for Computational Linguistics (ACL).
- IJCAI (2019, July 12). IJCAI 2018—Workshop on Explainable Artificial Intelligence (XAI). Available online: http://home.earthlink.net/~dwaha/research/meetings/faim18-xai/.
- CD-MAKE (2019, July 12). CD-MAKE 2019 Workshop on explainable Artificial Intelligence. Available online: https://cd-make.net/special-sessions/make-explainable-ai/.
- Lim, B., Smith, A., and Stumpf, S. (2018). ExSS 2018: Workshop on Explainable Smart Systems. CEUR Workshop Proceedings, City, University of London Institutional Repository. Available online: http://openaccess.city.ac.uk/20037/.
- ICAPS (2019, July 12). ICAPS 2018—Workshop on Explainable AI Planning (XAIP). Available online: http://icaps18.icaps-conference.org/xaip/.
- ICAPS (2019, July 12). ICAPS 2019—Workshop on Explainable AI Planning (XAIP). Available online: https://kcl-planning.github.io/XAIP-Workshops/ICAPS_2019.
- CVPR (2019, July 12). CVPR 19—Workshop on Explainable AI. Available online: https://explainai.net/.
- Lipton, Z.C. (2016). The mythos of model interpretability. arXiv.
- Holzinger, A., Biemann, C., Pattichis, C.S., and Kell, D.B. (2017). What do we need to build explainable AI systems for the medical domain?. arXiv.
- Kim, T.W. (2018). Explainable Artificial Intelligence (XAI), the goodness criteria and the grasp-ability test. arXiv.
- Ross, A., Lage, I., and Doshi-Velez, F. (2017, January 4–9). The neural lasso: Local linear sparsity for interpretable explanations. Proceedings of the Workshop on Transparent and Interpretable Machine Learning in Safety Critical Environments, 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA.
- Lee, M., He, X., Yih, W.t., Gao, J., Deng, L., and Smolensky, P. (2015). Reasoning in vector space: An exploratory study of question answering. arXiv.
- Xu, K., Park, D.H., Yi, C., and Sutton, C. (2018). Interpreting Deep Classifier by Visual Distillation of Dark Knowledge. arXiv.
- Koh, P.W., and Liang, P. (2017). Understanding black-box predictions via influence functions. arXiv.
- Bonnans, J.F., and Shapiro, A. (2013). Perturbation Analysis of Optimization Problems, Springer Science & Business Media.
Cited by
Items 701–800 of 1,007
The Methods and Approaches of Explainable Artificial Intelligence
Szczepański, M., Choraś, M., Pawlicki, M., & Pawlicka, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77970-2_1 · Cited by 14
Salih, A. M., Galazzo, I. B., Raisi‐Estabragh, Z., Petersen, S. E., Gkontra, P., Lekadir, K., Menegaz, G., & Radeva, P. I. (2021) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms52027.2021.00098 · Cited by 14
Artificial intelligence: Explainability, ethical issues and bias
Marshan, A. (2021) · Annals of Robotics and Automation · doi:10.17352/ara.000011 · Cited by 16
“[13], Rai [8], have classifi ed XAI models based on their generalisability and scope as well as the time of model creation: pre-model, in-model, and post-model [13]; resulting in the following classifi cation:”
BackgroundInfluentialUtkin, L. V., Kovalev, M. S., & Kasimov, E. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.02387 · Cited by 15
“Therefore, explainability of deep learning models is an topical direction of research nowadays, and, as a result, a lot of methods have been developed to address the interpretation problems and to get accurate explanations for obtained predictions [5, 11, 12, 21, 24, 37, 38, 39, 52].”
BackgroundZhang, R. S., Stolzenberg‐Solomon, R. Z., Lynch, S. M., & Urbanowicz, R. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.12844 · Cited by 14
“the inner workings are opaque such that even experts cannot fully understand the rationale behind their predictions [2].”
BackgroundTowards Interpretable ANNs: An Exact Transformation to Multi-Class Multivariate Decision TreesPreprint
Nguyen, D. T., Kasmarik, K., & Abbass, H. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04675 · Cited by 18
“[83], although these rules might be correct in majority of cases, the model is not stable as similar interpretation is expected for similar input instances.”
Rigorous validation of machine learning in laboratory medicine: guidance toward quality improvement
Miller, H. A., & Valdes, R. (2025) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2025.2488842 · Cited by 14
“Several meth-ods have been proposed to address the issue of explainability of complex models such as sensitivity analysis, feature summaries, reporting of internal model weights, surrogate modeling using an intrinsically interpretable model, and Shapley Additive Explanations (SHAP) values [100,102].”
Lee, D. H., Park, E. G., Lee, Y. J., Jeong, H.-S., Roh, H.-Y., Jeong, G.-R., Kim, S.-W., & Kim, H. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms262210925 · Cited by 16
Explainable hybrid intrusion detection for SCADA/ICS: a review and research agenda
Skrodelis, H. K., & Romānovs, A. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1724245 · Cited by 14
TunBERT: Pretraining BERT for Tunisian Dialect Understanding
Haddad, H., Rouhou, A. C., Messaoudi, A., Korched, A., Fourati, C., Sellami, A., HajHmida, M. B., & Ghriss, F. (2023) · SN Computer Science · doi:10.1007/s42979-022-01541-y · Cited by 13
“macro [12] and are referred to as (Acc.”
BackgroundEvaluating Self-attention Interpretability Through Human-Grounded Experimental Protocol
Bhan, M., Achache, N., Legrand, V., Blangero, A., & Chesneau, N. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44070-0_2 · Cited by 17
Natural Language Processing and Text Mining (Turning Unstructured Data into Structured)
Bagheri, A., Giachanou, A., Mosteiro, P. J., & Verberne, S. (2023) · Springer eBooks · doi:10.1007/978-3-031-36678-9_5 · Cited by 21
Opacity, Machine Learning and Explainable AI
Fernández, A. (2023) · The International library of ethics, law and technology · doi:10.1007/978-3-031-48135-2_3 · Cited by 16
Multi-horizon well performance forecasting with temporal fusion transformers
Maldonado-Cruz, E., & Pyrcz, M. J. (2024) · Results in Engineering · doi:10.1016/j.rineng.2024.101776 · Cited by 13
Wahab, S., Salami, B. A., AlAteah, A. H., Al-Tholaia, M. M. H., & Alahmari, T. S. (2024) · Case Studies in Construction Materials · doi:10.1016/j.cscm.2024.e03084 · Cited by 25
Cherry on Top or Real Need? A Review of Explainable Machine Learning in Kidney Transplantation
de Souza, A. A., Stubbs, A., Hesselink, D. A., Baan, C. C., & Boer, K. (2024) · Transplantation · doi:10.1097/tp.0000000000005063 · Cited by 15
How scanning probe microscopy can be supported by artificial intelligence and quantum computing?
Pręgowska, A., Roszkiewicz, A., Osial, M. E., & Giersig, M. (2024) · Microscopy Research and Technique · doi:10.1002/jemt.24629 · Cited by 15
“Besides, the AI technology complexity limits the possibility to understand and modify algorithms to adapt them to particular requirements (Carvalho et al., 2019).”
Al-Jamimi, H. A., BinMakhashen, G. M., & Saleh, T. A. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10187-1 · Cited by 13
“Based on the actual values ð y 1 ; y 2 ; :::; y n Þ and the anticipated values ð y 1 ; y 2 ; :::; y n Þ , 3 prediction error performance assessment indices are adopted in this work [41].”
Explainable AI: definition and attributes of a good explanation for health AI
Kyrimi, E., McLachlan, S., Wohlgemut, J. M., Perkins, Z., Lagnado, D. A., Marsh, W., the ExAIDSS Expert Group, Gimson, A. E., Shafti, A., Ercole, A., et al. (2025) · AI and Ethics · doi:10.1007/s43681-025-00668-x · Cited by 16
“Others also review evaluation methodologies for explainable systems [24].”
InfluentialDecoding the cry for help: AI's emerging role in suicide risk assessment
Esmaeilzadeh, P. (2025) · AI and Ethics · doi:10.1007/s43681-025-00758-w · Cited by 13
“Carvalho et al. [ 113] surveyed inter-pretability methods, finding that local interpretable model-agnostic explanations (LIME) techniques were particularly effective in explaining suicide risk assess - ments to non-technical healthcare providers”
So, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50334-5_28 · Cited by 13
An Evaluation of Backpropagation Interpretability for Graph Classification with Deep Learning
Shun, K. T. T., Limanta, E. E., & Khan, A. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9377827 · Cited by 14
“For surveys and recommendations, we refer to [6], [14], [4].”
BackgroundA branch & bound algorithm to determine optimal bivariate splits for oblique decision tree induction
Bollwein, F., & Westphal, S. (2021) · Applied Intelligence · doi:10.1007/s10489-021-02281-x · Cited by 13
Safe RuleFit: Learning Optimal Sparse Rule Model by Meta Safe Screening
Kato, H., Hanada, H., & Takeuchi, I. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3167993 · Cited by 15
“While complex black box models such as deep neural network models can be used in some fields, interpretable models such as rule models are desirable in fields that require an explanation of decision-making mechanisms such as medical diagnosis [2], [3].”
BackgroundZhou, H., Shan, M., Qin, L., & Cheng, G. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106379 · Cited by 15
A developer-oriented recommender model for the app store: A predictive network analytics approach
Davazdahemami, B., Kalgotra, P., Zolbanin, H. M., & Delen, D. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2023.113649 · Cited by 15
An Advanced Explainable and Interpretable ML-Based Framework for Educational Data Mining
Livieris, I. E., Karacapilidis, N. I., Domalis, G., & Tsakalidis, D. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-42134-1_9 · Cited by 13
Zhang, W., Cheng, W., Fujiwara, K., Evans, R. D., & Zhu, C. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3349353 · Cited by 16
“Theinterpretationofa black-boxmodelisanapproximateestimateoforiginalmodel, andcannothaveperfectfidelity[94],[95].”
Towards an understanding and explanation for mixed-initiative artificial scientific text detection
Weng, L., Liu, S., Zhu, H., Sun, J., Kam-Kwai, W., Han, D., Zhu, M., & Chen, W. (2024) · Information Visualization · doi:10.1177/14738716241240156 · Cited by 14
“Several surveys [3,11,25,26] have been proposed to provide a comprehensive overview of XAI approaches, including intrinsic and post-hoc interpretability [53].”
BackgroundSaifullah, S., Agne, S., Dengel, A. R., & Ahmed, S. (2024) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-024-00483-w · Cited by 17
“In recent years, a wide variety of approaches have been proposed that attempt to explain the predictions of black-box DL models [6, 20–24].”
InfluentialWhen Mitigating Bias is Unfair: Multiplicity and Arbitrariness in Algorithmic Group Fairness
Krčo, N., Laugel, T., Grari, V., Loubes, J., & Detyniecki, M. (2025) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00046 · Cited by 14
“For this purpose, we propose to leverage XAI techniques [56], in particular post-hoc methods, aiming to generate explanations for the predictions of a trained classifier.”
Explainable Neural Network-based Modulation Classification via Concept Bottleneck Models
Wong, L. J., & McPherson, S. (2021) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc51732.2021.9376108 · Cited by 19
“Resistance to the widespread adoption of ML and DL algorithms is not unique to RFML, and has yielded a breadth of work in areas such as verification, testing, and interpretation or explanation methods [4].”
Amini, M., Bagheri, A., & Delen, D. (2022) · Decision Analytics Journal · doi:10.1016/j.dajour.2022.100058 · Cited by 17
A Review of the Mitigating Methods against the Energy Conversion Decrease in Solar Panels
Szabó, G. S., Szabó, R., & Szabó, L. (2022) · Energies · doi:10.3390/en15186558 · Cited by 14
Interpretability of machine-learning models in physical sciencesPreprint
Ghiringhelli, L. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.10443 · Cited by 14
“The tools for addressing the interpretability of ML models vary with the complexity of the models[1-6] (see Fig.”
MethodDing, X., & Kwon, T. J. (2024) · Canadian Journal of Civil Engineering · doi:10.1139/cjce-2023-0410 · Cited by 13
Nakanishi, T., Chophuk, P., & Chinnasarn, K. (2024) · IEEE Access · doi:10.1109/access.2024.3418499 · Cited by 13
“…the review of this manuscript and approving it for publication was Pasquale De Meo . methods ([1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20]) that allow users to understand the reasons behind otherwise opaque ML predictions.”
BackgroundMachine learning in oncological pharmacogenomics: advancing personalized chemotherapy
Avcı, Ç. B., Bağca, B. G., Shademan, B., Takanlou, L. S., Takanlou, M. S., & Nourazarian, A. (2024) · Functional & Integrative Genomics · doi:10.1007/s10142-024-01462-4 · Cited by 16
“Despite its promise, ML faces challenges such as data quality, algorithmic transparency, and model interpretability (Carvalho et al. 2019 ).”
Ge, Z., Hu, Q. Y., Wang, R., Fei, H., Zhu, Y., & Wang, Z. (2024) · Coatings · doi:10.3390/coatings14121539 · Cited by 12
“…achieving exceptional accuracy Among these models, the NN exhibited superior predictive performance, achieving exceptional accuracy in capturing the complex and nonlinear relationships between texture parameters and tribological performance. in capturing the complex and nonlinear…”
Zhang, M., Deng, Y., Zhou, Q., Gao, J., Zhang, D., & Pan, X. (2024) · Environmental Science Processes & Impacts · doi:10.1039/d4em00662c · Cited by 15
Eight quick tips for biologically and medically informed machine learning
Oneto, L., & Chicco, D. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012711 · Cited by 15
Wang, Y., Zhu, C., Zhang, S., Xiang, C., Gao, Z., Zhu, G., Sun, J. F., Ding, X., Li, B., & Shen, X. (2025) · Advanced Science · doi:10.1002/advs.202413805 · Cited by 14
Li, X., Wu, W., Guo, H., Qiao, X., Lu, Y., Wu, Y. J., & Qiao, G. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.110414 · Cited by 12
AI-powered network optimization for next-generation wireless connectivity: exploring 5G/6G networks
AlQahtani, O. (2025) · Telecommunication Systems · doi:10.1007/s11235-025-01303-0 · Cited by 17
“This transparency enables network operators and stakeholders to understand the decision-making pathways of AI tools, thus improving trust and facilitating informed decision-making [15].”
Statistical analysis techniques in water quality monitoring: a review
Saab, C., & Zéhil, G. (2025) · Stochastic Environmental Research and Risk Assessment · doi:10.1007/s00477-025-03035-8 · Cited by 14
“…computational capabilities and effectiveness in handling large datas - ets (Saab and Zéhil 2023 ), they often suffer from limited interpretability, which hampers the understanding of causal relationships in human-comprehensible terms (Makridakis et al. 2018; Carvalho et al. 2019; Herm et al. 2023).”
Alam, K. N., Zadeh, P. B., & Sheikh-Akbari, A. (2025) · Electronics · doi:10.3390/electronics14153024 · Cited by 15
Jan, M. B., & Chai, M. (2025) · International Journal of Pressure Vessels and Piping · doi:10.1016/j.ijpvp.2025.105690 · Cited by 19
VISUAL STORYTELLING AND EXPLAINABLE INTELLIGENCE IN ORGANIZATIONAL CHANGE COMMUNICATION
Kashyap, S., Purohit, S., Kumar, A., Jawaid, F. I. M., Kumar, J. R. R., & Ajani, S. N. (2025) · ShodhKosh Journal of Visual and Performing Arts · doi:10.29121/shodhkosh.v6.i5s.2025.6965 · Cited by 13
An explainable data-driven optimization framework for industrial predictive maintenance scheduling
Danach, K. M., Harb, H., Issa, H., & Saker, L. (2026) · Results in Engineering · doi:10.1016/j.rineng.2026.109022 · Cited by 14
Kibria, M. R., Lafond, S., & Arslan, J. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104405 · Cited by 16
A SWOT Analysis of Human- and Machine Learning- Based Embryo Assessment
Tran, H. P., Tran, L. N.-H., Dang, H. T., Vu, T. D., Trinh, T. D., Bao, P. T., & Sang, V. N. T. (2020) · IEEE Access · doi:10.1109/access.2020.3045772 · Cited by 14
“A survey of ML model interpretability was introduced in a recent paper [81].”
The Epistemology of Machine Learning
Bai, H. (2022) · Filosofija Sociologija · doi:10.6001/fil-soc.v33i1.4668 · Cited by 12
“Both scientists and philosophers have addressed various facets of this black box dilemma of machine learning, which has also been labelled a problem with opacity, understanding, transparency and interpretability (Burrell 2016; Walmsley 2020; De Laat 2018; Carvalho et al. 2019; Krishnan 2019).”
BackgroundBuhl, M. (2022) · Diagnostics · doi:10.3390/diagnostics12020463 · Cited by 12
“An ideal CDSS should provide interpretability to experts [1,4,8] and must exploit data from different clinical databases.”
BackgroundMachine Learning Interpretability to Detect Fake Accounts in Instagram
Sallah, A., Alaoui, E. A. A., Agoujil, S., & Nayyar, A. (2022) · International Journal of Information Security and Privacy · doi:10.4018/ijisp.303665 · Cited by 12
Cascone, L., Pero, C., & Proença, H. P. M. C. (2023) · Image and Vision Computing · doi:10.1016/j.imavis.2023.104645 · Cited by 12
Deters, H., Droste, J., & Schneider, K. A. (2023) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3593434.3593444 · Cited by 14
“[3, 4, 11] M5 It is clear how the parameters that the user enters are connected to the events being explained.”
BackgroundMethodInfluentialMartínez, M. A. M., Nadj, M., Langner, M., Toreini, P., & Maedche, A. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3607145 · Cited by 16
“These models are inherently transparent, as their parameters directly reveal how the model works (e.g., attributes’ weights in linear regression models) [ 14 , 68 ].”
BackgroundMethodUnveiling the Black Box: A Unified XAI Framework for Signal-Based Deep Learning Models
Shojaeinasab, A., Jalayer, M., Baniasadi, A., & Najjaran, H. (2024) · Machines · doi:10.3390/machines12020121 · Cited by 12
Brunet-Gouet, É., Vidal, N., & Roux, P. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-55245-8_7 · Cited by 13
SIDEs: Separating Idealization from Deceptive 'Explanations' in xAI
Sullivan, E. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658999 · Cited by 17
Don’t get me wrong: How to apply deep visual interpretations to time series
Löffler, C., Lai, W.-C., Zanca, D., Schmidt, L., Eskofier, B. M., & Mutschler, C. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06798-3 · Cited by 13
“Considering this, we can’t help but ask what question the saliency metrics should answer [8, 73, 80].”
Clemens, T., & Viechtbauer-Gruber, M. (2020) · EAGE Conference and Exhibition incorporating SPE EUROPEC · doi:10.2118/200540-ms · Cited by 12
Can Big Data guide prognosis and clinical decisions in epilepsy?
Li, X., Cui, L., Zhang, G., & Lhatoo, S. D. (2021) · Epilepsia · doi:10.1111/epi.16786 · Cited by 16
Explainability of AI-predictions based on psychological profiling
Neugebauer, S., Rippitsch, L., Sobieczky, F., & Geiß, M. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.01.361 · Cited by 12
A fair classifier chain for multi‐label bank marketing strategy classification
Radovanović, S., Petrović, A., Delibašić, B., & Suknović, M. (2021) · International Transactions in Operational Research · doi:10.1111/itor.13059 · Cited by 14
Clinical Synthetic Data Generation to Predict and Identify Risk Factors for Cardiovascular Diseases
García-Vicente, C., Chushig-Muzo, D., Mora-Jiménez, I., Fabelo, H. A., Gram, I. T., Løchen, M., Granja, C., & Soguero-Ruíz, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-23905-2_6 · Cited by 12
An Explainable System for Diagnosis and Prognosis of COVID-19
Lu, J., Jin, R., Song, E., Alrashoud, M. R., Al-Mutib, K. N., & Al-Rakhami, M. S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3037915 · Cited by 15
Towards Explainable Evaluation Metrics for Machine TranslationPreprint
Leiter, C., Lertvittayakumjorn, P., Fomicheva, M., Zhao, W., Gao, Y., & Eger, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.13041 · Cited by 20
“…(e.g. Lipton 2016; Biran and Cotton 2017; Adadi and Berrada 2018; Doshi-Velez and Kim 2017; Doˇsilovi´c et al. 2018; Guidotti et al. 2018; Arya et al. 2019; Carvalho et al. 2019; Gilpin et al. 2019; Miller 2019; Barredo Arrieta et al. 2020; Bodria et al. 2023; Linardatos et al. 2021).”
BackgroundAtmaramani, R., Dreossi, T., Ford, K., Gan, L., Mitchell, J. M., Tu, S., Velayutham, J., Zeng, H., Chickering, M., Soare, T., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.01.04.574270 · Cited by 14
“We trained the classifier on a subset of cells, tested on the held-out cells, and evaluated performance using the “accuracy” metric of Carvalho et al. (Carvalho et al., 2019) .”
MethodResponsible AI measures dataset for ethics evaluation of AI systems
Rismani, S., Davis, L., Mingole, B., Rostamzadeh, N., Shelby, R. M., & Moon, A. (2025) · Scientific Data · doi:10.1038/s41597-025-06021-5 · Cited by 13
Ljubobratović, D., Vuković, M., Bakarić, M. B., Jemrić, T., & Matetić, M. (2021) · Electronics · doi:10.3390/electronics10243115 · Cited by 11
The Cambridge Handbook of Facial Recognition in the Modern State
Matulionytė, R., Žalnieriūtė, M., Selwyn, N., Andrejevic, M., O’Neill, C., Gu, X., Smith, G., Akbari, A., Taylor, S. M., Goldenfein, J., et al. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009321211 · Cited by 20
He, M., Wang, Y., & Fan, Y. (2024) · Journal of Physics Condensed Matter · doi:10.1088/1361-648x/ad4aab · Cited by 11
“Most ML approaches focus on average properties or lack interpretability, missing out on local details [120, 121].”
BackgroundHuman-annotated rationales and explainable text classification: a survey
Herrewijnen, E., Nguyen, D., Bex, F., & van Deemter, K. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1260952 · Cited by 22
FairMOE: counterfactually-fair mixture of experts with levels of interpretability
Germino, J., Moniz, N., & Chawla, N. V. (2024) · Machine Learning · doi:10.1007/s10994-024-06583-2 · Cited by 16
Data-Driven Approaches for Accident Analysis in Sociochemical Systems
Gholamizadeh, K., Zarei, E., Yazdi, M., & Amin, M. T. (2024) · Studies in systems, decision and control · doi:10.1007/978-3-031-62470-4_17 · Cited by 16
Artificial Intelligence Failures in Autonomous Vehicles: Causes, Implications, and Prevention
Joshi, H. C., & Kumar, S. (2024) · Computer · doi:10.1109/mc.2024.3449435 · Cited by 15
Paccotacya-Yanque, R. Y. G., Bissoto, A. E., & Avila, S. (2024) · International Symposium on Medical Information Processing and Analysis (SIPAIM) · doi:10.1109/sipaim62974.2024.10783606 · Cited by 12
“It is also referred to as comprehensibility [22], [26] and understandability [11], [27].”
Sushmitha, G., & Utukuru, S. (2025) · Iran Journal of Computer Science · doi:10.1007/s42044-024-00223-7 · Cited by 11
“How-G a novel deep learning-based system designed for long-term monitoring of age-based disease prediction, utilizing Convolutional Neural Networks (CNNs) [6] and Recurrent Neural Networks (RNNs) [7].”
Cong, J., Sun, H., Feng, D., Xie, Z., Tan, B., & Liu, K. (2025) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3721129 · Cited by 11
“CNN interpretation methods have been extensively studied to explain why CNN models predict what they predict in the computer vision domain [4].”
Evaluating Model Explanations without Ground Truth
Rawal, K., Fu, Z., Delaney, E. D., & Russell, C. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732219 · Cited by 15
“Previous work often refers to quality scores as fidelity or explanation faithfulness [8, 23, 35].”
Wang, Y., Lei, J., Jin, Z., Jiang, Y., Zhang, N., Lv, M., & Liu, T. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106036 · Cited by 12
Moradizeyveh, S., Hanif, A., Liu, S., Qi, Y., Beheshti, A., & Ieva, A. D. (2025) · Sensors · doi:10.3390/s25154575 · Cited by 11
Köhler, D., Rügamer, D., Boyle, L. J., Maloney, K. O., & Schmid, M. C. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00033-7 · Cited by 14
Marouf, A. A., Mottalib, M. M., Ridi, S. S., Jafarullah, O., Rokne, J. G., & Alhajj, R. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03253-8 · Cited by 12
“SHAP also enables transparent visualization of how individual features contribute to a prediction and can be aggregated between patients to reveal cohort-level risk drivers, thus bridging clinical interpretability and model transparency more effectively than alternative methods [66, 69, 70].”
Explainable AI for Stock Price Movement Prediction: A Comparative Study of Machine Learning Models
Rajab, A. A., Furaijl, H. B., Al-Mamouri, H., Srayyih, F. H., Harjan, S. A., Hasan, B. F., Shabeeb, A. L., Fadhil, N. A., & Ahmed, S. R. (2025) · International Conference on Business Analytics for Technology and Security (ICBATS) · doi:10.1109/icbats66542.2025.11258500 · Cited by 11
Nnadi, L. C., Isiwu, C. P., Ding, D., Muepu, D. M., & Watanobe, Y. (2026) · IEEE Access · doi:10.1109/access.2026.3652631 · Cited by 13
“Surveys highlight growing use of XAI in education and healthcare, but also note challenges of stability and domain fit [17], [35], [36], [37], [38].”
Yu, H., Evans, J. A., & Varshney, L. R. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14277 · Cited by 15
“While explainability has been widely studied in XAI, it fails to capture the communication of explanations to a human model.”
BackgroundInfluentialEthical Guidelines for Artificial Intelligence: A Systematic Literature Review
Farooq, M. S., Tahseen, R., & Omer, U. (2021) · VFAST Transactions on Software Engineering · doi:10.21015/vtse.v9i3.701 · Cited by 15
Tofighi, M., Asgary, A., Tofighi, G., Podloski, B., Cronemberger, F. A., Mukherjee, A., & Liu, X. (2021) · Applied Sciences · doi:10.3390/app112311198 · Cited by 11
Generating Artificial Texts as Substitution or Complement of Training Data
Claveau, V., Chaffin, A., & Ewa Kijak, C. (2022) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5n3kmjij5xgy · Cited by 11
Nagpal, S., Pinna, N. K., Pant, N., Singh, R. P., Srivastava, D., & Mande, S. S. (2022) · Journal of Molecular Biology · doi:10.1016/j.jmb.2022.167684 · Cited by 12
“Binary models were developed for each possible pair of graduated outcomes as well using all possible features and their combinations (Supplementary File 2,3 and Supplementary Table 11,12).”
InfluentialPredictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis
Jalali, A., Haslhofer, B., Kriglstein, S., & Rauber, A. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-37717-4_46 · Cited by 14
Baheri, A. (2023) · AI Magazine · doi:10.1002/aaai.12141 · Cited by 13
Synthetic Data Improve Survival Status Prediction Models in Early-Onset Colorectal Cancer
Kim, H., Jang, W. S., Sim, W. S., Kim, H. S., Choi, J. E., Baek, E. S., Park, Y. R., & Shin, S. J. (2024) · JCO Clinical Cancer Informatics · doi:10.1200/cci.23.00201 · Cited by 17
Empowering Learning through Intelligent Data-Driven Systems
Aldriwish, K. (2024) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.6675 · Cited by 11
Nakanishi, T. (2024) · IEEE Access · doi:10.1109/access.2024.3387547 · Cited by 13
“In recent years, substantial research has been conducted on ML interpretability and XAI, with numerous surveys and comprehensive descriptions [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22] addressing these issues.”
BackgroundNovel embeddings improve the prediction of risk perception
Hussain, Z., Mata, R., & Wulff, D. U. (2024) · EPJ Data Science · doi:10.1140/epjds/s13688-024-00478-x · Cited by 13
“Issues of interpretability are pivotal when considering applications in the real world that require understanding and justification of model performance [e.g., 29–31].”
Improving interpretability via regularization of neural activation sensitivity
Moshe, O., Fidel, G., Bitton, R., & Shabtai, A. (2024) · Machine Learning · doi:10.1007/s10994-024-06549-4 · Cited by 12
“Various recent studies [3, 8, 31] attempted to determine what properties an attribution-based explanation should have.”
Visualisation and knowledge discovery from interpretable models
Ghosh, S., Peter Tiňo, B., & Bunte, K. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206702 · Cited by 15
Metadata
- Authors
- Carvalho, D. V., Pereira, E. M., & Cardoso, J. S.
- Affiliations
- Deloitte Portugal, Manuel Bandeira Street, 43, 4150-479 Porto, Portugal
- Deloitte Portugal (Portugal) (PT)
- Universidade do Porto (PT)
- Venue
- Electronics
- Publisher
- MDPI AG
- Published
- 2019-07-26
- Volume
- 8
- Issue
- 8
- Pages
- 832
- ISSN
- 2079-9292 (ISSN-L)
- OpenAlex
- W2964303497
- Cited by
- 1,842
- References
- 106
- Citations by year
- 2026: 224 · 2025: 354 · 2024: 353 · 2023: 313 · 2022: 285 · 2021: 218 · 2020: 91 · 2019: 3
- FWCI
- 68.32
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Electronics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- model interpretability, explainable AI, explanation quality metrics, interpretable models, explanation methods, black-box models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.