Explainable AI Evaluation: A Top-Down Approach for Selecting Optimal Explanations for Black Box Models
Open accessThe DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
Explainable Artificial Intelligence (XAI) evaluation has grown significantly due to its extensive adoption, and the catastrophic consequence of misinterpreting sensitive data, especially in the medical field. However, the multidisciplinary nature of XAI research resulted in diverse scholars possessing significant challenges in designing proper evaluation methods. This paper proposes a novel framework of a three-layered top-down approach on how to arrive at an optimal explainer, accenting the persistent need for consensus in XAI evaluation. This paper also investigates a critical comparative evaluation of explanations in both model agnostic and specific explainers including LIME, SHAP, Anchors, and TabNet, aiming to enhance the adaptability of XAI in a tabular domain. The results demonstrate that TabNet achieved the highest classification recall followed by TabPFN, and XGBoost. Additionally, this paper develops an optimal approach by introducing a novel measure of relative performance loss with emphasis on faithfulness and fidelity of global explanations by quantifying the extent to which a model’s capabilities diminish when eliminating topmost features. This addresses a conspicuous gap in the lack of consensus among researchers regarding how global feature importance impacts classification loss, thereby undermining the trust and correctness of such applications. Finally, a practical use case on medical tabular data is provided to concretely illustrate the findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Another challenge is the lack of standardized metrics to evaluate the quality and usefulness of explanations. What is considered a good explanation depends on the clinical context, the user’s expertise, and the decision at hand [29,30].”
From Explainable AI in Clinical Decision Support Systems: A Meta-Analysis of Methods, Applications, and Usability Challenges (2025) · cites this paper as [30]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Narayan, V., Mall, P. K., Alkhayyat, A. H., Abhishek, K., Kumar, S., & Pandey, P. (2023) · Journal of Sensors · doi:10.1155/2023/8276738 · Cited by 58
RetractedSMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Building Predictive Models in R Using the caret Package
Kuhn, M. (2008) · Journal of Statistical Software · doi:10.18637/jss.v028.i05 · Cited by 8,863
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 1×
“The toolbox encompasses several explainers such as Saliency Map [44], Grad-CAM [29], Integrated Gradients [45], etc. Dalex [46] is a model-agnostic explanation toolbox that supports popular models such as Caret [47], random forests, and Gradient Boosting Machines.”
MethodAnchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 6×
“To address these challenges, Anchors introduced high-precision, rule-based, and model-agnostic explanations that utilize reinforcement learning by applying simple if-then rules [14,59].”
BackgroundMethodResultInfluentialMachine Learning Interpretability: A Survey on Methods and Metrics
Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019) · Electronics · doi:10.3390/electronics8080832 · Cited by 1,842 · cited 10×
“A study conducted by Carvalho et al. [6] showed that when two models contribute similar features to arrive at a prediction with consistent explanations, they exhibit a high degree of consistency, signifying a robust explanatory characteristic.”
BackgroundInfluentialTabNet: Attentive Interpretable Tabular Learning
Arik, S. Ö., & Pfister, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16826 · Cited by 1,771
Research works printed but not linked: 4
- Gurgul, 2022, A Parameters-Based Heart Disease Prediction Model, Age, 53, 28
- Thien, T.Q. Concept and Pattern Discovery for Reliable and Explainable AI. [PhD Thesis, University of Tsukuba].
- Salih, A., Raisi-Estabragh, Z., Galazzo, I.B., Radeva, P., Petersen, S.E., Menegaz, G., and Lekadir, K. (2023). Commentary on Explainable Artificial Intelligence Methods: SHAP and LIME. arXiv.
- Famiglini, L., Campagner, A., and Cabitza, F. (2023). ECAI 2023, IOS Press.
Cited by
Abbas, Q., Jeong, W., & Lee, S. W. (2025) · Healthcare · doi:10.3390/healthcare13172154 · Cited by 236
“What is considered a good explanation depends on the clinical context, the user’s expertise, and the decision at hand [29,30].”
Machucho-Cadena, R., & González, O. (2025) · Systems · doi:10.3390/systems13040264 · Cited by 142
Interpretable and explainable predictive machine learning models for data-driven protein engineering
Medina-Ortiz, D., Khalifeh, A., Anvari-Kazemabad, H., & Davari, M. D. (2024) · Biotechnology Advances · doi:10.1016/j.biotechadv.2024.108495 · Cited by 63
Muñoz-Ordóñez, J. F., Cobos, C., Vidal-Rojas, J.-C., & Herrera, F. (2025) · International Journal of Intelligent Systems · doi:10.1155/int/4934696 · Cited by 15
Compressive Strength of Geopolymer Concrete Prediction Using Machine Learning Methods
Stel’makh, S. A., Beskopylny, A. N., Shcherban’, E. M., Razveeva, I., Oganesyan, S., Shakhalieva, D. M., Chernil'nik, A. A., & Onore, G. (2025) · Algorithms · doi:10.3390/a18120744 · Cited by 12
Cohen, M. C., Kim, N., Ba, Y., Pan, A., Bhatti, S. A., Salehi, P., Sung, J., Blasch, E., Mancenido, M. V., & Chiou, E. K. (2025) · AI Magazine · doi:10.1002/aaai.70000 · Cited by 10
Effective Sensor Selection for Human Activity Recognition via Shapley Value
Borella, E., Çakmakçi, U. B., Gottardis, E., Buratto, A., Marchioro, T., & Badia, L. (2024) · IEEE International Workshop on Metrology for Living Environment (MetroLivEnv) · doi:10.1109/metrolivenv60384.2024.10615860 · Cited by 6
An in-depth analysis of KernelSHAP and SamplingSHAP: assessing robustness, error, and efficiency
Aydoğan, B., & Aytekin, T. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02541-z · Cited by 5
MDC-Net: Multimodal Detection and Captioning Network for Steel Surface Defects
Chazhoor, A. A. P., Hu, S., Gao, B., & Woo, W. L. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-59057-3_20 · Cited by 3
Self-citationLENs for Analyzing the Quality of Life of People with Intellectual Disability
Fraile-Parra, D., Costa, V., & Dellunde, P. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-71170-1_15 · Cited by 3
Advancements and Challenges in Artificial Intelligence Applications in Healthcare Delivery SystemsPreprint
Aftab, M., Nadeem, A., Zhang, C., Dong, Z., Jiang, Y., & Liu, K. (2024) · Preprints.org · doi:10.20944/preprints202407.0938.v1 · Cited by 5
ElBahy, S., Kaushik, A. K., Ramli, R., & Woo, W. L. (2025) · Energy and Buildings · doi:10.1016/j.enbuild.2025.116408 · Cited by 2
Self-citationA Review of Taxonomies of Explainable AI (XAI) Evaluation
Speith, T. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806537 · Cited by 3
Enhancing Cardiac AI Explainability Through Ontology-Based Evaluation
Tsolakis, N., Maga‐Nteve, C., Vrochidis, S., Bassiliades, N., & Μεδίτσκος, Γ. (2024) · International Conference on Information, Intelligence, Systems & Applications (IISA) · doi:10.1109/iisa62523.2024.10786663 · Cited by 1
Explainability in Fraud Detection: Trustworthy AI and Pattern Detection
Mehenni, H. A., Bellart, S., & Chamma, A. (2025) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-96522-7_14 · Cited by 1
MDC-Net: multimodal defect captioning network for surface steel defects
Chazhoor, A. A. P., Hu, S., Gao, B., & Woo, W. L. (2025) · The Visual Computer · doi:10.1007/s00371-025-04093-3 · Cited by 1
Self-citationExploring KANs: Theory and Applications for Binary Classification
Rodríguez, N. O. R., Alatriste, C. R., Alarcón-Paredes, A., Villuendas-Rey, Y., Yáñez-Márquéz, C., & Argüelles-Cruz, A.-J. (2024) · International Conference on Software Process Improvement (CIMPS) · doi:10.1109/cimps65195.2024.11095956 · Cited by 1
Kurcjusz, M., & Das, R. R. (2025) · Acta Scientiarum Polonorum Architectura · doi:10.22630/aspa.2025.24.28 · Cited by 3
Nasir, I. M., Tehsin, S., Damaševičius, R., Hussain, N., Bilal, M. A., & Maskeliūnas, R. (2025) · International Conference on Computational Intelligence, Security, and Artificial Intelligence (IntelliSecAI) · doi:10.1109/intellisecai66368.2025.11472877 · Cited by 1
Dafali, S. M., Kissi, M., & Beggar, O. E. (2026) · Results in Engineering · doi:10.1016/j.rineng.2026.112856 · Cited by 1
Explainable Tabular Transformer Model for Enhancing Security in Cryptocurrencies
Olusegun, R., & Yang, B. (2025) · International Conference on Artificial Intelligence, Robotics and Control (AIRC) · doi:10.1109/airc64931.2025.11077526
Review of NLP in EMR: abbreviation, diagnosis, and ICD classification
Basheer, N. A. B. I., Nordin, S., Kasim, S. S., Ali, A. M., & Hamid, N. H. A. (2025) · International Journal of Informatics and Communication Technology (IJ-ICT) · doi:10.11591/ijict.v14i3.pp881-891
How Unstable Is LIME? Evaluating Sensitivity and Trust in Explainable AI
Castilho, T. M. F. D. O. (2025) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Ensembled Tabnet-Based Model Approach for Diabetes Disease Classification
Obunge, D. O., Muriira, L. M., & Mbandu, V. (2025) · International Journal of Professional Practice · doi:10.71274/ijpp.v13i4.559
Bounded Rationality and AI Modeling Real-World Decision-Making
Musam, V. S., Musham, N. K., Mekala, R. R., & Fahmi, N. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-6691-3.ch012
Explainable Multilingual Aspect-Based Sentiment Analysis for Tourism Using SHAP and LIME
Pramono, B. A., Gernowo, R., & Sofwan, A. (2026) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.18774
Explainable AI for Mechanistic Toxicology and Regulatory-Grade Safety Assessment
Alzoubi, A. E., Wahed, M. K. Y. A., & Wahed, S. A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8562-4.ch011
Metadata
- Authors
- Mirzaei, S., Mao, H., Al-Nima, R. R. O., & Woo, W. L.
- Affiliations
- Northumbria University (GB)
- Northern Technical University (IQ)
- Corresponding author
- Woo, Wai Lok
- Venue
- Information
- Publisher
- MDPI AG
- Published
- 2023-12-20
- Volume
- 15
- Issue
- 1
- Pages
- 4
- ISSN
- 2078-2489 (ISSN-L)
- OpenAlex
- W4389994734
- Semantic Scholar
- 9132bec4065348584e0fc3ea944ca4c217868030
- Cited by
- 25
- References
- 54
- Citations by year
- 2026: 5 · 2025: 13 · 2024: 7
- FWCI
- 2.21
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Information (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Northumbria University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- black-box models, tabular data, lime, SHAP, anchors, TabNet, TabPFN, faithfulness, fidelity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.