Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics
Open accessThe DOI matches its Crossref record; 88 of the 88 references it cites were found in a registry or an index.
Abstract
The most successful Machine Learning (ML) systems remain complex black boxes to end-users, and even experts are often unable to understand the rationale behind their decisions. The lack of transparency of such systems can have severe consequences or poor uses of limited valuable resources in medical diagnosis, financial decision-making, and in other high-stake domains. Therefore, the issue of ML explanation has experienced a surge in interest from the research community to application domains. While numerous explanation methods have been explored, there is a need for evaluations to quantify the quality of explanation methods to determine whether and to what extent the offered explainability achieves the defined objective, and compare available explanation methods and suggest the best explanation from the comparison for a specific task. This survey paper presents a comprehensive overview of methods proposed in the current literature for the evaluation of ML explanations. We identify properties of explainability from the review of definitions of explainability. The identified properties of explainability are used as objectives that evaluation metrics should achieve. The survey found that the quantitative metrics for both model-based and example-based explanations are primarily used to evaluate the parsimony/simplicity of interpretability, while the quantitative metrics for attribution-based explanations are primarily used to evaluate the soundness of fidelity of explainability. The survey also demonstrated that subjective measures, such as trust and confidence, have been embraced as the focal point for the human-centered evaluation of explainable systems. The paper concludes that the evaluation of ML explanations is a multidisciplinary research topic. It is also not possible to define an implementation of evaluation metrics, which can be applied to all explanation methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“They found that some properties (clarity, broadness, and completeness) are still in shortage of appropriate metrics, and so is the class of explanations that are example-based.”
From Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021) · cites this paper as [141]
In the citing paper
“It is not possible to come up with quantitative metrics that are applicable for all interpretability methods [169].”
From Transparency of deep neural networks for medical image analysis: A review of interpretability methods (2022) · cites this paper as [169]
In the citing paper
“They conclude their survey by stating that”
From Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021) · cites this paper as [141]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 2×
“One way to objectively evaluate explanations is to verify whether the explanation mechanism satisfies (or does not satisfy) certain axioms, or properties [34].”
Background"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“[39] explained an ensemble of models, such as neural networks with a single model effectively.”
BackgroundCausality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“Explanation is also a domain-specific note, and there cannot be an all-purpose explanation [22], and different types of explanations might be useful.”
BackgroundPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 2×
“The machine learning explainability has been reviewed thoroughly in recent years [10,14,17,23,28,31,32].”
BackgroundLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · cited 4×
“…Main effect complexity [46] Level of (dis)agreement [82] Attribution-based explanations Monotonicity [80] Non-sensitivity [80,83], Sensitivity [84,85] Effective complexity [80] Remove and retrain [86] Recall of important features [33] Implementation invariance [85] Selectivity [87] Continuity…”
BackgroundInfluentialTowards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 4×
“The advantages of application-grounded and human-grounded evaluations are that they can provide direct and strong evidence of success of explanations [9].”
BackgroundMethodInfluential
Research works printed but not linked: 5
- Zhou, J., and Chen, F. (2018). Human and Machine Learning: Visible, Explainable, Trustworthy and Transparent, Springer.
- Lage, I., and Doshi-Velez, F. (2020, January 17). Human-in-the-Loop Learning of Interpretable and Intuitive Representations. Proceedings of the ICML Workshop on Human Interpretability in Machine Learning, Vienna, Austria.
- Becker, B., Kohavi, R., and Sommerfield, D. (2001). Visualizing the Simple Bayesian Classifier. Information Visualization in Data Mining and Knowledge Discovery, Morgan Kaufmann Publishers Inc.
- Poursabzi-Sangdeh, F., Goldstein, D.G., Hofman, J.M., Vaughan, J.W., and Wallach, H. (2019). Manipulating and Measuring Model Interpretability. arXiv.
- Yang, M., and Kim, B. (2019). Benchmarking Attribution Methods with Relative Feature Importance. arXiv, 1907.
Cited by
Items 301–400 of 676
User centric explanations: a breakthrough for explainable models
Hassan, M. A., Abdulhak, M., bin Sulaiman, R., & Kahtan, H. (2021) · doi:10.1109/icit52682.2021.9491641 · Cited by 7
Farahani, H., Blagojević, M., Azadfallah, P., Watson, P., Esrafilian, F., & Saljoughi, S. (2023) · Springer eBooks · doi:10.1007/978-3-031-31172-7_2 · Cited by 6
Wang, T., Ragodos, R., Feng, L., Yu, & -, H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.16987 · Cited by 7
Unbiasing fairness evaluation of radiology AI model
Liang, Y., Chao, H., Zhang, J., Wang, G., & Yan, P. (2024) · Meta-Radiology · doi:10.1016/j.metrad.2024.100084 · Cited by 8
A Survey of Explainable Reinforcement Learning: Targets, Methods and Needs
Saulières, L. (2026) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.20550 · Cited by 8
Path Optimization for Autonomous Driving using Deep Learning
Schitz, D., & Aschemann, H. (2022) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2022.10.560 · Cited by 6
Deng, F., Feng, C. H., Gao, N., & Zhang, L. (2025) · Transactions on Artificial Intelligence · doi:10.53941/tai.2025.100005 · Cited by 9
“…the multi-class and unbalanced nature of the data in this study, Balanced Accuracy and the Kappa statistic (Kappa), in addition to F1 Score (F1), Area Under the Curve (AUC), sensitivity (Recall), and specificity, were used to evaluate classification performance based on the test set [32,63,74–77].”
Machine learning-driven prediction of hospital admissions using gradient boosting and GPT-2
Zhang, X., Wang, H., Yu, G., & Zhang, W. (2025) · Digital Health · doi:10.1177/20552076251331319 · Cited by 7
“Additionally, we determined the optimal decision threshold for each model using the ROC curve, 21,22 which allowed us to balance sensitivity and speci fi city to maximize predictive performance.”
Stock, R., & Kegel, M. M. (2025) · International Journal of Social Robotics · doi:10.1007/s12369-025-01239-0 · Cited by 10
Canha, D., Kubler, S., Främling, K., & Fagherazzi, G. (2025) · ACM Computing Surveys · doi:10.1145/3737445 · Cited by 10
“…is directly achieved through constraints imposed on the model during training (e.g., white-box models are intrinsically explainable); (ii) Post-hoc methods: applied to provide black-box explanations after model training [12, 33], thus avoiding the explainability vs. accuracy tradeoff.”
InfluentialGender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methods
Dhaini, M., Erdoğan, E., Feldhus, N., & Kasneci, G. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732192 · Cited by 9
“These metrics include both quantitative measures, such as fidelity, stability, consistency, and plausibility [19, 39, 72, 77], as well as qualitative approaches, which involve human-based evaluations of the generated explanations [30, 36] to assess how humans perceive these explanations.”
Predicting Choices Driven by Emotional Stimuli Using EEG-Based Analysis and Deep Learning
Aldayel, M. S., Kharrat, A., & Al-Nafjan, A. (2023) · Applied Sciences · doi:10.3390/app13148469 · Cited by 6
Anitha, G., Manickam, J. M. L., & Mohan, S. K. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105523 · Cited by 6
Impact of Fidelity and Robustness of Machine Learning Explanations on User Trust
Wang, B., Zhou, J. J., Li, Y., & Chen, F. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8391-9_17 · Cited by 17
Self-citationUnlocking The Potential: Transforming Defense Through Human Capital Management
Sarjito, A. (2023) · West Science Interdisciplinary Studies · doi:10.58812/wsis.v1i11.333 · Cited by 6
“Metrics and Evaluation Methods: Measuring the impact of HCM in defense requires suitable metrics and evaluation methods [25].”
MethodAI-Driven Real-Time Incident Detection for Intelligent Transportation Systems
Γκιόκα, Γ., Domínguez, M., Tympakianaki, A., & Mentzas, G. N. (2024) · Advances in transdisciplinary engineering · doi:10.3233/atde240021 · Cited by 6
Generative AI as a Supportive Tool for Scientific Research
Weinberg, A. I. (2024) · Springer eBooks · doi:10.1007/978-3-031-46238-2_1 · Cited by 8
Machine Learning Based Aspect Category Detection for Hindi Data Using TF-IDF and Count Vectorization
Gupta, A., & Sharma, U. (2024) · International Conference on Device Intelligence, Computing and Communication Technologies (DICCT) · doi:10.1109/dicct61038.2024.10532960 · Cited by 6
“The experiments are evaluated by utilizing various assessments, encompassing precision, recall, F1-score, and Hamming Loss [15].”
MethodJaveed, A., Anderberg, P., Ghazi, A. N., Javeed, A., Dallora, A. L., & Berglund, J. S. (2024) · International Conference on Control, Automation and Diagnosis (ICCAD) · doi:10.1109/iccad60883.2024.10553890 · Cited by 6
“The performance of the ML models is tested by the researchers using several validation approaches [36]–[38].”
MethodRemusati, H., Caillec, J. L., Schneider, J.-Y., Petit-Frère, J., & Merlet, T. (2024) · Remote Sensing · doi:10.3390/rs16142569 · Cited by 6
Textile Surface Defects Analysis with Explainable AI
Soon, R. J., & Chui, C. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00248 · Cited by 6
Abu-Shaikha, M. F. (2024) · Asian Journal of Civil Engineering · doi:10.1007/s42107-024-01180-z · Cited by 6
“The model’s performance in this study was evaluated using key metrics, including Accuracy, Precision, Recall, F1-Score, and Root Mean Square Error (RMSE) (Al Yamani et al., 2023; Naidu et al., 2023; Zhou et al., 2021).”
Research on Beef Marbling Grading Algorithm Based on Improved YOLOv8x
Liu, J., Lian, W., Xu, H., Pi, J., & Wang, D. (2025) · Foods · doi:10.3390/foods14101664 · Cited by 7
Yokoyama, H., Shingaki, R., Nishino, K., Shimizu, S., & Pham, T. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228702 · Cited by 8
“However, since these methods do not consider the causal relationships between the prediction error, the target variable, and the explanatory variables in the ML method, existing heuristic attribution methods do not necessarily reflect the causality in the generating process of the prediction error…”
Model Agnostic Explainability Techniques in Ultrasound Image Analysis
Prentzas, N., Pitsiali, M., Kyriacou, E. C., Nicolaides, A. N., Kakas, A. C., & Pattichis, C. S. (2021) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe52308.2021.9635199 · Cited by 8
“Explainability properties have been suggested in [23] based on the argument that “a good explanation should be understandable to humans (interpretability) and accurately describe model behavior in the entire feature space (fidelity)” [24].”
BackgroundPriyanshu, A., Vardhan, A., Sivakumar, S., Vijay, S., & Chhabra, N. (2021) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2021.wnut-1.48 · Cited by 6
“Thus, it becomes unclear whether the degradation in MAELOSD comes from the distribution shift or due to the poor performance of the explainable model (Zhou et al., 2021).”
BackgroundThe HEIC application framework for implementing XAI-based socio-technical systems
Paredes, J. N., Teze, J. C., Martínez, M. V., & Simari, G. I. (2022) · Online Social Networks and Media · doi:10.1016/j.osnem.2022.100239 · Cited by 8
Schrills, T., Kargl, S., Bickel, M., & Franke, T. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/3n95u · Cited by 6
Nasarian, E., Alizadehsani, R., Acharya, U. R., & Tsui, K. (2023) · Research Square · doi:10.21203/rs.3.rs-3626164/v2 · Cited by 7
Fighting the disagreement in Explainable Machine Learning with consensusPreprint
Banegas‐Luna, A. J., Martínez-Cortés, C., & Pérez‐Sánchez, H. (2024) · Research Square · doi:10.21203/rs.3.rs-4106083/v1 · Cited by 7
Dakhli, R., & Barhoumi, W. (2024) · 2024 IEEE/ACS 21st International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa63423.2024.10912611 · Cited by 5
“Fidelity measures the agreement between an XAI explanation and the actual model prediction [3].”
Haas, S., Hegestweiler, K., Rapp, M. T., Muschalik, M., & Hüllermeier, E. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1471208 · Cited by 6
A typology of explanations to support Explainability-by-Design
Tsakalakis, N., Stalla-Bourdillon, S., Huynh, T. D., & Moreau, L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3708504 · Cited by 6
“However, there is no consensus on the metrics for evaluating explanation methods, and it is hard to compare them [61].”
Musa, A. M., Abu-Shaikha, M. F., & Al-Abed, R. Y. (2025) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.842 · Cited by 5
“…of the real-time and dynamic modes are PSO-based dynamic optimization approaches incorporating an optimization of external and variable parameters to prevent premature convergence of the PSO and to enhance the degree of approximation of neural network models utilized in wind energy forecasts [17].”
Visagreement: Visualizing and Exploring Explanations (Dis)Agreement
Silva, P., Guardieiro, V., Barr, B., Silva, C., & Nonato, L. G. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3558074 · Cited by 8
eXING-IoT conceptual framework for explainability integration in next generation-IoT
Vultureanu‐Albişi, A., Bădică, C., & Ivanović, M. D. (2025) · Connection Science · doi:10.1080/09540091.2025.2507180 · Cited by 6
Liu, X., Rastegari, S., Huang, Y., Cheong, S. C., Liu, W., Zhao, W., Tian, Q., Wang, H., Guo, Y., Zhou, S., et al. (2025) · Methods · doi:10.1016/j.ymeth.2025.07.004 · Cited by 6
Tobing, F. A. T., Kusnadi, A., Pane, I. Z., & Winantyo, R. (2025) · Indonesian Journal of Electrical Engineering and Computer Science · doi:10.11591/ijeecs.v39.i2.pp1092-1099 · Cited by 5
Explainable AI for remaining useful life prediction in industrial systems: a survey
Qu, Q., Zhang, L., & Zheng, Y. (2025) · Measurement Science and Technology · doi:10.1088/1361-6501/ae319f · Cited by 8
“Existing research [85, 86] classifies evaluation approaches into three categories, each with distinct applicability in RUL prediction scenarios.”
Explainability Metrics and Properties for Counterfactual Explanation Methods
Singh, V., Čyras, K., & Inam, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-15565-9_10 · Cited by 6
On Explanations for Hybrid Artificial Intelligence
Nolle, L., Stahl, F. T., & El-Mihoub, T. A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47994-6_1 · Cited by 6
Towards a Comprehensive Evaluation of Decision Rules and Decision Mining Algorithms Beyond Accuracy
Wais, B., & Rinderle‐Ma, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61057-8_24 · Cited by 5
“Parsimony is often used to evaluate the understandability of an explanation, e.g., [37].”
Multivariate pattern analysis of medical imaging-based Alzheimer's disease
Alarjani, M., & Almarri, B. A. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1412592 · Cited by 5
“The trained model’s performance is measured using evaluation metrics, with each implementation having a different preprocessing and classifier training (79).”
MethodExplainable paper classification system using topic modeling and SHAP
Shin, N., Lee, Y., Moon, H.-S., Kim, J., & Jung, H. (2024) · Intelligent Data Analysis · doi:10.3233/ida-240075 · Cited by 9
“The fidelity measures the degree to which the explanations approximate the original predictions of the model [92].”
Achilonu, O., Obaido, G., Ogbuokiri, B., Aruleba, K., Musenge, E., & Fabian, J. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1427845 · Cited by 6
Mitra, P., Schwalbe, G., & Klein, N. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00358 · Cited by 8
“Besides efficiency, the two main application-agnostic safety targets for computer vision CNNs are correct uncertainty calibration and robustness against naturally occurring cor-ruptions [37, 46, 62].”
BackgroundTransparency Metrics for Artificial Intelligence-Driven Applications in Healthcare
Nicolae, I.-E., Danciu, G. M., Nanou, C., Koulierakis, N., & Danilatou, V. (2024) · Hellenic Conference on Artificial Intelligence · doi:10.1145/3688671.3688782 · Cited by 5
“…into AI model behaviour, facilitating user understanding of predictions, researchers have also developed a diverse range of XAI evaluation metrics 10 to assess AI systems in terms of system’s performance, reliability, and user satisfaction, quantifying the quality of predictions or decisions…”
Werner, F., Silva, M. A. D. A. E., Ferreira, A. S., Neumaier, N., & Balbinot, A. A. (2018) · Colloquium Agrariae · doi:10.5747/ca.2018.v14.n2.a220 · Cited by 5
Enhancing Security in WhatsApp: A System for Detecting Malicious and Inappropriate Content
Kazimierczak, M., Thanapattheerakul, T., & Chan, J. H. (2023) · International Symposium on Information and Communication Technology · doi:10.1145/3628797.3628966 · Cited by 5
“Metrics for model evaluation are taken from Zhou et al. and M and M.N [15, 24].”
MethodTraffic Classification in Software Defined Networks based on Machine Learning Algorithms
mahgoub, S., Ashour, M., Yakout, M., & AbdElhalim, E. (2024) · International Journal of Telecommunications · doi:10.21608/ijt.2024.340441 · Cited by 5
“Accuracy, F1 score, Recall, Precision, and Training time were the used performance metrics [27].”
MethodKobayashi, K., Takamizawa, Y., Miyake, M., Ito, S., Gu, L., Nakatsuka, T., Akagi, Y., Harada, T., Kanemitsu, Y., & Hamamoto, R. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102929 · Cited by 8
Fazio, R. D., Giovannantonio, R. D., Bellini, E., & Marrone, S. (2022) · Information · doi:10.3390/info14010021 · Cited by 6
Lambers, L., & Uchitel, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30826-0 · Cited by 7
Finding the input features that reduce the entropy of a neural network’s prediction
Amanova, N., Martin, J., & Elster, C. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05277-5 · Cited by 6
“Some papers suggest a re-training strategy to measure a model change when important features are removed [20, 21].”
Porębski, A. (2023) · Research Handbook on Law and Technology · doi:10.4337/9781803921327.00037 · Cited by 4
Evaluating patient experience in maternity services using a Bayesian belief network model
Munassar, A. A. A., Simsekler, M. C. E., Saad, A. A., Qazi, A. J., & Omar, M. (2025) · PLoS ONE · doi:10.1371/journal.pone.0318612 · Cited by 5
“To achieve this, a Bayesian Belief Network (BBN) was employed to model and explore the interactions between these factors, as BBNs have proven effective in decision-making across key areas such as clinical decision support [15].”
Jaitawat, D. P. S., Singh, I., & Chauhan, S. B. (2025) · Current Computer Science · doi:10.2174/0129503779365887250312072050 · Cited by 4
Understanding the Effects of Explaining Predictive but Unintuitive Features in Human-XAI Interaction
Qu, J., Arguello, J., & Wang, Y. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732021 · Cited by 7
eXplainable AI Interfaces With (and for) Expert Operators: A Participatory Design Approach
Hashmati, N., Wärnberg, H., Brorsson, E., & Obaid, M. (2024) · Australian Conference on Human-Computer Interaction (OzCHI) · doi:10.1145/3726986.3727020 · Cited by 7
“Previous research shows that there is a need for features that enable users to give feedback to AI systems [37, 66, 74], and there are some suggestions on how to implement such features [63, 65, 66].”
Islam, M. R., Weber, R. O., Ahmed, M. U., & Begum, S. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114599 · Cited by 4
Algorithmic Knowability: A Unified Approach to Explanations in the AI Act
Sapienza, S., & Palmirani, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08317-3_9 · Cited by 8
Yang, C., Sheng, H., Kolbinson, K. T., Shaterian, H., Ashe, P. C., Gao, P., Zhang, W., Quilichini, T. D., & Xiang, D. (2025) · Plant Methods · doi:10.1186/s13007-025-01451-z · Cited by 4
On the Use of Machine Learning for Modern IoT ELF Malware Detection
Souza, C. H. M., & Batista, D. M. (2025) · IEEE Latin American Conference on Computational Intelligence (LA-CCI) · doi:10.1109/la-cci66231.2025.11270436 · Cited by 4
“These metrics help determine whether a model adequately fits the data and meets the specific objectives of the task at hand [18].”
EvalCouncil: A Committee-Based LLM Framework for Reliable and Unbiased Automated Grading
Anghel, C., Crăciun, M., Anghel, A. A., Cocu, A., Balau, A. S., Andrei, C. A., Maier, C., Dragosloveanu, Ş., Nedelea, D.-G., & Scheau, C. (2025) · Computers · doi:10.3390/computers14120530 · Cited by 5
Pellano, K. N., Strümke, I., Groos, D., Adde, L., Haugen, P., & Ihlen, E. A. F. (2026) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2026.3654400 · Cited by 4
“However, while these XAI metrics provide valuable insights into the mathematical reliability of the explanations [19], [20], [21], [22], [23], [24], they do not reveal how the identified significant and non-significant features relate to the underlying biomechanical features (e.g. joint angle or…”
Attribution Explanations for Deep Neural Networks: A Theoretical Perspective
Deng, H., Pei, H., Zhang, Q., & Du, M. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3667600 · Cited by 6
“While existing surveys on attri-bution explanations primarily focus on methodological tax-onomies [61], [62], [63], [64], [65], [66] or empirical evaluation protocols [58], [64], [67], [68], [69], [70], theoretical advancements remain fragmented and lack systematic organization.”
Risk-Based AI Assurance Framework
Muhammad, A. E., & Yow, K. (2026) · Information · doi:10.3390/info17030263 · Cited by 6
Visual Explanations and Perturbation-Based Fidelity Metrics for Feature-Based Models
Mozolewski, M., Bobek, S., & Nalepa, G. J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-63772-8_27 · Cited by 4
Enhancing Aviation Efficiency Through Big Data and Machine Learning for Flight Delay Prediction
Vonitsanos, G., Gounaridis, I., Kanavos, A., & Phivos Mylonas, B. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-73344-4_45 · Cited by 4
An Improved Hybrid Deep Learning Approach for Security Requirements Classification
Hassan, S., Li, Q., Zubair, M., Alsowail, R. A., & Umair, M. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.059832 · Cited by 4
Evaluating example-based explainable artificial intelligence methods for breast cancer diagnosis
Xu, C., Zhu, P., Fan, Z., & Ma, Y. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113222 · Cited by 5
Hidden Conflicts in Neural Networks and their Implications for Explainability
Dejl, A., Zhang, D., Ayoobi, H., Williams, M., & Toni, F. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732100 · Cited by 6
“In particular, CAFE may not always capture hypothetical effects of positive or negative features that are masked by the non-monotonicity of the considered activation function.”
Explainable clustering: Methods, challenges, and future opportunities
Dewoprabowo, R., Stefanus, L. Y., & Saptawijaya, A. (2025) · Journal of Intelligent Systems · doi:10.1515/jisys-2024-0477 · Cited by 7
From black box to glass box: algorithmic explainability as a strategic decision
Lambin, X., & Raizonville, A. (2025) · Information Economics and Policy · doi:10.1016/j.infoecopol.2025.101149 · Cited by 4
Sovrano, F., & Vitali, F. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_2 · Cited by 6
On the Definition of Appropriate Trust and the Tools that Come with it
Löfström, H. (2023) · Congress in Computer Science, Computer Engineering, and Applied Computing (CSCE) · doi:10.1109/csce60160.2023.00256 · Cited by 4
Emotion Detection System for Malayalam Text using Deep Learning and Transformers
Anuja, K., Raj, P. C. R., & Babu, K. R. R. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3663475 · Cited by 4
“Using test samples, the trained models were tested, and the predictions are evaluated using evaluation parameters like Precision, Recall, F1 score and accuracy [27][36][53].”
MethodChallenges in designing and developing a virtual biology laboratory: A systematic literature review
Agustin, M., Zubaidah, S., Susanto, H., Habiddin, H., & Bilad, M. R. (2024) · Multidisciplinary Reviews · doi:10.31893/multirev.2025116 · Cited by 4
Varghese, J., & Selvan, P. T. (2022) · International Journal on Recent and Innovation Trends in Computing and Communication · doi:10.17762/ijritcc.v10i7.5595 · Cited by 4
Are Graph Neural Network Explainers Robust to Graph Noises?
Li, Y., Verma, S., Yang, S., Zhou, J. J., & Chen, F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-22695-3_12 · Cited by 4
Self-citationMathematics for Machine Learning
Tiwari, S. (2024) · International Conference for Innovation in Technology (INOCON) · doi:10.1109/inocon60754.2024.10511475 · Cited by 4
“Whenever compared to testing data, you could obtain an accurate prediction about how the algorithm will behave and also how quickly this will operate [7].”
Machine learning based predictive analysis of DNA cleavage induced by diverse nanomaterials
Niu, J., Wang, X., Chen, J., Zhao, Y., Chen, X., Yang, B., Liu, N., & Wu, P. (2024) · Scientific Reports · doi:10.1038/s41598-024-73140-1 · Cited by 4
“In order to reveal the performance of the models, evaluation metrics for the prediction capability and evaluation methods for the learning process were adopted 29 .”
MethodExplaining Deep Learning Representations by Tracing the Training ProcessPreprint
Pfahler, L., & Morik, K. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.05880 · Cited by 4
“Finally we should note that evaluating explanations is a non-trivial task on its own (Zhou et al. 2021).”
BackgroundA Trustworthy View on XAI Method EvaluationPreprint
LI, D., Liu, Y., Huang, J. G., & Wang, Z. (2022) · doi:10.36227/techrxiv.21067438 · Cited by 4
A Trustworthy View on XAI Method EvaluationPreprint
LI, D., Liu, Y., Huang, J., & Wang, Z. (2022) · doi:10.36227/techrxiv.21067438.v1 · Cited by 5
The FIX Benchmark: Extracting Features Interpretable to eXpertsPreprint
Jin, H., Havaldar, S., Kim, C., Xue, A., You, W., Qu, H., Gatti, M., Hashimoto, D. A., Jain, B., Madani, A., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.13684 · Cited by 4
“In general, however, there is a lack of interpretability benchmarks that evaluate feature interpretability in real-world settings — a gap we aim to address with the FIX benchmark.”
BackgroundAn Automated Approach to Subjective Answer Evaluation Using ML and NLP
Metan, J., Kumar, D., Dhanya, A. N., & Kumar, H. (2024) · doi:10.1109/icait61638.2024.10690635 · Cited by 3
“In this work of Andreas et al. [9] evaluates whether explanations can improve human decision-making in practical scenarios of machine learning model development.”
BackgroundEnhancing Android Ransomware Detection Using an Ensemble Machine Learning Classifier
Vali, N., Portillo‐Dominguez, A. O., & Ayala-Rivera, V. (2024) · Programming and Computer Software · doi:10.1134/s0361768824700622 · Cited by 3
“Regarding evaluation metrics, we have selected the following standard ML metrics [30]: (1) Recall score: This metric measures the number of false negatives (i.e., ransomware wrongly labeled as not ransomware).”
Shuaib, S. E., Riyapan, P., & Muangprathub, J. (2025) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2025.200569 · Cited by 3
E-LENS: user requirement–oriented AI ethics assurance
Zhou, J. J., & Chen, F. (2025) · Human-Intelligent Systems Integration · doi:10.1007/s42454-025-00068-w · Cited by 4
“Various approaches can be used to enhance the transparency of an AI model or algorithm (Zhou et al. 2021), assess algorithmic bias for fairness in predictions (Bellamy et al. 2018), or improve privacy through algorithms (Kairouz et al. 2021).”
Self-citationMenzner, T., & Leidner, J. L. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130115 · Cited by 3
Maeda, H., & Lu, Y. (2025) · Journal of Educational Measurement · doi:10.1111/jedm.70017 · Cited by 6
A hybrid YOLOv8-EigenCAM model for accurate and interpretable plant disease classification
Balaha, H. M. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21382-8 · Cited by 4
Renewable energy management using explainable artificial intelligence
Albahdal, D., Ibrahim, D. M., Moulahi, T., & Omri, A. (2026) · Frontiers in Energy Research · doi:10.3389/fenrg.2026.1753200 · Cited by 4
Modification of Machine Learning Algorithms for Embedding in Image Compression Methods
Yuzkiv, R., & Gashnikov, M. V. (2022) · International Conference on Information Technology and Nanotechnology (ITNT) · doi:10.1109/itnt55410.2022.9848584 · Cited by 3
“The need for modification follows from the fact that when tuning, most machine learning methods use criteria that are not relevant for digital image compression [12, 13].”
BackgroundMethodRojas-López, A. G., Rodríguez-Molina, A., Uriarte-Arcia, A. V., & Villarreal-Cervantes, M. G. (2024) · Healthcare · doi:10.3390/healthcare12131324 · Cited by 3
“The tuning process employs accuracy as the scoring metric for these tests, which is a well-recognized and dependable metric in pattern recognition techniques [78,79].”
Behavior Matters: An Alternative Perspective on Promoting Responsible Data Science
Zhang, D., Patil, A., Shoda, Y., Battle, L., & Wall, E. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710932 · Cited by 7
“It is crucial to align the choice of evaluation metrics with the project’s specific objectives [82].”
XAI-Units: Benchmarking Explainability Methods with Unit Tests
Lee, J. R., Emami, S., Hollins, M. D., Wong, T. C. H., Sánchez, C. I. V., Toni, F., Zhang, D., & Dejl, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732186 · Cited by 5
“In response, a variety of metrics have been proposed for evaluating FA methods [30, 48].”
A Review of Methodologies for Photovoltaic Energy Generation Forecasting in the Building Sector
Pedram, O., Soares, A., & Moura, P. (2025) · Energies · doi:10.3390/en18185007 · Cited by 3
A taxonomy of explanations to support Explainability-by-DesignPreprint
Tsakalakis, N., Stalla-Bourdillon, S., Huynh, T. D., & Moreau, L. (2022) · ePrints Soton (University of Southampton) · doi:10.48550/arxiv.2206.04438 · Cited by 6
Metadata
- Authors
- Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A.
- Affiliations
- University of Technology Sydney (AU)
- Medical University of Graz (AT)
- Corresponding author
- Zhou, Jianlong
- Gandomi, Amir H.
- Venue
- Electronics
- Publisher
- MDPI AG
- Published
- 2021-03-04
- Volume
- 10
- Issue
- 5
- Pages
- 593
- ISSN
- 2079-9292 (ISSN-L)
- OpenAlex
- W3133543405
- Cited by
- 606
- References
- 80
- Citations by year
- 2026: 75 · 2025: 138 · 2024: 179 · 2023: 109 · 2022: 84 · 2021: 20
- FWCI
- 34.65
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Electronics (publishedVersion, cc-by)
- OPUS - Open Publications of UTS Scholars (University of Technology Sydney) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- University of Technology Sydney
- Austrian Science Fund (P 32554)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- explainable machine learning, explanation quality evaluation, evaluation metrics, explainability, model-based explanation, example-based explanations, interpretability, fidelity, human-centered evaluation, black-box models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.