A Literature Review on Applications of Explainable Artificial Intelligence (XAI)
Open accessThe DOI matches its Crossref record; 110 of the 110 references it cites were found in a registry or an index.
Abstract
As AI technologies, particularly deep learning models, have advanced, their inherent “black box” nature has raised significant concerns regarding accountability, fairness, and trust, especially in critical domains such as healthcare, finance, and criminal justice. We present a detailed exploration of XAI, emphasizing its essential role in improving the interpretability and transparency of complex AI systems in various application domains. Health-related applications were notably using XAI, emphasizing diagnostics, and medical imaging. Other notable domains of use of XAI is encompassed environmental and agricultural management, industrial optimization, cybersecurity, finance, transportation, and social media. Furthermore, nascent applications in law, education, and social care underscore the growing influence of XAI. The analysis indicates a prevalent application of local explanation techniques, especially SHAP and LIME, with a preference for SHAP due to its stability and mathematical assurances. Each technique is analysed for its strengths and limitations in providing clear, actionable insights into model decision-making processes, thereby aiding stakeholders in understanding AI behaviour. Ultimately, this document underscores the critical challenges for XAI in fostering user trust, enhancing decision-making processes, and ensuring that AI technologies are utilized responsibly and ethically across various applications, paving the way for a more transparent and accountable AI landscape. We believe that by serving as a guide for future studies in the area, our systematic review contributes to the body of literature on XAI.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“XAI provides tools such as SHAP, LIME, and attention mechanisms that can explain how models reach their decisions, improving both transparency and usability (Kalasampath et al., 2025).”
From Explainable AI: enhancing decision-making in the detection of cyber threats (2026) · cites this paper as Kalasampath et al., 2025
In the citing paper
“Kalasampath et al. (2025) emphasizes that this disconnect not only reduces the immediate value of XAI solutions but also undermines trust in AI more broadly, since practitioners may become sceptical of adopting systems that are perceived as impractical or irrelevant.”
From Explainable AI: enhancing decision-making in the detection of cyber threats (2026) · cites this paper as Kalasampath et al. (2025)
Citations
Cites
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“There are cases in which local justifications could be deceptive if the overall model performance varies greatly from particular cases, underlining the importance of a well-rounded strategy [37].”
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 1×
“Continuing research in XAI is focused on explaining AI capabilities, encouraging ethical and responsible use of technology, and empowering a knowledgeable society to interact with advancing AI systems [9], [10], [11].”
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 1×
“The emergence and increasing popularity of XAI tackle the issues of explainability, understandability, and interpretability linked to intricate black-box AI systems [15].”
Explanation in Artificial Intelligence: Insights from the Social Sciences
Miller, T. (2017) · arXiv · doi:10.48550/arxiv.1706.07269 · Cited by 5,338
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Kaplan, A., & Haenlein, M. (2019) · Business Horizons · doi:10.1016/j.bushor.2018.08.004 · Cited by 3,140 · cited 1×
“We define the term AI as a system’s capability to correctly interpret, to learn from external data provided, and to use that knowledge to meet particular goals by adjusting as necessary [1].”
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 1×
“A detailed analysis in 2020 highlights the importance of comprehending the decision-making processes of AI systems as they are used more in important fields, for accountability and ethical reasons [17].”
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 2×
“Methods like LIME and SHAP are categorized in this group because they produce explanations that emphasize the importance of features in individual predictions [27].”
Definitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“Stakeholders can improve their ability to evaluate the reliability of these systems and make informed decisions about their critical application by understanding how they reach their conclusions [14].”
The false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724 · cited 2×
“XAI techniques like SHAP and LIME clarify feature relevance and model transparency, building clinician trust by justifying the previously doubtful ‘black-box’ calculations [115].”
Samek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080 · cited 1×
“This enables users to comprehend how specific predictions align with the overall functioning of the model [38].”
Explainable Artificial Intelligence in education
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100074 · Cited by 863
Explainable artificial intelligence: an analytical review
Angelov, P. P., Soares, E. A., Jiang, R., Arnold, N. I., & Atkinson, P. M. (2021) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1424 · Cited by 843 · cited 1×
“Adapted from [118]. decisions are made, highlighting the significance of these ideas in the responsible deployment of AI [32].”
Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities
Saeed, W., & Omlin, C. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110273 · Cited by 783
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 1×
“As models develop, their reasoning strategies may change, leading to a possible decline in the significance of the methods used to explain them [102].”
Explainable Deep Learning Models in Medical Image Analysis
Singh, A., Sengupta, S., & Lakshminarayanan, V. (2020) · Journal of Imaging · doi:10.3390/jimaging6060052 · Cited by 718 · cited 1×
“Research was conducted on how saliency maps can enhance the interpretability of convolutional neural networks (CNNs) by visually displaying feature importance, leading to improved comprehension in fields such as medical image analysis [56].”
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711 · cited 1×
“As models develop, their reasoning strategies may change, leading to a possible decline in the significance of the methods used to explain them [102].”
A General Survey on Attention Mechanisms in Deep Learning
Brauwers, G., & Frasincar, F. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3126456 · Cited by 695 · cited 1×
“Both LIME and SHAP can be used to improve transparency in medical diagnosis systems, promoting trust and accountability in AI-based healthcare solutions [44].”
Interpretable Decision Sets: A Joint Framework for Description and Prediction
Lakkaraju, H., Bach, S. H., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939874 · Cited by 687
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · cited 1×
“Continuing research in XAI is focused on explaining AI capabilities, encouraging ethical and responsible use of technology, and empowering a knowledgeable society to interact with advancing AI systems [9], [10], [11].”
Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629 · cited 2×
“The choice of XAI methods should consider clinical relevance, data complexity, and computational feasibility to ensure effective, trustworthy AI integration [79], [83].”
Explainable Deep Learning: A Field Guide for the UninitiatedPreprint
Ras, G., Xie, N., van Gerven, M. V., & Doran, D. (2020) · Journal of Artificial Intelligence Research · doi:10.48550/arxiv.2004.14545 · Cited by 622 · cited 1×
“Research shows that local explanations are crucial for applications like criminal justice or medical diagnostics as they can improve user trust by offering context-specific insights, thus enhancing accountability [35].”
A Review of Explainable Artificial Intelligence in Healthcare
Sadeghi, Z., Alizadehsani, R., CIFCI, M. A., Kausar, S., Rehman, R., Mahanta, P., Bora, P. K., Almasri, A., Alkhawaldeh, R. S., Hussain, S., et al. (2024) · Computers and Electrical Engineering · doi:10.1016/j.compeleceng.2024.109370 · Cited by 598 · cited 4×
“Counterfactual explanations, providing ‘‘what-if’’ scenarios, support causal analysis in clinical settings but are challenging to implement with complex clinical data [77], [83].”
InfluentialSurvey of Explainable AI Techniques in Healthcare
Chaddad, A., Peng, J., Xu, J., & Bouridane, A. (2023) · Sensors · doi:10.3390/s23020634 · Cited by 591
On Interpretability of Artificial Neural Networks: A Survey
Fan, F.-L., Xiong, J., Li, M., & Wang, G. (2021) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2021.3066428 · Cited by 561 · cited 1×
“Interpretability, conversely, pertains to how well a person can understand the significance of a model’s predictions or choices [26].”
Machlev, R., Heistrene, L., Perl, M., Levy, K. Y., Belikov, J., Mannor, S., & Levron, Y. (2022) · Energy and AI · doi:10.1016/j.egyai.2022.100169 · Cited by 552
DARPA's explainable artificial intelligence (XAI) program
Gunning, D. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3308446 · Cited by 546
Explainable AI for Healthcare 5.0: Opportunities and Challenges
Saraswat, D., Bhattacharya, P., Verma, A., Prasad, V. K., Tanwar, S., Sharma, G., Bokoro, P. N., & Sharma, R. (2022) · IEEE Access · doi:10.1109/access.2022.3197671 · Cited by 523 · cited 2×
“LIME, on the other hand, offers flexible, model-agnostic insights suitable for general use but can be inconsistent with high-dimensional data [75], [80], [84].”
Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research
Zhang, Z., Hamadi, H. A., Damiani, E., Yeun, C. Y., & Taher, F. (2022) · IEEE Access · doi:10.1109/access.2022.3204051 · Cited by 450 · cited 1×
“Zhang et al. [88] endorse SHAP for malware detection, revealing feature importance to clarify model decisions.”
Explainable Deep Learning: A Field Guide for the Uninitiated
Ras, G., Xie, N., Van Gerven, M., & Doran, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13200 · Cited by 446
A., S., & R., S. (2023) · Decision Analytics Journal · doi:10.1016/j.dajour.2023.100230 · Cited by 432 · cited 2×
“Adapted from [116]. entertainment, and education.”
Ali, S., Akhlaq, F., Imran, A. S., Kastrati, Z., Daudpota, S. M., & Moosa, M. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107555 · Cited by 338 · cited 1×
“Counterfactual explanations, providing ‘‘what-if’’ scenarios, support causal analysis in clinical settings but are challenging to implement with complex clinical data [77], [83].”
Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare
Hulsen, T. (2023) · AI · doi:10.3390/ai4030034 · Cited by 305 · cited 2×
“As mentioned in [82], XAI enhances trust and usability in crop recommendation systems, as it provides farmers with the ability to interpret the model’s predictions based on specific agricultural needs.”
Weber, P., Carl, K. V., & Hinz, O. (2024) · Management Review Quarterly · doi:10.1007/s11301-023-00320-0 · Cited by 292 · cited 2×
“Weber, Carl, and Hinz, [84] offer the readers a comprehensive overview and detailed review of several existing techniques, including SHAP and LIME, all the while highlighting how each could be applied in specific portfolios and credit risk assessments.”
A survey on interpretable reinforcement learning
Glanois, C., Weng, P., Zimmer, M., Li, D., Yang, T., Hao, J., & Liu, W. (2024) · Machine Learning · doi:10.1007/s10994-024-06543-w · Cited by 245 · cited 1×
“A recent study (2021) proposes that enhancing human-AI collaboration could be improved by incorporating explainability into current AI systems [23].”
Survey on Explainable AI: From Approaches, Limitations and Applications Aspects
Yang, W., Wei, Y., Wei, H., Chen, Y., Huang, G., Li, X., Li, R., Yao, N., Wang, X., Gu, X., et al. (2023) · Human-Centric Intelligent Systems · doi:10.1007/s44230-023-00038-y · Cited by 240
Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
Kim, S. S. Y., Watkins, E. A., Russakovsky, O., Fong, R., & Monroy-Hernández, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581001 · Cited by 233
Shams, M. Y., Gamel, S. A., & Talaat, F. M. (2024) · Neural Computing and Applications · doi:10.1007/s00521-023-09391-2 · Cited by 224 · cited 1×
“This adds complexity to creating universal XAI tools, as they must be customized to meet different industry standards and user needs [98], [99].”
A Comparative Analysis of LIME and SHAP Interpreters With Explainable ML-Based Diabetes Predictions
Ahmed, S., Kaiser, M. S., Hossain, M. S., & Andersson, K. (2024) · IEEE Access · doi:10.1109/access.2024.3422319 · Cited by 221 · cited 1×
“LIME, on the other hand, offers flexible, model-agnostic insights suitable for general use but can be inconsistent with high-dimensional data [75], [80], [84].”
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 220 · cited 1×
“The data was obtained using queries that include keywords such as ‘Explainable AI,’ ’XAI,’ and related terms through Google Scholar’s advanced search [111], [120].”
Explainable AI in Fintech Risk Management
Bussmann, N., Giudici, P., Marinelli, D., & Papenbrock, J. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00026 · Cited by 214 · cited 1×
“Finally, Bussmann et al. [87] have presented SHAP-based models pertaining to the concept of lending platforms based on peer-to-peer wherein the Shapley values were used via XGBoost, rendering it highly valuable for in-depth insight generation into the creditworthiness of a borrower.”
Explainable AI under contract and tort law: legal incentives and technical challenges
Hacker, P., Krestel, R., Grundmann, S., & Naumann, F. (2020) · Artificial Intelligence and Law · doi:10.1007/s10506-020-09260-6 · Cited by 186 · cited 2×
“This creates a challenge in making the model simpler for increased clarity might result in lower effectiveness, especially in domains where precision is crucial [92], [93].”
Efficient and Interpretable Deep Blind Image Deblurring Via Algorithm Unrolling
Li, Y., Tofighi, M., Geng, J., Monga, V., & Eldar, Y. C. (2020) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2020.2964202 · Cited by 185 · cited 1×
“Interpretable models are crucial for implementing AI systems, stating that transparency can result in higher user trust and improved decision-making results [52].”
Stacked LSTM based deep recurrent neural network with kalman smoothing for blood glucose prediction
Rabby, M. F., Tu, Y., Hossen, M. I., Lee, I., Maida, A. S., & Hei, X. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01462-5 · Cited by 176 · cited 1×
“[2], [3], [4], [5], [6], [7], [8].”
Awosika, T., Shukla, R. M., & Pranggono, B. (2024) · IEEE Access · doi:10.1109/access.2024.3394528 · Cited by 175 · cited 1×
“Awosika et al. [85] present an FL and XAI framework for financial institutions to collectively detect fraudulent activities while respecting individual information privacy and enabling explainable predictive results.”
Enhanced Boundary Learning for Glass-like Object Segmentation
He, H., Li, X., Cheng, G., Shi, J., Tong, Y., Meng, G., Prinet, V., & Weng, L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01556 · Cited by 158 · cited 1×
“Nevertheless, the writers also point out that decision trees may experience overfitting if they become too deep, resulting in intricate models that could compromise their interpretability [46].”
Explainable AI and Law: An Evidential Survey
Richmond, K. M., Muddamsetty, S. M., Gammeltoft‐Hansen, T., Olsen, H. P., & Moeslund, T. B. (2023) · Digital Society · doi:10.1007/s44206-023-00081-z · Cited by 157 · cited 2×
“Similarly, Richmond et al. [93] highlight that XAI can facilitate the interpretation of AI-generated evidence, ensuring legal practitioners can scrutinize the reliability and relevance of AI-driven insights.”
Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems
Cobbe, J., Lee, M. S. A., & Singh, J. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445921 · Cited by 150 · cited 1×
“By comprehending the capabilities and restrictions of each kind of explanation, developers, and stakeholders can develop AI systems that are more efficient and reliable, thus advancing ethical and responsible AI practices [114].”
Sivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581075 · Cited by 141 · cited 1×
“They highlight the importance of establishing standardized metrics to evaluate the effectiveness of XAI models and fostering multidisciplinary collaboration [110], [119].”
Bommer, P. L., Kretschmer, M., Hedström, A., Bareeva, D., & Höhne, M. (2024) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-23-0074.1 · Cited by 139
Education in the age of Generative AI: Context and Recent DevelopmentsPreprint
Mello, R. F., Freitas, E., Pereira, F. D., Cabral, L., Tedesco, P. C. D. A. R., & Ramalho, G. L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.12332 · Cited by 138 · cited 1×
“Attaining this balance is crucial for the effectiveness [104], [105].”
Vale, D., El-Sharif, A., & Ali, M. (2022) · AI and Ethics · doi:10.1007/s43681-022-00142-y · Cited by 124 · cited 2×
“Models that are simpler to understand might be less intricate, but they often fail to deliver the same degree of performance [94].”
The Generative AI Paradox: "What It Can Create, It May Not Understand"
West, P., Lu, X., Dziri, N., Brahman, F., Li, L., Hwang, J. D., Jiang, L., Fisher, J., Ravichander, A., Chandu, K. R., et al. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2311.00059 · Cited by 122 · cited 1×
“A study has observed that enhancing the clarity of a model may occasionally reduce its performance or increase its complexity, prompting thoughtful deliberation in the design of AI systems [31].”
XAI for intrusion detection system: comparing explanations based on global and local scope
Hariharan, S., Robinson, R. R. R., Prasad, R. R., Thomas, C., & Balakrishnan, N. (2022) · Journal of Computer Virology and Hacking Techniques · doi:10.1007/s11416-022-00441-2 · Cited by 116 · cited 1×
“Permutation Importance (PI) was found to be useful in intrusion detection by Hariharan et al. [89], providing global explainability through the ranking of significant features.”
Social Media Hate Speech Detection Using Explainable Artificial Intelligence (XAI)
Mehta, H., & Passi, K. (2022) · Algorithms · doi:10.3390/a15080291 · Cited by 106 · cited 1×
“Some studies especially employed the LSTM and BERT for hate speech classification but with Local Interpretable Model-Agnostic Explanations to make the decision of the AI clearer since this is something important to handle in working with sensitive content [70].”
Anik, A. I., & Bunt, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445736 · Cited by 103 · cited 1×
“[2], [3], [4], [5], [6], [7], [8].”
The Promise of Explainable AI in Digital Health for Precision Medicine: A Systematic Review
Allen, B. (2024) · Journal of Personalized Medicine · doi:10.3390/jpm14030277 · Cited by 100 · cited 1×
“Additionally, interpretable models like Decision Trees and Bayesian Networks offer transparent decision paths, beneficial for precision medicine applications, yet they often lack scalability in larger datasets [81].”
Learning to Explain: An Information-Theoretic Perspective on Model Interpretation
Chen, J., Song, L., Wainwright, M. J., & Jordan, M. I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.07814 · Cited by 99 · cited 1×
“This need for openness mirrors the public’s desire for confidence in AI systems, particularly as they impact important choices in life [18].”
Technology‐based service proposal screening and decision‐making effectiveness
van Riel, A. C. R., Semeijn, J., Hammedi, W., & Henseler, J. (2011) · Management Decision · doi:10.1108/00251741111130841 · Cited by 92 · cited 1×
“Moreover, new technologies like interpretable reinforcement learning suggest a possibility of increasing transparency in even the most intricate decision-making processes [24].”
Analyzing and assessing explainable AI models for smart agriculture environments
Cartolano, A., Cuzzocrea, A., & Pilato, G. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17978-z · Cited by 92 · cited 1×
“The goal is to develop scalable methods that maintain high effectiveness while providing clear and timely explanations, even as models grow in complexity and size [97].”
Gongane, V. U., Munot, M. V., & Anuse, A. D. (2024) · Journal of Computational Social Science · doi:10.1007/s42001-024-00248-9 · Cited by 91 · cited 1×
“…role of XAI in transparency and interpretability for detecting misinformation and hate speech on social media is as follows: A survey of XAI models categorized the techniques for content moderation while emphasizing transparency as necessary to build user trust in sensitive AI applications [69].”
Agriculture 5.0 and Explainable AI for Smart Agriculture: A Scoping Review
Razak, A., Yogarayan, S., Sayeed, M. S., & Derafi, M. I. F. M. (2024) · Emerging Science Journal · doi:10.28991/esj-2024-08-02-024 · Cited by 91 · cited 1×
“For instance, a physician might struggle to grasp how a model arrives at its conclusions if it relies on intricate mathematical principles [100].”
Jia, R., Wu, F., Sun, X., Xu, J., Dao, D., Kailkhura, B., Zhang, C., Li, B., & Song, D. X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00814 · Cited by 90 · cited 1×
“A study reveals that LIME is efficient in multiple scenarios, proving its ability to offer a valuable understanding of the functioning of intricate models like deep neural networks and ensemble methods [40].”
Jung, J., Lee, H., Jung, H., & Kim, H. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e16110 · Cited by 87
A Survey on the Robustness of Feature Importance and Counterfactual ExplanationsPreprint
Mishra, S., Dutta, S., Long, J., & Magazzeni, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.00358 · Cited by 79 · cited 1×
“These techniques demonstrate how each characteristic contributes to the model’s determination for a certain input [34].”
Explainable Misinformation Detection Across Multiple Social Media Platforms
Joshi, G., Srivastava, A., Yagnik, B., Hasan, M. M., Saiyed, Z., Gabralla, L. A., Abraham, A., Walambe, R. A., & Kotecha, K. V. (2023) · IEEE Access · doi:10.1109/access.2023.3251892 · Cited by 74 · cited 1×
“Another DANN-based is used in developing cross-platform misinfor-mation detection systems integrating the LIME to increase explainability among different contexts found in social media systems [71].”
Explainable artificial intelligence in emergency medicine: an overview
Okada, Y., Ning, Y., & Ong, M. E. H. (2023) · Clinical and Experimental Emergency Medicine · doi:10.15441/ceem.23.145 · Cited by 72 · cited 1×
“They highlight the importance of establishing standardized metrics to evaluate the effectiveness of XAI models and fostering multidisciplinary collaboration [110], [119].”
Kisten, M., Ezugwu, A. E.-S., & Olusanya, M. O. (2024) · IEEE Access · doi:10.1109/access.2024.3365586 · Cited by 70 · cited 1×
“This adds complexity to creating universal XAI tools, as they must be customized to meet different industry standards and user needs [98], [99].”
Explainable AI In Education : Current Trends, Challenges, And Opportunities
Rachha, A., & Seyam, M. (2023) · SoutheastCon 2023 · doi:10.1109/southeastcon51012.2023.10115140 · Cited by 68 · cited 1×
“Likewise, another research study discusses the growing trend of XAI in LMS and EDM while underscoring explain-ability demands for the learning and teaching parties as well [66].”
Li, H., Vulova, S., Rocha, A. D., & Kleinschmit, B. (2024) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2024.170330 · Cited by 66 · cited 1×
“Another research [64] uses XAI to reproduce European wildfire events, and SHAP identifies the environmental factors, such as land surface temperature and solar radiation, which determine the likelihood of wildfires, and such factors vary by region in Europe.”
Representation Engineering: A Top-Down Approach to AI Transparency
Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, X., Mazeika, M., Dombrowski, A.-K., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01405 · Cited by 65 · cited 1×
“These models can be taught to maintain the simplicity of simpler models, yet still reach the efficiency of conventional deep neural networks [51].”
Explainable AI for Soil Fertility Prediction
Chandra, H., Pawar, P. M., Elakkiya, R., Tamizharasan, P. S., Muthalagu, R., & Panthakkan, A. (2023) · IEEE Access · doi:10.1109/access.2023.3311827 · Cited by 54 · cited 1×
“Restrictions apply. them to react favorably to the insights presented by the AI system [101].”
A survey of AI ethics in business literature: Maps and trends between 2000 and 2021
Daza, M. T., & Ilozumba, U. J. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1042661 · Cited by 47 · cited 1×
“New research conducted in late 2020 shows how these methods can clarify intricate models, leading to increased trust from users [19].”
Dimensionality Reduction based on SHAP Analysis: A Simple and Trustworthy Approach
Kumar, C. S., Choudary, M. N. S., Bommineni, V. B., Tarun, G., & Anjali, T. (2020) · doi:10.1109/iccsp48568.2020.9182109 · Cited by 44 · cited 1×
“A study conducted highlights that SHAP offers interpretability on both a local and global scale by combining insights from individual features to comprehend the overall behaviour of a model [41].”
Cybersecurity Knowledge Extraction Using XAI
Šarčević, A., Pintar, D., Vranić, M., & Krajna, A. (2022) · Applied Sciences · doi:10.3390/app12178669 · Cited by 41 · cited 1×
“Šarčević et al. [90] applied if-then decision tree rules in network security, providing rule-based classifications for complex datasets.”
Local Interpretable Model-agnostic Explanations (LIME)
Biecek, P., & Burzykowski, T. (2021) · Explanatory Model Analysis · doi:10.1201/9780429027192-11 · Cited by 38 · cited 1×
“Adapted from [118]. decisions are made, highlighting the significance of these ideas in the responsible deployment of AI [32].”
NCART: Neural Classification and Regression Tree for tabular data
Luo, J., & Xu, S. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110578 · Cited by 37
Gaba, A., Kaufman, Z., Cheung, J., Shvakel, M., Hall, K. W., Brun, Y., & Xiong, C. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3327192 · Cited by 34 · cited 1×
“The authors stress that successful visualizations can narrow the divide between intricate model results and user understanding, ultimately building trust in AI systems [54].”
Gong, Y., Shang, L., & Wang, D. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3404236 · Cited by 34 · cited 1×
“In this, the approach brings forth an understanding of the higher social influences that may find a role in spreading the information and enhances the reliability of AI decisions about its users [72].”
McGrath, S., Mehta, P., Zytek, A., Lage, I., & Lakkaraju, H. (2023) · Transactions on machine learning research · doi:10.48550/arxiv.2011.06167 · Cited by 33 · cited 1×
“Developing flexible and adaptable explanation techniques that can keep up with these alterations is essential for ensuring the clarity and dependability of AI systems over their entire lifespan [103].”
Validation Methods to Promote Real-world Applicability of Machine Learning in Medicine
Rafiq, R. B., Modave, F. P., Guha, S., & Albert, M. V. (2020) · International Conference on Digital Medicine and Image Processing (DMIP) · doi:10.1145/3441369.3441372 · Cited by 31 · cited 1×
“On the other hand, even though SHAP offers reliable and theoretically grounded explanations for important decision-making processes, its computational demands may be significant, especially for models with a large number of features [42].”
On the Art and Science of Machine Learning Explanations
Hall, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02909 · Cited by 30 · cited 1×
“Methods like feature importance plots, partial dependence plots, and decision boundary visualizations help users understand the connection between input features and output predictions efficiently [53].”
A study of data and label shift in the LIME frameworkPreprint
Rahnama, A. H. A., & Boström, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.14421 · Cited by 27 · cited 1×
“It suggests modifications to SHAP that utilize sampling methods to decrease computational burden without compromising accuracy, enabling the use of SHAP in extensive applications [43].”
"Explanation" is Not a Technical Term: The Problem of Ambiguity in XAIPreprint
Gilpin, L. H., Paley, A., Alam, M. A., Spurlock, S., & Hammond, K. J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.00007 · Cited by 27 · cited 1×
“Recent research conducted in 2023 highlights that even if a model is transparent, it may not be interpretable if it learns relationships that are too intricate for humans to understand [28], [113].”
Individual Explanations in Machine Learning Models: A Survey for PractitionersPreprint
Carrillo, A., Cantú, L. F., & Noriega-Campero, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.04144 · Cited by 26 · cited 1×
“The latest studies highlight the importance of global explanations in grasping model behaviour as a whole, especially in fields such as finance and healthcare where recognizing systemic biases is crucial [33].”
Challenges and Solutions in AI for AllPreprint
Shams, R. A., Zowghi, D., & Bano, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.10600 · Cited by 23 · cited 1×
“The success of an explanation relies on elements such as its clarity, significance, and the user’s comprehension [106].”
Hoque, M. T., Islam, A., Ahmed, E., Mamun, K. A., & Huda, M. N. (2019) · International Conference on Electrical, Computer and Communication Engineering · doi:10.1109/ecace.2019.8679272 · Cited by 21 · cited 1×
“[2], [3], [4], [5], [6], [7], [8].”
Robust Transparency Against Model Inversion Attacks
Alufaisan, Y., Kantarcıoğlu, M., & Zhou, Y. (2020) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2020.3019508 · Cited by 19 · cited 1×
“Recent research conducted in 2023 highlights that even if a model is transparent, it may not be interpretable if it learns relationships that are too intricate for humans to understand [28], [113].”
Visualization for Trust in Machine Learning Revisited: The State of the Field in 2023
Chatzimparmpas, A., Kucher, K., & Kerren, A. (2024) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2024.3360881 · Cited by 19 · cited 1×
“A common approach for visual interpretations involves utilizing saliency maps, especially when utilizing deep learning models on image data [55].”
Safe AI for CPS (Invited Paper)
Fulton, N., & Platzer, A. (2018) · IEEE International Test Conference (ITC) · doi:10.1109/test.2018.8624774 · Cited by 18
Explainable AI to understand study interest of engineering students
Ghosh, S., Kamal, M. S., Chowdhury, L., Neogi, B., Dey, N., & Sherratt, R. S. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11943-x · Cited by 18 · cited 1×
“Other XAI methods, such as Belief Rule Base (BRB) and SP-LIME, have been applied in determining factors that affect the study interests of engineering students, with PCA clustering of students based on interest levels, which helps in the design of curricula [68].”
Abdullah, Ashraf, F. B., & Momo, N. S. (2019) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt45670.2019.8944850 · Cited by 15 · cited 1×
“[2], [3], [4], [5], [6], [7], [8].”
Does Dataset Complexity Matters for Model Explainers?
Ribeiro, J., da Silva, R. L. S., Cardoso, L., & Alves, R. C. D. O. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671630 · Cited by 15 · cited 1×
“Although global explanations give a general understanding of model behaviour, they may lack the detailed insights necessary for precise predictions [36].”
Improving Stability in Decision Tree ModelsPreprint
Bertsimas, D., & Digalakis, V. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.17299 · Cited by 15 · cited 1×
“A study conducted by Van der Waa and colleagues in 2023 emphasizes the importance of transparency in establishing trust among users, especially in critical fields such as healthcare and finance, where comprehending decision-making processes is crucial [25].”
Deep Learning in IoT-Based Healthcare Applications
Zobaed, S., Hassan, M., Islam, M. U., & Haque, M. E. (2021) · CRC Press eBooks · doi:10.1201/9781003032175-9 · Cited by 13
Seven challenges for harmonizing explainability requirementsPreprint
Chen, J., & Storchan, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.05390 · Cited by 13
Small, E. A., Xuan, Y., Hettiachchi, D., & Sokol, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.00934 · Cited by 12 · cited 1×
“Creating dependable, context-oriented metrics is crucial for enhancing and assessing XAI techniques, guaranteeing they address the requirements of both developers and users [107], [109] VIII.”
Lee, M. H., Ng, R. B. X., Choo, S., & Thilarajah, S. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia241044 · Cited by 9
Explaining Neural Networks Using Attentive Knowledge Distillation
Lee, H., & Kim, S. (2021) · Sensors · doi:10.3390/s21041280 · Cited by 7 · cited 1×
“One instance is the utilization of attention mechanisms in neural networks, which enables models to emphasize important characteristics during prediction [48].”
Theoretical and Empirical Advances in Forest PruningPreprint
Dorador, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.05535 · Cited by 7 · cited 1×
“Decision trees are interpretable because of their visual layout, with every node symbolizing a feature and each branch symbolizing a decision rule [45].”
Survey of Information Visualization Techniques for Enhancing Visual Analysis
Fernandez, R., & Fetais, N. (2017) · International Conference on Computer and Applications (ICCA) · doi:10.1109/comapp.2017.8079755 · Cited by 6 · cited 1×
“It was discovered that interactive visualizations can boost model comprehension by allowing users to assess how different inputs affect outputs, leading to increased trust and understanding of the models [57].”
Chen, H., & Mason, C. (2024) · IEEE Access · doi:10.1109/access.2024.3430317 · Cited by 6
Towards glass-box CNNsPreprint
Piduguralla, M., & Bhatt, J. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.10443 · Cited by 6 · cited 1×
“These models have a structure that increases the transparency of their decision-making processes [50].”
Host trait prediction from human microbiome data for Colorectal Cancer
Ashraf, F. B., Shafi, M. S. R., & Kabir, M. R. (2020) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit51783.2020.9392731 · Cited by 5 · cited 1×
“[2], [3], [4], [5], [6], [7], [8].”
DiabeDetect: A Novel Decision Tree-Based Approach for Early Prognosis of Diabetes
Islam, M. U., Hossain, M. M., Hossain, I., & Kashem, M. A. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-981-16-2597-8_48 · Cited by 5 · cited 1×
“[2], [3], [4], [5], [6], [7], [8].”
Computing SHAP Efficiently Using Model Structure InformationPreprint
Hu, L., & Wang, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.02417 · Cited by 5 · cited 1×
“On the other hand, industries such as finance could gain advantages from worldwide explanations that pin-point systemic risks or biases across extensive datasets [39].”
A Survey on XAI for 5G and Beyond Security: Technical Aspects, Challenges and Research Directions
Senevirathna, T., La, V. H., Marchal, S., Siniarski, B., Liyanage, M., & Wang, S. (2022) · arXiv · doi:10.48550/arxiv.2204.12822 · cited 1×
“Creating dependable, context-oriented metrics is crucial for enhancing and assessing XAI techniques, guaranteeing they address the requirements of both developers and users [107], [109] VIII.”
Research works printed but not linked: 13
- Dhanorkar, Tutorial on explainability for natural language processing, 447
- Arrieta, 2020, Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and threats, Inf. Sci., 570, 100
- Doshi-Velez, Towards a rigorous science of interpretable machine learning, 70, 2786
- 2021, Proposal for a Regulation Laying Down Harmonized Rules on Artificial Intelligence (Artificial Intelligence Act)
- 2023, Requirements of Trustworthy AI
- Cheng, 2024, Interactive analysis of LLMs using meaningful counterfactuals, arXiv:2405.00708 · arXiv
- 2024, Explainable ai for personalized learning: Improving student outcomes, Int. J. Multidisciplinary Innov. Res. Methodol., 2068, 198
- Zhang, 2022, Citation, Appl. Explainable Artif. Intell. Diagnosis Surg., 12, 237
- Soundararajan, 2022, Enhancing financial decision-making through explainable AI and blockchain integration: Improving transparency and trust in predictive models, Educ. Admin., Theory Pract., 30, 9341
- 2017, Explainable Artificial Intelligence—The New Frontier in Legal Informatics
- Lim, 2020, Judicial Decision-Making and Explainable Artificial Intelligence
- 10.1145/3411764.3445315 · DOI · Paper page
- Sharma, 2024, Explainable AI (XAI): Model Interpretability, Feature Attribution, and Model Explainability
Cited by
Khan, F. S., Mazhar, S. S., Mazhar, K., AlSaleh, D., & Mazhar, A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11215-9 · Cited by 151
Alsabah, M. Q., Naser, M. A., Albahri, A. S., Albahri, A. S., Alamoodi, A. H., Abdulhussain, S. H., & Alzubaidi, L. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11342-3 · Cited by 112
Khan, N., Ahmad, K., Tamimi, A. A., Alani, M. M., Bermak, A., & Khalil, I. M. (2025) · Information · doi:10.3390/info16121036 · Cited by 140
Explainable AI for industrial fault diagnosis: A systematic review
Cação, J., Santos, J., & Antunes, M. (2025) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2025.100905 · Cited by 85
Živić, F., Kaplarevic-Malisic, A. M., Grujović, N., Stojanovic, B. S., & Ivanovic, M. R. (2025) · Materials Today Communications · doi:10.1016/j.mtcomm.2025.113525 · Cited by 65
Metadata
- Authors
- Kalasampath, K., Spoorthi, K. N., Sajeev, S., Kuppa, S. S., Ajay, K., & Maruthamuthu, A.
- Affiliations
- Vellore Institute of Technology University (IN)
- School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India
- ORCID
- Venue
- IEEE Access
- Published
- 2025
- Volume
- 13
- Pages
- 41111-41140
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4408047551
- Cited by
- 180
- References
- 107
- Citations by year
- 2026: 128 · 2025: 52
- FWCI
- 72.20
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Vellore Institute of Technology, Vellore, India
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- explainable AI, interpretability, transparency, black-box models, SHAP, lime, diagnosis, diagnostic imaging, industrial optimization, cybersecurity, transportation, social media, application of law, social care, trust in AI, accountability, fairness, ethical AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.