Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities
Open accessAbstract
The past decade has seen significant progress in artificial intelligence (AI), which has resulted in algorithms being adopted for resolving a variety of problems. However, this success has been met by increasing model complexity and employing black-box AI models that lack transparency. In response to this need, Explainable AI (XAI) has been proposed to make AI more transparent and thus advance the adoption of AI in critical domains. Although there are several reviews of XAI topics in the literature that identified challenges and potential research directions in XAI, these challenges and research directions are scattered. This study, hence, presents a systematic meta-survey for challenges and future research directions in XAI organized in two themes: (1) general challenges and research directions in XAI and (2) challenges and research directions in XAI based on machine learning life cycle's phases: design, development, and deployment. We believe that our meta-survey contributes to XAI literature by providing a guide for future exploration in the XAI area.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
SLR has been prominently utilized in numerous scholarly surveys, as evidenced by Murtaza et al. (2020); Saeed and Omlin (2023).
“Offering a more expansive and accurate insight compared to conventional literature reviews, this approach has been prominently utilized in numerous scholarly surveys, as evidenced by Murtaza et al. (2020); Saeed and Omlin (2023).”
From A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions (2025) · cites this paper as Saeed and Omlin (2023)
According to recent findings, most XAI methods used in medical image analysis apply local post hoc explanations rather than global model-based ones.
“According to recent findings in [8, 9], most XAI methods used in medical image analysis apply local post hoc explanations rather than global model-based ones.”
From Evaluating Explainable Artificial Intelligence (XAI) techniques in chest radiology imaging through a human-centered Lens (2024) · cites this paper as [9]
Visual explainability holds great promise in AI-based tools for medical imaging, but several usability issues may arise that can hinder their effective adoption in clinical settings.
“While visual explainability holds great promise in AI-based tools for medical imaging, several usability issues may arise that can hinder their effective adoption in clinical settings [9, 12].”
From Evaluating Explainable Artificial Intelligence (XAI) techniques in chest radiology imaging through a human-centered Lens (2024) · cites this paper as [9]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · see citing works
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · cited 8× · see citing works
“While both terms are often used interchangeably in the literature, some examples from the selected papers distinguish them [20, 2, 21, 22, 23, 24].”
BackgroundMethodInfluentialPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524 · see citing works
Definitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167 · see citing works
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · see citing works
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477 · see citing works
Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
van der Velden, B. H. M., Kuijf, H. J., Gilhuijs, K. G. A., & Viergever, M. A. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102470 · Cited by 1,328 · see citing works
Samek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080 · see citing works
Markus, A. F., Kors, J. A., & Rijnbeek, P. R. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103655 · Cited by 811 · see citing works
Explainable Recommendation: A Survey and New Perspectives
Zhang, Y., & Chen, X. (2020) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000066 · Cited by 755 · see citing works
Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629 · see citing works
Accountability of AI Under the Law: The Role of Explanation
Doshi-Velez, F., Kortz, M., Budish, R., Bavitz, C., Gershman, S. J., O'Brien, D., Scott, K., Shieber, S., Waldo, J., Weinberger, D., et al. (2017) · doi:10.2139/ssrn.3064761 · Cited by 457 · cited 1× · see citing works
“Therefore, costs associated with algorithmic explanations should be incurred when the benefits of the explanations outweigh the costs [83].”
BackgroundStepin, I., Alonso, J. M., Catala, A., & Pereira-Farina, M. (2021) · IEEE Access · doi:10.1109/access.2021.3051315 · Cited by 376 · see citing works
Interpretable and explainable AI (XAI) model for spatial drought prediction
Dikshit, A., & Pradhan, B. (2021) · Science of The Total Environment · doi:10.1016/j.scitotenv.2021.149797 · Cited by 272 · see citing works
Mueller, S. T., Hoffman, R. R., Clancey, W., Emrey, A., & Klein, G. (2019) · arXiv · doi:10.48550/arxiv.1902.01876 · Cited by 215 · see citing works
Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI
Tomsett, R., Preece, A., Braines, D., Cerutti, F., Chakraborty, S., Srivastava, M., Pearson, G., & Kaplan, L. (2020) · Patterns · doi:10.1016/j.patter.2020.100049 · Cited by 186 · see citing works
Explainable Agency for Intelligent Autonomous Systems
Langley, P., Meadows, B., Sridharan, M., & Choi, D. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i2.19108 · Cited by 156 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Review of Explainable Artificial Intelligence in Healthcare
Sadeghi, Z., Alizadehsani, R., CIFCI, M. A., Kausar, S., Rehman, R., Mahanta, P., Bora, P. K., Almasri, A., Alkhawaldeh, R. S., Hussain, S., et al. (2024) · Computers and Electrical Engineering · doi:10.1016/j.compeleceng.2024.109370 · Cited by 598 · see citing works
A literature review on applications of explainable artificial intelligence (XAI)
Kalasampath, K., Spoorthi, K. N., Sajeev, S., Kuppa, S. S., Ajay, K., & Maruthamuthu, A. (2025) · IEEE Access · doi:10.1109/access.2025.3546681 · Cited by 180 · see citing works
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212 · see citing works
Aderibigbe, A. O., Ohenhen, P. E., Nwaobia, N. K., Gidiagba, J. O., & Ani, E. C. (2023) · Computer Science & IT Research Journal · doi:10.51594/csitrj.v4i3.629 · Cited by 172 · see citing works
Mathew, D. E., Ebem, D. U., Ikegwu, A. C., Ukeoma, P. E., & Dibiaezue, N. F. (2025) · Neural Processing Letters · doi:10.1007/s11063-025-11732-2 · Cited by 168 · see citing works
A comprehensive survey on relation extraction: recent advances and new frontiers
Zhao, X., Deng, Y., Yang, M., Wang, L., Zhang, R., Cheng, H., Lam, W., Shen, Y., & Xu, R. (2024) · ACM Computing Surveys · doi:10.1145/3674501 · Cited by 168 · see citing works
Survey on Explainable AI: Techniques, challenges and open issues
Abusitta, A., Li, M. Q., & Fung, B. C. M. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124710 · Cited by 98 · see citing works
Farrow, R. (2023) · Learning, Media and Technology · doi:10.1080/17439884.2023.2185630 · Cited by 81 · see citing works
A multifaceted vision of the human-AI collaboration: a comprehensive review
Puerta-Beldarrain, M., Gómez-Carmona, O., Sánchez-Corcuera, R., Casado-Mansilla, D., López-de-Ipiña, D., & Chen, L. (2025) · IEEE Access · doi:10.1109/access.2025.3536095 · Cited by 73 · see citing works
E. Ihongbe, I., Fouad, S., F. Mahmoud, T., Rajasekaran, A., & Bhatia, B. (2024) · PLOS ONE · doi:10.1371/journal.pone.0308758 · Cited by 75 · see citing works
Abas Mohamed, Y., Ee Khoo, B., Shahrimie Mohd Asaari, M., Ezane Aziz, M., & Rahiman Ghazali, F. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105689 · Cited by 57 · see citing works
Kabir, S., Hossain, M. S., & Andersson, K. (2025) · Algorithms · doi:10.3390/a18090556 · Cited by 44 · see citing works
Mirzaei, S., Mao, H., Al-Nima, R. R. O., & Woo, W. L. (2023) · Information · doi:10.3390/info15010004 · Cited by 25 · see citing works
A survey of post-hoc xai methods from a visualization perspective: Challenges and opportunities
Bhati, D., Amiruzzaman, M., Zhao, Y., Guercio, A., & Le, T. (2025) · IEEE Access · doi:10.1109/access.2025.3581136 · Cited by 24 · see citing works
Beyond single systems: how multi-agent ai is reshaping ethics in radiology
Salehi, S., Singh, Y., Habibi, P., & Erickson, B. J. (2025) · Bioengineering · doi:10.3390/bioengineering12101100 · Cited by 21 · see citing works
Fresz, B., Dubovitskaya, E., Brajovic, D., Huber, M. F., & Horz, C. (2024) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v7i1.31648 · Cited by 17 · see citing works
Klüver, C., Greisbach, A., Kindermann, M., & Püttmann, B. (2024) · Security and Safety · doi:10.1051/sands/2024020 · Cited by 8 · see citing works
“Quality models for AI products, such as ISO/IEC 25059:2023 (will be replaced by ISO/IEC AWI 25059 Software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Quality model for AI systems) need to be extended, e.g. , to meet the requirements of the AI Act.”
Suzuki, S., Takatsu, H., Matsuura, R., Koyama, M., Saeki, M., & Matsuyama, Y. (2025) · Language Testing · doi:10.1177/02655322251348725 · Cited by 6 · see citing works
Pawar, L., Patil, S., Dhotre, S., & Sonawane, S. (2024) · 2024 8th International Artificial Intelligence and Data Processing Symposium (IDAP) · doi:10.1109/idap64064.2024.10710903 · Cited by 5 · see citing works
- Authors
- Saeed, W., & Omlin, C.
- Affiliations
- University of Agder (NO)
- Venue
- Knowledge-Based Systems
- Publisher
- Elsevier BV
- Published
- 2023-03
- Volume
- 263
- Pages
- 110273
- ISSN
- 0950-7051 (ISSN-L)
- OpenAlex
- W3211639647
- Semantic Scholar
- 92259fcde062eeb232214c75e849dc5d9094599c
- Cited by
- 783
- References
- 22
- Citations by year
- 2026: 177 · 2025: 275 · 2024: 257 · 2023: 69 · 2022: 5
- FWCI
- 73.34
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Knowledge-Based Systems (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DMU Open Research Archive (De Montfort University) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Universitetet i Agder
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- explainable AI, trust in AI, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
