CoChat

A Review of Explainable Artificial Intelligence from the Perspectives of Challenges and Opportunities

Open access

The DOI matches its Crossref record; 172 of the 185 references it cites were found in a registry or an index, and 13 are not checked yet.

Abstract

The widespread adoption of Artificial Intelligence (AI) in critical domains, such as healthcare, finance, law, and autonomous systems, has brought unprecedented societal benefits. Its black-box (sub-symbolic) nature allows AI to compute prediction without explaining the rationale to the end user, resulting in lack of transparency between human and machine. Concerns are growing over the opacity of such complex AI models, particularly deep learning architectures. To address this concern, explainability is of paramount importance, which has triggered the emergence of Explainable Artificial Intelligence (XAI) as a vital research area. XAI is aimed at enhancing transparency, trust, and accountability of AI models. This survey presents a comprehensive overview of XAI from the dual perspectives of challenges and opportunities. We analyze the foundational concepts, definitions, terminologies, and taxonomy of XAI methods. We then review several application domains of XAI. Special attention is given to various challenges of XAI, such as no universal definition, trade-off between accuracy and interpretability, and lack of standardized evaluation metrics. We conclude by outlining the future research directions of human-centric design, interactive explanation, and standardized evaluation frameworks. This survey serves as a resource for researchers, practitioners, and policymakers to navigate the evolving landscape of interpretable and responsible AI.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/a18090556

Cites

172 of 185 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 27

  1. Lundberg, S.M., and Lee, S.I. (2017, January 4–9). A Unified Approach to Interpreting Model Predictions. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA. Available online: https://arxiv.org/abs/1705.07874. · arXiv
  2. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
  3. Kulesza, T., Stumpf, S., Burnett, M., Wong, W.K., and Striegel, A. (May, January 27). Principles of Explanatory Debugging to Assist Machine Learning. Proceedings of the 2013 CHI Conference on Human Factors in Computing Systems (CHI ’13), Paris, France.
  4. Stumpf, S., Rajaram, V., Li, L., Burnett, M., Dietterich, T.G., Sullivan, E., and Drummond, R. (2009, January 8–11). Interacting with Machine Learning Systems: The Case for Explanation and Beyond. Proceedings of the 14th International Conference on Intelligent User Interfaces (IUI ’09), Sanibel Island, FL, USA.
  5. Shortliffe, E. (2012). Computer-Based Medical Consultations: MYCIN, Elsevier.
  6. Barocas, S., Friedler, S., Hardt, M., Kroll, J., Venkatasubramanian, S., and Wallach, H. (2025, June 09). The FAT-ML Workshop Series on Fairness, Accountability, and Transparency in Machine Learning. Available online: http://www.fatml.org/.
  7. Binns, R. (2018, January 23–24). Fairness in Machine Learning: Lessons from Political Philosophy. Proceedings of the 2018 Conference on Fairness, Accountability and Transparency (FAT), New York, NY, USA.
  8. Goldstein, A., Kapelner, A., Bleich, J., and Pitkin, E. (2015, January 10–13). Predicting Consumer Choices from Complex Behavioral Data. Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, Australia.
  9. Amershi, S., Chickering, M., Drucker, S.M., Lee, B., Simard, P., and Suh, J. (May, January 26). ModelTracker: Redesigning Performance Analysis Tools for Machine Learning. Proceedings of the CHI ’14: SIGCHI Conference on Human Factors in Computing Systems, Toronto, ON, Canada.
  10. Sarti, G., Feldhus, N., Qi, J., Nissim, M., and Bisazza, A. (2024, January 17–19). Democratizing Advanced Attribution Analyses of Generative Language Models with the Inseq Toolkit. Proceedings of the Joint of the 2nd World Conference on eXplainable Artificial Intelligence Late-Breaking Work, Demos and Doctoral Consortium, xAI-2024: LB/D/DC, Valletta, Malta.
  11. Simuni, 2024, Explainable AI in ML: The Path to Transparency and Accountability, Int. J. Recent Adv. Multidiscip. Res., 11, 10531
  12. Rajkomar, 2019, Ensuring Fairness in Machine Learning for Healthcare, Lancet, 393, 110
  13. Uchida, S., Barney, E., and Eglin, V. (2022). Sequence-to-Sequence Models for Extracting Information from Registration and Legal Documents. Document Analysis Systems. DAS 2022, Springer. Lecture Notes in Computer Science, Vol. 13237.
  14. Zafar, 2024, Artificial Intelligence in Autonomous Systems: Challenges and Opportunities, Res. Corrid. J. Eng. Sci., 1, 182
  15. Gartner, 2017, Autonomy and Accountability in Autonomous Systems, IEEE Intell. Syst., 32, 8
  16. Dastin, J. (2025, August 28). Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women. Reuters, 10 October 2018. Available online: https://www.taylorfrancis.com/chapters/edit/10.1201/9781003278290-44/amazon-scraps-secret-ai-recruiting-tool-showed-bias-women-jeffrey-dastin. · doi:10.1201/9781003278290-44/amazon-scraps-secret-ai-recruiting-tool-showed-bias-women-jeffrey-dastin
  17. Veeramachaneni, 2025, Large Language Models: A Comprehensive Survey on Architectures, Applications, and Challenges, Adv. Innov. Comput. Program. Lang., 7, 20
  18. Polignano, 2024, XAI.it 2024: An Overview on the Future of AI in the Era of Large Language Models, XAI.it, 3839, 1
  19. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q.V., and Zhou, D. (2022). Chain of Thought Prompting Elicits Reasoning in Large Language Models. arXiv.
  20. Krause, S., and Stolzenburg, F. Commonsense Reasoning and Explainable Artificial Intelligence Using Large Language Models. Proceedings of the European Conference on Artificial Intelligence.
  21. Randl, K., Pavlopoulos, J., Henriksson, A., and Lindgren, T. Evaluating the Reliability of Self-Explanations in Large Language Models. Proceedings of the International Conference on Discovery Science.
  22. Chen, J., Song, L., and Yoon, J. (2018, January 3–8). Explaining Explanations: Axiomatic Feature Interactions for Deep Networks. Proceedings of the 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montreal, QC, Canada.
  23. Molnar, C. (2020). Interpretable Machine Learning, Springer. [2nd ed.].
  24. Ziethmann, P., Stieler, F., Pfrommer, R., Schlögl-Flierl, K., and Bauer, B. (July, January 29). Towards a Framework for Interdisciplinary Studies in Explainable Artificial Intelligence. Proceedings of the International Conference on Human-Computer Interaction, Washington, DC, USA.
  25. Lei, Y., Li, Z., Li, Y., Zhang, J., and Shan, H. (2023, January 10–16). LICO: Explainable Models with Language-Image Consistency. Proceedings of the Thirty-Seventh Conference on Neural Information Processing Systems (NeurIPS 2023), New Orleans, LA, USA. Available online: https://arxiv.org/abs/2310.09821. · arXiv
  26. Serrurier, 2023, On the Explainable Properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective, Adv. Neural Inf. Process. Syst., 36, 54645
  27. Burger, C., Walter, C., and Le, T. (2024). The Effect of Similarity Measures on Accurate Stability Estimates for Local Surrogate Models in Text-Based Explainable AI. arXiv.

Cited by

See all 45

Metadata

Authors
Kabir, S., Hossain, M. S., & Andersson, K.
Affiliations
  • Luleå University of Technology (SE)
  • University of Chittagong (BD)
Corresponding author
  • Kabir, Sami
  • Andersson, Karl
Venue
Algorithms
Publisher
MDPI AG
Published
2025-09-03
Volume
18
Issue
9
Pages
556
ISSN
1999-4893 (ISSN-L)
OpenAlex
W4413951932
Cited by
44
References
167
Citations by year
2026: 42 · 2025: 2
FWCI
17.65
Type
journal article
Language
en
Open access
Locations
  • Algorithms (publishedVersion, cc-by)
  • KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
VINNOVA (Sweden’s Innovation Agency) through the Digital Stadsutveckling Campus Skellefteå project (2022-01188)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords
explainable AI, interpretability, black-box models, trust in AI, accountability, accuracy-interpretability trade-off, evaluation metrics, interactive explanations, human-centered design, responsible AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.