A Review of Explainable Artificial Intelligence from the Perspectives of Challenges and Opportunities
Open accessThe DOI matches its Crossref record; 172 of the 185 references it cites were found in a registry or an index, and 13 are not checked yet.
Abstract
The widespread adoption of Artificial Intelligence (AI) in critical domains, such as healthcare, finance, law, and autonomous systems, has brought unprecedented societal benefits. Its black-box (sub-symbolic) nature allows AI to compute prediction without explaining the rationale to the end user, resulting in lack of transparency between human and machine. Concerns are growing over the opacity of such complex AI models, particularly deep learning architectures. To address this concern, explainability is of paramount importance, which has triggered the emergence of Explainable Artificial Intelligence (XAI) as a vital research area. XAI is aimed at enhancing transparency, trust, and accountability of AI models. This survey presents a comprehensive overview of XAI from the dual perspectives of challenges and opportunities. We analyze the foundational concepts, definitions, terminologies, and taxonomy of XAI methods. We then review several application domains of XAI. Special attention is given to various challenges of XAI, such as no universal definition, trade-off between accuracy and interpretability, and lack of standardized evaluation metrics. We conclude by outlining the future research directions of human-centric design, interactive explanation, and standardized evaluation frameworks. This survey serves as a resource for researchers, practitioners, and policymakers to navigate the evolving landscape of interpretable and responsible AI.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“2012 Deep Learning [41] Despite high predictive accuracy, DNN is an opaque black-box model.”
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.105906 · Cited by 11,748
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“These models offer transparency by exposing the internal logic directly to the user [54].”
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Research works printed but not linked: 27
- Lundberg, S.M., and Lee, S.I. (2017, January 4–9). A Unified Approach to Interpreting Model Predictions. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA. Available online: https://arxiv.org/abs/1705.07874. · arXiv
- Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
- Kulesza, T., Stumpf, S., Burnett, M., Wong, W.K., and Striegel, A. (May, January 27). Principles of Explanatory Debugging to Assist Machine Learning. Proceedings of the 2013 CHI Conference on Human Factors in Computing Systems (CHI ’13), Paris, France.
- Stumpf, S., Rajaram, V., Li, L., Burnett, M., Dietterich, T.G., Sullivan, E., and Drummond, R. (2009, January 8–11). Interacting with Machine Learning Systems: The Case for Explanation and Beyond. Proceedings of the 14th International Conference on Intelligent User Interfaces (IUI ’09), Sanibel Island, FL, USA.
- Shortliffe, E. (2012). Computer-Based Medical Consultations: MYCIN, Elsevier.
- Barocas, S., Friedler, S., Hardt, M., Kroll, J., Venkatasubramanian, S., and Wallach, H. (2025, June 09). The FAT-ML Workshop Series on Fairness, Accountability, and Transparency in Machine Learning. Available online: http://www.fatml.org/.
- Binns, R. (2018, January 23–24). Fairness in Machine Learning: Lessons from Political Philosophy. Proceedings of the 2018 Conference on Fairness, Accountability and Transparency (FAT), New York, NY, USA.
- Goldstein, A., Kapelner, A., Bleich, J., and Pitkin, E. (2015, January 10–13). Predicting Consumer Choices from Complex Behavioral Data. Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, Australia.
- Amershi, S., Chickering, M., Drucker, S.M., Lee, B., Simard, P., and Suh, J. (May, January 26). ModelTracker: Redesigning Performance Analysis Tools for Machine Learning. Proceedings of the CHI ’14: SIGCHI Conference on Human Factors in Computing Systems, Toronto, ON, Canada.
- Sarti, G., Feldhus, N., Qi, J., Nissim, M., and Bisazza, A. (2024, January 17–19). Democratizing Advanced Attribution Analyses of Generative Language Models with the Inseq Toolkit. Proceedings of the Joint of the 2nd World Conference on eXplainable Artificial Intelligence Late-Breaking Work, Demos and Doctoral Consortium, xAI-2024: LB/D/DC, Valletta, Malta.
- Simuni, 2024, Explainable AI in ML: The Path to Transparency and Accountability, Int. J. Recent Adv. Multidiscip. Res., 11, 10531
- Rajkomar, 2019, Ensuring Fairness in Machine Learning for Healthcare, Lancet, 393, 110
- Uchida, S., Barney, E., and Eglin, V. (2022). Sequence-to-Sequence Models for Extracting Information from Registration and Legal Documents. Document Analysis Systems. DAS 2022, Springer. Lecture Notes in Computer Science, Vol. 13237.
- Zafar, 2024, Artificial Intelligence in Autonomous Systems: Challenges and Opportunities, Res. Corrid. J. Eng. Sci., 1, 182
- Gartner, 2017, Autonomy and Accountability in Autonomous Systems, IEEE Intell. Syst., 32, 8
- Dastin, J. (2025, August 28). Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women. Reuters, 10 October 2018. Available online: https://www.taylorfrancis.com/chapters/edit/10.1201/9781003278290-44/amazon-scraps-secret-ai-recruiting-tool-showed-bias-women-jeffrey-dastin. · doi:10.1201/9781003278290-44/amazon-scraps-secret-ai-recruiting-tool-showed-bias-women-jeffrey-dastin
- Veeramachaneni, 2025, Large Language Models: A Comprehensive Survey on Architectures, Applications, and Challenges, Adv. Innov. Comput. Program. Lang., 7, 20
- Polignano, 2024, XAI.it 2024: An Overview on the Future of AI in the Era of Large Language Models, XAI.it, 3839, 1
- Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q.V., and Zhou, D. (2022). Chain of Thought Prompting Elicits Reasoning in Large Language Models. arXiv.
- Krause, S., and Stolzenburg, F. Commonsense Reasoning and Explainable Artificial Intelligence Using Large Language Models. Proceedings of the European Conference on Artificial Intelligence.
- Randl, K., Pavlopoulos, J., Henriksson, A., and Lindgren, T. Evaluating the Reliability of Self-Explanations in Large Language Models. Proceedings of the International Conference on Discovery Science.
- Chen, J., Song, L., and Yoon, J. (2018, January 3–8). Explaining Explanations: Axiomatic Feature Interactions for Deep Networks. Proceedings of the 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montreal, QC, Canada.
- Molnar, C. (2020). Interpretable Machine Learning, Springer. [2nd ed.].
- Ziethmann, P., Stieler, F., Pfrommer, R., Schlögl-Flierl, K., and Bauer, B. (July, January 29). Towards a Framework for Interdisciplinary Studies in Explainable Artificial Intelligence. Proceedings of the International Conference on Human-Computer Interaction, Washington, DC, USA.
- Lei, Y., Li, Z., Li, Y., Zhang, J., and Shan, H. (2023, January 10–16). LICO: Explainable Models with Language-Image Consistency. Proceedings of the Thirty-Seventh Conference on Neural Information Processing Systems (NeurIPS 2023), New Orleans, LA, USA. Available online: https://arxiv.org/abs/2310.09821. · arXiv
- Serrurier, 2023, On the Explainable Properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective, Adv. Neural Inf. Process. Syst., 36, 54645
- Burger, C., Walter, C., and Le, T. (2024). The Effect of Similarity Measures on Accurate Stability Estimates for Local Surrogate Models in Text-Based Explainable AI. arXiv.
Cited by
Aravindkumar, A., Ramadoss, M., Ahmed, S. A. F., Sampath, V., & Lakshminarayanan, K. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1749527 · Cited by 50
Öz, M., Üstüner, E., Çifci, S., Budak, F., İleri, E., & Dikel, S. (2025) · Aquaculture International · doi:10.1007/s10499-025-02352-7 · Cited by 33
Khajavian, M., Jang, J.-H., Kwon, J.-Y., Lee, J.-M., Lee, S., Hwang, M.-H., Yang, E., Jang, J., & Chae, K. (2026) · Inorganic Chemistry Communications · doi:10.1016/j.inoche.2026.116269 · Cited by 25
Machine Learning in Next-Generation Polymer Composites: Recent Advances and Perspectives
Olhan, S., Antil, B., & Maimí, P. (2026) · Applied Composite Materials · doi:10.1007/s10443-025-10437-y · Cited by 26
Dang, M., Liu, B., Chen, Y., & Zhang, Z. (2026) · Psychoradiology · doi:10.1093/psyrad/kkag007 · Cited by 10
Karras, A., Giannaros, A., Amasiadi, N., & Karras, C. (2026) · Future Internet · doi:10.3390/fi18020083 · Cited by 10
Samra, R. M. A., & Alí, R. R. (2026) · AgriEngineering · doi:10.3390/agriengineering8070270 · Cited by 11
Hybrid intelligence systems as ontological mirrors of human cognition
Ahmed, M. F., & Albino, J. S. (2026) · AI and Ethics · doi:10.1007/s43681-026-01032-3 · Cited by 6
Qi, J., Li, M., Liu, J., Shu, Y., Yu, D. S. F., Ma, S., Cui, W., Zhao, Y., Chen, Y., Jiang, R., et al. (2026) · Academia AI and Applications · doi:10.20935/acadai8260 · Cited by 24
Gonçalves, A., & Correia, A. C. (2026) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp6020043 · Cited by 11
Haupt, M., & Maurer, M. H. (2026) · European Journal of Radiology Artificial Intelligence · doi:10.1016/j.ejrai.2026.100098 · Cited by 8
Afif, M. M., Noman, A. A., Kabir, K. M. T., Seemanta, S., Ahmmed, M. M., Rahman, M. O., Uddin, J., & Fung, W.-K. (2026) · IEEE Access · doi:10.1109/access.2026.3664510 · Cited by 2
Gaikwad, A. S., B.Pawar, S., & Rajguru, M. K. (2026) · Journal of Pharmaceutical Innovation · doi:10.1007/s12247-026-10841-2 · Cited by 2
Ritu, R, H. K. G., Mehrotra, S., & Nagappan, P. (2026) · International Conference on Augmented Intelligence and Sustainable Systems (ICAISS) · doi:10.1109/icaiss68683.2026.11525437 · Cited by 1
Górecki, S., Tatka, A., & Osmola, M. (2026) · Cancers · doi:10.3390/cancers18121905 · Cited by 1
Recent advances in smart nanosensors and IoT-based systems for real-time diagnostic monitoring
Omiyale, B. O., Ogbeyemi, A., Odeyemi, J., Ashraf, M. A., Olugbade, T. O., Song, K., & Zhang, W. (2026) · Analytical Methods · doi:10.1039/d6ay01081d · Cited by 1
Ali, J. M., Mohammed, A. M., Obodo, K. O., & Mekonnen, Y. S. (2026) · Journal of CO2 Utilization · doi:10.1016/j.jcou.2026.103557 · Cited by 2
AI-enhanced Paediatric Dentistry: new horizons driven by artificial intelligence
Aura‐Tormos, J. I. (2026) · Revista Odontología Pediátrica · doi:10.20960/odontolpediatr.00076
Harnessing the Affordances of Artificial Intelligence to Foster Deep Learning
Ramaila, S. M. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/404026
Lukhele, N., & Mostafa, F. B. (2026) · medRxiv · doi:10.64898/2026.04.25.26351733
Explainable artificial intelligence 5.0 for emerging technologies of responsible innovation
Daovisan, H. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01250-y
Rodrigues, R. B., & Creen, C. M. (2026) · Journal of Aerospace Technology and Management · doi:10.1590/jatm.v18.1437 · Cited by 2
R, S. T., Sripriya, N., Avita, M., J, A. P., -, P. V. S., & Sruthi, M. (2026) · International Conference on Trends in Electronics and Informatics (ICOEI) · doi:10.1109/icoei68323.2026.11541986
Wang, S., Yoo, B., & Li, H. (2026) · Universal Access in the Information Society · doi:10.1007/s10209-026-01350-7
AI Revolution: Raising GenAI, XAI, and Agentic AI Using Quantum Computing
Saeed, S. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-2601-4.ch001 · Cited by 1
İş Sağlığı ve Güvenliğinde Yapay Zekâ Tabanlı Risk Analizi: Bütünleşik Bir Mimari Yaklaşım Önerisi
Çınar, C., & Yavuz, S. (2026) · Bilgi Teknolojileri ve İletişim Dergisi · doi:10.66643/btkdergi.1939549
Artificial Intelligence for Engineering Solutions: Trends, Challenges, and Future Directions
Avilés, M., Aceves-Fernández, M. A., Odry, Á., Álvarez-Alvarado, J. M., & Rodríguez‐Reséndiz, J. (2026) · Eng—Advances in Engineering · doi:10.3390/eng7070317
Artificial Intelligence Technologies for Secure and Sustainable E-Commerce
Huseynov, F., & Özdenizci, B. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-9048-2.ch007
Human-Centric, AI-Enabled Concurrent Engineering: Towards a cross-domain framework
Holzinger, A., Müller, H., & Stjepandić, J. (2026) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2026.101192 · Cited by 2
Explainable Gradient Boosting for GPA Prediction with SHAP Aggregation
Fernanda, J. W., Akhyar, M. K., Zuraidah, N., Rahmawati, N. R., & Asfarina, L. M. A. (2026) · Jurnal Statistika dan Komputasi · doi:10.32665/statkom.v5i1.6381
Sriwichien, W., Narkbunnum, W., & Wisaeng, K. (2026) · Information · doi:10.3390/info17070672
Iyer, D. S. S., Raji, D. B., Subramanian, R., & Arora, R. (2026) · International Journal of Latest Technology in Engineering Management & Applied Science · doi:10.51583/ijltemas.2026.150600099
Mateo-Elgueda, F., Rivera, M. E., Su, Y., & del Campo-Hitschfeld, M. L. (2026) · Electronics · doi:10.3390/electronics15143200 · Cited by 1
Theodorakopoulos, L., & Theodoropoulou, A. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70116 · Cited by 3
Gardashova, L. A., & Mirzazada, A. (2026) · Himičeskaâ tehnologiâ. Kontrolʹ i upravlenie/Chemical Technology. Control and Management · doi:10.59048/2181-1105.1804
Das, K., Chowdhury, A., & Chowdhury, A. (2026) · International Conference on Circuit, Systems and Communication (ICCSC) · doi:10.1109/iccsc71566.2026.11650369
Privacy and Trust in AI Personalization Strategy: A Systematic Literature Review
Almas, R., & Dirgantara, I. M. B. (2025) · Entrepreneurship and Innovation Review · doi:10.63208/210113-538
Explainable Sentiment-Based Reputational Risk Modelling in Ugandan Megaprojects
Ngambeki, A. K., Chongomweru, H., & Kivunike, F. N. (2026) · Information · doi:10.3390/info17100987
Solano, R. P., Berlanga, A., Molina, J. M., & Julio, Y. R. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8100319
zhi, F. (2026) · Research Square · doi:10.21203/rs.3.rs-11093106/v1
Lukhele, N., & Mostafa, F. B. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1952669
Post-Processing Matters: Enhancing Post-Hoc Explanation Methods in Medical Imaging
Repetto, S., Ciná, A. E., Anguita, D., Roli, F., & Oneto, L. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11728537
Application of interpretable deep learning in brain age prediction: a comprehensive review
Wang, Z., Gao, H., Lin, L., Li, Y., & Zhu, X. (2026) · Reviews in the Neurosciences · doi:10.1515/revneuro-2026-0186
Urooj, B., Guruge, D. B., & Gyasi-Agyei, A. (2026) · Data and Information Management · doi:10.1016/j.dim.2026.100144
An Explainable Imbalance-Aware Random Forest Framework for Employee Turnover Prediction
Saiful, M., Soeleman, A., & Rijati, N. (2026) · International Seminar on Application for Technology of Information and Communication (iSemantic) · doi:10.1109/isemantic71495.2026.11742582
Metadata
- Authors
- Kabir, S., Hossain, M. S., & Andersson, K.
- Affiliations
- Luleå University of Technology (SE)
- University of Chittagong (BD)
- Corresponding author
- Kabir, Sami
- Andersson, Karl
- Venue
- Algorithms
- Publisher
- MDPI AG
- Published
- 2025-09-03
- Volume
- 18
- Issue
- 9
- Pages
- 556
- ISSN
- 1999-4893 (ISSN-L)
- OpenAlex
- W4413951932
- Cited by
- 44
- References
- 167
- Citations by year
- 2026: 42 · 2025: 2
- FWCI
- 17.65
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Algorithms (publishedVersion, cc-by)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- VINNOVA (Sweden’s Innovation Agency) through the Digital Stadsutveckling Campus Skellefteå project (2022-01188)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- explainable AI, interpretability, black-box models, trust in AI, accountability, accuracy-interpretability trade-off, evaluation metrics, interactive explanations, human-centered design, responsible AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.