A Survey of Post-Hoc XAI Methods From a Visualization Perspective: Challenges and Opportunities
Open accessThe DOI matches its Crossref record; 73 of the 76 references it cites were found in a registry or an index.
Abstract
XAI (eXplainable AI) has become a pivotal area of research with the advancement of deep learning (DL) technologies and applications. Post-hoc explanation methods interpret deep learning predictions by uncovering the significance of input features, while visualization tools can contribute to a deep understanding of AI model reasoning based on these methods. In this paper, we survey a broad spectrum of post-hoc explanation methods and the visual analytics work based on them. First, we categorize the computational methods into four main types: perturbation-based, gradient-based, decomposition-based, and concept-based. While the first three focus on attributing the model’s output to specific regions of the input image, concept-based methods provide global explanations by mapping human-understandable concepts to high-level features. Then, we examine the methodologies, features, strengths, and limitations of each approach. Moreover, we review existing visualization-focused work based on these computational methods. Finally, we discuss further research challenges and opportunities for XAI visualization with post-hoc explanation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“By contrast, feature- or concept-based explanations appeared only in systems incorporating conventional predictive components amenable to attribution, such as saliency maps, concept bottleneck representations, or SHAP analyses over structured variables [31,32,33].”
From Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review (2026) · cites this paper as [33]
In the citing paper
“For instance, while models like deep neural networks often demonstrate high detection accuracy, their interpretability is limited, requiring post-hoc XAI tools that can sometimes be computationally expensive (Bhati et al., 2025).”
From Explainable AI: enhancing decision-making in the detection of cyber threats (2026) · cites this paper as Bhati et al., 2025
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 8×
“This scalable tool aggregates neuron activations, enabling users to visually explore how different features influence model predictions, particularly for large datasets like ImageNet [1].”
InfluentialGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 2×
“SHAP assigns each feature an importance score by measuring its contribution to the difference between the actual prediction and a baseline prediction, ensuring consistent and theoretically grounded model interpretation by evaluating all possible feature combinations [45].”
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 4×
“A significant draw-back of Occlusion is the computational power required, as inference is needed for each occluded image, which increases with higher resolution images and more detailed heatmaps [43].”
InfluentialLearning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · cited 2×
“Illustration of the class activation map (CAM) generation process in [64].”
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 2×
“Focusing specifically on surveys related to post-hoc explanation, Arrieta et al. [7] and Adadi and Berrada [8] comprehensively examine XAI assessment methods and challenges relevant to image classification tasks.”
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 4×
“By weighting the gradient with the input features, the model is locally linearized, providing insight into feature importance [46].”
InfluentialOn Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524 · cited 3×
“Specifically, PatternAttribution employs a class ‘signal’ estimator, trained on the first half of the ImageNet training dataset [1], to estimate the root point for DTD [49].”
Research works printed but not linked: 17
- Nguyen, 2020, Artificial intelligence in the battle against coronavirus (COVID-19): A survey and future research directions, arXiv:2008.07343 · arXiv
- 2025, Google Scholar
- 2025, IEEE Xplore Digital Library
- 2025, Association for Computing Machinery (ACM), 23
- 2025, Springer—International Publisher Science, Technology, Medicine
- 2025, MDPI—Multidisciplinary Digital Publishing Institute
- 2025, Elsevier—Academic Publishing Company
- 2025, Scopus—Abstract and Citation Database
- 2025, ScienceDirect—Elsevier Journal and Book Platform
- 2025, ArXiv—Open-access Preprint Repository
- 2025, Inceptionism: Going Deeper into Neural Networks—Google AI Blog
- 2025, DeepDream Implementation—GitHub Repository by M. Fatin Akarsu
- 2025, DeepDream - dream.ipynb—GitHub Repository by Google
- 2015, Journey Through the Layers of the Mind—Memo Akten
- 2025, Inside Deep Dreams: How Google Made Its Computers Go Crazy—Wired
- 10.1109/ICCV.2019.00304 · DOI
- Anders, 2021, Software for dataset-wide XAI: From local explanations to global insights with zennit, CoRelAy, and ViRelAy, arXiv:2106.13200 · arXiv
Cited by
Osman, K. A. Y. A., Shah, A. F. M. S., Karabulut, M. A., Karahan, S. N., Osmanca, M. S., & Acır, N. (2026) · IEEE Access · doi:10.1109/access.2026.3663161 · Cited by 36
Almegdadi, O., Marcelino, J., Fakhreddine, S., Manso, J., & Marques, N. C. (2025) · Ecological Informatics · doi:10.1016/j.ecoinf.2025.103542 · Cited by 10
“Explainable AI (XAI) methods such as SHAP and LIME complement this perspective by providing post-hoc explanations for models whose internal structure is not directly interpretable (Bhati et al., 2025; Vale et al., 2022; Rudin, 2019; Huang et al., 2024; Taskin et al., 2024; Mia et al., 2023).”
Chen, T., Chiu, M.-C., & Lin, Y.-C. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2610447 · Cited by 6
“In addition, XAI methods can also be divided into several categories such as visualization XAI methods (Bhati et al., 2025; So, 2020), attribute importance evaluation XAI methods, and relationship approximation XAI methods (Chen, 2024).”
Explainable AI: enhancing decision-making in the detection of cyber threats
Prasad, P. W. C., Sayeed, M. S., Nguyen, D.-M., Hutabarat, D. P., & Mohiuddin, G. M. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1762332 · Cited by 16
Reimagining cancer treatments in the era of generative AI
Derbal, Y. (2025) · DIGITAL HEALTH · doi:10.1177/20552076251394631 · Cited by 4
“90,91 However, it has been argued that explanation of GenAI outputs alone is not suf fi - cient to engender trust and that efforts to improve transparency should be focused on the socio-technical system (e.g., treatment delivery system) that embed the AI models.”
Metadata
- Authors
- Bhati, D., Amiruzzaman, M., Zhao, Y., Guercio, A., & Le, T.
- Affiliations
- Kent State University (US)
- West Chester University (US)
- Venue
- IEEE Access
- Published
- 2025
- Volume
- 13
- Pages
- 120785-120806
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4411406312
- Cited by
- 24
- References
- 76
- Citations by year
- 2026: 21 · 2025: 3
- FWCI
- 9.59
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Kent State University’s Open Access APC Support Fund
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- explainable AI, post-hoc explanation, visual analytics, deep learning interpretability, perturbation-based explanation, gradient-based explanation, concept-based explanations, feature attribution, global explanations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Survey of Post-Hoc XAI Methods From a Visualization Perspective: Challenges and Opportunities. https://app.cochat.ai/papers/10.1109/access.2025.3581136
- The paper
- Bhati, D., Amiruzzaman, M., Zhao, Y., Guercio, A., & Le, T. (2025). A Survey of Post-Hoc XAI Methods From a Visualization Perspective: Challenges and Opportunities. IEEE Access. https://doi.org/10.1109/access.2025.3581136
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.