CoChat

Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence

Open access

The DOI matches its Crossref record; 180 of the 180 references it cites were found in a registry or an index; 20 of the 74 sources its claims cite were checked for support.

Abstract

This review surveys explainable AI, emphasizing transparency, interpretability, trust, and evaluation across high-stakes domains.

Overview

CoChat Summary

## TL;DR

This paper is a survey of explainable artificial intelligence (XAI) that organizes the field around why XAI is needed, how explanations are produced, and where the main challenges lie. It argues that black-box AI systems need explainability for trust, accountability, regulation, and safer deployment across domains such as healthcare, finance, and transportation.

## Key findings

  • The review proposes a revised hierarchical taxonomy for XAI and says it is intended to serve as a building block for future researchers.
  • It distinguishes two broad interpretability classes: perceptive interpretability and interpretability by mathematical structures; it also separates explanations by scope (global vs. local) and by stage (ante hoc vs. post hoc).
  • The paper summarizes model-agnostic methods such as PDP, ICE, ALE, LIME, and counterfactuals, and reports their main trade-offs; for example, PDP is intuitive but valid for at most three features, ICE can become unreadable due to overcrowding, and ALE is described as unbiased for correlated features and faster to compute.
  • In the paper’s escitalopram example, the interpretable version of the model achieved approximately 70% accuracy in identifying patients more likely to benefit from the treatment.
  • It also notes a reported example where CrystalCandle explanations increased a LinkedIn software sales team’s subscription revenue by 8%.

## Methods

  • Narrative review / survey of XAI literature; the authors state that they “careful[ly]” selected and analyzed current state-of-the-art XAI research.
  • The paper synthesizes prior work into a taxonomy and organizes the discussion into conceptual background, need for XAI, evaluation frameworks, challenges, discussion/reflection, and future directions.
  • The analysis is qualitative and comparative, using cited examples from multiple application domains and explaining the strengths and weaknesses of different explanation methods.

## Limitations

  • The paper states that a “thorough and systematic survey is still lacking despite the large number of surveys” in the area, which indicates the field remains fragmented.
  • It explicitly says XAI is still in its infancy and that a more formal approach is needed to address diverse user questions; it also notes that guidelines alone do not guarantee success.
  • Evident from the text: no new datasets were generated or analyzed in this study, so the conclusions are based on literature synthesis rather than original empirical testing.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s12559-023-10179-8

Cites

180 of 180 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

15 Valid · 25 Not valid · 49 Not checked · 59 Own results and interpretations

27 more claims

Create a free account to see all 148 claims and to verify the 21 references still open.

See all claims — free account
  • Probably not a substantial claim. 108
  • It may state two separate claims as one. 37

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

2 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 180 references

Research works printed but not linked: 9

  1. Li XH, Cao CC, Shi Y, Bai W, Gao H, Qiu L, Wang C, Gao Y, Zhang S, Xue X, Chen L. A survey of data-driven and knowledge-aware explainable ai. IEEE Trans Knowl Data Eng. 2022;34(1):29–49.
  2. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv. 2014.
  3. Sarkar S, Weyde T, d’Avila Garcez AS, Slabaugh GG, Dragicevic S, Percy C. Accuracy and interpretability trade-offs in machine learning applied to safer gambling. In: CoCo@NIPS. 2016.
  4. Green D, Kern H. Modeling heterogeneous treatment effects in large-scale experiments using Bayesian additive regression trees. Iowa City: The Annual Summer Meeting of the Society of Political Methodology; 2010.
  5. Su J, Vargas D, Sakurai K. One pixel attack for fooling deep neural networks. IEEE Trans Evolut Comput. 2017.
  6. Sadiku M, Shadare A, Musa S, Akujuobi C, Perry R. Data visualization. Int J Eng Res Adv Technol (IJERAT). 2016;12:2454–6135.
  7. Chang J, Boyd-Graber J, Gerrish S, Wang C, Blei D. Reading tea leaves: how humans interpret topic models. vol 32. 2009. p. 288–96.
  8. Ballav A, Ghosh M. Human factors of human machine interaction: Analyzing future trends through the past and the present. Int J Res. 2017;4:138–44.
  9. Xiong W, Fan H, Ma L, Wang C. Challenges of human—machine collaboration in risky decision-making. Front Eng Manag. 2022;9(1):89–103. · DOI · Paper page

Cited by

See all 2,495

Metadata

Authors
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A.
Affiliations
  • KIIT University (IN)
  • Birla Institute of Technology and Science, Pilani (IN)
  • Jaypee Institute of Information Technology (IN)
  • Duke Kunshan University (CN)
  • Sapienza University of Rome (IT)
  • Istituto Nazionale di Fisica Nucleare, Sezione di Roma I (IT)
  • Nottingham Trent University (GB)
  • Edinburgh Napier University (GB)
Corresponding author
  • Hassija, Vikas
  • Chamola, Vinay
  • Singal, Abhinandan
  • Goel, Divyansh
  • Huang, Kaizhu
  • Scardapane, Simone
  • Spinelli, Indro
  • Mahmud, Mufti
  • Hussain, Amir
Venue
Cognitive Computation
Published
2023-08-24
Volume
16
Issue
1
Pages
45-74
ISSN
  • 1866-9956 (ISSN-L)
  • 1866-9964
OpenAlex
W4386142022
Cited by
2,495
References
139
Citations by year
2026: 1109 · 2025: 1030 · 2024: 338 · 2023: 18
FWCI
177.44
Type
journal article
Language
en
Open access
Licence
Locations
  • Cognitive Computation (publishedVersion, cc-by)
  • IRIS Research product catalog (Sapienza University of Rome) (submittedVersion, other-oa)
  • Edinburgh Napier Research Repository (Edinburgh Napier University) (submittedVersion, other-oa)
Funders
  • European Commission (618615-EPP-1-2020-1-UKEPPKA2-CBHEJP, 2020-1-UK01-KA201-079167)
  • Erasmus+ (618615-EPP-1-2020-1-UK-EPPKA2-CBHE-JP, 2020-1-UK01-KA201-079167)
  • CHIST-ERA (CHIST-ERA-19-XAI-009)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
explainable AI, black-box models, model interpretability, model transparency, decision explanation, machine learning model evaluation, deep learning model interpretability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence. https://app.cochat.ai/papers/10.1007/s12559-023-10179-8
The paper
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024). Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence. Cognitive Computation. https://doi.org/10.1007/s12559-023-10179-8

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.