Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index; 9 of the 15 sources its claims cite were checked for support.

Study on the Helpfulness of Explainable Artificial Intelligence

Open access
Labarta, Tobias, Kulicheva, Elizaveta, Froelian, Ronja, Geißler, Christian, Melman, Xenia, von Klitzing, Julian · 2024 · Communications in Computer and Information Science

Abstract

Explainable Artificial Intelligence (XAI) is essential for building advanced machine learning-powered applications, especially in critical domains such as medical diagnostics or autonomous driving. Legal, business, and ethical requirements motivate using effective XAI, but the increasing number of different methods makes it challenging to pick the right ones. Further, as explanations are highly context-dependent, measuring the effectiveness of XAI methods without users can only reveal a limited amount of information, excluding human factors such as the ability to understand it. We propose to evaluate XAI methods via the user's ability to successfully perform a proxy task, designed such that a good performance is an indicator for the explanation to provide helpful information. In other words, we address the helpfulness of XAI for human decision-making. Further, a user study on state-of-the-art methods was conducted, showing differences in their ability to generate trust and skepticism and the ability to judge the rightfulness of an AI decision correctly. Based on the results, we highly recommend using and extending this approach for more objective-based human-centered user studies to measure XAI performance in an end-to-end fashion.

Overview

CoChat Summary

TL;DR

The paper proposes an objective, human-centered proxy-task methodology for evaluating XAI by measuring whether explanations help users judge, trust, and question AI image-classification decisions. In a user study with 139 participants and six XAI methods, Confidence Scores gave the highest accuracy, while no single method performed best across accuracy, sensitivity, and specificity.

Key findings

  • Mean accuracy in judging whether the model was correct was highest for Confidence Scores (≈0.698), followed by GradCAM (≈0.603), LRP (≈0.583), SHAP (≈0.558), LIME (≈0.545), and Integrated Gradients (≈0.532); all were significantly above the 0.5 random baseline, and only Confidence Scores was significantly higher than all other methods.
  • For sensitivity (trusting correct AI decisions), GradCAM had the highest mean (≈0.784), followed by Confidence Scores (≈0.752), LRP (≈0.748), LIME (≈0.590), Integrated Gradients (≈0.561), and SHAP (≈0.335); only GradCAM had a large positive effect.
  • For specificity (questioning incorrect AI decisions), SHAP had the highest mean (≈0.781), followed by Confidence Scores (≈0.644), Integrated Gradients (≈0.504), LIME (=0.5), GradCAM (≈0.421), and LRP (≈0.417); SHAP helped users most to question an AI decision with a strong effect.
  • The paper concludes that no explanation method overperformed in all cases and recommends a diverse, interactive framework of explanation methods rather than relying on a single monolithic approach.

Methods

  • The authors designed an objective proxy task: participants saw an input image, the ground-truth label, and one local post-hoc XAI explanation, but not the model’s prediction or whether it was correct; they had to judge whether the model predicted correctly.
  • The user study used two models (AlexNet and VGG16) on the ImageNetV2-matched-frequency dataset and compared six XAI methods: LRP, GradCAM, LIME, SHAP, Integrated Gradients, and Confidence Scores.
  • 139 participants completed the survey; each participant answered four questions per XAI method. Responses were analyzed as a binary confusion matrix, with accuracy, sensitivity, and specificity, one-sample t-tests against a 0.5 baseline, Cohen’s d effect sizes, and paired t-tests for pairwise method comparisons (α = 0.05).

Limitations

  • The paper states that the scenario was relatively staged because users did not know the model decision, whereas real users usually know the decision but may not know whether it is correct; it also notes the method is costly because it requires a user study.
  • The paper states that potential model bias could not be fully excluded because there was no documented list of identified model biases for VGG16 or AlexNet on ImageNet, and some participants lacked domain knowledge for certain ImageNet classes.
  • The paper states possible study-population bias because the survey was shared in a university context without demographic selection criteria; no statistically significant impact was found for tracked demographics, but untracked demographic criteria could still bias results. The text also notes that heuristic explanations like Confidence Scores may oversimplify and lead to overreliance.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/978-3-031-63803-9_16

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Overview of the objective methodology for evaluating XAI. It starts with the classification of an image. The model makes a prediction hidden from the user, that will be checked by its correctness in the background. Then, a local posthoc XAI method is applied to generate an explanation that will be presented to the user, together with the input image and ground truth. Based on this information, the user has to predict the model prediction.
Fig. 1 Overview of the objective methodology for evaluating XAI. It starts with the classification of an image. The model makes a prediction hidden from the user, that will be checked by its correctness in the background. Then, a local posthoc XAI method is applied to generate an explanation that will be presented to the user, together with the input image and ground truth. Based on this information, the user has to predict the model prediction.CC-BY · From the PDF
Image 2
CC-BY · From the PDF
Fig. 2 Example Survey Question
Fig. 2 Example Survey QuestionCC-BY · From the PDF
Fig. 3 The mean accuracy of participants in determining the correctness of the AI's predictions varied by XAI method used. Confidence Scores consistently outperformed all other methods, while the remaining methods showed minimal to no differences in results.
Fig. 3 The mean accuracy of participants in determining the correctness of the AI's predictions varied by XAI method used. Confidence Scores consistently outperformed all other methods, while the remaining methods showed minimal to no differences in results.CC-BY · From the PDF
Image 5
CC-BY · From the PDF
Fig. 5 Right prediction based on a wrong feature
Fig. 5 Right prediction based on a wrong featureCC-BY · From the PDF
Image 7
CC-BY · From the PDF
Image 8
CC-BY · From the PDF
Image 9
CC-BY · From the PDF
Fig. A.3: Difference in accuracy between VGG16 and AlexNet
Fig. A.3: Difference in accuracy between VGG16 and AlexNetCC-BY · From the PDF
Fig. A.4: Summary of means for accuracy, sensitivity, and specificity for all examined XAI methods compared to the random baseline.
Fig. A.4: Summary of means for accuracy, sensitivity, and specificity for all examined XAI methods compared to the random baseline.CC-BY · From the PDF

Verification

Claims the paper makes

5 Valid · 2 Not valid · 4 Not checked · 70 Own results and interpretations

47 more claims

Create a free account to see all 81 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 19
  • Probably not a substantial claim. 14

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

5 of 6 · the rest aren’t loaded

1 more finding

Create a free account to see every finding of this check.

See all findings — free account

Citations

Cites

Show all 39 references

Cited by

See all 8

Metadata

Authors
Labarta, T., Kulicheva, E., Froelian, R., Geißler, C., Melman, X., & von Klitzing, J.
Affiliations
  • Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
  • Technische Universität Berlin (DE)
Corresponding author
Labarta, Tobias
Venue
Communications in Computer and Information Science
Published
2024
Pages
294-312
ISSN
  • 1865-0929 (ISSN-L)
  • 1865-0937
OpenAlex
W4400439128
Cited by
8
References
39
Citations by year
2026: 1 · 2025: 7
FWCI
2.35
Type
conference paper
Language
en
Open access
Licence
Locations
  • Communications in computer and information science (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
Keywords
explainable AI, XAI evaluation, proxy tasks, human decision-making, user-centered evaluation, human-centered AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Study on the Helpfulness of Explainable Artificial Intelligence. https://app.cochat.ai/papers/10.1007/978-3-031-63803-9_16
The paper
Labarta, T., Kulicheva, E., Froelian, R., Geißler, C., Melman, X., & von Klitzing, J. (2024). Study on the Helpfulness of Explainable Artificial Intelligence. Communications in Computer and Information Science. https://doi.org/10.1007/978-3-031-63803-9_16

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.