The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index; 9 of the 15 sources its claims cite were checked for support.
Study on the Helpfulness of Explainable Artificial Intelligence
Open accessAbstract
Explainable Artificial Intelligence (XAI) is essential for building advanced machine learning-powered applications, especially in critical domains such as medical diagnostics or autonomous driving. Legal, business, and ethical requirements motivate using effective XAI, but the increasing number of different methods makes it challenging to pick the right ones. Further, as explanations are highly context-dependent, measuring the effectiveness of XAI methods without users can only reveal a limited amount of information, excluding human factors such as the ability to understand it. We propose to evaluate XAI methods via the user's ability to successfully perform a proxy task, designed such that a good performance is an indicator for the explanation to provide helpful information. In other words, we address the helpfulness of XAI for human decision-making. Further, a user study on state-of-the-art methods was conducted, showing differences in their ability to generate trust and skepticism and the ability to judge the rightfulness of an AI decision correctly. Based on the results, we highly recommend using and extending this approach for more objective-based human-centered user studies to measure XAI performance in an end-to-end fashion.
Overview
CoChat Summary
TL;DR
The paper proposes an objective, human-centered proxy-task methodology for evaluating XAI by measuring whether explanations help users judge, trust, and question AI image-classification decisions. In a user study with 139 participants and six XAI methods, Confidence Scores gave the highest accuracy, while no single method performed best across accuracy, sensitivity, and specificity.
Key findings
- Mean accuracy in judging whether the model was correct was highest for Confidence Scores (≈0.698), followed by GradCAM (≈0.603), LRP (≈0.583), SHAP (≈0.558), LIME (≈0.545), and Integrated Gradients (≈0.532); all were significantly above the 0.5 random baseline, and only Confidence Scores was significantly higher than all other methods.
- For sensitivity (trusting correct AI decisions), GradCAM had the highest mean (≈0.784), followed by Confidence Scores (≈0.752), LRP (≈0.748), LIME (≈0.590), Integrated Gradients (≈0.561), and SHAP (≈0.335); only GradCAM had a large positive effect.
- For specificity (questioning incorrect AI decisions), SHAP had the highest mean (≈0.781), followed by Confidence Scores (≈0.644), Integrated Gradients (≈0.504), LIME (=0.5), GradCAM (≈0.421), and LRP (≈0.417); SHAP helped users most to question an AI decision with a strong effect.
- The paper concludes that no explanation method overperformed in all cases and recommends a diverse, interactive framework of explanation methods rather than relying on a single monolithic approach.
Methods
- The authors designed an objective proxy task: participants saw an input image, the ground-truth label, and one local post-hoc XAI explanation, but not the model’s prediction or whether it was correct; they had to judge whether the model predicted correctly.
- The user study used two models (AlexNet and VGG16) on the ImageNetV2-matched-frequency dataset and compared six XAI methods: LRP, GradCAM, LIME, SHAP, Integrated Gradients, and Confidence Scores.
- 139 participants completed the survey; each participant answered four questions per XAI method. Responses were analyzed as a binary confusion matrix, with accuracy, sensitivity, and specificity, one-sample t-tests against a 0.5 baseline, Cohen’s d effect sizes, and paired t-tests for pairwise method comparisons (α = 0.05).
Limitations
- The paper states that the scenario was relatively staged because users did not know the model decision, whereas real users usually know the decision but may not know whether it is correct; it also notes the method is costly because it requires a user study.
- The paper states that potential model bias could not be fully excluded because there was no documented list of identified model biases for VGG16 or AlexNet on ImageNet, and some participants lacked domain knowledge for certain ImageNet classes.
- The paper states possible study-population bias because the survey was shared in a university context without demographic selection criteria; no statistically significant impact was found for tracked demographics, but untracked demographic criteria could still bias results. The text also notes that heuristic explanations like Confidence Scores may oversimplify and lead to overreliance.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures











Verification
Claims the paper makes
5 Valid · 2 Not valid · 4 Not checked · 70 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The only approach combining human involvement with quantitative methods was by Achtibat et al. [1], where users were asked to assess if the model relied on artifacts in the prediction process.”
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > 3 An objective Methodology for evaluating XAI
What the sources say
- Partly supportedcites [1] Achtibat, R., et al.: From attribution maps to human-understandable explanations through concept relevance propagation. Nature Mach. Intell. 5(9), 1006–1019 (2023) sourceExperimental — accuracy not yet measured
The Human evaluation study says participants assessed whether a known border artefact influenced model predictions, and it reports quantitative accuracy and detection rates. This supports the description of Achtibat et al.'s approach, but the paper does not establish that it was the only approach combining human involvement with quantitative methods. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“This approach is a very interesting step in the right direction for quantitative XAI user-studies, but also focused on a sub-task, namely detection of model bias.”
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > 3 An objective Methodology for evaluating XAI
What the sources say
- Supportedcites [1] Achtibat, R., et al.: From attribution maps to human-understandable explanations through concept relevance propagation. Nature Mach. Intell. 5(9), 1006–1019 (2023) source
The Human evaluation study section describes a user-study task in which participants judged whether a known data artefact—a black border—influenced a model’s prediction. That supports the characterization of the study as focused on the sub-task of detecting artefact-driven model bias. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Approaches without human feedback, for example, because they use an artificial benchmark with defined ground truth for the desired explanations [18,32,36], suffer from a direct proof of being transferable to real-world use cases.”
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > 2.1 Approaches for measuring explainability
What the sources say
- Could not verifycites [18] Hedström, A., et al.: Quantus: an explainable ai toolkit for responsible evaluation of neural network explanations. arXiv preprint arXiv:2202.06861 (2022) sourceExperimental — accuracy not yet measured
The Introduction (pp. 1–2) discusses missing ground-truth explanations and limitations of both human and quantitative evaluation. Library Design (p. 4) says evaluation must be calibrated to its application and stakeholders, but our copy does not say that approaches without human feedback lack direct proof of transfer to real-world use cases. Searches for artificial benchmarks and real-world transfer found no passage making that link. (LLM, conf=0.73)
- Partly supportedcites [32] Müller, H., Holzinger, A.: Kandinsky patterns. Artif. Intell. 300, 103546 (2021) sourceExperimental — accuracy not yet measured
The Introduction describes Kandinsky Patterns as synthetic test data with defined ground truth, supporting the citation as an example of an artificial benchmark. Neither that passage nor the Data sets and challenges section says that approaches without human feedback lack direct proof of transferability to real-world use cases; the paper also emphasizes that its patterns are distinguishable by human observers. (LLM, conf=0.78)
- Supportedcites [36] Salewski, L., Koepke, A.S., Lensch, H.P.A., Akata, Z.: CLEVR-X: a visual reasoning dataset for natural language explanations. In: Holzinger, A., Goebel, R., Fong, R., Moon, T., Müller, K.-R., Samek, W. (eds.) xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pp. 69–88. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-031-04083-2_5 sourceExperimental — accuracy not yet measured
CLEVR-X describes a synthetic visual-reasoning benchmark whose explanations are constructed from scene graphs and used to evaluate explanation-generation methods. That supports citing it as an example of an artificial benchmark with defined ground truth, though the paper does not explicitly state the broader conclusion about transfer to real-world use cases. CLEVR-X also reports a human study of whether people can understand its explanations, so describing the paper as having no human feedback would be inaccurate; that study does not itself demonstrate real-world transfer. The unscoped match reproduces the citing sentence in the citing paper, not an independent source for the claim. (LLM, conf=0.80)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The mean accuracy of participants in determining the correctness of the AI's predictions varied by XAI method used.”
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > XAI Method
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The highest mean over all participants was achieved by Grad-CAM with ≈ 0.784, followed by Confidence Scores with ≈ 0.752, LRP ≈ 0.748, LIME ≈ 0.590, Integrated Gradients ≈ 0.561 and SHAP ≈ 0.335.”
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > 4.1 Questionnaire responses
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
47 more claims
Create a free account to see all 81 claims and every source verdict.
- It may state two separate claims as one. 19
- Probably not a substantial claim. 14
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > Disclosure of Interests
“SHAP performs best for sensitivity and worst of all methods for specificity.”
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > 4.1 Questionnaire responses
“The highest mean over all participants was achieved by Grad-CAM with ≈ 0.784, followed by Confidence Scores with ≈ 0.752, LRP ≈ 0.748, LIME ≈ 0.590, Integrated Gradients ≈ 0.561 and SHAP ≈ 0.335.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > Evaluation Metrics
“The sensitivity is also known as true positive rate T P R = T P T P +F P .”
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > Disclosure of Interests
“| model output was correct | True Positive (TP) | False Negative (FN) |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > Disclosure of Interests
“Confidence Scores perform the best in both measures, besides SHAP.”
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > 4.1 Questionnaire responses
“The highest mean over all participants was achieved by Grad-CAM with ≈ 0.784, followed by Confidence Scores with ≈ 0.752”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > Evaluation Metrics
“Accuracy = T P +T N T P +T N +F P +T N expresses the participant's ability to correctly judge model predictions.”
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > Disclosure of Interests
“| model output was correct | True Positive (TP) | False Negative (FN) | | model output was false | False Positive (FP) | True Negative (TN) |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > Evaluation Metrics
“The specificity is also known as the true negative rate T N R = T N T N +F N .”
§ Study on the Helpfulness of Explainable Artificial Intelligence (XAI) > Disclosure of Interests
“| model output was false | False Positive (FP) | True Negative (TN) |”
1 more finding
Create a free account to see every finding of this check.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“To avoid model-specific biases, two different AI models, AlexNet [22] and VGG16 [38], were selected for classification.”
MethodStatistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · cited 1×
“Ribiero et al. [35] present an explanation method that is evaluated in this study, LIME .”
MethodFrom local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · cited 1×
“…Layer-wise relevance propagation ( LRP ) [2,31], Gradient class activation mapping ( GradCAM ) [7,10], Local In-terpretable Model-Agnostic Explanations ( LIME ) [15], SHapley Additive exPla-nations ( SHAP ) [28,29,37,40], Integrated Gradients ( Integrated Gradients ) [41] and Confidence Scores .”
Background“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524 · cited 1×
“Following these restrictions, six XAI methods were selected: Layer-wise relevance propagation ( LRP ) [2,31], Gradient class activation mapping ( GradCAM ) [7,10], Local In-terpretable Model-Agnostic Explanations ( LIME ) [15], SHapley Additive exPla-nations ( SHAP ) [28,29,37,40], Integrated…”
MethodAxiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087
Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks
Chattopadhay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00097 · Cited by 3,639 · cited 1×
“…these restrictions, six XAI methods were selected: Layer-wise relevance propagation ( LRP ) [2,31], Gradient class activation mapping ( GradCAM ) [7,10], Local In-terpretable Model-Agnostic Explanations ( LIME ) [15], SHapley Additive exPla-nations ( SHAP ) [28,29,37,40], Integrated Gradients (…”
Method
Cited by
Ansari, Z. A., Tripathi, M. M., & Ahmed, R. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00307-8 · Cited by 64
“Nonetheless, Grad-CAM is less comprehensible for non-experts, as it predominantly visualizes neural network activations, necessitating expertise for accurate contextualization [82].”
Autonomy by Design: Preserving Human Autonomy in AI Decision-Support
Buijsman, S., Carter, S. E., & Bermúdez, J. P. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00932-2 · Cited by 49
“They are also not able to better anticipate what the AI system will do for new inputs (Poursabzi-Sangdeh et al., 2021).”
Design for operator contestability: control over autonomous systems by introducing defeaters
Veluwenkamp, H., & Buijsman, S. (2025) · AI and Ethics · doi:10.1007/s43681-025-00657-0 · Cited by 9
Sass, R. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00948-8 · Cited by 4
“Indeed, Buijsman et al. (2025) reference one study in particular (Labarta et al., 2024), on which a variety of XAI methods were tested; only one method in the study improved user accuracy on a classification task, but another improved specificity.”
A review on empirical studies in explainable artificial intelligence
Jahn, T., Hühn, P., & Reichert, M. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11595-6 · Cited by 2
Metadata
- Authors
- Labarta, T., Kulicheva, E., Froelian, R., Geißler, C., Melman, X., & von Klitzing, J.
- Affiliations
- Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
- Technische Universität Berlin (DE)
- Corresponding author
- Labarta, Tobias
- ORCID
- Geißler, Christian
- Venue
- Communications in Computer and Information Science
- Publisher
- Springer Nature Switzerland
- Published
- 2024
- Pages
- 294-312
- ISSN
- 1865-0929 (ISSN-L)
- 1865-0937
- OpenAlex
- W4400439128
- Semantic Scholar
- 0eb798d103b98e83af1321d15062283bde93d47c
- Cited by
- 8
- References
- 39
- Citations by year
- 2026: 1 · 2025: 7
- FWCI
- 2.35
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Communications in computer and information science (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- explainable AI, XAI evaluation, proxy tasks, human decision-making, user-centered evaluation, human-centered AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Study on the Helpfulness of Explainable Artificial Intelligence. https://app.cochat.ai/papers/10.1007/978-3-031-63803-9_16
- The paper
- Labarta, T., Kulicheva, E., Froelian, R., Geißler, C., Melman, X., & von Klitzing, J. (2024). Study on the Helpfulness of Explainable Artificial Intelligence. Communications in Computer and Information Science. https://doi.org/10.1007/978-3-031-63803-9_16
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.