The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making
Open accessAbstract
This paper contributes to the growing literature in empirical evaluation of explainable AI (XAI) methods by presenting a comparison on the effects of a set of established XAI methods in AI-assisted decision making. Specifically, based on our review of previous literature, we highlight three desirable properties that ideal AI explanations should satisfy—improve people’s understanding of the AI model, help people recognize the model uncertainty, and support people’s calibrated trust in the model. Through randomized controlled experiments, we evaluate whether four types of common model-agnostic explainable AI methods satisfy these properties on two types of decision making tasks where people perceive themselves as having different levels of domain expertise in (i.e., recidivism prediction and forest cover prediction). Our results show that the effects of AI explanations are largely different on decision making tasks where people have varying levels of domain expertise in, and many AI explanations do not satisfy any of the desirable properties for tasks that people have little domain expertise in. Further, for decision making tasks that people are more knowledgeable, feature contribution explanation is shown to satisfy more desiderata of AI explanations, while the explanation that is considered to resemble how human explain decisions (i.e., counterfactual explanation) does not seem to improve calibrated trust. We conclude by discussing the implications of our study for improving the design of XAI methods to better support human decision making.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, the previous research discusses the issues of these feature-based explanations on overtrust in AI [29, 37, 64].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [64]
In the citing paper
“Overall, our findings indicate user's overreliance on 'wrong' AI outputs, which follows the previous studies that describe salient feature explanations increases user's overreliance on the AI model [6, 64].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [64]
In the citing paper
“Other works that utilize AI/ML models focused on tasks that do not require domain experts, such as judges or clinicians [5, 38, 64].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [64]
In the citing paper
“Wang and Yin [64] conducted the randomized experiment using four types of common model-agnostic explainable AI methods and discussed the effect of AI explanations could be largely different where people have varying levels of domain expertise.”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [64]
In the citing paper
“However, across many domains, empirical studies of explanations have produced mixed results [4,11,49,53,58,68,82,87], with some suggesting that explanations increase decisionmakers' tendency to rely on AI predictions even when the AI system is wrong [4,68,82,87]-a phenomenon often referred to as overreliance [10,78].”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [82]
16 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,411
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 4×
“For example, sophisticated techniques such as LIME [60] and SHAP [48] have been used to illustrate how much each feature contributes to a model’s final prediction.”
MethodInfluential"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · cited 5×
“For example, sophisticated techniques such as LIME [60] and SHAP [48] have been used to illustrate how much each feature contributes to a model’s nal prediction.”
MethodInfluential(2011) · Cited by 14,317
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · cited 1×
“…interpretable models such as generalized additive models and decision sets [15, 34, 43, 66], as well as visualizing what deep neural network has learned in its intermediate layers and how its predictions are affected by different part of the inputs (e.g., through saliency map) [37, 61, 68].”
BackgroundPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 1×
“havior of the entire AI model is categorized as global explanations, while methods that provide reasons for specic model predictions are categorized as local explanations [2, 24, 26].”
MethodLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 1×
“Typical methods include ask people to rank the input data features based on their inuence to overall predictions [18, 33], to indicate the direction of change in the model’s prediction when a feature’s value is altered [16, 18, 29], to simulate the model’s predictions [16, 24, 40, 47, 59], to answer “what-if” question…”
MethodDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 1×
“…interpretable models such as generalized additive models and decision sets [15, 34, 43, 66], as well as visualizing what deep neural network has learned in its intermediate layers and how its predictions are affected by different part of the inputs (e.g., through saliency map) [37, 61, 68].”
Background
Research works printed but not linked: 7
- Dennis Collaris Leo M Vink and Jarke J van Wijk. 2018. Instance-level explanations for fraud detection: A case study. arXiv preprint arXiv:1806.07129(2018). Dennis Collaris Leo M Vink and Jarke J van Wijk. 2018. Instance-level explanations for fraud detection: A case study. arXiv preprint arXiv:1806.07129(2018). · arXiv
- 10.2139/ssrn.3465622 · DOI · Paper page
- Isaac Lage Emily Chen Jeffrey He Menaka Narayanan Been Kim Sam Gershman and Finale Doshi-Velez. 2019. An evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1902.00006(2019). Isaac Lage Emily Chen Jeffrey He Menaka Narayanan Been Kim Sam Gershman and Finale Doshi-Velez. 2019. An evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1902.00006(2019). · arXiv
- Jeff Larson , Surya Mattu , Lauren Kirchner , and Julia Angwin . 2016. How we analyzed the COMPAS recidivism algorithm. ProPublica (5 2016) 9, 1 ( 2016 ). Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. 2016. How we analyzed the COMPAS recidivism algorithm. ProPublica (5 2016) 9, 1 (2016).
- Maranda McBride and Shona Morgan . 2010. Trust calibration for automated decision aids . Institute for Homeland Security Solutions.[Online] . Available: https://www. ihssnc. org/portals/0/Documents/VIMSDocuments/McBride_Research_Brief. pdf(2010). Maranda McBride and Shona Morgan. 2010. Trust calibration for automated decision aids. Institute for Homeland Security Solutions.[Online]. Available: https://www. ihssnc. org/portals/0/Documents/VIMSDocuments/McBride_Research_Brief. pdf(2010).
- Forough Poursabzi-Sangdeh Daniel G Goldstein Jake M Hofman Jennifer Wortman Vaughan and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810(2018). Forough Poursabzi-Sangdeh Daniel G Goldstein Jake M Hofman Jennifer Wortman Vaughan and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810(2018). · arXiv
- Jennifer Wortman Vaughan and Hanna Wallach . 2020. A Human-Centered Agenda for Intelligible Machine Learning. Machines We Trust: Getting Along with Artificial Intelligence ( 2020 ). Jennifer Wortman Vaughan and Hanna Wallach. 2020. A Human-Centered Agenda for Intelligible Machine Learning. Machines We Trust: Getting Along with Artificial Intelligence (2020).
Cited by
Explanations Can Reduce Overreliance on AI Systems During Decision-Making
Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., & Krishna, R. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579605 · Cited by 402
“Future work should explore our theory on high-stakes tasks such as recidivism prediction [92], where the consequences of a wrong decision are much higher.”
BackgroundChen, H., Gomez, C., Huang, C.-M., & Unberath, M. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00699-2 · Cited by 300
“…images, plots) [20], channel ( e. g. graphical interface) [21], and interactivity ( e. g. whether users can provide feedback or refine model outputs) can drastically affect users’ experience and performance [22–24], and therefore, must be aligned with the goal of transparent system development.”
BackgroundTowards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278
“The agreement rate of users with the model decisions is commonly used [9, 11, 12, 25] as a measure of observed trust.”
BackgroundInfluentialGomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 209
“…et al., 2022; Schemmer et al., 2023; Mackeprang et al., 2019); image labeling (Cau et al., 2023) 8 Law Recidivism prediction (Grgic-Hlaca et al., 2019; Wang and Yin, 2021; Alufaisan et al., 2021; Liu et al., 2021); criminal referral decision (Zytek et al., 2022); penal sentence prediction (Kahr…”
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204
“The most commonly used metric is forward simulation [1, 2, 15, 21, 24, 36, 48, 80, 81, 83, 87, 105, 111], by asking participants to simulate a model’s predictions on unseen instances. and what design impacts the perception is an active research area.”
BackgroundMethodInfluential
Metadata
- Authors
- Wang, X., & Yin, M.
- Affiliations
- Purdue University West Lafayette (US)
- ORCID
- Venue
- 26th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2021-04-14
- Pages
- 318-328
- OpenAlex
- W3156106752
- Semantic Scholar
- c79ba865574cb9635dbca135cec99ee56125778c
- Cited by
- 320
- References
- 66
- Citations by year
- 2026: 63 · 2025: 76 · 2024: 82 · 2023: 56 · 2022: 34 · 2021: 9
- FWCI
- 20.90
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion, cc-by)
- Funders
- National Science Foundation (1850335)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- explainable AI, AI-assisted decision-making, model-agnostic explanation, counterfactual explanations, model uncertainty, domain knowledge, recidivism prediction, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making. https://app.cochat.ai/papers/10.1145/3397481.3450650
- The paper
- Wang, X., & Yin, M. (2021). Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making. 26th International Conference on Intelligent User Interfaces. https://doi.org/10.1145/3397481.3450650
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.