Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making
Open accessThe DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
This paper contributes to the growing literature in empirical evaluation of explainable AI (XAI) methods by presenting a comparison on the effects of a set of established XAI methods in AI-assisted decision making. Specifically, based on our review of previous literature, we highlight three desirable properties that ideal AI explanations should satisfy—improve people’s understanding of the AI model, help people recognize the model uncertainty, and support people’s calibrated trust in the model. Through randomized controlled experiments, we evaluate whether four types of common model-agnostic explainable AI methods satisfy these properties on two types of decision making tasks where people perceive themselves as having different levels of domain expertise in (i.e., recidivism prediction and forest cover prediction). Our results show that the effects of AI explanations are largely different on decision making tasks where people have varying levels of domain expertise in, and many AI explanations do not satisfy any of the desirable properties for tasks that people have little domain expertise in. Further, for decision making tasks that people are more knowledgeable, feature contribution explanation is shown to satisfy more desiderata of AI explanations, while the explanation that is considered to resemble how human explain decisions (i.e., counterfactual explanation) does not seem to improve calibrated trust. We conclude by discussing the implications of our study for improving the design of XAI methods to better support human decision making.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, the previous research discusses the issues of these feature-based explanations on overtrust in AI [29, 37, 64].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [64]
In the citing paper
“Overall, our findings indicate user's overreliance on 'wrong' AI outputs, which follows the previous studies that describe salient feature explanations increases user's overreliance on the AI model [6, 64].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [64]
In the citing paper
“Other works that utilize AI/ML models focused on tasks that do not require domain experts, such as judges or clinicians [5, 38, 64].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [64]
In the citing paper
“Wang and Yin [64] conducted the randomized experiment using four types of common model-agnostic explainable AI methods and discussed the effect of AI explanations could be largely different where people have varying levels of domain expertise.”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [64]
In the citing paper
“However, across many domains, empirical studies of explanations have produced mixed results [4,11,49,53,58,68,82,87], with some suggesting that explanations increase decisionmakers' tendency to rely on AI predictions even when the AI system is wrong [4,68,82,87]-a phenomenon often referred to as overreliance [10,78].”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [82]
16 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,370
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 4×
“For example, sophisticated techniques such as LIME [60] and SHAP [48] have been used to illustrate how much each feature contributes to a model’s final prediction.”
MethodInfluential"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · cited 5×
“For example, sophisticated techniques such as LIME [60] and SHAP [48] have been used to illustrate how much each feature contributes to a model’s nal prediction.”
MethodInfluential(2011) · Cited by 14,317
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“the training dataset or counterfactual examples to explain model predictions [36, 39, 65], and conducting model distillation [10, 32].”
BackgroundGauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · cited 1×
“…interpretable models such as generalized additive models and decision sets [15, 34, 43, 66], as well as visualizing what deep neural network has learned in its intermediate layers and how its predictions are affected by different part of the inputs (e.g., through saliency map) [37, 61, 68].”
BackgroundPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 1×
“havior of the entire AI model is categorized as global explanations, while methods that provide reasons for specic model predictions are categorized as local explanations [2, 24, 26].”
MethodLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 1×
“Typical methods include ask people to rank the input data features based on their inuence to overall predictions [18, 33], to indicate the direction of change in the model’s prediction when a feature’s value is altered [16, 18, 29], to simulate the model’s predictions [16, 24, 40, 47, 59], to answer “what-if” question…”
MethodDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 1×
“…interpretable models such as generalized additive models and decision sets [15, 34, 43, 66], as well as visualizing what deep neural network has learned in its intermediate layers and how its predictions are affected by different part of the inputs (e.g., through saliency map) [37, 61, 68].”
Background
Research works printed but not linked: 6
- Dennis Collaris Leo M Vink and Jarke J van Wijk. 2018. Instance-level explanations for fraud detection: A case study. arXiv preprint arXiv:1806.07129(2018). Dennis Collaris Leo M Vink and Jarke J van Wijk. 2018. Instance-level explanations for fraud detection: A case study. arXiv preprint arXiv:1806.07129(2018). · arXiv
- 10.2139/ssrn.3465622 · DOI · Paper page
- Isaac Lage Emily Chen Jeffrey He Menaka Narayanan Been Kim Sam Gershman and Finale Doshi-Velez. 2019. An evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1902.00006(2019). Isaac Lage Emily Chen Jeffrey He Menaka Narayanan Been Kim Sam Gershman and Finale Doshi-Velez. 2019. An evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1902.00006(2019). · arXiv
- Jeff Larson , Surya Mattu , Lauren Kirchner , and Julia Angwin . 2016. How we analyzed the COMPAS recidivism algorithm. ProPublica (5 2016) 9, 1 ( 2016 ). Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. 2016. How we analyzed the COMPAS recidivism algorithm. ProPublica (5 2016) 9, 1 (2016).
- Maranda McBride and Shona Morgan . 2010. Trust calibration for automated decision aids . Institute for Homeland Security Solutions.[Online] . Available: https://www. ihssnc. org/portals/0/Documents/VIMSDocuments/McBride_Research_Brief. pdf(2010). Maranda McBride and Shona Morgan. 2010. Trust calibration for automated decision aids. Institute for Homeland Security Solutions.[Online]. Available: https://www. ihssnc. org/portals/0/Documents/VIMSDocuments/McBride_Research_Brief. pdf(2010).
- Jennifer Wortman Vaughan and Hanna Wallach . 2020. A Human-Centered Agenda for Intelligible Machine Learning. Machines We Trust: Getting Along with Artificial Intelligence ( 2020 ). Jennifer Wortman Vaughan and Hanna Wallach. 2020. A Human-Centered Agenda for Intelligible Machine Learning. Machines We Trust: Getting Along with Artificial Intelligence (2020).
Cited by
Items 101–200 of 332
Reassuring, Misleading, Debunking: Comparing Effects of XAI Methods on Human Decisions
Humer, C., Hinterreiter, A., Leichtmann, B., Mara, M., & Streit, M. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3665647 · Cited by 31
“The risk of negative consequences due to wrong AI recommendations, for example, is relatively low in age prediction from images [15] or in the prediction of forest cover types [82]—contexts in which the effects of XAI have recently been studied.”
InfluentialInterpretable Directed Diversity: Leveraging Model Explanations for Iterative Crowd Ideation
Wang, Y., Venkatesh, P., & Lim, B. Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517551 · Cited by 21
“Prior research has shown that XAI could increase users’ trust and understanding of AI system [7,83].”
BackgroundMethodSecure Explainable-AI Approach for Brake Faults Prediction in Heavy Transport
Khan, M. A., Khan, M., Dawood, H., Dawood, H., & Daud, A. Y. (2024) · IEEE Access · doi:10.1109/access.2024.3444907 · Cited by 29
“This is accomplished through discovering internal processes of AI algorithms [8], [9].”
Evaluating explainable social choice-based aggregation strategies for group recommendation
Barile, F., Draws, T., Inel, O., Rieger, A., Najafian, S., Fard, A. E., Hada, R., & Tintarev, N. (2023) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-023-09363-0 · Cited by 28
SenseMate: An Accessible and Beginner-Friendly Human-AI Platform for Qualitative Data Analysis
Overney, C., Saldías-Fuentes, B., Dimitrakopoulou, D., & Roy, D. K. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645194 · Cited by 46
The impact of AI explanations on clinicians’ trust and diagnostic accuracy in breast cancer
Rezaeian, O., Asan, O., & Bayrak, A. E. (2025) · Applied Ergonomics · doi:10.1016/j.apergo.2025.104577 · Cited by 30
Stepin, I., Alonso, J. M., Catalá, A., & Pereira-Fariña, M. (2022) · Information Sciences · doi:10.1016/j.ins.2022.10.098 · Cited by 26
Kim, T., & Im, I. (2024) · Information & Management · doi:10.1016/j.im.2024.104061 · Cited by 23
Measuring the effect of mental workload and explanations on appropriate AI reliance using EEG
Zhang, Z. T., Argın, S. K., Bilen, M. B., Urgun, D., Deniz, S. M., Liu, Y., & Hassib, M. E. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2431055 · Cited by 24
“One possible factor could be the difficulty of a decision, as overreliance and the ineffectiveness of explanations to mitigate it have been particularly observed in tasks that are challenging for humans [12,13,16,17].”
From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis
Li, Z., Zhu, H., Lu, Z., Xiao, Z., & Yin, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713133 · Cited by 38
Self-citationDo Expressions Change Decisions? Exploring the Impact of AI's Explanation Tone on Decision-Making
Okoso, A., Yang, M., & Baba, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713744 · Cited by 32
“Model-related information includes model accuracy [37, 69], global feature importance [16, 68], and the data used for training [7].”
InfluentialBridging Generations using AI-Supported Co-Creative Activities
Kim, C. Y., Sato, A. J., White, N. T., Ho, H.-R., Lee, C. P., Hwang, Y., & Mutlu, B. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713718 · Cited by 35
“A pivotal approach to calibrating human trust is to convey AI’s capability to humans [8, 68, 83, 88].”
Eckhardt, S., Kühl, N., Dolata, M., & Schwabe, G. (2025) · ACM Computing Surveys · doi:10.1145/3776528 · Cited by 45
Lammert, O., Richter, B., Schütze, C., Thommes, K., & Wrede, B. (2024) · Frontiers in Behavioral Economics · doi:10.3389/frbhe.2024.1377075 · Cited by 30
To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems
He, G., Bharos, A., & Gadiraju, U. (2024) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3648188.3675130 · Cited by 24
IRIS: Interpretable Rubric-Informed Segmentation for Action Quality Assessment
Matsuyama, H., Kawaguchi, N., & Lim, B. Y. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584048 · Cited by 25
“Wang and Yin identified several important desiderata of XAI for non-experts [41] and Dodge et al.”
BackgroundWang, X., Chen, L., & Yin, M. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/343 · Cited by 27
“, 2020] and understand the AI model [Wang and Yin, 2021; Poursabzi-Sangdeh et al., 2021], and how their decision accuracy is affected by their usage of the AI-based decision aids [Lai and Tan, 2019; Bansal et al.”
BackgroundSelf-citationNavigating the landscape of concept-supported XAI: Challenges, innovations, and future directions
Khoozani, Z. S., Sabri, A. Q. M., Seng, W. C., Seera, M., & Eg, K. Y. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17666-y · Cited by 22
REX: Designing User-centered Repair and Explanations to Address Robot Failures
Lee, C. P., Praveena, P., & Mutlu, B. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661559 · Cited by 21
“Previous research has demonstrated that explanations help improve people’s understanding of AI models, recognition of uncertainty, and trust in the model [72].”
BackgroundMitigating Gender Stereotypes Toward AI Agents Through an eXplainable AI (XAI) Approach
Duan, W., McNeese, N. J., Freeman, G., & Li, L. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686969 · Cited by 24
“For instance, Wang and Yin [90] found that feature contribution, the explanation technique that indicates and quantifies the importance of input features and their positive or negative influence toward the outcome can raise participants’ awareness of AI decisions’ biases in a recidivism prediction…”
InfluentialOn Explainable and Adaptable Detection of Distributed Denial-of-Service Traffic
Feng, Y., Li, J., Sisodia, D., & Reiher, P. L. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2023.3301293 · Cited by 20
“…evaluate the explainability of our approach, we conducted a questionnaire survey, which is a formal and effective method in Human-Computer Interaction (HCI) research [59], to collect feedback from participants and assess their understanding of the system’s functionality and decision-making process.”
MethodMuijlwijk, H., Willemsen, M. C., Smyth, B., & IJsselsteijn, W. A. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645205 · Cited by 20
“Other work suggests that users with larger (self-reported) domain expertise typically compare the explanation of an AI with their own rationale, resulting in a more critical judgement on the AI compared to novices [25, 64].”
BackgroundKaur, H., Conrad, M. R., Rule, D., Lampe, C. A. C., & Gilbert, É. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637354 · Cited by 33
Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process
Zhang, Z. T., Feger, S. S., Dullenkopf, L., Liao, R., Süsslin, L. E., Liu, Y., & Butz, A. M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687024 · Cited by 30
Guided By AI: Navigating Trust, Bias, and Data Exploration in AI‐Guided Visual Analytics
Ha, S., Monadjemi, S., & Ottley, A. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15108 · Cited by 26
“It is important to consider the prior work showing that the quality of the explanations provided by the AI plays a pivotal role [LT19, WY21] in the users’ interactions with AI suggestions.”
InfluentialExplainable AI and stakes in medicine: A user study
Baron, S., Latham, A. J., & Varga, S. (2025) · Artificial Intelligence · doi:10.1016/j.artint.2025.104282 · Cited by 18
“This underscores why research into user perceptions of explanations is so vital, leading to a now burgeoning field [13,21,29,35,37,39,40,43,59,61,65,68,74,75].”
Combining uncertainty information with AI recommendations supports calibration with domain knowledge
Subramanian, H. V., Canfield, C., Shank, D. B., & Kinnison, M. (2023) · Journal of Risk Research · doi:10.1080/13669877.2023.2259406 · Cited by 17
“Most research on AI recommendations aims to calibrate trust, rather than confidence (Buçinca et al. 2020; Wang and Yin 2021; Zhang, Liao, and Bellamy 2020).”
BackgroundThe Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features
Goyal, N., Baumler, C., Nguyen, T. T., & Daumé, H. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645210 · Cited by 27
Information that matters: Exploring information needs of people affected by algorithmic decisions
Schmude, T., Koesten, L. M., Möller, T. B., & Tschiatschek, S. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103380 · Cited by 20
“Good explanations of AI systems are aligned with the information needs of the stakeholder audience [19, 34, 71, 73] and are a means to help them achieve their aims, which can depend on the stakeholder’s role [11, 56], AI literacy [65], and domain knowledge [97].”
BackgroundYun, S., & Lim, Y.-K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713347 · Cited by 34
“Previous studies have explored strategies to encourage users to move beyond passively accepting AI system explanations and instead critically evaluate AI reasoning through analytical thinking [3, 6, 34, 57, 68, 73].”
Tutul, A. A., Nirjhar, E. H., & Chaspari, T. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479926 · Cited by 16
“Wang & Yin conducted experiments on Amazon Mechanical Turk to understand the extent to which various explain-able AI methods and models can improve people’s understanding of the decision making task and help calibrate their trust to the AI [28].”
MethodResultHouston, A. D., Cosma, G., Turner, P., & Bennett, A. (2021) · Scientific Reports · doi:10.1038/s41598-021-03825-4 · Cited by 13
“However a recent evaluation of the effects of such measures found that whilst existing methods are successful in improving an end user’s understanding, minimal evidence supported the idea that they empower the end-user to trust a model appropriately 29 .”
BackgroundOn the Impact of Explanations on Understanding of Algorithmic Decision-Making
Schmude, T., Koesten, L. M., Möller, T. B., & Tschiatschek, S. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594054 · Cited by 31
“, input influence, sensitivity, counterfactuals, case-based, and white box explanations [14, 18, 51, 57, 61].”
BackgroundMethodInfluentialDebiased-CAM to mitigate image perturbations with faithful visual explanations of machine learning
Zhang, W., Dimiccoli, M., & Lim, B. Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517522 · Cited by 30
“User studies of model explanations aim to show that explanations can improve user understanding and trust [46, 60, 66, 91, 99].”
BackgroundThe Influences of Task Design on Crowdsourced Judgement: A Case Study of Recidivism Risk Evaluation
Duan, X., Ho, C.-J., & Yin, M. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512239 · Cited by 16
“The advance of AI technologies recently also makes it possible to provide AI-powered agents’ recommendations on judgement tasks to crowd workers as the feedback, which is also shown to increase the quality of the crowd’s judgement in many cases [27, 29, 45, 61].”
BackgroundSelf-citationAI-overdependence and human cognitive decline: Hazards, evidence, and mitigation strategies
Noorbehbahani, F., & Oyibo, K. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101102 · Cited by 12
Dissenting Explanations: Leveraging Disagreement to Reduce Model Overreliance
Reingold, O., Shen, J. H., & Talati, A. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30151 · Cited by 18
Fair Machine Guidance to Enhance Fair Decision Making in Biased People
Yang, M., Arai, H., Yamashita, N., & Baba, Y. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642627 · Cited by 19
“Research on human-AI decision making has focused on assessing the information that is useful for humans when using AI and making them trust it [32, 55, 60, 65, 69, 82, 84].”
BackgroundMethodInfluentialSalimzadeh, S., & Gadiraju, U. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3627043.3659567 · Cited by 19
Towards Understanding AI Delegation: The Role of Self-Efficacy and Visual Processing Ability
Westphal, M., Hemmer, P., Vössing, M., Schemmer, M., Vetter, S., & Satzger, G. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3696423 · Cited by 16
“The human makes the final decision and can, therefore, either accept or override the recommendation [85, 101].”
Waterschoot, C., Yera, R., Tintarev, N., & Barile, F. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3699682.3728345 · Cited by 18
“Per participant, these scores were summed up and divided by the number of tasks to obtain their accuracy rate: • Objective understanding (continuous): The accuracy rate of each participant’s answers to the objective understanding questionnaire (six questions in total).”
InfluentialEffects of Fairness and Explanation on Trust in Ethical AI
Angerschmid, A., Theuermann, K., Holzinger, A., Chen, F., & Zhou, J. J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-14463-9_4 · Cited by 19
Grgić-Hlača, N., Castelluccia, C., & Gummadi, K. P. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.21989 · Cited by 26
Brachman, M., Pan, Q., Do, H. J., Dugan, C., Chaudhary, A., Johnson, J. M., Rai, P., Chakraborti, T., Gschwind, T., Laredo, J. A., et al. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584088 · Cited by 11
“Mental model accuracy can also be affected by a variety of contextual factors, like type of explanation, expertise [107, 117, 129], personal characteristics [86, 93], and order of seeing system strengths and weaknesses [92].”
BackgroundGentile, D., Donmez, B., & Jamieson, G. A. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103945 · Cited by 15
Evaluating Self-attention Interpretability Through Human-Grounded Experimental Protocol
Bhan, M., Achache, N., Legrand, V., Blangero, A., & Chesneau, N. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44070-0_2 · Cited by 17
Counterfactual Explanations May Not Be the Best Algorithmic Recourse Approach
Upadhyay, S., Lakkaraju, H., & Gajos, K. Z. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712095 · Cited by 21
“Namely, Wang and Yin [52] evaluated the efect of counterfactual and feature-based explanations on model understanding, uncertainty awareness, and trust, in familiar (recidivism prediction) and unfamiliar (forest cover prediction) domains.”
Breckner, K., Neumayr, T., Mara, M., Streit, M., & Augstein, M. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2482742 · Cited by 16
Morrison, K., Jain, M., Hammer, J., & Perer, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610064 · Cited by 10
“However, in the past few years, numerous studies have focused on evaluating XAI techniques using human-centered methods [4, 23, 31, 34, 60, 63].”
BackgroundCreating a User Model to Support User-specific Explanations of AI Systems
Chambers, O., Cohen, R. Z., Grossman, M. R., & Chen, Q. (2022) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3511047.3537678 · Cited by 13
“This may occur, for instance, with a user significantly lacking in domain knowledge; in this case, the user may have difficulty comprehending how the features of the explanation and of the model are relevant to their particular situation [13].”
BackgroundAshktorab, Z., Desmond, M., Pan, Q., Johnson, J., Brachman, M., Dugan, C., Danilevsky, M., & Geyer, W. (2025) · Proceedings of the 4th Annual Symposium on Human-Computer Interaction for Work · doi:10.1145/3729176.3729179 · Cited by 10
Do, H. J., Kong, H., Tetali, P., Karahalios, K. G., & Bailey, B. P. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610192 · Cited by 23
“These findings echo research findings in XAI that the explanation technique improves people’s understanding of the AI model [94].”
BackgroundSeo, H., Lee, S., Lee, D., & Xiong, A. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31397 · Cited by 13
“In XAI, a bar chart has been often used to explain features that impact AI decision-making (Cheng et al. 2019; Wang and Yin 2021).”
MethodHuman-AI Teamwork Interface Design Using Patterns of Interactions
Momose, K., Mehta, R., Moukpe, J., Weekes, T. R., & Eskridge, T. C. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2389350 · Cited by 13
“…in collaborative decision-making by providing feature contribution (Bansal et al., 2021), example-based (Yang et al., 2020), counterfactual (Wang & Yin, 2021) explanations, describing AI behaviors (Cabrera et al., 2023), tailoring AI recommendations based on human expertise levels (Calisto…”
The Amplifying Effect of Explainability in AI-assisted Decision-making in Groups
Duarte, R. D. B., Abreu, M. C., Campos, J., & Paiva, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713534 · Cited by 19
“On the other hand, some studies suggest that explanations emphasizing feature importance may induce over-reliance [5, 11, 16, 65], whereas other types of explanations may have little or no noticeable effect on decision-making performance and trust in the AI [10, 16, 50].”
Papantonis, I., & Belle, V. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1560448 · Cited by 14
Ferguson, S. A., Aoyagui, P. A., & Kuzminykh, A. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585727 · Cited by 14
“However, in open-to-interpretation scenarios, without ground truth labels, more subjective evaluation scales are required beyond accuracy [20], such as perceived trust [28], usefulness [22] and user satisfaction [12] to name a few - thus indicating Human-AI hybrid teams’ success might also rely on more parameters that…”
BackgroundAllioui, H., Allioui, A., & Mourdi, Y. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19038-6 · Cited by 18
Generating Recommendations with Post-Hoc Explanations for Citizen Science
Zaken, D. B., Segal, A., Cavalier, D., Shani, G., & Gal, K. (2022) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3503252.3531290 · Cited by 10
“In fact, most of the works we explored, including more recent papers [6, 11, 15, 49] do not include online evaluation with a real world system.”
BackgroundHuman-centred explanations for artificial intelligence systems
Baber, C., Kandola, P., Apperly, I. A., & McCormick, E. (2024) · Ergonomics · doi:10.1080/00140139.2024.2334427 · Cited by 14
“Approaches which concentrate on explaining the algo-rithm might not be beneficial to users (Alufaisan et al. 2021; Bansal et al. 2021; Carton, Mei, and Resnick 2020; Edwards and Veale 2017; Ehrlich et al. 2011; Wang and Yin 2021).”
Practising Appropriate Trust in Human-Centred AI Design
Degachi, C., Mehrotra, S., Yurrita, M., Niforatos, E., & Tielman, M. L. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650825 · Cited by 10
“However, calibrated trust necessarily involves a process of trust calibration or trust alignment that corrects for over/under — trust through repeated interactions [40].”
XAI in Automated Fact-Checking? The Benefits Are Modest and There's No One-Explanation-Fits-All
Lim, G., & Perrault, S. T. (2023) · Australian Conference on Computer-Human Interaction (OzCHI) · doi:10.1145/3638380.3638388 · Cited by 13
“While there are works comparing different types of explanations [68, 73], it is unclear how explanations are understood and may influence users’ behavior in the automated fact-checking context.”
BackgroundResultVanNostrand, P. M., Hofmann, D. M., Ma, L., & Rundensteiner, E. A. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658997 · Cited by 26
“[62] show that point-based counterfactuals can improve user understanding of an automated decision system and Bove et al .”
BackgroundMethodInfluentialUnderstanding the User Perception and Experience of Interactive Algorithmic Recourse Customization
Koh, S., Kim, B. H., & Jo, S. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3674503 · Cited by 12
“Unfortunately, such a high cognitive workload has been associated with the problem of XAI pitfalls—the unexpected negative effects of AI explanations—such as over-reliance [4, 39, 58, 63].”
Ishizu, N., Yeoh, W. L., Okumura, H., & Fukuda, O. (2024) · Applied Sciences · doi:10.3390/app14167192 · Cited by 9
“However, Wang et al. found that the AI explanations are less effective in tasks in areas where the user has no expertise [13].”
The Influence of Curiosity Traits and On-Demand Explanations in AI-Assisted Decision-Making
Cau, F. M., & Spano, L. D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712165 · Cited by 15
Peters, U., & Carman, M. (2023) · IEEE Intelligent Systems · doi:10.1109/mis.2023.3320433 · Cited by 15
Understanding and Designing for Trust in AI-Powered Developer Tooling
Murillo, A., Elizondo, A., D’Angelo, S., Brown, A., Kumar, U., Madison, Q., & Macvean, A. (2024) · IEEE Software · doi:10.1109/ms.2024.3439108 · Cited by 9
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4772689 · Cited by 8
Self-citationBi, N., Huang, Y., Han, C.-C., & Hsu, J. Y.-J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579593 · Cited by 10
“In this sense, our findings extended Wang and Yin’s [76] results in terms of people’s previous domain expertise (in our case, meme literacy) will affect the performance of explanations.”
BackgroundResultZhang, T., Yang, X. J., & Li, B. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2364986 · Cited by 15
“Perhaps surprisingly, despite the research effort, a number of recent studies [9, 61, 66, 86, 117, 129] find that the explanations often do not resolve user confusion regarding the neural networks they are purported to explain.”
InfluentialTrustworthy human computation: a survey
Kashima, H., Oyama, S., Arai, H., & Mori, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10974-1 · Cited by 11
“[173] identify three desirable properties of AI explanations in the context of human–AI decision-”
Coalesce: An Accessible Mixed-Initiative System for Designing Community-Centric Questionnaires
Overney, C., Kessler, D. T., Fulay, S. P., Jasim, M., & Roy, D. K. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712118 · Cited by 14
“Trust calibration ensures users neither over-rely on incorrect model recommendations nor ignore correct ones [11, 107, 112].”
Decision Making in Human-Computer Interaction
Revankar, V. N., Nivashini, G., Selvakumar, P., Meera, K. L., Rambabu, U., Arun, M. R., & Manjunath, T. C. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-5702-7.ch003 · Cited by 10
Is Overreliance on AI Provoked by Study Design?
Zhang, Z. T., Tong, S., Liu, Y., & Butz, A. M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42286-7_3 · Cited by 11
“This conforms to measures commonly used in related work (e.g. [1,3,11,12,15,17]).”
BackgroundMethodMatarese, M., Cocchella, F., Rea, F., & Sciutti, A. (2023) · IEEE RO-MAN · doi:10.1109/ro-man57019.2023.10309636 · Cited by 12
“During the no exp phase’s five games, iCub could participate in the decision-making; this was similar to the one used in [36].”
ResultZhang, Z. T., Buchner, F., Liu, Y., & Butz, A. (2024) · Proceedings of Mensch und Computer 2024 · doi:10.1145/3670653.3670660 · Cited by 6
“Wang and Yin [38] studied the effect of different explanations in two decision-making tasks, one where participants had more domain knowledge in, and one where they had less.”
InfluentialDelaunay, J., Galárraga, L. A., Largouët, C., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711011 · Cited by 14
Schäfer, U., Sipos, L., & Müller-Birn, C. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757451 · Cited by 13
Kuhl, U., & Bush, A. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08333-3_12 · Cited by 12
“A number of user-based-evaluations fail to demonstrate significant improvements in trust driven by CEs [41, 34].”
Schömbs, S., Zhang, Y., Gonçalves, J., & Johal, W. (2025) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3765766.3765795 · Cited by 19
“LeeandSee[40]describedtrust as“anattitudethatanagentwillachieveanindividual’sgoalina situationcharacterisedbyuncertaintyandvulnerability.””
Walke, F., Bennek, L., & Winkler, T. J. (2025) · Lecture notes in information systems and organisation · doi:10.1007/978-3-031-80122-8_8 · Cited by 13
Jeon, Y., Hwang, C., & Chen, X. '. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712161 · Cited by 10
Predicting the need for XAI from high-granularity interaction data
de Santana, V. F., Fucs, A., Segura, V., de Moraes, D. B., & Cerqueira, R. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2023.103029 · Cited by 8
Schrills, T., Gruner, M., Peuscher, H., & Franke, T. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35748-0_23 · Cited by 7
Joshi, B., Liu, Z., Ramnath, S., Chan, A., Tong, Z., Nie, S., Wang, Q., Choi, Y., & Ren, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.392 · Cited by 45
An Architecture to Support Graduated Levels of Trust for Cancer Diagnosis with AI
Rezaeian, O., Bayrak, A. E., & Asan, O. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-61966-3_37 · Cited by 11
Outlier Explanation Through Masking Models
Angiulli, F., Fassetti, F., Nisticò, S., & Palopoli, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-15740-0_28 · Cited by 7
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialLess or More: Towards Glanceable Explanations for LLM Recommendations Using Ultra-Small Devices
Wang, X., Yu, M., Nguyen, H., Iuzzolino, M. L., Wang, T., Tang, P., Lynova, N., Tran, C., Zhang, T., Sendhilnathan, N., et al. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712074 · Cited by 17
“In addition, to ensure transparency and control during human-AI interaction, there has been a growing interest in Explainable AI (XAI) techniques that can help users understand and trust the AI assistance they receive [48, 49, 66, 72, 78].”
Self-citationChen, R., Wang, R., Sadeh, N., & Fang, F. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732050 · Cited by 13
Wang, W.-F., Lee, T.-Y., Lu, C.-T., Hsu, C.-W., Campanyà, N. P. I., Chen, Y., Chen, M. Y., & Chen, B.-Y. (2025) · ACM Symposium on User Interface Software and Technology · doi:10.1145/3746059.3747774 · Cited by 13
Exploring the impact of explainable AI and cognitive capabilities on users’ decisions
Cau, F. M., & Spano, L. D. (2025) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-025-09438-0 · Cited by 20
“…XAI literature has focused on feature-based explanations (Lai et al. 2023a), and there is insufficient evidence regarding their impact on users’ accuracy, particularly with tabular data (Zhang et al. 2020; Wang and Yin 2021; Chen et al. 2023; Ma et al. 2023; Cau et al. 2023b; Cau and Spano…”
InfluentialFaas, C., Uth, R., Sterz, S., Langer, M., & Feit, A. M. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812265 · Cited by 27
The Impact of Artificial Intelligence on Human Resource Management: Barriers and Benefits
Maota, T., & Naidoo, V. (2025) · IGI Global eBooks · doi:10.4018/979-8-3693-8679-8.ch016 · Cited by 5
Conversational Explanations: Discussing Explainable AI with Non-AI Experts
Zhang, T., Zhang, M., Low, W. Y., Yang, X. J., & Li, B. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712143 · Cited by 18
“To handle data scarcity, we propose a novel method in this work to develop conversational explanations by training large vision language models (VLMs) on synthetic conversations.”
Human-Centered Evaluation of Explanations
Boyd-Graber, J. L., Carton, S., Shi, F., Liao, Q. V., Lombrozo, T., Smith, A., & Tan, C. (2022) · doi:10.18653/v1/2022.naacl-tutorials.4 · Cited by 27
“Additional references: (Liao and Varshney, 2021; Zhang et al., 2020; Wang and Yin, 2021; McKnight et al., 2002; Cheng et al., 2019; Lai et al., 2021; Kaur et al., 2020; Jacobs et al., 2020).”
Resilience Through Appropriation: Pilots’ View on Complex Decision Support
Zhang, Z. T., Storath, C., Liu, Y., & Butz, A. M. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584056 · Cited by 13
“A frequent focus of these studies is trust calibration via explanations of AI outputs [51, 58, 60, 63], i.”
BackgroundThe Role of Response Time for Algorithm Aversion in Fast and Slow Thinking Tasks
Lebedeva, A., Kornowicz, J., Lammert, O., & Papenkordt, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35891-3_9 · Cited by 8
Metadata
- Authors
- Wang, X., & Yin, M.
- Affiliations
- Purdue University West Lafayette (US)
- ORCID
- Venue
- 26th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2021-04-14
- Pages
- 318-328
- OpenAlex
- W3156106752
- Semantic Scholar
- c79ba865574cb9635dbca135cec99ee56125778c
- Cited by
- 320
- References
- 66
- Citations by year
- 2026: 63 · 2025: 76 · 2024: 82 · 2023: 56 · 2022: 34 · 2021: 9
- FWCI
- 20.90
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion, cc-by)
- Funders
- National Science Foundation (1850335)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- explainable AI, AI-assisted decision-making, model-agnostic explanation, counterfactual explanations, model uncertainty, domain knowledge, recidivism prediction, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.