Effects of AI and Logic-Style Explanations on Users’ Decisions Under Different Levels of Uncertainty
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Existing eXplainable Artificial Intelligence (XAI) techniques support people in interpreting AI advice. However, although previous work evaluates the users’ understanding of explanations, factors influencing the decision support are largely overlooked in the literature. This article addresses this gap by studying the impact of user uncertainty , AI correctness , and the interaction between AI uncertainty and explanation logic-styles for classification tasks. We conducted two separate studies: one requesting participants to recognize handwritten digits and one to classify the sentiment of reviews. To assess the decision making, we analyzed the task performance, agreement with the AI suggestion, and the user’s reliance on the XAI interface elements. Participants make their decision relying on three pieces of information in the XAI interface (image or text instance, AI prediction, and explanation). Participants were shown one explanation style (between-participants design) according to three styles of logical reasoning (inductive, deductive, and abductive). This allowed us to study how different levels of AI uncertainty influence the effectiveness of different explanation styles. The results show that user uncertainty and AI correctness on predictions significantly affected users’ classification decisions considering the analyzed metrics. In both domains (images and text), users relied mainly on the instance to decide. Users were usually overconfident about their choices, and this evidence was more pronounced for text. Furthermore, the inductive style explanations led to overreliance on the AI advice in both domains—it was the most persuasive, even when the AI was incorrect. The abductive and deductive styles have complex effects depending on the domain and the AI uncertainty levels.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524 · cited 1×
“Other famous methods that are not gradient based are SHAP [51], LIME [60], and LRP (Layer-wise Relevance Propagation) [7].”
MethodKincaid, J. P., Fishburne, Jr., Robert P., R., Richard L., C., & Brad, S. (1975) · doi:10.21236/ada006655 · Cited by 3,497 · cited 1×
“We tested two reading difficulty metrics, Flesch-Reading-Ease [21] and Flesch-Kincaid metrics [41], but they were too sensitive to sentence length, finding “difficult” instances that were easy to classify for users.”
MethodTowards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings
Friedman, M. (1940) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177731944 · Cited by 2,785 · cited 1×
“For H1 (reliance), we assess the results with the Friedman test [22, 23], analyzing user uncertainty values (low and high) separately to find significant differences in the factors’ distributions.”
MethodLearning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761
Aleatory or epistemic? Does it matter?
Kiureghian, A. D., & Ditlevsen, O. (2009) · Structural Safety · doi:10.1016/j.strusafe.2008.06.020 · Cited by 2,454 · cited 1×
“In Bayesian modeling, we can characterize two types of uncertainty: epistemic and aleatoric [14].”
MethodSmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467 · cited 1×
“The large majority of these methods rely on gradients, such as Grad-CAM, [64], SmoothGrad [69], and GBP (Guided Back Propagation) [70].”
MethodTask complexity: Definition of the construct
Wood, R. E. (1986) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(86)90044-0 · Cited by 1,393 · cited 1×
“These measures are related to the objective and subjective task complexity distinction: the first considers task complexity exclusively based on their characteristics and independently from the per-former [83].”
BackgroundSanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 1×
“Among the subjective measures, we focus on the Reliance factor, which is generally used in the literature to measure users’ trust in a system when it makes errors [17, 33], also considering the effect of different levels of user uncertainty.”
BackgroundCer, D., Yang, Y., Kong, S.-Y., Hua, N., Limtiaco, N., John, R. S., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., et al. (2018) · arXiv · doi:10.48550/arxiv.1803.11175 · Cited by 1,267 · cited 2×
“…tasks: Bidirectional Encoder Representations from Transformers (BERT) [15] with one additional output layer (same settings as in the work of Kenny and Keane [39]), and Yoon Kim CNN-multichannel [40] united with Transformer Universal Sentence Encoder (USE_T) [11] for features extraction.”
MethodZhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783
Evaluating the quality of machine learning explanations: a survey on methods and metrics
Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A. (2021) · Electronics · doi:10.3390/electronics10050593 · Cited by 606 · cited 2×
“Zhou et al. [89] propose a comprehensive overview of the state-of-the-art methods for the evaluation of machine learning explanations.”
BackgroundUnderstanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 1×
“Conversely, studies that use real tasks evaluate the cooperation between users and AI: users mainly focus on the task and may decide to use the AI advice to complete it [9, 18, 86].”
BackgroundTask complexity: A review and conceptualization framework
Liu, P., & Li, Z. (2012) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2012.09.001 · Cited by 438
Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., & Baesens, B. (2011) · Decision Support Systems · doi:10.1016/j.dss.2010.12.003 · Cited by 425 · cited 4×
“Another metric we consider is the agreement, already used in the literature to quantify response times in user studies, as well as the agreement between annotators and model predictions [34, 45, 61, 62].”
BackgroundMethodInfluentialEvaluating XAI: A comparison of rule-based and example-based explanations
van der Waa, J., Nieuwburg, E., Cremers, A., & Neerincx, M. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2020.103404 · Cited by 393
Multimodal Neurons in Artificial Neural Networks
Goh, G., Cammarata, N., Voss, C., Carter, S., Petrov, M., Schubert, L., Radford, A., & Olah, C. (2021) · Distill · doi:10.23915/distill.00030 · Cited by 330 · cited 2×
“A possible solution for creating deductive visual explanations could be the solution in the work of Goh et al. [26], supporting users in exploring the intermediate layers and neurons activations of the neural network.”
MethodProxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307 · cited 9×
“We found comparisons limited to a subset of the logical reasoning styles, and they do not extend the result beyond a single domain [9, 75].”
BackgroundMethodInfluentialExplanation in Case-Based Reasoning—Perspectives and Goals
Sørmo, F., Cassens, J., & Aamodt, A. (2005) · Artificial Intelligence Review · doi:10.1007/s10462-005-4607-7 · Cited by 239 · cited 1×
“Another way is to leverage the Twin Systems approach [35] by coupling a black-box with case-based reasoning (CBR) system [71] and map the feature weights from the former to the latter to find cases that explain the networks’ outputs.”
BackgroundKenny, E. M., Ford, C., Quinn, M., & Keane, M. T. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103459 · Cited by 211 · cited 8×
“Inspired by the findings of Kenny et al. [38], showing that explanations impact people’s judgment of AI errors, we expect that explanations may indirectly expose the level of AI uncertainty to the users who, in turn, may change their decision accordingly.”
BackgroundMethodInfluentialEvaluating saliency map explanations for convolutional neural networks: a user study
Alqaraawi, A., Schuessler, M., Weiß, P., Costanza, E., & Berthouze, N. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377519 · Cited by 163
Narayanan, M., Chen, E., He, J., Kim, B., Gershman, S., & Doshi-Velez, F. (2018) · arXiv · doi:10.48550/arxiv.1802.00682 · Cited by 153
Weitz, K., Schiller, D., Schlagowski, R., Huber, T., & André, E. (2019) · Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents · doi:10.1145/3308532.3329441 · Cited by 136 · cited 1×
“Subjective measures consider the personal experience of the user on tasks and AI explanations, such as trust, confidence, and preference [81, 91].”
BackgroundAnchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems
Nourani, M., Roy, C., Block, J. E., Honeycutt, D. R., Rahman, T., Ragan, E., & Gogate, V. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450639 · Cited by 107 · cited 6×
“Most notably, Nourani et al. [56] studied how ordering biases can affect users’ mental models and reliance/confidence formation in intelligent systems and the role the explanations play with such biases.”
BackgroundInfluentialFaithful Multimodal Explanation for Visual Question Answering
Wu, J., & Mooney, R. (2019) · Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w19-4812 · Cited by 73
Cited by
de Jong, S., Paananen, V., Tag, B., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710946 · Cited by 32
Don’t Be Fooled: The Misinformation Effect of Explanations in Human–AI Collaboration
Spitzer, P., Holstein, J., Morrison, K., Holstein, K., Satzger, G., & Kühl, N. (2026) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2574511 · Cited by 12
“Correct AI advice [3, 39, 42, 55, 68, 76, 77, 94, 102, 104] [17, 61] Incorrect AI advice [4, 15, 16, 18, 20, 30, 53, 54, 72, 75] [56, 61, 64]”
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5
“…96, 98, 108, 112, 118, 123, 127, 132, 133, 144–146, 156], Over-reliance [10, 27, 29, 32, 93, 96, 98, 103, 133, 144, 145, 151, 167], Under-reliance [27, 32, 93, 96, 98, 103, 133, 167], Human Error [10], Weight of Advice [13, 33, 117, 146], Absolute Percentage Error [33], RAIR [13, 60–63, 103, 127,…”
InfluentialFrom Trust to Appropriate Reliance: Measurement Constructs in Human-AI Decision-Making
Raees, M., & Papangelis, K. (2026) · arXiv · doi:10.48550/arxiv.2604.23896 · Cited by 5
Metadata
- Authors
- Cau, F. M., Hauptmann, H., Spano, L. D., & Tintarev, N.
- Venue
- ACM Transactions on Interactive Intelligent Systems
- Published
- 2023-12-08
- Volume
- 13
- Issue
- 4
- Pages
- 1-42
- ISSN
- 2160-6455 (ISSN-L)
- 2160-6463
- OpenAlex
- W4327591912
- Cited by
- 41
- References
- 34
- FWCI
- 3.60
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.