To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making
Open accessThe DOI matches its DataCite record; 75 of the 79 references it cites were found in a registry or an index, and 4 are not checked yet; 2 of the 50 sources its claims cite were checked for support.
Abstract
People supported by AI-powered decision support tools frequently overrely on the AI: they accept an AI's suggestion even when that suggestion is wrong. Adding explanations to the AI decisions does not appear to reduce the overreliance and some studies suggest that it might even increase it. Informed by the dual-process theory of cognition, we posit that people rarely engage analytically with each individual AI recommendation and explanation, and instead develop general heuristics about whether and when to follow the AI suggestions. Building on prior research on medical decision-making, we designed three cognitive forcing interventions to compel people to engage more thoughtfully with the AI-generated explanations. We conducted an experiment (N=199), in which we compared our three cognitive forcing designs to two simple explainable AI approaches and to a no-AI baseline. The results demonstrate that cognitive forcing significantly reduced overreliance compared to the simple explainable AI approaches. However, there was a trade-off: people assigned the least favorable subjective ratings to the designs that reduced the overreliance the most. To audit our work for intervention-generated inequalities, we investigated whether our interventions benefited equally people with different levels of Need for Cognition (i.e., motivation to engage in effortful mental activities). Our results show that, on average, cognitive forcing interventions benefited participants higher in Need for Cognition more. Our research suggests that human cognitive motivation moderates the effectiveness of explainable AI solutions.
Verified 2026-10-11 · 4 references not checked yet
Overview
CoChat Summary
TL;DR
In an MTurk experiment with 199 participants on a nutrition replacement task, cognitive forcing functions reduced overreliance on incorrect AI suggestions compared with simple explainable AI, but they did not improve performance overall enough to match the AI alone. The gains came with a subjective cost: the more effective interfaces were also rated as less preferred, more mentally demanding, and more complex.
Key findings
- Across all task instances, both simple explainable AI (SXAI) and cognitive forcing functions (CFF) improved performance over no AI on overall performance, carb source detection, carb reduction, and flavor similarity, but there were no significant differences between CFF and SXAI on these aggregate metrics.
- When the AI’s top prediction was incorrect, CFF reduced overreliance and improved correctness relative to SXAI: the paper reports overall performance of 0.27 (0.03) for CFF vs 0.08 (0.03) for SXAI, and carb source detection of 0.09 (0.01) vs 0.03 (0.02), with significant differences.
- In the main results, CFF led to significantly more correct decisions than SXAI on incorrect AI predictions, and significantly less overreliance for carb source detection; however, CFF still did not eliminate overreliance, and human+AI performance remained below the AI alone (AI accuracy was set to 75%).
- Subjective ratings showed a trade-off: compared with no AI, participants preferred AI-assisted conditions more, but CFF/SXAI were rated as less mentally demanding than no AI; CFF was perceived as more complex than SXAI, and in the NFC analysis high-NFC participants especially benefited from CFF on incorrect predictions.
Methods
- Mixed between- and within-subject online experiment on Amazon Mechanical Turk with 199 retained participants from an initial 260 recruited adults in the United States; each participant interacted with two of nine conditions across 26 questions in two blocks.
- Participants completed a nutrition task: for meal images, they selected the ingredient highest in carbohydrates to replace and then chose a low-carb, flavor-similar replacement from a list of about 10 options.
- Conditions included no AI, two SXAI baselines (explanation, uncertainty), and three CFF designs (on demand, update, wait). Analysis used mixed-effects models/ANOVA, with category as a fixed effect and participant as a random effect; post-hoc comparisons used t-tests with Holm-Bonferroni correction, and correlations used Pearson’s r.
Limitations
- The paper states that the study was conducted on a single non-critical decision-making task, so generalization to other domains and settings is uncertain.
- The paper notes that CFF improved performance at the cost of reduced perceived usability, which may limit adoption; this is a limitation stated by the authors.
- Evident from the design: the AI was simulated rather than a trained machine-learning model, and the task used a nutrition domain with lay participants on MTurk, so ecological validity for high-stakes expert settings is limited (evident from the text).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 Valid · 2 Not valid · 25 Not checked · 98 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Yet, performance of participants that completed the task with no AI assistanceno AI category-was significantly higher than that of participants' in either cognitive forcing functions or simple explainable AI categories when they saw incorrect model predictions.”
plain- Asserted: unclear — It is unclear whether the authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When analyzing performance on all task instances (both those where the top AI predictions were correct and those where they were incorrect), both simple explainable AI conditions and cognitive forcing functions improved participants' performance on all measures (overall performance, carb source detection, carb reduction, and flavor similarity) compared to the no AI baseline.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For all the metrics, there were no significant differences among conditions within either category (i.e., simple explainable AI and cognitive forcing functions ).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When the top AI predictions were incorrect, however, cognitive forcing functions improved the objective metrics (i.e., overall performance, carb source detection performance, carb reduction and flavor similarity) significantly more compared to simple explainable AI .”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When the top AI predictions were incorrect, however, cognitive forcing functions improved the objective metrics (i.e., overall performance, carb source detection performance, carb reduction and flavor similarity) significantly more compared to simple explainable AI .”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
83 more claims
Create a free account to see all 126 claims and every source verdict.
- It may state two separate claims as one. 20
- The quote only partly states this claim. 4
- Probably not a substantial claim. 18
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The sequential task setup could induce an anchoring effect which prevents the human to more actively take the AI into account [9].”
From Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations (2023) · cites this paper as [9]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, they also reduce reliance on correct model predictions, yielding no significant differences when humans are not aided by an AI.”
From Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2023) · cites this paper as [9]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, unlike in [16], where cognitive effort significantly increased under-reliance (i.e., humans under high cognitive effort may blindly ignore AI's suggestions), our findings reveal no such adverse effect.”
From Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making (2025) · cites this paper as [16]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A recent work by Buçinca et al. [10] called out that when interacting with AI decision support, there is tension between cognitive engagement, which can lead to better decision outcomes, and subjective user experience, which may be influenced by ease of use and efficiency.”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as Buçinca et al. [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, recent research has shown that the recommendation-driven explainable/interpretable AI has little effect on decision making [1, 6-8, 28, 32], although this is not always the case [22,23,41].”
From Explainable AI is Dead, Long Live Explainable AI!: Hypothesis-driven Decision Support using Evaluative AI (2023) · cites this paper as [1]
Experimental — accuracy not yet measured
71 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 5 ETHICAL CONSIDERATIONS: INDIVIDUAL DIFFERENCES IN COGNITIVE MOTIVATION
“| performance | incorrect AI predictions | {SXAI} < {CFF} | 0.02 (0.02) | 0.11 (0.02) | 𝐹 1 , 100 . 6 = 10 . 55, 𝑝 = . 002, 𝑑 = . 57 | {SXAI, CFF} | 0.03 (0.02) | 0.06 (0.02) | 𝐹 1 , 99 . 37 = 0 . 94, 𝑛.𝑠. , 𝑑 = . 15 |”
§ 5 ETHICAL CONSIDERATIONS: INDIVIDUAL DIFFERENCES IN COGNITIVE MOTIVATION
“| correct | {SXAI} <{CFF} | 0.02 (0.02) | 0.11 (0.02) | 𝐹 1 , 100 . 6 = 10 . 55, 𝑝 = . 002, 𝑑 = . 57 | {SXAI, CFF} | 0.03 (0.02) | 0.07 (0.02) | 𝑛.𝑠. , 𝑑 = . 15 |”
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810 · cited 1×
“These heuristics are highly useful and mostly effective, however they can lead to systemic and predictable errors [34].”
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Random effects structure for confirmatory hypothesis testing: Keep it maximal
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013) · Journal of Memory and Language · doi:10.1016/j.jml.2012.11.001 · Cited by 10,328 · cited 1×
“, all the measurements were not statistically independent) [3].”
BackgroundThe Elaboration Likelihood Model of Persuasion
Petty, R. E., & Cacioppo, J. T. (1986) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60214-2 · Cited by 8,822
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 1×
“Thus, numerous studies have highlighted the importance of calibrated trust in improving AI-assisted decision-making [7, 9, 31, 44, 51, 53, 67].”
BackgroundCacioppo, J. T., & Petty, R. E. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.42.1.116 · Cited by 5,359 · cited 1×
“In psychology, it is captured through the concept of Need for Cognition (NFC), a stable personality trait that reflects how much a person enjoys engaging in cognitively demanding activities [10, 52].”
BackgroundPetty, R. E., & Cacioppo, J. T. (1986) · doi:10.1007/978-1-4612-4964-1 · Cited by 5,219
Judgment under uncertainty: Heuristics and biases
Tversky, A., & Kahneman, D. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.002 · Cited by 3,643
(2018) · Cited by 3,510
Representativeness revisited: Attribute substitution in intuitive judgment.
Kahneman, D., & Frederick, S. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.004 · Cited by 3,476
The Elaboration Likelihood Model of Persuasion
Petty, R. E., & Cacioppo, J. T. (1986) · Communication and Persuasion · doi:10.1007/978-1-4612-4964-1_1 · Cited by 3,202
The Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913 · cited 2×
“After the first block, they also completed a four-item Need for Cognition questionnaire, consisting the four items with the highest factor loading from [11].”
MethodRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 1×
“Therefore, we conducted a limited internal audit [54] of our work to check for inequalities that might emerge and to understand how different groups of people are affected by the introduction of cognitive forcing functions into the AI-assisted decision-making processes.”
MethodThe Importance of Cognitive Errors in Diagnosis and Strategies to Minimize Them
Croskerry, P. (2003) · Academic Medicine · doi:10.1097/00001888-200308000-00003 · Cited by 1,675 · cited 1×
“However, because prior research provides ample evidence that even experts making critical decisions resort to heuristic thinking (and fall prey to cognitive biases) [4, 6, 17, 42], we have some confidence that our results will generalize broadly.”
BackgroundProceedings of the 2nd International Conference on Neural Information Processing Systems
Touretzky, D. S. (1989) · Cited by 1,439
Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1821936116 · Cited by 1,371 · cited 1×
“But when evaluated objectively, their actual learning and performance was better with the more cognitively demanding, active instruction [18].”
BackgroundOverconfidence as a Cause of Diagnostic Error in Medicine
Berner, E. S., & Graber, M. L. (2008) · The American Journal of Medicine · doi:10.1016/j.amjmed.2008.01.001 · Cited by 1,034 · cited 1×
“However, because prior research provides ample evidence that even experts making critical decisions resort to heuristic thinking (and fall prey to cognitive biases) [4, 6, 17, 42], we have some confidence that our results will generalize broadly.”
BackgroundWestfall, J., Kenny, D. A., & Judd, C. M. (2014) · Journal of Experimental Psychology: General · doi:10.1037/xge0000014 · Cited by 928
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · cited 3×
“Thus, numerous studies have highlighted the importance of calibrated trust in improving AI-assisted decision-making [7, 9, 31, 44, 51, 53, 67].”
BackgroundDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 5×
“Indeed, some studies demonstrate that explanations are interpreted as a general signal of competence—rather than being evaluated individually for their content—and just by their presence can increase the trust in and overreliance on the AI [2].”
BackgroundInfluentialDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Good intentions are not enough: how informatics interventions can worsen inequality
Veinot, T. C., Mitchell, H., & Ancker, J. S. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy052 · Cited by 704 · cited 2×
“IGIs arise when the benefits of an intervention disproportionately accrue to a group that is already privileged in a particular context [60].”
BackgroundMethodHaugtvedt, C. P., Petty, R. E., & Cacioppo, J. T. (1992) · Journal of Consumer Psychology · doi:10.1016/s1057-7408(08)80038-1 · Cited by 627 · cited 1×
“The impact of NFC on cognitive engagement with information has been studied in fields such as advertising and purchasing behavior [27, 43, 45], skill acquisition [13], web usage [56, 58], team function [37], health communication [61, 65], and AI-assisted decision-making [24].”
BackgroundReactance to Recommendations: When Unsolicited Advice Yields Contrary Responses
Fitzsimons, G. J., & Lehmann, D. R. (2004) · Marketing Science · doi:10.1287/mksc.1030.0033 · Cited by 604 · cited 1×
“There is evidence that showing unsolicited advice that contradicts a person’s initial idea may trigger reactance (resistance to the advice) [21].”
BackgroundWHEN AND HOW DIVERSITY BENEFITS TEAMS: THE IMPORTANCE OF TEAM MEMBERS' NEED FOR COGNITION
Kearney, E., Gebert, D., & Voelpel, S. C. (2009) · Academy of Management Journal · doi:10.5465/amj.2009.41331431 · Cited by 591 · cited 1×
“The impact of NFC on cognitive engagement with information has been studied in fields such as advertising and purchasing behavior [27, 43, 45], skill acquisition [13], web usage [56, 58], team function [37], health communication [61, 65], and AI-assisted decision-making [24].”
BackgroundWason, P. C., & Evans, J. S. B. T. (1974) · Cognition · doi:10.1016/0010-0277(74)90017-1 · Cited by 575 · cited 1×
“According to the dual-process theory, humans mostly operate on System 1 thinking, which employs heuristics and shortcuts when making decisions [32, 63].”
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 1×
“This condition captures another common approach for designing interfaces for human-AI collaboration [2, 40, 66].”
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5285 · Cited by 466 · cited 1×
“A leading view to explain the inferior team performance is that humans overtrust the AI [1].”
BackgroundCognitive interventions to reduce diagnostic error: a narrative review
Graber, M. L., Kissam, S., Payne, V. L., Meyer, A. N. D., Sorensen, A., Lenfestey, N., Tant, E., Henriksen, K., LaBresh, K., & Singh, H. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000149 · Cited by 449 · cited 1×
“That is because evidence suggests that very often diagnostic errors are not cases of lack of knowledge, but rather of biases and faulty information synthesis [25].”
BackgroundThe Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · cited 4×
“Consistent with prior research [7, 9, 26, 41], our results demonstrate that people aided by simple explainable AI interfaces performed better overall than people who completed the task without any AI support.”
BackgroundMethodResultInfluentialCognitive forcing strategies in clinical decisionmaking
Croskerry, P. (2003) · Annals of Emergency Medicine · doi:10.1067/mem.2003.22 · Cited by 440 · cited 1×
“Interventions that seek to mitigate these types of errors by invoking deliberative thinking can be categorized into educational strategies and cognitive forcing functions [16].”
BackgroundCombining human and machine intelligence in large-scale crowdsourcing
Kamar, E., Hacker, S., & Horvitz, E. (2012) · Adaptive Agents and Multi-Agent Systems · doi:10.65109/hmcs1623 · Cited by 412 · cited 1×
“, human+AI teams) were expected to perform better than either people or AIs alone [35, 36].”
BackgroundThe Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392 · cited 4×
“Prior studies in human-AI decision-making have shown the anchoring bias that occurs by presenting people with AI’s recommendation before allowing them to make a decision first [26].”
BackgroundResultInfluentialToo much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388 · cited 2×
“Hence, in addition to tuning explanation attributes such as soundness and completeness [39], or faithfulness, sensitivity, and complexity [5], explainable AI researchers should ensure that people will exert effort to attend to those explanations.”
BackgroundEFFECTS OF INTERACTIVITY IN A WEB SITE: The Moderating Effect of Need for Cognition
Sicilia, M., Ruiz, S., & Munuera, J. L. (2005) · Journal of Advertising · doi:10.1080/00913367.2005.10639202 · Cited by 388 · cited 1×
“The impact of NFC on cognitive engagement with information has been studied in fields such as advertising and purchasing behavior [27, 43, 45], skill acquisition [13], web usage [56, 58], team function [37], health communication [61, 65], and AI-assisted decision-making [24].”
BackgroundLai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377 · cited 4×
“Consistent with prior research [7, 9, 26, 41], our results demonstrate that people aided by simple explainable AI interfaces performed better overall than people who completed the task without any AI support.”
BackgroundResultInfluentialChecklists to Reduce Diagnostic Errors
Ely, J. W., Graber, M. L., & Croskerry, P. (2011) · Academic Medicine · doi:10.1097/acm.0b013e31820824cd · Cited by 366 · cited 1×
“, System 1) decision-making process [14, 15, 19, 20, 42, 50], but may not necessarily be referred to as cognitive forcing functions even in the healthcare domain.”
BackgroundBornstein, B. H., & Emler, A. C. (2001) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.2001.00284.x · Cited by 353 · cited 1×
“However, because prior research provides ample evidence that even experts making critical decisions resort to heuristic thinking (and fall prey to cognitive biases) [4, 6, 17, 42], we have some confidence that our results will generalize broadly.”
BackgroundCombining human and machine intelligence in large-scale crowdsourcing
Kamar, E. S., Hacker, S., & Horvitz, E. (2012) · doi:10.5555/2343576.2343643 · Cited by 318
Slowing Down When You Should: A New Model of Expert Judgment
Moulton, C.-A. E., Regehr, G., Mylopoulos, M., & MacRae, H. M. (2007) · Academic Medicine · doi:10.1097/acm.0b013e3181405a76 · Cited by 314 · cited 1×
“Indeed, successful use of pattern matching and heuristics that arise from extensive experience are important components of expertise [49].”
BackgroundProxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307 · cited 6×
“However, evidence suggests that explainable systems, also, have not had substantial success in reducing human overreliance on the AIs: when the AI suggests incorrect or suboptimal solutions, people still on average make poorer final decisions than they would have without AI’s assistance [2, 7, 30, 40, 67].”
BackgroundResultInfluentialSelf-citationKool, W., & Botvinick, M. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0401-9 · Cited by 288 · cited 1×
“Because evaluating every explanation requires substantial cognitive effort, which humans are averse to [38], this assumption is likely incorrect.”
BackgroundTo Trust Or Not To Trust A Classifier
Jiang, H., Kim, B., Guan, M. Y., & Gupta, M. R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11783 · Cited by 262 · cited 1×
“Thus, numerous studies have highlighted the importance of calibrated trust in improving AI-assisted decision-making [7, 9, 31, 44, 51, 53, 67].”
BackgroundTuten, T. L., & Bosnjak, M. (2001) · Social Behavior and Personality: an international journal · doi:10.2224/sbp.2001.29.4.391 · Cited by 249 · cited 1×
“The impact of NFC on cognitive engagement with information has been studied in fields such as advertising and purchasing behavior [27, 43, 45], skill acquisition [13], web usage [56, 58], team function [37], health communication [61, 65], and AI-assisted decision-making [24].”
BackgroundJacobs, M., Pradier, M. F., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01224-x · Cited by 247 · cited 5×
“However, evidence suggests that explainable systems, also, have not had substantial success in reducing human overreliance on the AIs: when the AI suggests incorrect or suboptimal solutions, people still on average make poorer final decisions than they would have without AI’s assistance [2, 7, 30, 40, 67].”
BackgroundInfluentialSelf-citationDual-process cognitive interventions to enhance diagnostic reasoning: a systematic review
Lambe, K. A., O'Reilly, G., Kelly, B. D., & Curristan, S. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004417 · Cited by 215 · cited 7×
“However, because prior research provides ample evidence that even experts making critical decisions resort to heuristic thinking (and fall prey to cognitive biases) [4, 6, 17, 42], we have some confidence that our results will generalize broadly.”
BackgroundInfluentialCognitive bias research in forensic science: A systematic review
Cooper, G. S., & Meterko, V. (2019) · Forensic Science International · doi:10.1016/j.forsciint.2019.01.016 · Cited by 212 · cited 2×
“Therefore, shifting people’s reasoning from System 1 to a more deliberative and rational, System 2 thinking remains a challenge that researchers have tackled in numerous high-stakes domains [14, 15, 42].”
BackgroundFlavorDB: a database of flavor molecules
Garg, N., Sethupathy, A., Tuwani, R., NK, R., Dokania, S., Iyer, A., Gupta, A., Agrawal, S., Singh, N., Shukla, S., et al. (2018) · Nucleic Acids Research · doi:10.1093/nar/gkx957 · Cited by 200 · cited 1×
“We used a flavor database comprised of flavor molecules representing an array of tastes and odors associated with 936 ingredients [23].”
MethodRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Evaluating and Aggregating Feature-based Model Explanations
Bhatt, U., Weller, A., & Moura, J. M. F. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/417 · Cited by 176 · cited 2×
“Hence, in addition to tuning explanation attributes such as soundness and completeness [39], or faithfulness, sensitivity, and complexity [5], explainable AI researchers should ensure that people will exert effort to attend to those explanations.”
BackgroundDegradation in Spatial Knowledge Acquisition When Using Automatic Navigation Systems
Parush, A., Ahuvia, S., & Erev, I. (2007) · Springer eBooks · doi:10.1007/978-3-540-74788-8_15 · Cited by 161 · cited 1×
“Thus, numerous studies have highlighted the importance of calibrated trust in improving AI-assisted decision-making [7, 9, 31, 44, 51, 53, 67].”
BackgroundBenefitting InfoVis with Visual Difficulties
Hullman, J., Adar, E., & Shah, P. (2011) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2011.175 · Cited by 159 · cited 1×
“Visualization literature reveals that visual difficulties, while not necessarily preferred, improve participants’ comprehension and recall of the displayed content [29].”
BackgroundAn exploratory cognitive DSS for strategic decision making
Chen, J. Q., & Lee, S. M. (2003) · Decision Support Systems · doi:10.1016/s0167-9236(02)00139-2 · Cited by 159
Directions in hybrid intelligence: complementing AI systems with human intelligence
Kamar, E. S. (2016) · International Joint Conference on Artificial Intelligence · Cited by 154
Millecamp, M., Htun, N. N., Conati, C., & Verbert, K. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302313 · Cited by 148 · cited 1×
“However, there is also some evidence that people with high NFC benefit less from explanations (in terms of confidence in their decisions) when using recommender systems than people with low NFC [47] although there appear to be limits to the generalizability of this result [48].”
BackgroundWagner, A. R., Borenstein, J., & Howard, A. (2018) · Communications of the ACM · doi:10.1145/3241365 · Cited by 146 · cited 1×
“[62] in the context of people overtrusting robots, suggest avoiding features that may nudge the user towards the anthropomorphization of robots, or for self-driving cars, they advise developing tools that understand when the driver is not paying attention.”
BackgroundWilliams-Piehota, P. A., Schneider, T. R., Pizarro, J., Mowad, L. Z., & Salovey, P. (2003) · Health Communication · doi:10.1207/s15327027hc1504_01 · Cited by 144 · cited 1×
“The impact of NFC on cognitive engagement with information has been studied in fields such as advertising and purchasing behavior [27, 43, 45], skill acquisition [13], web usage [56, 58], team function [37], health communication [61, 65], and AI-assisted decision-making [24].”
Background"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
Lai, V., Liu, H., & Tan, C. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376873 · Cited by 123
Analysis of Unbalanced Data by Mixed Linear Models Using the mixed Procedure of the SAS System
Spilke, J., Piepho, H. P., & Hu, X. (2005) · Journal of Agronomy and Crop Science · doi:10.1111/j.1439-037x.2004.00120.x · Cited by 116 · cited 1×
“Mixed-effects models also properly handle the imbalance in our data [57], due to participants being randomly assigned to conditions and not all participants having”
MethodIndividual Differences in the Calibration of Trust in Automation
Pop, V. L., Shrewsbury, A., & Durso, F. T. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814564422 · Cited by 109 · cited 1×
“Thus, numerous studies have highlighted the importance of calibrated trust in improving AI-assisted decision-making [7, 9, 31, 44, 51, 53, 67].”
BackgroundSherbino, J., Kulasegaram, K., Howey, E., & Norman, G. (2014) · CJEM · doi:10.2310/8000.2013.130860 · Cited by 108 · cited 1×
“Studies indicate that cognitive forcing functions, as interventions at the time of decision-making, are more effective than educational strategies in increasing accuracy of diagnostic processes in clinical decision making [42, 55].”
BackgroundExplainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers
Ghai, B., Liao, Q. V., Zhang, Y., Bellamy, R., & Mueller, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3432934 · Cited by 106 · cited 2×
“Because prior work suggested that people with high Need for Cognition (NFC, a stable personality trait that captures one’s motivation to engage in effortful mental activities) tend to benefit more from complex user interface features [12, 22, 24], we did so by disaggregating our results by NFC level.”
BackgroundThe effects of online reviews on purchasing intention: The moderating role of need for cognition
Lin, C.-L., Lee, S.-H., & Horng, D.-J. (2011) · Social Behavior and Personality An International Journal · doi:10.2224/sbp.2011.39.1.71 · Cited by 100 · cited 1×
“The impact of NFC on cognitive engagement with information has been studied in fields such as advertising and purchasing behavior [27, 43, 45], skill acquisition [13], web usage [56, 58], team function [37], health communication [61, 65], and AI-assisted decision-making [24].”
BackgroundBiased: Uncovering the Hidden Prejudice that Shapes What We See, Think, and Do
Bridgeforth, A. C. (2021) · AmeriQuests · doi:10.15695/cyrf8056 · Cited by 90 · cited 1×
“, System 1) decision-making process [14, 15, 19, 20, 42, 50], but may not necessarily be referred to as cognitive forcing functions even in the healthcare domain.”
BackgroundNeed for Cognition, Causal Processing, and Memory for Behavior
Lassiter, G. D., Briggs, M. A., & Slaw, R. D. (1991) · Personality and Social Psychology Bulletin · doi:10.1177/0146167291176013 · Cited by 72 · cited 1×
“The impact of NFC on cognitive engagement with information has been studied in fields such as advertising and purchasing behavior [27, 43, 45], skill acquisition [13], web usage [56, 58], team function [37], health communication [61, 65], and AI-assisted decision-making [24].”
BackgroundA Slow Algorithm Improves Users' Assessments of the Algorithm's Accuracy
Park, J. S., Barber, R., Kirlik, A., & Karahalios, K. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359204 · Cited by 65 · cited 3×
“Informed by prior work that a slow algorithm improves user’s accuracy [50], we also hypothesized this is a form of cognitive forcing function.”
BackgroundNeed for Cognition and Approaches to Learning among University Students
Cazan, A., & Indreica, E.-S. (2014) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2014.03.227 · Cited by 60 · cited 1×
“The impact of NFC on cognitive engagement with information has been studied in fields such as advertising and purchasing behavior [27, 43, 45], skill acquisition [13], web usage [56, 58], team function [37], health communication [61, 65], and AI-assisted decision-making [24].”
BackgroundThe Influence of Personality Traits and Cognitive Load on the Use of Adaptive User Interfaces
Gajos, K. Z., & Chauncey, K. (2017) · Proceedings of the 22nd International Conference on Intelligent User Interfaces · doi:10.1145/3025171.3025192 · Cited by 57 · cited 3×
“Because prior work suggested that people with high Need for Cognition (NFC, a stable personality trait that captures one’s motivation to engage in effortful mental activities) tend to benefit more from complex user interface features [12, 22, 24], we did so by disaggregating our results by NFC level.”
BackgroundMethodSelf-citationVidrine, J. I., Simmons, V. N., & Brandon, T. H. (2007) · Journal of Applied Social Psychology · doi:10.1111/j.0021-9029.2007.00149.x · Cited by 56
What's in a User? Towards Personalising Transparency for Music Recommender Interfaces
Millecamp, M., Htun, N. N., Conati, C., & Verbert, K. (2020) · Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3340631.3394844 · Cited by 50
Need for Cognition and Advertising
Haugtvedt, C. P., Petty, R. E., & Cacioppo, J. T. (1992) · Journal of Consumer Psychology · doi:10.1207/s15327663jcp0103_03 · Cited by 50
The effects of mixing machine learning and human judgment
Vaccaro, M., & Waldo, J. (2019) · Communications of the ACM · doi:10.1145/3359338 · Cited by 49 · cited 1×
“Recent studies, however, show that although human+AI teams typically outperform people working alone, their performance is usually inferior to the AI’s [2, 7, 9, 26, 30, 41, 59].”
An analysis of the influence of need for cognition on dynamic queries usage
Carenini, G. (2001) · CHI '01 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/634067.634293 · Cited by 12 · cited 2×
“Because prior work suggested that people with high Need for Cognition (NFC, a stable personality trait that captures one’s motivation to engage in effortful mental activities) tend to benefit more from complex user interface features [12, 22, 24], we did so by disaggregating our results by NFC level.”
BackgroundThe Effects of Mixing Machine Learning and Human Judgment
Vaccaro, M. A., & Waldo, J. (2019) · Queue · doi:10.1145/3358955.3363293 · Cited by 3
(2017) · doi:10.9728/dcs.2024.25.7.1847 · Cited by 2 · cited 2×
“These feature based explanations (i.e., carb reduction and flavor similarity) were intended to be comparable to commonly used explanations generated by techniques such as SHAP [46].”
Method
Cited by
Zhai, C., Wibowo, S., & Li, L. D. (2024) · Smart Learning Environments · doi:10.1186/s40561-024-00316-7 · Cited by 1,567
“Students can inadvertently become overly dependent on AI-generated assistance, potentially detracting from their ability to make independent, well-informed decisions (Buçinca et al., 2021).”
BackgroundWhen combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570
Abulibdeh, A., Zaidan, E., & Abulibdeh, R. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2023.140527 · Cited by 641
Trust in AI: progress, challenges, and future directions
Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04044-8 · Cited by 507
“People often over-rely on arti fi cial intelligence systems, trusting them too much (Buçinca et al., 2021).”
InfluentialLee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713778 · Cited by 490
Metadata
- Authors
- Buçinca, Z., Malaya, M. B., & Gajos, K. Z.
- Affiliations
- Harvard University (US)
- Lodz University of Technology (PL)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Volume
- 5
- Issue
- CSCW1
- Pages
- 1-21
- ISSN
- 2573-0142 (ISSN-L)
- arXiv
- 2102.09692
- OpenAlex
- W3159250634
- Semantic Scholar
- 65772de56676a6a437c8abaeccef34224f62ee13
- Cited by
- 1,124
- References
- 68
- Citations by year
- 2026: 475 · 2025: 285 · 2024: 164 · 2023: 151 · 2022: 44 · 2021: 5
- FWCI
- 32.17
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- overreliance on AI, explainable AI, dual-process theory, heuristics, clinical decision support, decision support system, need for cognition, subjective ratings, trust in AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.