Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its DataCite record; 75 of the 79 references it cites were found in a registry or an index, and 4 are not checked yet; 2 of the 50 sources its claims cite were checked for support.

To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making

Open access

Abstract

People supported by AI-powered decision support tools frequently overrely on the AI: they accept an AI's suggestion even when that suggestion is wrong. Adding explanations to the AI decisions does not appear to reduce the overreliance and some studies suggest that it might even increase it. Informed by the dual-process theory of cognition, we posit that people rarely engage analytically with each individual AI recommendation and explanation, and instead develop general heuristics about whether and when to follow the AI suggestions. Building on prior research on medical decision-making, we designed three cognitive forcing interventions to compel people to engage more thoughtfully with the AI-generated explanations. We conducted an experiment (N=199), in which we compared our three cognitive forcing designs to two simple explainable AI approaches and to a no-AI baseline. The results demonstrate that cognitive forcing significantly reduced overreliance compared to the simple explainable AI approaches. However, there was a trade-off: people assigned the least favorable subjective ratings to the designs that reduced the overreliance the most. To audit our work for intervention-generated inequalities, we investigated whether our interventions benefited equally people with different levels of Need for Cognition (i.e., motivation to engage in effortful mental activities). Our results show that, on average, cognitive forcing interventions benefited participants higher in Need for Cognition more. Our research suggests that human cognitive motivation moderates the effectiveness of explainable AI solutions.

Verified 2026-10-11 · 4 references not checked yet

Overview

CoChat Summary

TL;DR

In an MTurk experiment with 199 participants on a nutrition replacement task, cognitive forcing functions reduced overreliance on incorrect AI suggestions compared with simple explainable AI, but they did not improve performance overall enough to match the AI alone. The gains came with a subjective cost: the more effective interfaces were also rated as less preferred, more mentally demanding, and more complex.

Key findings

  • Across all task instances, both simple explainable AI (SXAI) and cognitive forcing functions (CFF) improved performance over no AI on overall performance, carb source detection, carb reduction, and flavor similarity, but there were no significant differences between CFF and SXAI on these aggregate metrics.
  • When the AI’s top prediction was incorrect, CFF reduced overreliance and improved correctness relative to SXAI: the paper reports overall performance of 0.27 (0.03) for CFF vs 0.08 (0.03) for SXAI, and carb source detection of 0.09 (0.01) vs 0.03 (0.02), with significant differences.
  • In the main results, CFF led to significantly more correct decisions than SXAI on incorrect AI predictions, and significantly less overreliance for carb source detection; however, CFF still did not eliminate overreliance, and human+AI performance remained below the AI alone (AI accuracy was set to 75%).
  • Subjective ratings showed a trade-off: compared with no AI, participants preferred AI-assisted conditions more, but CFF/SXAI were rated as less mentally demanding than no AI; CFF was perceived as more complex than SXAI, and in the NFC analysis high-NFC participants especially benefited from CFF on incorrect predictions.

Methods

  • Mixed between- and within-subject online experiment on Amazon Mechanical Turk with 199 retained participants from an initial 260 recruited adults in the United States; each participant interacted with two of nine conditions across 26 questions in two blocks.
  • Participants completed a nutrition task: for meal images, they selected the ingredient highest in carbohydrates to replace and then chose a low-carb, flavor-similar replacement from a list of about 10 options.
  • Conditions included no AI, two SXAI baselines (explanation, uncertainty), and three CFF designs (on demand, update, wait). Analysis used mixed-effects models/ANOVA, with category as a fixed effect and participant as a random effect; post-hoc comparisons used t-tests with Holm-Bonferroni correction, and correlations used Pearson’s r.

Limitations

  • The paper states that the study was conducted on a single non-critical decision-making task, so generalization to other domains and settings is uncertain.
  • The paper notes that CFF improved performance at the cost of reduced perceived usability, which may limit adoption; this is a limitation stated by the authors.
  • Evident from the design: the AI was simulated rather than a trained machine-learning model, and the task used a nutrition domain with lay participants on MTurk, so ecological validity for high-stakes expert settings is limited (evident from the text).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2102.09692

Cites

75 of 79 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

1 Valid · 2 Not valid · 25 Not checked · 98 Own results and interpretations

83 more claims

Create a free account to see all 126 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 20
  • The quote only partly states this claim. 4
  • Probably not a substantial claim. 18

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

71 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 68 references

Cited by

See all 1,124

Metadata

Authors
Buçinca, Z., Malaya, M. B., & Gajos, K. Z.
Affiliations
  • Harvard University (US)
  • Lodz University of Technology (PL)
Venue
arXiv
Publisher
arXiv
Published
2021
Volume
5
Issue
CSCW1
Pages
1-21
ISSN
2573-0142 (ISSN-L)
OpenAlex
W3159250634
Cited by
1,124
References
68
Citations by year
2026: 475 · 2025: 285 · 2024: 164 · 2023: 151 · 2022: 44 · 2021: 5
FWCI
32.17
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Keywords
overreliance on AI, explainable AI, dual-process theory, heuristics, clinical decision support, decision support system, need for cognition, subjective ratings, trust in AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making. https://app.cochat.ai/papers/10.48550/arxiv.2102.09692
The paper
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making. arXiv. https://doi.org/10.48550/arxiv.2102.09692

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.