Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 133 of the 133 references it cites were found in a registry or an index; 23 of the 39 sources its claims cite were checked for support.

The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers

Open access
Lee, Hao-Ping (Hank), Sarkar, Advait, Tankelevitch, Lev, Drosos, Ian, Rintel, Sean, Banks, Richard, Wilson, Nicholas · 2025 · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems

Abstract

The rise of Generative AI (GenAI) in knowledge workflows raises questions about its impact on critical thinking skills and practices.We survey 319 knowledge workers to investigate 1) when and how they perceive the enaction of critical thinking when using GenAI, and 2) when and why GenAI affects their effort to do so.Participants shared 936 first-hand examples of using GenAI in work tasks.Quantitatively, when considering both task-and user-specific factors, a user's task-specific self-confidence and confidence in GenAI are predictive of whether critical thinking is enacted and the effort of doing so in GenAI-assisted tasks.Specifically, higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more critical thinking.Qualitatively, GenAI shifts the nature of critical thinking toward information verification, response integration, and task stewardship.Our insights reveal new design challenges and opportunities for developing GenAI tools for knowledge work.

Overview

CoChat Summary

TL;DR

A survey of 319 knowledge workers who use GenAI at work at least weekly, drawing on 936 real-world examples, finds that higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more. GenAI also shifts perceived critical-thinking effort toward information verification, response integration, and task stewardship.

Key findings

  • Participants reported enacting critical thinking in 555 of 936 examples (59.29%). Higher confidence in AI doing the task negatively correlated with perceived enaction of critical thinking (β = -0.69, p < 0.001); confidence in doing the task oneself (β = 0.26, p = 0.026), confidence in evaluating AI responses (β = 0.31, p = 0.046), and overall tendency to reflect on work (β = 0.52, p < 0.001) positively correlated with perceived enaction.
  • Perceived effort decreased in the majority of examples: “much less effort” or “less effort” comprised 72% for Knowledge, 79% for Comprehension, 69% for Application, 72% for Analysis, 76% for Synthesis, and 55% for Evaluation. Higher confidence in AI negatively correlated with perceived effort for five of six activities (all except Application): Knowledge (β = -0.11, p = 0.029), Comprehension (β = -0.13, p = 0.014), Analysis (β = -0.15, p = 0.003), Synthesis (β = -0.12, p = 0.026), and Evaluation (β = -0.23, p < 0.001).
  • Qualitatively, GenAI shifted critical-thinking effort in three ways: for Knowledge and Comprehension, from information gathering to information verification; for Application, from problem-solving to AI response integration; and for Analysis, Synthesis, and Evaluation, from task execution to task stewardship.
  • Motivators for critical thinking included work quality (74/319), potential negative outcomes (116/319), and skill development (13/319). Inhibitors included awareness barriers such as secondary use of GenAI (14/319), trivial tasks (55/319), and trust/reliance on GenAI (83/319); motivation barriers such as lack of time (44/319) and not being part of job responsibilities (11/319); and ability barriers such as barriers to inspecting AI responses (58/319) and revising queries/improving responses (72/319).

Methods

  • Online survey on Prolific of 319 knowledge workers who self-reported using GenAI tools at work at least once per week. The study received 333 responses and excluded 14 for low response quality; retained participants spent an average of 43.19 minutes (SD = 23.13) completing the survey and were compensated GBP £10.
  • Participants shared three real-world GenAI work examples each. The study collected 957 examples, removed 11 lacking sufficient detail and 11 duplicated or non-GenAI examples, retaining 936: 374 Creation (39.96%), 303 Information (32.37%), and 259 Advice (27.67%).
  • Quantitative analysis used one random-intercepts logistic regression for perceived enaction of critical thinking and six random-intercepts linear regressions for perceived effort across Bloom’s taxonomy activities, with Participant ID as a random intercept. Benjamini-Hochberg correction was applied across 98 hypothesised predictors, yielding a corrected p-value threshold of 0.007. Qualitative analysis used open coding of free-text responses; one researcher initially coded 50 responses with three others to build a codebook, then two researchers coded the remaining 269 responses.

Limitations

  • The paper states that participants occasionally conflated reduced effort in using GenAI with reduced effort in critical thinking with GenAI, potentially leading to inaccurate self-reporting; future studies could use think-aloud protocols or task-based assessments.
  • The paper states that subjective task confidence may not be well-calibrated with objective expertise, and that the survey was English-only, with no representation of non-English-speaking or multilingual contexts.
  • The paper states that the sample was biased toward younger, more technologically skilled participants who use GenAI at work at least weekly, and that GenAI tools are constantly evolving; the task taxonomy was coarse-grained, and future longitudinal or telemetry/experience-sampling work is suggested. The text also makes evident that the analysis does not establish causation, as the paper itself notes.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3706598.3713778

Cites

133 of 133 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

12 Valid · 5 Not valid · 9 Not checked · 146 Own results and interpretations

78 more claims

Create a free account to see all 172 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 68
  • It may state two separate claims as one. 35

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

20 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 118 references

Research works printed but not linked: 11

  1. Benjamin S Bloom, Max D Engelhart, Edward J Furst, Walquer H Hill, David R Krathwohl, et al. 1956. Taxonomy of educational objetives: the classification of educational goals: handbook I: cognitive domain. Technical Report. New York, US: D. Mckay.
  2. Ooi Yan Chiew An Qi Lai and Wen Xin Liew. 2020. Digital technology overuse as a predictor of digital amnesia and productivity. Ph. D. Dissertation. UTAR.
  3. Amir Dirin Ari Alamäki and Jyrki Suomala. 2019. Digital amnesia and personal dependency in smart devices: A challenge for AI. Proceedings of Fake Intelligence Online Summit 2019 (2019).
  4. Peter A Facione Carol A Sanchez Noreen C Facione and Joanne Gainen. 1995. The disposition toward critical thinking. The Journal of General Education 44 1 (1995) 1–25.
  5. Hong Jiao and Robert W Lissitz. 2020. Application of Artificial Intelligence to Assessment. IAP.
  6. Deanna Kuhn. 1993. Connecting scientific and informal reasoning. Merrill-Palmer Quarterly (1982-) (1993) 74–103.
  7. George Lakoff and Mark Johnson. 2008. Metaphors we live by. University of Chicago press.
  8. Samir Passi and Mihaela Vorvoreanu. 2022. Overreliance on AI literature review. Microsoft Research (2022).
  9. Richard Paul and Linda Elder. 2020. The miniature guide to critical thinking concepts and tools (8th edition ed.). Rowman & Littlefield, Lanham, Md. OCLC: on1132213785.
  10. Advait Sarkar. 2024. Large Language Models Cannot Explain Themselves. In ACM CHI 2024 Workshop on Human-Centered Explainable AI (HCXAI).
  11. Donna Wilson and Marcus Conyers. 2016. Teaching students to drive their brains: Metacognitive strategies, activities, and lesson ideas. Ascd.

Cited by

See all 517

Metadata

Authors
Lee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N.
Affiliations
  • Carnegie Mellon University (US)
  • Microsoft Research (United Kingdom) (GB)
Venue
Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2025-04-25
Pages
1-22
OpenAlex
W4409885395
Cited by
490
References
118
Citations by year
2026: 328 · 2025: 166
FWCI
573.88
Type
conference paper
Language
en
Open access
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Social Sciences › Education › Education and Critical Thinking Development
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
Keywords
generative AI, critical thinking, knowledge workers, self-confidence, cognitive effort, fact-checking, response integration, AI tool design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. https://app.cochat.ai/papers/10.1145/3706598.3713778
The paper
Lee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025). The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3706598.3713778

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.