The DOI matches its Crossref record; 133 of the 133 references it cites were found in a registry or an index; 23 of the 39 sources its claims cite were checked for support.
The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers
Open accessAbstract
The rise of Generative AI (GenAI) in knowledge workflows raises questions about its impact on critical thinking skills and practices.We survey 319 knowledge workers to investigate 1) when and how they perceive the enaction of critical thinking when using GenAI, and 2) when and why GenAI affects their effort to do so.Participants shared 936 first-hand examples of using GenAI in work tasks.Quantitatively, when considering both task-and user-specific factors, a user's task-specific self-confidence and confidence in GenAI are predictive of whether critical thinking is enacted and the effort of doing so in GenAI-assisted tasks.Specifically, higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more critical thinking.Qualitatively, GenAI shifts the nature of critical thinking toward information verification, response integration, and task stewardship.Our insights reveal new design challenges and opportunities for developing GenAI tools for knowledge work.
Overview
CoChat Summary
TL;DR
A survey of 319 knowledge workers who use GenAI at work at least weekly, drawing on 936 real-world examples, finds that higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more. GenAI also shifts perceived critical-thinking effort toward information verification, response integration, and task stewardship.
Key findings
- Participants reported enacting critical thinking in 555 of 936 examples (59.29%). Higher confidence in AI doing the task negatively correlated with perceived enaction of critical thinking (β = -0.69, p < 0.001); confidence in doing the task oneself (β = 0.26, p = 0.026), confidence in evaluating AI responses (β = 0.31, p = 0.046), and overall tendency to reflect on work (β = 0.52, p < 0.001) positively correlated with perceived enaction.
- Perceived effort decreased in the majority of examples: “much less effort” or “less effort” comprised 72% for Knowledge, 79% for Comprehension, 69% for Application, 72% for Analysis, 76% for Synthesis, and 55% for Evaluation. Higher confidence in AI negatively correlated with perceived effort for five of six activities (all except Application): Knowledge (β = -0.11, p = 0.029), Comprehension (β = -0.13, p = 0.014), Analysis (β = -0.15, p = 0.003), Synthesis (β = -0.12, p = 0.026), and Evaluation (β = -0.23, p < 0.001).
- Qualitatively, GenAI shifted critical-thinking effort in three ways: for Knowledge and Comprehension, from information gathering to information verification; for Application, from problem-solving to AI response integration; and for Analysis, Synthesis, and Evaluation, from task execution to task stewardship.
- Motivators for critical thinking included work quality (74/319), potential negative outcomes (116/319), and skill development (13/319). Inhibitors included awareness barriers such as secondary use of GenAI (14/319), trivial tasks (55/319), and trust/reliance on GenAI (83/319); motivation barriers such as lack of time (44/319) and not being part of job responsibilities (11/319); and ability barriers such as barriers to inspecting AI responses (58/319) and revising queries/improving responses (72/319).
Methods
- Online survey on Prolific of 319 knowledge workers who self-reported using GenAI tools at work at least once per week. The study received 333 responses and excluded 14 for low response quality; retained participants spent an average of 43.19 minutes (SD = 23.13) completing the survey and were compensated GBP £10.
- Participants shared three real-world GenAI work examples each. The study collected 957 examples, removed 11 lacking sufficient detail and 11 duplicated or non-GenAI examples, retaining 936: 374 Creation (39.96%), 303 Information (32.37%), and 259 Advice (27.67%).
- Quantitative analysis used one random-intercepts logistic regression for perceived enaction of critical thinking and six random-intercepts linear regressions for perceived effort across Bloom’s taxonomy activities, with Participant ID as a random intercept. Benjamini-Hochberg correction was applied across 98 hypothesised predictors, yielding a corrected p-value threshold of 0.007. Qualitative analysis used open coding of free-text responses; one researcher initially coded 50 responses with three others to build a codebook, then two researchers coded the remaining 269 responses.
Limitations
- The paper states that participants occasionally conflated reduced effort in using GenAI with reduced effort in critical thinking with GenAI, potentially leading to inaccurate self-reporting; future studies could use think-aloud protocols or task-based assessments.
- The paper states that subjective task confidence may not be well-calibrated with objective expertise, and that the survey was English-only, with no representation of non-English-speaking or multilingual contexts.
- The paper states that the sample was biased toward younger, more technologically skilled participants who use GenAI at work at least weekly, and that GenAI tools are constantly evolving; the task taxonomy was coarse-grained, and future longitudinal or telemetry/experience-sampling work is suggested. The text also makes evident that the analysis does not establish causation, as the paper itself notes.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
12 Valid · 5 Not valid · 9 Not checked · 146 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These findings suggest that a reflective approach toward the use of GenAI tools, which can lead to what prior work refers to as”
§ The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers > 4.2.1 Task Factors.
What the sources say
- Partly supportedcites [20] 20. Valerie Chen, Q. Vera Liao, Jennifer Wortman Vaughan, and Gagan Bansal. 2023. Understanding the Role of Human Intuition on Reliance in Human- AI Decision-Making with Explanations. Proceedings of the ACM on Human- Computer Interaction 7, CSCW2 (Sept. 2023), 1-32. https://doi.org/10.1145/ 3610219 sourceExperimental — accuracy not yet measured
Section 4.1 (page unavailable) reports that participants with strong intuition sometimes discounted an AI prediction, supporting a link between confidence in their own judgment and critical use of AI. But it also says they sometimes decided quickly without checking explanations when the AI agreed with them. The study concerns two AI-assisted prediction tasks, not GenAI use by knowledge workers, and does not establish the claim’s general likelihood comparison. (LLM, conf=0.86)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Quantitatively, when considering both task-and user-specific factors, a user's task-specific self-confidence and confidence in GenAI are predictive of whether critical thinking is enacted and the effort of doing so in GenAI-assisted tasks.”
§ The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers > Abstract
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Specifically, higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more critical thinking.”
§ The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers > Abstract
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The data shows a shift in cognitive effort as knowledge workers increasingly move from task execution to oversight when using GenAI.”
§ The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the study reveals that knowledge workers engage in critical thinking when using GenAI tools primarily to ensure the quality of their work.”
§ The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers > 1 Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
78 more claims
Create a free account to see all 172 claims and every source verdict.
- Probably not a substantial claim. 68
- It may state two separate claims as one. 35
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“our findings show that GenAI may undermine students’ motivation to engage in the cognitive effort that is necessary for deeper intellectual enjoyment (contra Lee et al., 2025; see also Stadler et al., 2024).”
From If ChatGPT can do it, where is my creativity? generative AI boosts performance but diminishes experience in creative writing (2025) · cites this paper as Lee et al., 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We might also be concerned about mechanised convergence in the increased use of GenAI; that AI generation tends to result in less diverse textual output (Lee et al., 2025).”
From If ChatGPT can do it, where is my creativity? generative AI boosts performance but diminishes experience in creative writing (2025) · cites this paper as Lee et al., 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Further, GenAI tools can increase developers' perceived productivity [Ziegler et al. 2024] and level of understanding [Prather et al. 2024] without delivering actual improvements, potentially leading to a false sense of confidence [Lee et al. 2025] among novices.”
From How Do Programming Students Use Generative AI? (2025) · cites this paper as Lee et al. 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Lee et al. [8] found that professionals using GenAI tools reported reduced cognitive effort but increased confidence in the quality of their outputs, revealing a mismatch between perceived and actual engagement.”
From From Offloading to Engagement: An Experimental Study on Structured Prompting and Critical Reasoning with Generative AI (2025) · cites this paper as [8]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In the present study, this same pattern was evident in the unguided condition. Although participants often believed they had 'used AI to check their ideas,' interview data revealed that many had in fact accepted the AI's structure wholesale.”
From From Offloading to Engagement: An Experimental Study on Structured Prompting and Critical Reasoning with Generative AI (2025) · cites this paper as Lee et al. [8]
Experimental — accuracy not yet measured
20 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 1×
“To correct for multiple comparisons, we apply the Benjamini–Hochberg procedure [9] with a total of 98 hypothesised predictors across the seven models, yielding a corrected p-value threshold of 0.007.”
InfluentialBasics of Qualitative Research
Ozanne, J. L., Strauss, A., & Corbin, J. (1992) · Journal of Marketing Research · doi:10.2307/3172751 · Cited by 17,850
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242 · cited 1×
“To mitigate this, using GenAI for individualised, content-focused feedback may help novice writers develop writing skills while improving productivity [58, 86, 144].”
Binder, J. R., Desai, R. H., Graves, W. W., & Conant, L. L. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhp055 · Cited by 4,060 · cited 1×
“Cognitive science shows that effective learning requires “grounding” information through multiple perspectives and examples [10, 11, 68], and GenAI can offer personalised analogies to aid this process [77, 90].”
Dewey, J. (1910) · D C Heath eBooks · doi:10.1037/10903-000 · Cited by 3,998
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · cited 1×
“We measure participants’ overall trust in GenAI, which has been shown to correlate with users’ attitudes and adoption of the use of the technologies [43, 76].”
Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence
Noy, S., & Zhang, W. (2023) · doi:10.2139/ssrn.4375283 · Cited by 2,157
The neurobiology of semantic memory
Binder, J. R., & Desai, R. H. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.10.001 · Cited by 2,114 · cited 1×
“Cognitive science shows that effective learning requires “grounding” information through multiple perspectives and examples [10, 11, 68], and GenAI can offer personalised analogies to aid this process [77, 90].”
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490 · cited 1×
“Proactive systems take the initiative [52] to interrupt the user to highlight the need and opportunity for critical thinking in situations where it is likely to be overlooked; a reactive approach would allow the user to explicitly request critical thinking assistance when it is consciously needed.”
Generative artificial intelligence enhances creativity but reduces the diversity of novel content
Doshi, A., & Hauser, O. (2023) · doi:10.2139/ssrn.4535536 · Cited by 1,350
Research works printed but not linked: 11
- Benjamin S Bloom, Max D Engelhart, Edward J Furst, Walquer H Hill, David R Krathwohl, et al. 1956. Taxonomy of educational objetives: the classification of educational goals: handbook I: cognitive domain. Technical Report. New York, US: D. Mckay.
- Ooi Yan Chiew An Qi Lai and Wen Xin Liew. 2020. Digital technology overuse as a predictor of digital amnesia and productivity. Ph. D. Dissertation. UTAR.
- Amir Dirin Ari Alamäki and Jyrki Suomala. 2019. Digital amnesia and personal dependency in smart devices: A challenge for AI. Proceedings of Fake Intelligence Online Summit 2019 (2019).
- Peter A Facione Carol A Sanchez Noreen C Facione and Joanne Gainen. 1995. The disposition toward critical thinking. The Journal of General Education 44 1 (1995) 1–25.
- Hong Jiao and Robert W Lissitz. 2020. Application of Artificial Intelligence to Assessment. IAP.
- Deanna Kuhn. 1993. Connecting scientific and informal reasoning. Merrill-Palmer Quarterly (1982-) (1993) 74–103.
- George Lakoff and Mark Johnson. 2008. Metaphors we live by. University of Chicago press.
- Samir Passi and Mihaela Vorvoreanu. 2022. Overreliance on AI literature review. Microsoft Research (2022).
- Richard Paul and Linda Elder. 2020. The miniature guide to critical thinking concepts and tools (8th edition ed.). Rowman & Littlefield, Lanham, Md. OCLC: on1132213785.
- Advait Sarkar. 2024. Large Language Models Cannot Explain Themselves. In ACM CHI 2024 Workshop on Human-Centered Explainable AI (HCXAI).
- Donna Wilson and Marcus Conyers. 2016. Teaching students to drive their brains: Metacognitive strategies, activities, and lesson ideas. Ascd.
Cited by
Tian, J., & Zhang, R. (2025) · Acta Psychologica · doi:10.1016/j.actpsy.2025.105725 · Cited by 232
Liu, X., & Zhong, B. (2025) · Educational Research Review · doi:10.1016/j.edurev.2025.100741 · Cited by 78
Rodger, D., Mann, S. P., Earp, B., Savulescu, J., Bobier, C., & Blackshaw, B. P. (2025) · Nurse Education in Practice · doi:10.1016/j.nepr.2025.104461 · Cited by 64
“Furthermore, there is evidence from other domains showing that the use of GenAI can diminish critical thinking and independent problem-solving skills over time (Lee et al., 2025).”
Mei, P., Brewis, D. N., Nwaiwu, F., Sumanathilaka, D., Alva-Manchego, F., & Demaree-Cotton, J. (2025) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2025.100140 · Cited by 50
AI deskilling is a structural problem
Ferdman, A. (2025) · AI & Society · doi:10.1007/s00146-025-02686-z · Cited by 104
Metadata
- Authors
- Lee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N.
- Affiliations
- Carnegie Mellon University (US)
- Microsoft Research (United Kingdom) (GB)
- ORCID
- Venue
- Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2025-04-25
- Pages
- 1-22
- OpenAlex
- W4409885395
- Cited by
- 490
- References
- 118
- Citations by year
- 2026: 328 · 2025: 166
- FWCI
- 573.88
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Education › Education and Critical Thinking Development
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- generative AI, critical thinking, knowledge workers, self-confidence, cognitive effort, fact-checking, response integration, AI tool design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. https://app.cochat.ai/papers/10.1145/3706598.3713778
- The paper
- Lee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025). The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3706598.3713778
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.