The DOI matches its DataCite record; 45 of the 45 references it cites were found in a registry or an index; none of the 58 sources its claims cite is checked for support yet.
Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension
Open accessAbstract
Software developers spend significant time reading and understanding code, increasingly supplemented by generative AI. While these tools aim to reduce the difficulty of reading code, interacting with them introduces a dual-task environment that may impose extraneous cognitive load that is challenging to capture. This study investigates how access to generative AI affects developers' real-time cognitive workload, code comprehension performance, and self-perceived code comprehension strategies. We conducted a 3x2 within-subjects study with 20 novice CS students reviewing Java code snippets of varying lengths with or without AI assistance, followed by an AI-unassisted comprehension quiz. Cognitive workload was continuously captured using a Detection-Response Task (tDRT) during both reading and quiz phases, and compared to post-quiz NASA-TLX and SEQ scores. AI assistance significantly increased objective cognitive workload, raising tDRT reaction times by 11.2%, yet participants' perceived workload remained unchanged. Code comprehension scores were high under both conditions, with no evidence that AI assistance improved comprehension at the difficulty levels evaluated; a ceiling effect limited power to detect benefits on harder tasks. Qualitatively, participants reported trusting the AI conditionally -verifying its output against their own reading for more complex snippets, but directly trusting it for simpler code- and, while many felt their reading process was unchanged, a notable subset shifted toward skimming or reading the AI's explanations in place of the code itself. These findings reveal that developers do not accurately perceive the cognitive resources they expend when managing AI tools. This study provides evidence of added cognitive effort associated with using generative AI, pointing towards a need to account for this dual-task penalty in software engineering tools and education.
Overview
CoChat Summary
TL;DR
In a 3×2 within-subjects study with 20 novice computer science students, AI assistance during Java code comprehension increased objective cognitive workload (tDRT reaction times +11.2%) without changing perceived workload or improving quiz accuracy. The paper reports a dual-task penalty that persisted into an AI-unassisted quiz phase and qualitative shifts toward verification, skimming, or displacement.
Key findings
- AI assistance significantly increased objective cognitive workload: tDRT reaction times were 11.2% higher overall (χ² = 17.37, p < .001). The AI × Code Size interaction was significant (p = .016), with elevations of 8.9% for small, 15.2% for medium, and 9.5% for large snippets; AI × Java Experience was not significant (p = .828). Subjective workload did not change significantly on NASA-TLX (p = .180) or SEQ (p = .161).
- The workload elevation persisted after AI was removed: during the quiz phase, the AI condition still showed reaction times 6.93% higher than control (p = .007), while during reading the increase was 12.7% (p < .001); the AI × Task Phase interaction was significant (χ² = 7.10, p = .008).
- AI assistance did not improve code comprehension: mean accuracy was AI M = 3.22, SD = 0.99 vs. No-AI M = 3.23, SD = 1.08; the planned mixed-effects model found no significant effect (p = .982), and an ordinal sensitivity analysis agreed (p = .692). Scores clustered near the maximum, producing a ceiling effect.
- Qualitative analysis identified three strategies—Resistance, Scaffolding, and Displacement—with trust generally conditional and scaled to perceived task complexity; some participants shifted toward skimming code or reading the AI’s explanations instead of the code itself.
Methods
- A 3×2 within-subjects experiment with N = 20 novice computer science students (22 completed; 2 excluded) recruited from two Java-based courses at Colorado State University. Participants completed six Java comprehension tasks across small (~30 lines), medium (~45 lines), and large (~70 lines) snippets, each with and without AI assistance; order was randomized/counterbalanced.
- Cognitive workload was continuously captured with a tactile Detection-Response Task (tDRT) during reading and quiz phases, supplemented by post-task NASA-TLX and SEQ questionnaires. Comprehension was assessed with four multiple-choice questions per task covering maintenance, summarization, mental execution, and structural mapping; exit survey open questions addressed trust and reading-behavior changes.
- Analysis used linear mixed-effects models in R with log2-transformed reaction times and likelihood ratio tests across six models; RQ2 also used an ordinal cumulative-link mixed model as sensitivity analysis; RQ3 used inductive thematic analysis with affinity diagramming.
Limitations
- Stated external validity: the student sample may not generalize to professional software engineers; results likely capture relatively novice AI adoption; the external browser interface may differ from IDE-integrated tools.
- Stated internal validity: typing could mechanically interfere with the finger-mounted tDRT, so typing intervals were removed (2.6% of observations); the highest workload occurred during quiz phases where no typing occurred.
- Stated conclusion validity: elevated cognitive load is not inherently detrimental and could reflect engagement; the ceiling effect in performance data limits determining whether increased load would benefit or harm comprehension on more complex tasks. Evident from the text: small N = 20 with sensitivity to about a 5.9% RT change, a single AI tool (Gemini), short snippets, and no prompt-engineering training.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
0 Valid · 1 Not valid · 40 Not checked · 106 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“retaining those confounded trials would have artificially inflated the estimated AI effect (increasing the coefficient from 0.1536 [11.2%] to 0.1828 [13.5%])”
§ Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension > A. Participants and Recruitment
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“no evidence that AI assistance improved comprehension at the difficulty levels evaluated”
- Also stated in: § I. INTRODUCTION · plain
“we found no evidence that AI assistance improved comprehension at the difficulty levels evaluated”
§ Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § I. INTRODUCTION · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while many felt their reading process was unchanged”
- Also stated in: § I. INTRODUCTION · plain
“while many participants felt their habits remained unchanged between conditions”
§ Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § I. INTRODUCTION · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“a notable subset shifted toward skimming or reading the AI's explanations in place of the code itself”
- Also stated in: § I. INTRODUCTION · plain
“a notable subset shifted towards skimming the code during the AI-assisted condition, or relying entirely on the AI's natural language explanation”
§ Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § I. INTRODUCTION · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Code comprehension scores were high under both conditions”
- Also stated in: § Abstract · plain
“scores were high under both conditions”
§ Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
79 more claims
Create a free account to see all 147 claims and every source verdict.
- It may state two separate claims as one. 43
- Probably not a substantial claim. 29
- The quote only partly states this claim. 1
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentz = 2.47reported p = .036 · recomputed p = .014
“(z = 2.47, p = .036”
The paper states a correction for multiple tests
- Consistentz = 2.44reported p = .039 · recomputed p = .015
“(z = 2.44, p = .039”
The paper states a correction for multiple tests
- Consistentz = 0.25reported p = .968 · recomputed p = .803
“(z = 0.25, p = .968”
The paper states a correction for multiple tests
- Consistentz = 0.4reported p = .692 · recomputed p = .689
“,z = 0.40, p = .692”
- Consistentz = 0.25reported p = .968 · recomputed p = .803
“z = 0.25, p = .968”
The paper states a correction for multiple tests
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension > A. Participants and Recruitment
“overall hit rates remained consistently high (above 95%).”
§ Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension > A. Participants and Recruitment
“hit rates averaged 93% in the AI-assisted condition compared to 97% in the non-AI condition.”
Citations
Cites
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297
Monsell, S. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00028-7 · Cited by 2,984
SIMR : an R package for power analysis of generalized linear mixed models by simulation
Green, P., & MacLeod, C. J. (2016) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12504 · Cited by 2,467
Complacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660
Program comprehension during software maintenance and evolution
Von Mayrhauser, A., & Vans, A. M. (1995) · Computer · doi:10.1109/2.402076 · Cited by 575
Revisiting methodological issues in transcript analysis: Negotiated coding and reliability
Garrison, D. R., Cleveland-Innes, M., Koole, M., & Kappelman, J. (2006) · The Internet and Higher Education · doi:10.1016/j.iheduc.2005.11.001 · Cited by 520
Using an LLM to Help With Code Understanding
Nam, D., Macvean, A., Hellendoorn, V., Vasilescu, B., & Myers, B. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639187 · Cited by 487
Research works printed but not linked: 2
- --, "Dynamic code cognition behaviors for large scale code," in Proceedings 1994 IEEE 3rd Workshop on Program Comprehension- WPC'94. IEEE, 1994, pp. 74-81.
- K. Holtzblatt, J. B. Wendell, and S. Wood, Rapid contextual design: a how-to guide to key techniques for user-centered design. Elsevier, 2004.
Cited by
Still being checked.
Metadata
- Authors
- White, M., Rose, M., Buckman, M., McDonnell, A., Ortega, F., & Trinkenreich, B.
- Affiliations
- Colorado State University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2611.12019
- OpenAlex
- W7225693140
- Cited by
- 0
- References
- 45
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Keywords
- cognitive load, generative AI, program comprehension, AI-assisted programming, perceived workload, NASA-TLX, detection response task, novice programmers, trust in AI, software engineering education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension. https://app.cochat.ai/papers/10.48550/arxiv.2611.12019
- The paper
- White, M., Rose, M., Buckman, M., McDonnell, A., Ortega, F., & Trinkenreich, B. (2026). Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension. arXiv. https://doi.org/10.48550/arxiv.2611.12019
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.