Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its DataCite record; 45 of the 45 references it cites were found in a registry or an index; none of the 58 sources its claims cite is checked for support yet.

Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension

Open access

Abstract

Software developers spend significant time reading and understanding code, increasingly supplemented by generative AI. While these tools aim to reduce the difficulty of reading code, interacting with them introduces a dual-task environment that may impose extraneous cognitive load that is challenging to capture. This study investigates how access to generative AI affects developers' real-time cognitive workload, code comprehension performance, and self-perceived code comprehension strategies. We conducted a 3x2 within-subjects study with 20 novice CS students reviewing Java code snippets of varying lengths with or without AI assistance, followed by an AI-unassisted comprehension quiz. Cognitive workload was continuously captured using a Detection-Response Task (tDRT) during both reading and quiz phases, and compared to post-quiz NASA-TLX and SEQ scores. AI assistance significantly increased objective cognitive workload, raising tDRT reaction times by 11.2%, yet participants' perceived workload remained unchanged. Code comprehension scores were high under both conditions, with no evidence that AI assistance improved comprehension at the difficulty levels evaluated; a ceiling effect limited power to detect benefits on harder tasks. Qualitatively, participants reported trusting the AI conditionally -verifying its output against their own reading for more complex snippets, but directly trusting it for simpler code- and, while many felt their reading process was unchanged, a notable subset shifted toward skimming or reading the AI's explanations in place of the code itself. These findings reveal that developers do not accurately perceive the cognitive resources they expend when managing AI tools. This study provides evidence of added cognitive effort associated with using generative AI, pointing towards a need to account for this dual-task penalty in software engineering tools and education.

Overview

CoChat Summary

TL;DR

In a 3×2 within-subjects study with 20 novice computer science students, AI assistance during Java code comprehension increased objective cognitive workload (tDRT reaction times +11.2%) without changing perceived workload or improving quiz accuracy. The paper reports a dual-task penalty that persisted into an AI-unassisted quiz phase and qualitative shifts toward verification, skimming, or displacement.

Key findings

  • AI assistance significantly increased objective cognitive workload: tDRT reaction times were 11.2% higher overall (χ² = 17.37, p < .001). The AI × Code Size interaction was significant (p = .016), with elevations of 8.9% for small, 15.2% for medium, and 9.5% for large snippets; AI × Java Experience was not significant (p = .828). Subjective workload did not change significantly on NASA-TLX (p = .180) or SEQ (p = .161).
  • The workload elevation persisted after AI was removed: during the quiz phase, the AI condition still showed reaction times 6.93% higher than control (p = .007), while during reading the increase was 12.7% (p < .001); the AI × Task Phase interaction was significant (χ² = 7.10, p = .008).
  • AI assistance did not improve code comprehension: mean accuracy was AI M = 3.22, SD = 0.99 vs. No-AI M = 3.23, SD = 1.08; the planned mixed-effects model found no significant effect (p = .982), and an ordinal sensitivity analysis agreed (p = .692). Scores clustered near the maximum, producing a ceiling effect.
  • Qualitative analysis identified three strategies—Resistance, Scaffolding, and Displacement—with trust generally conditional and scaled to perceived task complexity; some participants shifted toward skimming code or reading the AI’s explanations instead of the code itself.

Methods

  • A 3×2 within-subjects experiment with N = 20 novice computer science students (22 completed; 2 excluded) recruited from two Java-based courses at Colorado State University. Participants completed six Java comprehension tasks across small (~30 lines), medium (~45 lines), and large (~70 lines) snippets, each with and without AI assistance; order was randomized/counterbalanced.
  • Cognitive workload was continuously captured with a tactile Detection-Response Task (tDRT) during reading and quiz phases, supplemented by post-task NASA-TLX and SEQ questionnaires. Comprehension was assessed with four multiple-choice questions per task covering maintenance, summarization, mental execution, and structural mapping; exit survey open questions addressed trust and reading-behavior changes.
  • Analysis used linear mixed-effects models in R with log2-transformed reaction times and likelihood ratio tests across six models; RQ2 also used an ordinal cumulative-link mixed model as sensitivity analysis; RQ3 used inductive thematic analysis with affinity diagramming.

Limitations

  • Stated external validity: the student sample may not generalize to professional software engineers; results likely capture relatively novice AI adoption; the external browser interface may differ from IDE-integrated tools.
  • Stated internal validity: typing could mechanically interfere with the finger-mounted tDRT, so typing intervals were removed (2.6% of observations); the highest workload occurred during quiz phases where no typing occurred.
  • Stated conclusion validity: elevated cognitive load is not inherently detrimental and could reflect engagement; the ceiling effect in performance data limits determining whether increased load would benefit or harm comprehension on more complex tasks. Evident from the text: small N = 20 with sensitivity to about a 5.9% RT change, a single AI tool (Gemini), short snippets, and no prompt-engineering training.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2611.12019

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 tDRT Set Up
Fig. 1 tDRT Set UpCC-BY · From the PDF
Fig. 2 Study Procedure Diagram
Fig. 2 Study Procedure DiagramCC-BY · From the PDF
Fig. 3 Study Set Up
Fig. 3 Study Set UpCC-BY · From the PDF
Fig. 4 Main Effect of AI Assistance on Cognitive Workload
Fig. 4 Main Effect of AI Assistance on Cognitive WorkloadCC-BY · From the PDF
Fig. 5 Interaction between AI Assistance and Task Phase on Cognitive Workload
Fig. 5 Interaction between AI Assistance and Task Phase on Cognitive WorkloadCC-BY · From the PDF
Fig. 6 Interaction between AI Assistance and Code Size on Cognitive Workload
Fig. 6 Interaction between AI Assistance and Code Size on Cognitive WorkloadCC-BY · From the PDF

Verification

Claims the paper makes

0 Valid · 1 Not valid · 40 Not checked · 106 Own results and interpretations

79 more claims

Create a free account to see all 147 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 43
  • Probably not a substantial claim. 29
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentz = 2.47reported p = .036 · recomputed p = .014

    “(z = 2.47, p = .036”

    The paper states a correction for multiple tests

  • Consistentz = 2.44reported p = .039 · recomputed p = .015

    “(z = 2.44, p = .039”

    The paper states a correction for multiple tests

  • Consistentz = 0.25reported p = .968 · recomputed p = .803

    “(z = 0.25, p = .968”

    The paper states a correction for multiple tests

  • Consistentz = 0.4reported p = .692 · recomputed p = .689

    “,z = 0.40, p = .692”

  • Consistentz = 0.25reported p = .968 · recomputed p = .803

    “z = 0.25, p = .968”

    The paper states a correction for multiple tests

5 of 5

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 45 references

Research works printed but not linked: 2

  1. --, "Dynamic code cognition behaviors for large scale code," in Proceedings 1994 IEEE 3rd Workshop on Program Comprehension- WPC'94. IEEE, 1994, pp. 74-81.
  2. K. Holtzblatt, J. B. Wendell, and S. Wood, Rapid contextual design: a how-to guide to key techniques for user-centered design. Elsevier, 2004.

Cited by

Still being checked.

Metadata

Authors
White, M., Rose, M., Buckman, M., McDonnell, A., Ortega, F., & Trinkenreich, B.
Affiliations
Colorado State University (US)
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
OpenAlex
W7225693140
Cited by
0
References
45
Type
posted content
Language
en
Open access
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
Keywords
cognitive load, generative AI, program comprehension, AI-assisted programming, perceived workload, NASA-TLX, detection response task, novice programmers, trust in AI, software engineering education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension. https://app.cochat.ai/papers/10.48550/arxiv.2611.12019
The paper
White, M., Rose, M., Buckman, M., McDonnell, A., Ortega, F., & Trinkenreich, B. (2026). Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension. arXiv. https://doi.org/10.48550/arxiv.2611.12019

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.