Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index.

Performance, Workload, Emotion, and Self-Efficacy of Novice Programmers Using AI Code Generation

Open access
Gardella, Nicholas, Pettit, Raymond, Riggs, Sara L. · 2024 · Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1

Abstract

Artificial Intelligence-driven Development Environments (AIDEs) offer developers revolutionary computer programming assistance. There is great potential in incorporating AIDEs into Computer Science education; however, the effects of these tools should be fully examined before doing so. Here, a within-subjects study was conducted to compare the programming performance, workload, emotion, and self-efficacy of seventeen novices coding with and without use of the GitHub Copilot AIDE under time pressure. Results showed that using the AIDE significantly increased programming efficiency and reduced effort and mental workload but did not significantly impact emotion or self-efficacy. However, participants' performance improved with more experience using the AI, and their self-efficacy followed. The results suggest that students who try AIDEs will likely be tempted to use them for time-sensitive work. There is no evidence that providing AIDEs will aid struggling students, but there is a clear need for students to practice with AI to become competent and confident using it.

Overview

Integrity signals

Registry

Crossref · 10.1145/3649217.3653615

Cites

34 of 35 references verified1 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

The text we found for this paper looks like another document, so its claims and self-check weren’t run. The registry and citation checks still apply.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The text we found for this paper looks like another document, so the self-check wasn’t run.

Citations

Cites

Show all 28 references

Research works printed but not linked: 6

  1. 10.1007/s40593-015-0069--5 · doi:10.1007/s40593-015-0069--5
  2. Torsten Hothorn, Kurt Hornik, Mark A. van de Wiel, Henric Winell, and Achim Zeileis. 2015. Conditional inference procedures in a permutation test framework. Coin Package R Version 31 3 Retrieved April 6, (2015), 2016.
  3. Boris Iglewicz and David C. Hoaglin. 1993. Volume 16: how to detect and handle outliers. Quality Press.
  4. Laerd Statistics. 2015. Tutorials for SPSS Statistics. Retrieved December 12, 2023 from https://statistics.laerd.com/
  5. Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. Codegen: An open large language model for code with multi-turn program synthesis. ArXiv Prepr. ArXiv220313474 (2022).
  6. Julie Pallant. 2007. SPSS Survival Manual. Open University Press New York, NY, USA.

Cited by

See all 16

Metadata

Authors
Gardella, N., Pettit, R., & Riggs, S. L.
Affiliations
University of Virginia (US)
Venue
Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1
Publisher
ACM
Published
2024-07-03
Pages
290-296
OpenAlex
W4400268466
Cited by
15
References
28
Citations by year
2026: 9 · 2025: 6
FWCI
6.58
Type
conference paper
Language
en
Open access
Locations
ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) (publishedVersion, cc-by)
Funders
  • National Science Foundation (1842490 and 1829004)
  • Commonwealth Cyber Initiative
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Social Sciences › Education › Artificial Intelligence in Education
Keywords
AI-assisted code generation, GitHub Copilot, novice programmers, programming education, mental workload, emotions, self-efficacy, programming performance, time pressure

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A second model labelled each claim on 2026-10-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Performance, Workload, Emotion, and Self-Efficacy of Novice Programmers Using AI Code Generation. https://app.cochat.ai/papers/10.1145/3649217.3653615
The paper
Gardella, N., Pettit, R., & Riggs, S. L. (2024). Performance, Workload, Emotion, and Self-Efficacy of Novice Programmers Using AI Code Generation. Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1. https://doi.org/10.1145/3649217.3653615

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.