The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Computing Education in the Era of Generative AI
Open accessAbstract
Challenges and opportunities faced by computing educators and students adapting to LLMs capable of generating accurate source code from natural-language problem descriptions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Researchers such as Denny et al. [4], Khalil and Er [10], and Susnjak [13] also address the limitations of current plagiarism detection tools, which fail to recognize the original essays produced by ChatGPT, indicating a need for advancements in detection technologies and updated classroom policies.”
From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Several authors lament the current state of popular AI detection tools, such as iThenticate, MOSS, and Turnitin, stating they are inadequate for the needs of educators [4,9,10].”
From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Generative AI, such as GPT-4o CoPilot and OpenAI's GPT-4, enables the generation of sophisticated code from natural-language prompts, raising concerns about student overreliance and potential academic misconduct. This issue is reported by Denny et al. [4].”
From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [4]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Generative AI, such as GPT-4o CoPilot and OpenAI's GPT-4, enables the generation of sophisticated code from natural-language prompts, raising concerns about student overreliance and potential academic misconduct. This issue is reported by Denny et al. [4].”
From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“This model helps reconcile the “performance–learning paradox” noted in recent empirical and review work (Yan et al., 2025; Denny et al., 2024).”
From Tool, tutor, or crutch?: A grounded theory of cognitive scaffolding and offloading in AI-assisted programming education (2026) · cites this paper as Denny et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · cited 2×
“For example, in a deep discussion of LLMs, Bommasani et al. highlight that it will become “much more complex for teachers to understand the extent of a student’s contribution” and to “regulate ineffective collaborations and detect plagiarism” [7].”
BackgroundEvaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · cited 2×
“Can a student with no programming knowledge, but who is armed with a code-generating LLM, pass typical programming assessments?”
BackgroundCompetition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043 · cited 4×
“Similarly, the developers of AlphaCode suggest such tools have “the potential for a positive, transformative impact on society, with a wide range of applications including computer science education” [21].”
BackgroundInfluentialPrompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
Reynolds, L., & McDonell, K. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451760 · Cited by 952 · cited 1×
“It is well known that the outputs produced by large language models are very sensitive to their inputs [32].”
BackgroundVaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660 · cited 1×
“Depending on the prompt, the resulting programs used varied programmatic structures, while ultimately favoring expected methods for each problem variation (i.e., for-loops for processing lists, and while-loops for processing standard input).”
BackgroundAutomatic Generation of Programming Exercises and Code Explanations Using Large Language Models
Sarsa, S., Denny, P., Hellas, A., & Leinonen, J. (2022) · Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 1 · doi:10.1145/3501385.3543957 · Cited by 497
Self-citationAsleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions
Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., & Karri, R. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833571 · Cited by 476
The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming
Finnie-Ansley, J., Denny, P., Becker, B. A., Luxton-Reilly, A., & Prather, J. (2022) · Proceedings of the 24th Australasian Computing Education Conference · doi:10.1145/3511861.3511863 · Cited by 463 · cited 6×
“This can facilitate discussions about different approaches and the quality of solutions, and provide opportunities for refactoring exercises [15].”
BackgroundResultInfluentialSelf-citationGrounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2023) · Proceedings of the ACM on Programming Languages · doi:10.1145/3586030 · Cited by 436 · cited 1×
“Other recent work has shown that developers are more successful working with Copilot when they decompose larger programming statements into smaller tasks and then explicitly prompt Copilot for each of the subtasks [1, 18].”
BackgroundThe Robots Are Here: Navigating the Generative AI Revolution in Computing Education
Prather, J., Denny, P., Leinonen, J., Becker, B. A., Albluwi, I., Craig, M., Keuning, H., Kiesler, N., Kohn, T., Luxton-Reilly, A., et al. (2023) · Proceedings of the 2023 Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3623762.3633499 · Cited by 396 · cited 1×
“For a more technical overview of the historical developments and future trends of language models, we direct the reader to the CACM article by Li [20].”
MethodSelf-citation
Research works printed but not linked: 4
- Porter, L. and Zingaro, D. Learn AI-Assisted Python Programming With GitHub Copilot and ChatGPT. Manning, Shelter Island, NY, USA (2023); https://bit.ly/4aE7CRa
- Sarkar A. et al. What is it like to program with artificial intelligence? (2022); arXiv:cs.HC/2208.06213
- Thompson, E., Whalley, J., Lister, R., and Simon, B. Code classification as a learning and assessment exercise for novice programmers. In Proceedings of the 19th Annual Conf. of the National Advisory Committee on Computing Qualifications (2006), 291--298.
- Wang, S., Piech, C., and Mitchell, J.C. A large scale RCT on effective error messages in CS1. In Proceedings of the 55th ACM Technical Symp. on Computer Science Education 1, Association for Computing Machinery (2023), 7.
Cited by
Sorted by citations
Artificial intelligence in teaching and teacher professional development: A systematic review
Tan, X., Cheng, G., & Ling, M. H. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100355 · Cited by 420
Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda
Bittle, K., & El-Gayar, O. (2025) · Information · doi:10.3390/info16040296 · Cited by 305
“Several authors lament the current state of popular AI detection tools, such as iThenti-cate, MOSS, and Turnitin, stating they are inadequate for the needs of educators [4,9,10].”
InfluentialComputer Science Curricula 2023
Kumar, A. N., Raj, R. K., Aly, S. G., Anderson, M., Becker, B. A., Blumenthal, R. L., Eaton, E., Epstein, S. L., Goldweber, M., Jalote, P., et al. (2024) · ACM eBooks · doi:10.1145/3664191 · Cited by 235
Self-citationThe Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers
Prather, J., Reeves, B. N., Leinonen, J., MacNeil, S., Randrianasolo, A. S., Becker, B. A., Kimmel, B., Wright, J., & Briggs, B. (2024) · Proceedings of the 2024 ACM Conference on International Computing Education Research - Volume 1 · doi:10.1145/3632620.3671116 · Cited by 196
Self-citationSun, L., & Zhou, L. (2024) · Journal of Educational Computing Research · doi:10.1177/07356331241277937 · Cited by 129
“As students become more pro fi cient with Gen-AI, they can learn better, thus improving their academic achievement (Denny et al., 2023).”
Metadata
- Authors
- Denny, P., Prather, J., Becker, B. A., Finnie-Ansley, J., Hellas, A., Leinonen, J., Luxton-Reilly, A., Reeves, B. N., Santos, E. A., & Sarsa, S.
- Affiliations
- University of Auckland (NZ)
- Abilene Christian University (US)
- University College Dublin (IE)
- Aalto University (FI)
- ORCID
- Venue
- Communications of the ACM
- Published
- 2024-01-25
- Volume
- 67
- Issue
- 2
- Pages
- 56-67
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W4390985631
- Cited by
- 316
- References
- 38
- Citations by year
- 2026: 136 · 2025: 102 · 2024: 70
- FWCI
- 116.20
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Communications of the ACM (publishedVersion)
- Aaltodoc (Aalto University) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- generative AI, large language models, computing education, code generation, natural language programming, programming education, artificial intelligence in education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
