Cited by 321Teaching and Learning ProgrammingArtificial Intelligence in EducationSoftware Engineering Research

Computing Education in the Era of Generative AI

Open access
Denny, Paul, Prather, James, Becker, Brett A., Finnie-Ansley, James, Hellas, Arto, Leinonen, Juho, Luxton-Reilly, Andrew, Reeves, Brent N. +2 more·2024·Communications of the ACM

Abstract

Challenges and opportunities faced by computing educators and students adapting to LLMs capable of generating accurate source code from natural-language problem descriptions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3624720

Cites

42 of 42 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Current plagiarism detection tools fail to recognize original essays produced by ChatGPT.

    “Researchers such as Denny et al. [4], Khalil and Er [10], and Susnjak [13] also address the limitations of current plagiarism detection tools, which fail to recognize the original essays produced by ChatGPT, indicating a need for advancements in detection technologies and updated classroom policies.”

    From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [4]

    The Academic integrity passage (indexed under “Metacognition”; page unavailable) says common plagiarism detection tools are often ineffective against AI-generated solutions in computing coursework. It does not establish that the tools fail to recognize ChatGPT-produced essays specifically, and “often ineffective” is narrower than an unqualified failure. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • Popular AI detection tools such as iThenticate, MOSS, and Turnitin are inadequate for the needs of educators.

    “Several authors lament the current state of popular AI detection tools, such as iThenticate, MOSS, and Turnitin, stating they are inadequate for the needs of educators [4,9,10].”

    From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [4]

    The Metacognition section says common plagiarism-detection tools are often ineffective against AI-generated solutions, creating concerns for educators monitoring academic integrity. It does not identify iThenticate, MOSS, or Turnitin or assess those tools individually. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • Generative AI such as GPT-4o CoPilot and OpenAI's GPT-4 enables the generation of sophisticated code from natural-language prompts.

    “Generative AI, such as GPT-4o CoPilot and OpenAI's GPT-4, enables the generation of sophisticated code from natural-language prompts, raising concerns about student overreliance and potential academic misconduct. This issue is reported by Denny et al. [4].”

    From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [4]

    The Abstract says that tools including GPT-4 and GitHub Copilot can generate source code from natural-language prompts. The sections on Plentiful learning resources and Academic misconduct discuss student overreliance and integrity concerns. Our copy does not mention GPT-4o, so it does not support that specific example. (LLM, conf=0.87)

    Experimental — accuracy not yet measured
  • The generation of sophisticated code from natural-language prompts raises concerns about student overreliance and potential academic misconduct.

    “Generative AI, such as GPT-4o CoPilot and OpenAI's GPT-4, enables the generation of sophisticated code from natural-language prompts, raising concerns about student overreliance and potential academic misconduct. This issue is reported by Denny et al. [4].”
  • This model helps reconcile the performance-learning paradox noted in recent empirical and review work.

    “This model helps reconcile the “performance–learning paradox” noted in recent empirical and review work (Yan et al., 2025; Denny et al., 2024).”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. [1]1. research self-paced learning environment. In Proceedings of the 23 rd Koli Calling Conf. on Computing Education Research (2023), 10.
  2. [33]33. Kazemitabaar, M. et al. How novices use LLM-based code generators to solve CS1 coding tasks in a 33. Sarkar, A. et al. What is it like to program with artificial intelligence? (2022); arXiv:cs. HC/2208.06213
  3. [38]38. Thompson, E., Whalley, J., Lister, R., and Simon, B. Code classification as a learning and assessment exercise for novice programmers. In Proceedings of the 19 th Annual Conf. of the National Advisory Committee on Computing Qualifications (2006), 291-298.
  4. [40]40. Wang, S., Piech, C., and Mitchell, J.C. A large scale RCT on effective error messages in CS1. In Proceedings of the 55 th ACM Technical Symp. on Computer Science Education 1, Association for Computing Machinery (2023), 7.
  5. [43]43. Zilles, C.B., West, M., Herman, G.L., and Bretl, T. Every university should have a computer-based testing facility. In Proceedings of the 11 th Intern. Conf. on Computer Supported Education, 1.

Cited by

Sorted by citations

Authors
Denny, P., Prather, J., Becker, B. A., Finnie-Ansley, J., Hellas, A., Leinonen, J., Luxton-Reilly, A., Reeves, B. N., Santos, E. A., & Sarsa, S.
Affiliations
  • University of Auckland (NZ)
  • Abilene Christian University (US)
  • University College Dublin (IE)
  • Aalto University (FI)
Venue
Communications of the ACM
Published
2024-01-25
Volume
67
Issue
2
Pages
56-67
ISSN
  • 0001-0782 (ISSN-L)
  • 1557-7317
OpenAlex
W4390985631
Cited by
316
References
38
Citations by year
2026: 136 · 2025: 102 · 2024: 70 · 2023: 8
FWCI
116.20
Type
journal article
Language
en
Open access
Locations
  • Communications of the ACM (publishedVersion)
  • Aaltodoc (Aalto University) (submittedVersion, other-oa)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
Keywords
generative AI, large language models, computing education, code generation, natural language programming, programming education, artificial intelligence in education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review