Transformed by Transformers: Navigating the AI Coding Revolution for Computing Education: An ITiCSE Working Group Conducted by Humans
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
The recent advent of highly accurate and scalable large language models (LLMs) has taken the world by storm. From art to essays to computer code, LLMs are producing novel content that until recently was thought only humans could produce. Recent work in computing education has sought to understand the capabilities of LLMs for solving tasks such as writing code, explaining code, creating novel coding assignments, interpreting programming error messages, and more. However, these technologies continue to evolve at an astonishing rate leaving educators little time to adapt. This working group seeks to document the state-of-the-art for code generation LLMs, detail current opportunities and challenges related to their use, and present actionable approaches to integrating them into computing curricula.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models
Sarsa, S., Denny, P., Hellas, A., & Leinonen, J. (2022) · Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 1 · doi:10.1145/3501385.3543957 · Cited by 497 · cited 1×
“creating educational resources such as novel, personalized programming exercises [2, 10], code explanations that could help support students when they are working on exercises [6, 8], and enhanced programming error messages that might be easier to understand for novice programmers [7].”
BackgroundSelf-citationThe Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming
Finnie-Ansley, J., Denny, P., Becker, B. A., Luxton-Reilly, A., & Prather, J. (2022) · Proceedings of the 24th Australasian Computing Education Conference · doi:10.1145/3511861.3511863 · Cited by 463 · cited 6×
“This working group seeks to document the state-of-the-art for code generation LLMs, detail current opportunities and challenges related to their use, and present actionable approaches to integrating them into computing curricula.”
BackgroundInfluentialSelf-citationStudying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming
Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., & Grossman, T. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580919 · Cited by 366 · cited 1×
“Students using these tools can also quickly become lost while reading code they didn’t write [5] or lazily drift from code suggestion to code suggestion without understanding what they are doing [9].”
BackgroundPrather, J., Reeves, B. N., Denny, P., Becker, B. A., Leinonen, J., Luxton-Reilly, A., Powell, G., Finnie-Ansley, J., & Santos, E. A. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3617367 · Cited by 256 · cited 1×
“Students using these tools can also quickly become lost while reading code they didn’t write [5] or lazily drift from code suggestion to code suggestion without understanding what they are doing [9].”
BackgroundSelf-citationUsing Large Language Models to Enhance Programming Error Messages
Leinonen, J., Hellas, A., Sarsa, S., Reeves, B., Denny, P., Prather, J., & Becker, B. A. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569770 · Cited by 235
Self-citationMacNeil, S., Tran, A., Hellas, A., Kim, J., Sarsa, S., Denny, P., Bernstein, S., & Leinonen, J. (2022) · arXiv · doi:10.48550/arxiv.2211.02265 · Cited by 195
Self-citationMacNeil, S., Tran, A., Hellas, A., Kim, J., Sarsa, S., Denny, P., Bernstein, S., & Leinonen, J. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569785 · Cited by 195 · cited 1×
“creating educational resources such as novel, personalized programming exercises [2, 10], code explanations that could help support students when they are working on exercises [6, 8], and enhanced programming error messages that might be easier to understand for novice programmers [7].”
BackgroundSelf-citationUsing GitHub Copilot to Solve Simple Programming Problems
Wermelinger, M. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569830 · Cited by 191
Becker, B. A., Denny, P., Finnie-Ansley, J., Luxton-Reilly, A., Prather, J., & Santos, E. A. (2022) · arXiv · doi:10.48550/arxiv.2212.01020 · Cited by 36
Self-citationRobosourcing Educational Resources -- Leveraging Large Language Models for Learnersourcing
Denny, P., Sarsa, S., Hellas, A., & Leinonen, J. (2022) · arXiv · doi:10.48550/arxiv.2211.04715 · Cited by 21 · cited 1×
“creating educational resources such as novel, personalized programming exercises [2, 10], code explanations that could help support students when they are working on exercises [6, 8], and enhanced programming error messages that might be easier to understand for novice programmers [7].”
BackgroundSelf-citation
Cited by
The Robots Are Here: Navigating the Generative AI Revolution in Computing Education
Prather, J., Denny, P., Leinonen, J., Becker, B. A., Albluwi, I., Craig, M., Keuning, H., Kiesler, N., Kohn, T., Luxton-Reilly, A., et al. (2023) · Proceedings of the 2023 Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3623762.3633499 · Cited by 396
Self-citationGenerative AI in Computing Education: Perspectives of Students and Instructors
Zastudil, C., Rogalska, M., Kapp, C., Vaughn, J., & MacNeil, S. (2023) · 2023 IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie58773.2023.10343467 · Cited by 136
“These concerns have been echoed by position papers [7], [12], a working group [13], and an investigation into instructor perspectives [14].”
BackgroundSelf-citationThe Effects of Generative AI on Computing Students’ Help-Seeking Preferences
Hou, I., Mettille, S., Man, O., Li, Z., Zastudil, C., & MacNeil, S. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636248 · Cited by 97
“Recently, there has been significant excitement about the potential for using generative AI in computing classrooms [33, 51, 70] with researchers already demonstrating many remarkable capabilities of generative AI including generating code explanations [34, 39, 42], enhancing error messages [35],…”
BackgroundSelf-citationKumar, H., Xiao, R., Lawson, B., Musabirov, I., Shi, J., Wang, X., Luo, H., Williams, J. J., Rafferty, A. N., Stamper, J., et al. (2024) · Proceedings of the Eleventh ACM Conference on Learning @ Scale · doi:10.1145/3657604.3662042 · Cited by 53
“LLMs that are capable of generating computer code with relative ease have in particular believed to be highly applicable to aiding computer science students or students in other fields who lack a computer science background [38].”
MethodMailach, A., Gorgosch, D., Siegmund, N., & Siegmund, J. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10561-6 · Cited by 13
“Closest to our work are studies conducted by Prather and others, who observe how students interact with LLMs in a CS1 course (Prather et al. 2023), and Kazemitabaar and others, who extract interaction patterns from students using Codex in a self-paced learning environment (Kazemitabaar et al. 2023).”
Metadata
- Authors
- Prather, J., Denny, P., Leinonen, J., Becker, B. A., Albluwi, I., Caspersen, M. E., Craig, M., Keuning, H., Kiesler, N., Kohn, T., Luxton-Reilly, A., MacNeil, S., Petersen, A., Pettit, R., Reeves, B. N., & Savelka, J.
- Venue
- Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 2
- Publisher
- ACM
- Published
- 2023-06-29
- Pages
- 561-562
- OpenAlex
- W4382515993
- Cited by
- 26
- References
- 11
- FWCI
- 8.07
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Transformed by Transformers: Navigating the AI Coding Revolution for Computing Education: An ITiCSE Working Group Conducted by Humans. https://app.cochat.ai/papers/10.1145/3587103.3594206
- The paper
- Prather, J., Denny, P., Leinonen, J., Becker, B. A., Albluwi, I., Caspersen, M. E., Craig, M., Keuning, H., Kiesler, N., Kohn, T., Luxton-Reilly, A., MacNeil, S., Petersen, A., Pettit, R., Reeves, B. N., & Savelka, J. (2023). Transformed by Transformers: Navigating the AI Coding Revolution for Computing Education: An ITiCSE Working Group Conducted by Humans. Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 2. https://doi.org/10.1145/3587103.3594206
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.