Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
Advances in natural language processing have resulted in large language models (LLMs) that can generate code and code explanations. In this paper, we report on our experiences generating multiple code explanation types using LLMs and integrating them into an interactive e-book on web software development. Three different types of explanations -- a line-by-line explanation, a list of important concepts, and a high-level summary of the code -- were created. Students could view explanations by clicking a button next to code snippets, which showed the explanation and asked about its utility. Our results show that all explanation types were viewed by students and that the majority of students perceived the code explanations as helpful to them. However, student engagement varied by code snippet complexity, explanation type, and code snippet length. Drawing on our experiences, we discuss future directions for integrating explanations generated by LLMs into CS classrooms.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This has been inspired by many other instances where LLMs have already demonstrated remarkable effectiveness in generating novel language artifacts. These include course readings [22], code explanations [23, 28, 29, 41], model solutions [9, 35, 42], feedback [34], or responses to help requests [18, 26, 46, 47].”
From A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education (2024) · cites this paper as [28]
In the citing paper
“On the other hand, recent work has shown that AI tools can be used to produce high-quality learning resources, including programming exercises [32], explanations of code [24,28], and even on-demand feedback to students [27].”
From Instructor Perceptions of AI Code Generation Tools - A Multi-Institutional Interview Study (2024) · cites this paper as [28]
In the citing paper
“MacNeil et al. found that students generally found code explanations generated by LLMs useful for learning, but the perceived utility of the explanations and students' engagement with them varied by explanation type [137].”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [137]
In the citing paper
“However, non-code models such as GPT-3 have been found to be more apt at creating higher level code explanations [137].”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [137]
In the citing paper
“Students rated these explanations on a 5-point Likert scale, confirming that explanations matched the code and were useful for learning in general, whereas line-by-line explanations were rated as least helpful.”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [137]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660
Review of recent systems for automatic assessment of programming assignments
Ihantola, P., Ahoniemi, T., Karavirta, V., & Seppälä, O. (2010) · Proceedings of the 10th Koli Calling International Conference on Computing Education Research · doi:10.1145/1930464.1930480 · Cited by 554 · cited 1×
“through the use of automated grading systems [1, 18, 30].”
MethodAutomatic Generation of Programming Exercises and Code Explanations Using Large Language Models
Sarsa, S., Denny, P., Hellas, A., & Leinonen, J. (2022) · Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 1 · doi:10.1145/3501385.3543957 · Cited by 497 · cited 8×
“Explanations of code can also be generated by LLMs [24, 37].”
BackgroundMethodInfluentialSelf-citationOnline python tutor: embeddable web-based program visualization for cs education
Guo, P. J. (2013) · Proceeding of the 44th ACM technical symposium on Computer science education · doi:10.1145/2445196.2445368 · Cited by 396
Designing the whyline: a debugging interface for asking questions about program behavior
Ko, A. J., & Myers, B. A. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985712 · Cited by 361
PeerWise: students sharing their multiple choice questions
Denny, P., Hamer, J., Luxton-Reilly, A., & Purchase, H. (2008) · Proceedings of the Fourth international Workshop on Computing Education Research · doi:10.1145/1404520.1404526 · Cited by 211
Self-citationPair programming in education: a literature review
Hanks, B., Fitzgerald, S., McCauley, R., Murphy, L., & Zander, C. (2011) · Computer Science Education · doi:10.1080/08993408.2011.579808 · Cited by 205 · cited 1×
“For example in pair programming, students work together which requires them to often explain their code and their thinking process to their partner [16].”
BackgroundGenerating Diverse Code Explanations using the GPT-3 Large Language Model
MacNeil, S., Tran, A., Mogil, D., Bernstein, S., Ross, E., & Huang, Z. (2022) · Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 2 · doi:10.1145/3501709.3544280 · Cited by 200 · cited 6×
“In computer science classrooms, LLMs are beginning to be used to generate assignments [36], help students to write their code [3, 40], and to generate explanations of code snippets to facilitate learning [23].”
BackgroundMethodInfluentialSelf-citationWheel-Spinning: Students Who Fail to Master a Skill
Beck, J. E., & Gong, Y. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-39112-5_44 · Cited by 161 · cited 1×
“However, these grading systems have become increasingly focused on providing formative feedback such as hints [31, 34] error-specific feedback [24, 34] to help students avoid getting stuck in counterproductive behaviors when their answers are incorrect [4].”
BackgroundiSnap: Towards Intelligent Tutoring in Novice Programming Environments
Price, T. W., Dong, Y., & Lipovac, D. (2017) · Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education · doi:10.1145/3017680.3017762 · Cited by 140
Research works printed but not linked: 12
- Kirsti M Ala-Mutka. 2005. A survey of automated assessment approaches for programming assignments. Computer science education, Vol. 15, 2 (2005), 83--102.
- Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski David Dohan Ellen Jiang Carrie Cai Michael Terry Quoc Le and Charles Sutton. 2021. Program Synthesis with Large Language Models. arxiv: 2108.07732 [cs.PL] · arXiv
- Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in neural information processing systems. 1877--1901.
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021). · arXiv
- James Finnie-Ansley, Paul Denny, Brett A Becker, Andrew Luxton-Reilly, and James Prather. 2022. The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming. In Australasian Computing Education Conf. 10--19.
- Katy Ilonka Gero, Vivian Liu, and Lydia Chilton. 2022. Sparks: Inspiration for Science Writing Using Language Models. In Designing Interactive Systems Conf. (Virtual Event, Australia) (DIS '22). ACM, New York, NY, USA, 1002--1019.
- Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586 (2021). · arXiv
- José Carlos Paiva, José Paulo Leal, and Álvaro Figueira. 2022. Automated Assessment in Computer Science Education: A State-of-the-Art Review. ACM Transactions on Computing Education (TOCE) (2022).
- Jaanus Pöial. 2020. Challenges of Teaching Programming in StackOverflow Era. In International Conf. on Interactive Collaborative Learning. Springer, 703--710.
- Wengran Wang, Yudong Rao, Rui Zhi, Samiha Marwan, Ge Gao, and Thomas W. Price. 2020. Step Tutor: Supporting Students through Step-by-Step Example-Based Feedback. In Proceedings of the 2020 ACM Conf. on Innovation and Technology in Computer Science Education. ACM, New York, NY, USA, 391--397.
- Jacqueline L. Whalley, Raymond Lister, Errol Thompson, Tony Clear, Phil Robbins, P. K. Ajith Kumar, and Christine Prasad. 2006. An Australasian Study of Reading and Comprehension Skills in Novice Programmers, Using the Bloom and SOLO Taxonomies. In Proc. of the 8th Australasian Conf. on Computing Education - Volume 52. Australian Computer Society, Inc., AUS, 243--252.
- Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022. Wordcraft: Story Writing With Large Language Models. In 27th International Conf. on Intelligent User Interfaces. 841--852.
Cited by
Items 101–200 of 271
Interactions with Prompt Problems: A New Way to Teach Programming with Large Language ModelsPreprint
Prather, J. E., Denny, P. C., Leinonen, J., Smith, D. H., Reeves, B. N., MacNeil, S., Becker, B. A., Luxton-Reilly, A., Amarouche, T., & Kimmel, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.10759 · Cited by 27
Self-citationGPT-Powered Elicitation Interview Script Generator for Requirements Engineering Training
Görer, B., & Aydemir, F. B. (2024) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re59067.2024.00044 · Cited by 31
“For content creation, recent research efforts have focused on tasks closely related to coding, including the development of programming exercises [21] and the generation of code explanations [6].”
BackgroundHu, R., Peng, C., Ren, J., Jiang, B., Meng, X., Wu, Q., Gao, P., Wang, X., & Gao, C. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730262 · Cited by 22
Understanding the Role of Temperature in Diverse Question Generation by GPT-4
Agarwal, A., Mittal, K., Doyle, A., Sridhar, P., Wan, Z., Doughty, J. A., Šavelka, J., & Sakr, M. F. (2024) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3626253.3635608 · Cited by 21
“One of the key parameters influencing the outcomes of the natural language generation of GPT models is temperature .”
BackgroundExplaining Explanations: An Empirical Study of Explanations in Code Reviews
Widyasari, R., Zhang, T., Bouraffa, A., Maalej, W., & Lo, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708518 · Cited by 30
Prather, J., Reeves, B. N., Denny, P., Leinonen, J., MacNeil, S., Luxton-Reilly, A., Orvalho, J., Alipour, A., Alfageeh, A., Amarouche, T., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.12800 · Cited by 20
“MacNeil et al. reported that students found LLM-generated code explanations useful and helpful for their learning [44].”
Self-citationGenerative AI in Computer Science Education: Challenges and Opportunities
Franklin, D., Denny, P., Gonzalez-Maldonado, D. A., & Tran, M. (2025) · doi:10.1017/9781009581738 · Cited by 20
Self-citationNaamati-Schneider, L., & Alt, D. (2025) · Nurse Education in Practice · doi:10.1016/j.nepr.2025.104572 · Cited by 23
Agarwal, V., Garg, M. K., Dharmavaram, S., & Kumar, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.01687 · Cited by 27
Qiao, Y., Shihab, M. I. H., & Hundhausen, C. D. (2026) · ACM Transactions on Computing Education · doi:10.1145/3785366 · Cited by 31
“MacNeil et al. [55] recorded the timestamps when students opened and closed an explanation panel for a given code snippet, enabling calculation of total viewing time and number of views.”
InfluentialThe Rest of the Robots: Generative AI in Post-introductory Computing Education
Bouvier, D. J., Cipriano, B. P., Glassey, R. J., Petrovska, O., Anderson, E., Birillo, A., Dougherty, R. E., Pettit, R., Pombo, N. G. C. C., Rahimi, E., et al. (2025) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3760545.3783970 · Cited by 21
“6.14.3 Web Development Course: MacNeil et al. [74] discuss an experiment where different types of code explanations were generated using LLMs and incorporated into an interactive e-book for web software development.”
InfluentialGenerative AI And Software Variability - A Research Vision
Greiner, S., Schmid, K., Berger, T., Krieter, S., & Meixner, K. (2024) · International Workshop on Variability Modelling of Software-Intensive Systems (VaMoS) · doi:10.1145/3634713.3634722 · Cited by 18
Black, G., Vaidyan, V. M., & Comert, G. (2024) · IEEE Access · doi:10.1109/access.2024.3484947 · Cited by 23
“In response to these challenges, recent advances in machine learning—particularly the development of large language models (LLMs)—offer promising solutions [4], [5], [6].”
BackgroundFeng, T. H., Denny, P., Wuensche, B., Luxton-Reilly, A., & Hooper, S. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636263 · Cited by 16
Self-citationCompiler-Integrated, Conversational AI for Debugging CS1 Programs
Renzella, J., Vassar, A., Solano, L. L., & Taylor, A. (2025) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3641554.3701827 · Cited by 16
Liu, Y., Wen, Z., Weng, L., Woodman, O., Yang, Y., & Chen, W. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3410523 · Cited by 18
“For instance, MacNeil et al. incorporate LLM-generated code explanations within a web software development e-book [30].”
Using Large Language Models to Automatically Identify Programming Concepts in Code Snippets
Tran, A., Li, L., Rama, E., Angelikas, K., & Macneil, S. (2023) · Proceedings of the 2023 ACM Conference on International Computing Education Research - Volume 2 · doi:10.1145/3568812.3603482 · Cited by 13
Self-citationAnalysis of Generative AI Policies in Computing Course Syllabi
Ali, A., Collier, A. H., Dewan, U., McDonald, N., & Johri, A. (2025) · Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3641554.3701823 · Cited by 14
“For instance, research has explored how to use explanations to support computing learning [20].”
Kohn, T. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70043 · Cited by 15
Towards LLM-Generated Code Tours for Onboarding
Balfroid, M., Vanderose, B., & Devroey, X. (2024) · doi:10.1145/3643787.3648033 · Cited by 14
“We argue that LLMs might help generate code tours since they are already used for generating documentation and explaining code [11, 13].”
InfluentialSupervised Fine-Tuning LLMs to Behave as Pedagogical Agents in Programming EducationPreprint
Ross, E., Kansal, Y., Renzella, J., Vassar, A., & Taylor, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.20527 · Cited by 21
“In CS1 contexts, applications include solving simple coding problems [7, 10, 8, 11, 37], generating explanations for code [25, 21], and for assistance with resolving compilation errors [35, 18, 19].”
Mailach, A., Gorgosch, D., Siegmund, N., & Siegmund, J. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10561-6 · Cited by 13
“In our study, we focus on qualitatively analyzing how this usage can look in detail.”
Nowrozy, R. (2024) · Informatics · doi:10.3390/informatics11030045 · Cited by 15
MacNeil, S., Rogalska, M., Leinonen, J., Denny, P. C., Hellas, A., & Crosland, X. (2024) · ACM Virtual Global Computing Education Conference (SIGCSE Virtual) · doi:10.1145/3649165.3690100 · Cited by 20
Self-citationAuthoring Worked Examples for JAVA Programming with Human AI Collaboration
Hassany, M., Ke, J., Brusilovsky, P. L., Narayanan, A. B. L., & Akhüseyinoğlu, K. (2024) · ACM Symposium on Applied Computing · doi:10.1145/3605098.3636160 · Cited by 14
“Although there have been several attempts to use LLMs such as OpenAI Codex or ChatGPT [2, 8, 9] to generate code explanations, our work is the first attempt to integrate the power of LLM into an example authoring tool that enables prospective example authors to produce fully explained worked…”
Navigating the Ethical and Societal Impacts of Generative AI in Higher Computing Education
Mak, J., Nakatumba-Nabende, J., Clear, T., Clear, A., Abluwi, I., Andrei, O., Angeli, L., MacNeil, S., Oyelere, S. S., Hale Rattigan, M., et al. (2025) · Proceedings of the 2025 Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3760545.3783975 · Cited by 12
Self-citationŠavelka, J., Agarwal, A., Bogart, C., & Sakr, M. F. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-53656-4_8 · Cited by 21
Qiao, S., Denny, P. C., & Giacaman, N. (2025) · Australasian Computing Education Conference · doi:10.1145/3716640.3716651 · Cited by 15
Self-citationLarge Language Models (LLMs) in Programming Learning: The Current Research State and Agenda
Fu, Q., Zhao, Y., Jia, Z., & Zheng, Y. (2025) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2025.3622043 · Cited by 19
SocraticAI: Transforming LLMs into Guided CS Tutors Through Scaffolded InteractionPreprint
Sunil, K., & Thakkar, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.03501 · Cited by 15
“Prior research indicates that AI tutors can enhance learning when students remain actively engaged and reflective [8,7,12,6,10,9,3].”
Feng, H., Denny, P. C., Wuensche, B. C., Luxton-Reilly, A., & Whalley, J. L. (2024) · doi:10.1145/3680533.3697064 · Cited by 15
“Additionally, LLMs are capable of assisting CS educators by generating educational material [Leinonen et al. 2023; Liffiton et al. 2023; MacNeil et al. 2023], providing many new opportunities [Bernstein et al. 2024; Denny et al. 2024a].”
Self-citationBrown, N. C. C., Weill-Tessier, P., Leinonen, J., Denny, P. C., & Kölling, M. (2025) · ACM Transactions on Computing Education · doi:10.1145/3737885 · Cited by 15
Self-citationOpen, Small, Rigmarole – Evaluating Llama 3.2 3B’s Feedback for Programming Exercises
Azaiz, I., Kiesler, N., Strickroth, S., & Zhang, A. (2025) · International Journal of Engineering Pedagogy (iJEP) · doi:10.3991/ijep.v15i5.55359 · Cited by 11
Collini, E., Kurniadi, F. I., Nesi, P., & Pantaleo, G. (2025) · IEEE Access · doi:10.1109/access.2025.3614553 · Cited by 13
“Many industries, including healthcare [4], [5], education [6], [7], [8], and finance [9], [10], [11], now use The associate editor coordinating the review of this manuscript and approving it for publication was Ayman El-Baz .”
Show Me the Mastery Learning! Obstacles to Adoption and Opportunities for New Solutions
Álvarez, C., Falkner, N., Kinnunen, P., Šavelka, J., & Zhang, L. (2025) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3724363.3729104 · Cited by 11
Song, T., Tian, Q., Xiao, Y., & Liu, S. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-0730-0_28 · Cited by 15
Prompts First, FinallyPreprint
Reeves, B. N., Prather, J. E., Denny, P. C., Leinonen, J., MacNeil, S., Becker, B. A., & Luxton-Reilly, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.09231 · Cited by 11
“The production of learning resources has been well studied already, for example, with Balse et al. [2] and MacNeil et al. [16] demonstrating the effectiveness of LLM-generated code explanations in supporting student learning, and Sarsa et al. [22] and Logacheva et al. [15] documenting the potential…”
BackgroundSelf-citationThe Robots are Here: Navigating the Generative AI Revolution in Computing Education
Prather, J., Denny, P., Leinonen, J., Becker, B. A., Albluwi, I., Craig, M., Keuning, H., Kiesler, N., Kohn, T., Luxton-Reilly, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.00658 · Cited by 9
Self-citationMapping the Design Space of Interactions in Human-AI Text Co-creation Tasks
Ding, Z., & Chan, J. (2023) · arXiv · doi:10.48550/arxiv.2303.06430 · Cited by 9
“Previous research has indicated that Large language models (LLMs) can effectively handle well-defined, content curation tasks such as text summarization [9, 15], content refinement [17], and code explanation [18, 19].”
BackgroundAli, M., & Dasgupta, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713624 · Cited by 10
SE Perspective on LLMs: Biases in Code Generation, Code Interpretability, and Code Security Risks
Krasniqi, R., Xu, D., & Vieira, M. P. A. (2025) · ACM Computing Surveys · doi:10.1145/3774324 · Cited by 14
“Bias affects interpretability by complicating how models generate code explanations and justify internal decisions [32].”
Investigating Student Mistakes in Introductory Data Science Programming
Singh, A., Fariha, A., Brooks, C. A., Soares, G., Henley, A. Z., Tiwari, A., Chethan, M. N., Choi, H., & Gulwani, S. (2024) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3626252.3630884 · Cited by 14
Evaluating GPT for use in K-12 Block Based CS Instruction Using a Transpiler and Prompt Engineering
Gonzalez-Maldonado, D., Liu, J., & Franklin, D. M. (2025) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3641554.3701910 · Cited by 9
AI in Criminal Justice Education: Assessing Educator’s Attitudes and Experiences
Tyler, D. H. F., Gagnon, A., & George, S. S. (2024) · Journal of Criminal Justice Education · doi:10.1080/10511253.2024.2413995 · Cited by 8
“For example, it has been used to explain various aspects of computer code (macneil et al., 2023) and is showing promise in its potential to clarify language used in the legal and medical fields (Harrington, 2023; Sun & Hoelscher, 2023).”
Ionescu, A. C., Titov, S., & Izadi, M. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728611 · Cited by 10
A systematic review of research on large language models for computer programming education
Zhu, M., Xu, L., & Ericson, B. J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.21818 · Cited by 10
Deception at Scale: Deceptive Designs in 1K LLM-Generated E-Commerce Components
Chen, Z., Shen, J., Luna, Zhang, H., & Vaccaro, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791063 · Cited by 7
“The membership sign-up component collects customer names and email without explaining how it will be used or protected ( interface interference in Gray’s taxonomy [29]).”
Njuguna, G. N., & Qingfei, M. (2025) · Technology in Society · doi:10.1016/j.techsoc.2025.103080 · Cited by 7
GenAI Voice Mode in Programming Education
Jacobs, S., & Kiesler, N. (2025) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3769994.3770001 · Cited by 7
Qi, J., Zhao, H., Xiang, R., Zhang, Y., Zhang, X., Wang, C., Jian, L., & Ji, F. (2026) · Journal of Research on Technology in Education · doi:10.1080/15391523.2025.2601067 · Cited by 9
Novice Learners of Programming and Generative AI - Prior Knowledge Matters
Kiesler, N., Scholz, I., Albrecht, J., Stappert, F., & Wienkop, U. (2024) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3699538.3699580 · Cited by 8
On the Opportunities of Large Language Models for Programming Process Data
Edwards, J., Hellas, A., & Leinonen, J. (2025) · Australasian Computing Education Conference · doi:10.1145/3716640.3716652 · Cited by 10
Self-citationCognitive Apprenticeship and Artificial Intelligence Coding Assistants
Poitras, E., Crane, B. G. C., Dempsey, D., Bragg, T. A., Siegel, A. A., & Lin, M. P.-C. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-1066-3.ch013 · Cited by 8
A Case Study in Engineering a Conversational Programming Assistant's PersonaPreprint
Ross, S., Muller, M. J., Martinez, F., Houde, S., & Weisz, J. D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.10016 · Cited by 10
“[19] engineer prompts to generate different forms of code explanations.”
Teaching and Learning AI in K-12 Informatics Education
Barendsen, E., Lonati, V., Quille, K., Altın, R., Divitini, M., Hooshangi, S., Karnalim, O., Kiesler, N., Melton, M., Montero, C. S., et al. (2024) · doi:10.1145/3708550.3730560 · Cited by 8
“…of GenAI tools (e.g., ChatGPT-3, 3.5 and 4) for programming education contexts are certainly their ability to provide individual feedback to learners’ input [3, 57, 68], generate next-step hints [112], localize errors in program code [146], and to enhance programming error messages [63, 77, 116].”
InfluentialPădurean, V.-A., Gotovos, A., Ghosh, A., Denny, P. C., Leinonen, J., Luxton-Reilly, A., Prather, J., & Singla, A. K. (2026) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3803400.3809312 · Cited by 6
Self-citationUnderstanding and Enhancing CS Students’ Interaction Experience with AI Coding Assistant Tools
Long, X., Tan, X. L., Zhu, Y., Jiang, J., & Zhang, L. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3785479 · Cited by 10
“Their findings indicate that Copilot improves security in solutions for harder problems but has no significant effect on easier tasks.”
Exploring Large Language Models for Method Name Prediction
Qian, H., Xu, T., Ding, Z., Liu, W., & Zhu, S. (2024) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs62785.2024.00028 · Cited by 10
“…by Codex supports a variety of automated programming tasks and has shown remarkable performance, particularly in the areas of efficient and accurate code generation [38], [39], error and vulnerability identification [40], [41] and resolution, and the provision of code segment summaries [42], [43].”
BackgroundTowards Personalized and Contextualized Code Explanations
Brachman, M., Goldberg, A., Anderson, A. W., Houde, S., Muller, M. J., & Weisz, J. D. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3733681 · Cited by 9
Tran, M. (2023) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3568812.3603453 · Cited by 8
“In CS classroom, LLMs have shown potential in helping students write code [14], generating code explanations [9], parsing non-compiling code and generating programming error messages [8].”
BackgroundAssessing the impact of hints in learning formal specification
Cunha, A., Macedo, N., Campos, J. C., Margolis, I., & de Sousa, E. A. F. (2024) · IEEE/ACM International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) · doi:10.1145/3639474.3640050 · Cited by 6
Software Engineering Educational Experience in Building an Intelligent Tutoring System
Fan, Z., Noller, Y., Dandekar, A., & Roychoudhury, A. (2025) · IEEE International Conference on Software Engineering Education and Training (CSEE&T) · doi:10.1109/cseet66350.2025.00015 · Cited by 7
“While [23], [28], [30], [32] tried to build LLM-based feedback generation systems that are capable of handling general students’ questions, they heavily rely on specifically curated prompts, and it remains unknown how well these systems can be adapted by worldwide educators.”
Semantic Similarity Loss for Neural Source CodeSummarizationPreprint
Su, C., & McMillan, C. (2023) · Research Square · doi:10.21203/rs.3.rs-3240092/v1 · Cited by 6
ASSERTIFY: Utilizing Large Language Models to Generate Assertions for Production CodePreprint
Torkamani, M. J., Sharma, A., Mehrotra, N., & Purandare, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.16927 · Cited by 8
“The effectiveness, correctness, and accuracy of these generated code summaries have been corroborated by recent studies, e.g., [21], illustrating the benefits of employing LLMs for generating method explanations.”
The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers
Prather, J., Reeves, B., Leinonen, J., MacNeil, S., Randrianasolo, A. S., Becker, B., Kimmel, B., Wright, J., & Briggs, B. (2024) · arXiv · Cited by 196
Self-citationLocalized explanations for automatically synthesized network configurations
Nazari, A., Zhang, Y., Raghothaman, M., & Chen, H. (2024) · Proceedings of the 23rd ACM Workshop on Hot Topics in Networks · doi:10.1145/3696348.3696888 · Cited by 5
“In contrast to code explanation, which primarily focus on clarifying the code logic or structure given a concrete implementation [17, 22], subspecifications man-date the minimal conditions for a device to satisfy the global specifications, which is more general and flexible.”
Behmanush, H., Akhtari, F., Nooripour, R., Weber, I. G., & Cannanure, V. K. (2025) · ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3715335.3735479 · Cited by 7
“In introductory programming, GenAI has proven helpful by assisting with student help requests [40] and providing natural language explanations for code [60, 62, 85].”
ÖZGÜL, E. (2026) · Kastamonu Eğitim Dergisi · doi:10.24106/kefdergi.1878122 · Cited by 6
McCabe, A. T., Bjorkman, M., Engström, J., Kuang, P., Söderberg, E., & Church, L. (2024) · International Conference on Art, Science, and Engineering of Programming (<Programming>) · doi:10.1145/3660829.3660835 · Cited by 5
Leveling up Learning: Serious Games for Computing Education - Long-Term Opportunities and Risks
Wuensche, B. C., Hooper, S., Whalley, J. L., Straand, I. J., Denny, P. C., Crow, T. V. H., Lange-Nawka, D. C., Luxton-Reilly, A., & Thompson, S. E. R. (2025) · Australasian Computing Education Conference · doi:10.1145/3716640.3716655 · Cited by 5
Self-citationNavigating Pitfalls: Evaluating LLMs in Machine Learning Programming EducationPreprint
Kumar, S., Lones, M. A., Maarek, M., & Zantout, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.18220 · Cited by 6
“…CodeQwen (Qwen, 2024) are revolutionizing core software engineering related tasks like code generation (Austin et al., 2021), code interpretation (MacNeil et al., 2023), testcase generation (Liu et al., 2024), code review (Becker et al., 2023), code debugging (Wermelinger, 2023) and program…”
Modeling Student Knowledge Progression Across Concepts in Intelligent Tutoring Interactions
Mittal, K., O’Neill, A., Schlegel, H., Ranade, G. V., & Norouzi, N. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98417-4_8 · Cited by 4
Manakina, O., & Lung, C. (2025) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac65507.2025.00310 · Cited by 4
Adeyemi, I. O., Giwa, K., Giwa, F., Hassan, R., & Gabriel, A. (2025) · Artificial Intelligence in Education · doi:10.1108/aiie-11-2024-0041 · Cited by 4
Chapagain, J., Sajib, M. I., Prodan, R., & Rus, V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72315-5_5 · Cited by 8
Fowler, M., Emeka, C., Chen, B., Smith, D. H., West, M., & Zilles, C. B. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3774752 · Cited by 5
Language-Model Assisted Learning How to Program?
Leidner, J. L., & Reiche, M. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-50485-3_41 · Cited by 4
“4 In the context of a next-generation programming education e-book, the same group investigated LMs’ power to explain code in an educational context [9]; they let human students rate the usefulness of the automated output.”
Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For?Preprint
Scholl, A., & Kiesler, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.17258 · Cited by 4
“Other studies revealed they can enhance programming error messages (Leinonen et al., 2023), and effectively generate code explanations (MacNeil et al., 2023; Sarsa et al., 2022).”
Solano, L. L., Koutcheme, C., Leinonen, J., Vassar, A., & Renzella, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.05305 · Cited by 5
Self-citationTowards Implementing and Evaluating AI-Assisted Pull Requests in Software Engineering Education
Parra, E., & Willingham, S. (2025) · IEEE International Conference on Software Engineering Education and Training (CSEE&T) · doi:10.1109/cseet66350.2025.00008 · Cited by 4
“General purpose LLMs such as ChatGPT and coding-specific LLMs such as GitHub Copilot [13] can understand and explain source code [30] with demonstrated practical applications that provide help in writing and suggesting code [48].”
Automatic generation of introductory programming exercises with large language models
Duong, T. N. B., Nguyen, H. L., & Gottipati, S. (2025) · Research and Practice in Technology Enhanced Learning · doi:10.58459/rptel.2026.21025 · Cited by 3
Exploring the quality and effectiveness of AI-generated feedback in introductory programming
Qian, Y., Liu, M., & Zhu, L. (2026) · Humanities and Social Sciences Communications · doi:10.1057/s41599-026-07920-7 · Cited by 4
“Unfortunately, existing studies on using AI-generated feedback to support teaching introductory programming did not show signi fi cant positive effects (Kimmel et al. 2024; MacNeil et al. 2023; Wang et al. 2024).”
Borys, D., Macura, J., Sikora, B., & Wróbel, Ł. (2025) · Applied System Innovation · doi:10.3390/asi8060174 · Cited by 3
“In the literature several examples of custom learning applications have been described in which AI was used to explain code [21], present step-by-step solutions [22], provide assistance to the learner [23], and generate exercises [12,24].”
Solano, L. L., Koutcheme, C., Leinonen, J., Vassar, A., & Renzella, J. (2026) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3770762.3772576 · Cited by 4
Self-citationDesign Implications for Student and Educator Needs in AI-Supported Programming Learning Tools
Ma, B., Xie, Y., Li, H., Li, G., Chen, L. Y. E., Shimada, A., & Konomi, S. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2730083 · Cited by 3
“Bespoke Bots”: Diverse Instructor Needs for Customizing Generative AI Classroom Chatbots
Hou, I., Xiong, Z., Guo, P. J., & Wang, A. Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791491 · Cited by 4
“Across higher education, generative AI chat-bots have been used to answer student questions, provide formative feedback, generate practice and assessment materials, and even integrate with learning management systems [9, 16, 26, 36, 39].”
Artificial intelligence for web development: Perspectives from the industry
Pohjalainen, P., & Vepsäläinen, J. (2026) · AI Magazine · doi:10.1002/aaai.70051 · Cited by 3
Textbooks for Writing Play Scripts Based on Local Culture in Higher Education
Suryani, I., Izar, J., Wulandari, S., & Helty (2025) · Journal for Lesson and Learning Studies · doi:10.23887/jlls.v8i3.104413 · Cited by 2
From Code Generation to Conceptual Learning: Student Use of LLMs in a Web Programming Course
Isa, H.-Y., Weston, M., Wellyanto, M. R., Karna, I., Talton, J. O., & Kumar, R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3793207 · Cited by 4
Bartoš, P. (2026) · Discover Education · doi:10.1007/s44217-026-02172-8 · Cited by 2
“In programming education, they can explain error messages, prompt debugging, and generate examples, but they can also provide complete solutions, hallucinate APIs, obscure uncertainty, and weaken assessment validity [4, 6, 7, 18, 19, 25].”
Generating CS1 Coding Questions using OpenAI
Naringrekar, P. D., Akhmetov, I., & Stroulia, E. (2023) · Western Canadian Conference on Computing Education · doi:10.1145/3593342.3593348 · Cited by 3
“Although LLMs pose a significant threat to computing education, they also offer new possibilities that could enrich education and provide new opportunities [1, 2].”
BackgroundSemantic similarity loss for neural source code summarization
Su, C., & McMillan, C. (2024) · Journal of Software Evolution and Process · doi:10.1002/smr.2706 · Cited by 2
Teaching Model-Driven Low-Code Development Platforms
Charles, J., Michael, J., Netz, L., & Rumpe⋆, B. (2024) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3652620.3687778 · Cited by 3
Incorporating AI in the Teaching of Requirements Tracing Within Software Engineering
Couder, J. O., Pate, W. C., Machado, D. A., & Ochoa, O. (2024) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10892858 · Cited by 3
“LLMs have been used to explain code for students in a way they could easily understand what a code is doing [43].”
IIITD, A., Sethi, D., Gupta, N., Sharma, S., Jain, S., Singhal, U., & Kumar, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.16805 · Cited by 3
“Existing research on the integration of LLMs in computing education has explored the challenges and opportunities that educators and students encounter when adapting to LLMs’ capabilities, particularly in generating code from natural language descriptions [2, 4–6, 8–11, 17, 18, 20, 21, 23–27, 38].”
BackgroundInfluentialAscertaining trustworthiness of AI systems in telecommunications
Sen, R., Vasudevan, S., Britto, R., & Prasath, M. (2024) · ITU Journal on Future and Evolving Technologies · doi:10.52953/wibx7049 · Cited by 1
Kamberovic, M. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3699682.3727571 · Cited by 1
Creative Learning Using Generative AI: Empowering Problem-Solving and Programming Skills
Elanwar, R. (2025) · Springer series on cultural computing · doi:10.1007/978-3-031-86551-0_7 · Cited by 2
Metadata
- Authors
- MacNeil, S., Tran, A., Hellas, A., Kim, J., Sarsa, S., Denny, P., Bernstein, S., & Leinonen, J.
- Affiliations
- Temple University (US)
- Aalto University (FI)
- University of Auckland (NZ)
- ORCID
- Venue
- Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1
- Publisher
- ACM
- Published
- 2023-03-02
- Pages
- 931-937
- OpenAlex
- W4323033814
- Cited by
- 195
- References
- 30
- Citations by year
- 2026: 27 · 2025: 59 · 2024: 79 · 2023: 29
- FWCI
- 58.66
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Technical Symposium on Computer Science Education (SIGCSE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Aaltodoc (Aalto University) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- code explanation, large language models, interactive e-books, computer science education, student engagement, explanation types, code generation, code complexity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.