Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language
Open accessThe DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
GitHub Copilot is an artificial intelligence tool for automatically generating source code from natural language problem descriptions. Since June 2022, Copilot has officially been available for free to all students as a plug-in to development environments like Visual Studio Code. Prior work exploring OpenAI Codex, the underlying model that powers Copilot, has shown it performs well on typical CS1 problems thus raising concerns about its potential impact on how introductory programming courses are taught. However, little is known about the types of problems for which Copilot does not perform well, or about the natural language interactions that a student might have with Copilot when resolving errors. We explore these questions by evaluating the performance of Copilot on a publicly available dataset of 166 programming problems. We find that it successfully solves around half of these problems on its very first attempt, and that it solves 60% of the remaining problems using only natural language changes to the problem description. We argue that this type of prompt engineering, which we believe will become a standard interaction between human and Copilot when it initially fails, is a potentially useful learning activity that promotes computational thinking skills, and is likely to change the nature of code writing skill development.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This has been inspired by many other instances where LLMs have already demonstrated remarkable effectiveness in generating novel language artifacts. These include course readings [22], code explanations [23, 28, 29, 41], model solutions [9, 35, 42], feedback [34], or responses to help requests [18, 26, 46, 47].”
From A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education (2024) · cites this paper as [9]
In the citing paper
“The phrasing of the prompt and problem statement can significantly impact whether the LLM will solve it correctly [Denny et al. 2023].”
From How Do Programming Students Use Generative AI? (2025) · cites this paper as Denny et al. 2023
In the citing paper
“Our examination of the prompts indicated that this discrepancy may arise because students face challenges in constructing effective queries or revising them to obtain useful information, aligning with findings in the literature on code generation using LLMs [13].”
From Using an LLM to Help With Code Understanding (2024) · cites this paper as [13]
In the citing paper
“Hence, the findings reported in [46] appear to be challenged by the more capable GPT-4 model as well as the findings reported in [7, 10, 11, 38].”
From Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses (2023) · cites this paper as [7]
In the citing paper
“Hence, the strengths and limitations reported in prior work [7, 10, 11, 38, 46] still hold to a certain degree.”
From Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses (2023) · cites this paper as [7]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“While their ability to generate novel human-like outputs is on the one hand fascinating, their rapidly increasing deployment has caused alarm among some researchers and led to calls for better understanding of their implications and risks [3, 24].”
BackgroundOn the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
Reynolds, L., & McDonell, K. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451760 · Cited by 952 · cited 1×
“Indeed, it is well known that language model outputs are very sensitive to their inputs [19].”
BackgroundVaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660
Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models
Sarsa, S., Denny, P., Hellas, A., & Leinonen, J. (2022) · Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 1 · doi:10.1145/3501385.3543957 · Cited by 497 · cited 1×
“applied Codex to the task of generating novel programming exercises given a single example as input [20].”
MethodSelf-citationThe Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming
Finnie-Ansley, J., Denny, P., Becker, B. A., Luxton-Reilly, A., & Prather, J. (2022) · Proceedings of the 24th Australasian Computing Education Conference · doi:10.1145/3511861.3511863 · Cited by 463 · cited 3×
“Nevertheless, across the 166 problems we evaluated, overall we observe similar rates of initial success reported in earlier studies [11].”
BackgroundResultSelf-citationGrounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2023) · Proceedings of the ACM on Programming Languages · doi:10.1145/3586030 · Cited by 436
GitHub Copilot AI pair programmer: Asset or Liability?
Dakhel, A. M., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M. C., Ming, Z., & Jiang (2022) · arXiv · doi:10.48550/arxiv.2206.15331 · Cited by 417
Research works printed but not linked: 4
- Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski David Dohan Ellen Jiang Carrie Cai Michael Terry Quoc Le et al. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732 (2021). · arXiv
- 10.5555/3495724.3495883 · doi:10.5555/3495724.3495883
- Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586 (2021). · arXiv
- Leonard Tang, Elizabeth Ke, Nikhil Singh, Bo Feng, Derek Austin, Nakul Verma, and Iddo Drori. 2022. Solving Probability and Statistics Problems by Probabilistic Program Synthesis at Human Level and Predicting Solvability. In Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners' and Doctoral Consortium,, Maria Mercedes Rodrigo, Noburu Matsuda, Alexandra I. Cristea, and Vania Dimitrova (Eds.). Springer International Publishing, Cham, 612--615.
Cited by
Items 201–300 of 398
Do AI Assistants Help Students Write Formal Specifications? A Study with ChatGPT and the B-Method
Capozucca, A., Yampolskyi, D., Goldberg, A., & Cristiá, M. (2025) · IEEE International Conference on Software Engineering Education and Training (CSEE&T) · doi:10.1109/cseet66350.2025.00009 · Cited by 8
From Prompts to Propositions: A Logic-Based Lens on Student-LLM Interactions
Alfageeh, A., Zarkouei, S. A. K., Nam, D., Prol, D., Amoozadeh, M., Chattopadhyay, S., Prather, J. E., Denny, P. C., Leinonen, J., Hilton, M. L., et al. (2025) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3769994.3770004 · Cited by 7
Self-citationSmith, D. H., & Zilles, C. B. (2024) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3626253.3635542 · Cited by 7
A Systematic Review of Gen-AI Applications in Education: Rewards, Challenges and Future Prospects
Wanyonyi, E. N., & Murithi, M. (2025) · Pan-African journal of education and social sciences · doi:10.56893/pajes2025v06i01.01 · Cited by 7
Yang, S., Chen, M., & Schneider, B. (2025) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3769994.3770009 · Cited by 10
Evaluating Copilot on CS1 Code Writing Problems with Suppressed Specifications
Venkatesh, V., Venkatesh, V., & Kumar, V. (2023) · Bangalore Annual Compute Conference · doi:10.1145/3627217.3627235 · Cited by 8
Self-citationCognitive Apprenticeship and Artificial Intelligence Coding Assistants
Poitras, E., Crane, B. G. C., Dempsey, D., Bragg, T. A., Siegel, A. A., & Lin, M. P.-C. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-1066-3.ch013 · Cited by 8
Oh, S.-Y., Cao, Y. L., Katz, A., & Zhao, J. (2024) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10893102 · Cited by 10
A systematic review of research on large language models for computer programming education
Zhu, M., Xu, L., & Ericson, B. J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.21818 · Cited by 10
A Case Study in Engineering a Conversational Programming Assistant's PersonaPreprint
Ross, S., Muller, M. J., Martinez, F., Houde, S., & Weisz, J. D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.10016 · Cited by 10
Benchmarking Educational Program RepairPreprint
Koutcheme, C., Dainese, N., Sarsa, S., Leinonen, J., Hellas, A., & Denny, P. C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.05347 · Cited by 11
Self-citationHelpful or Harmful? Exploring the Efficacy of Large Language Models for Online Grooming Prevention
Prosser, E., & Edwards, M. (2024) · European Interdisciplinary Cybersecurity Conference · doi:10.1145/3655693.3655694 · Cited by 6
Garg, A., & Rajendran, R. (2024) · Proceedings of the 16th International Conference on Computer Supported Education · doi:10.5220/0012693000003693 · Cited by 7
“An experimental study showcased how the programming tool Codex, powered by generative AI, outperformed learners in a CS1 class on a rainfall problem, ranking in the top quartile (Denny, P. et al., 2023).”
Ali, M., Ma, M., Muneeb, M., & Wong, G. K. W. (2025) · Metrics · doi:10.3390/metrics2040023 · Cited by 6
The Convergence of AI/ML and DevSecOps: Revolutionizing Software Development
Pakalapati, N., Venkatasubbu, S., & Sistla, S. M. K. (2023) · Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (online) · doi:10.60087/jklst.vol2.n2.p212 · Cited by 6
Understanding and Enhancing CS Students’ Interaction Experience with AI Coding Assistant Tools
Long, X., Tan, X. L., Zhu, Y., Jiang, J., & Zhang, L. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3785479 · Cited by 10
Kaputa, Z. G., Rajaram, A., Feliciano, V. A., Lyu, Z., Agrawala, M., & Subramonyam, H. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791514 · Cited by 6
Path to Personalization: A Systematic Review of GenAI in Engineering Education
Khan, R. R., Bhaduri, S., Mackenzie, T., Paul, A., Sankalp, K. J., & Sen, I. (2024) · HAL · Cited by 23
Generating Effective Distractors for Introductory Programming Challenges: LLMs vs Humans
Hassany, M., Brusilovsky, P. L., Šavelka, J., Narayanan, A. B. L., Akhüseyinoğlu, K., Agarwal, A., & Hendrawan, R. A. (2025) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3706468.3706529 · Cited by 11
Evaluating ChatGPT to Answer Multi-Modal Exercises in Computer Science Education
Ouh, E. L., Tan, K. W., Lo, S. L., & Gan, B. K. S. (2025) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3724363.3729056 · Cited by 7
Tran, M. (2023) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3568812.3603453 · Cited by 8
Toward Personalised Learning Experiences: Beyond Prompt Engineering
Kruis, J., Pera, M. S., ten Napel, Z., Landoni, M., Murgia, E., Huibers, T. W. C., & Feskens, R. C. W. (2024) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3628516.3659367 · Cited by 6
Incremental Repair Feedback on Automated Assessment of Programming Assignments
Paiva, J. C., Leal, J. P., & Figueira, Á. (2025) · Electronics · doi:10.3390/electronics14040819 · Cited by 6
The Recent Trends of Research on GitHub Copilot: A Systematic Review
Ani, Z. C., Hamid, Z. A., & Zhamri, N. N. (2024) · Communications in Computer and Information Science · doi:10.1007/978-981-99-9589-9_27 · Cited by 5
Jhonnerie, R., Oktorini, Y., Darlis, V. V., Pebriandi, P., Prianto, E., Fatmawati, R., Nofrizal, N., Ramses, R., Miswadi, M., & Rahmatdillah, R. (2024) · BIO Web of Conferences · doi:10.1051/bioconf/202413603002 · Cited by 6
Aruleba, K., Oyelere, S. S., Obaido, G., & Sanusi, I. T. (2025) · Conference on United Kingdom & Ireland Computing Education Research (UKICER) · doi:10.1145/3754508.3754517 · Cited by 6
Chapagain, J., Sajib, M. I., Prodan, R., & Rus, V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72315-5_5 · Cited by 8
Analyzing the uses and perceptions of computer science students towards generative AI tools
HASAN, A., Simion, B., & Pop, F. (2024) · Proceedings of the... International Conference on Virtual Learning · doi:10.58503/icvl-v19y202430 · Cited by 5
Leveling up Learning: Serious Games for Computing Education - Long-Term Opportunities and Risks
Wuensche, B. C., Hooper, S., Whalley, J. L., Straand, I. J., Denny, P. C., Crow, T. V. H., Lange-Nawka, D. C., Luxton-Reilly, A., & Thompson, S. E. R. (2025) · Australasian Computing Education Conference · doi:10.1145/3716640.3716655 · Cited by 5
Self-citationTowards the Integration of Large Language Models in an Object-Oriented Programming Course
Cipriano, B. P. (2024) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3649405.3659473 · Cited by 7
Use Me Wisely: AI-Driven Assessment for LLM Prompting Skills DevelopmentPreprint
Ognibene, D., Donabauer, G., Theophilou, E., Koyutürk, C., Yavari, M., Buršić, S., Telari, A., Testa, A., Boiano, R., Taibi, D., et al. (2025) · arXiv (Cornell University) · doi:10.30191/ets.202507_28(3).sp12 · Cited by 7
Ayouni, S., Khan, R. A., Maddeh, M., Alwageed, H. S., Keshta, I. M., & Almagrabi, A. O. (2026) · Egyptian Informatics Journal · doi:10.1016/j.eij.2026.100915 · Cited by 6
Tholander, J., & Jönsson, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791847 · Cited by 5
Yılmaz, F. G. K., Olpak, Y. Z., & Yılmaz, R. (2026) · Journal of Research on Technology in Education · doi:10.1080/15391523.2026.2683793 · Cited by 6
Smith, D. H., Fowler, M., Denny, P. C., & Zilles, C. B. (2025) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3724363.3729045 · Cited by 7
Self-citationExploring Direct Instruction and Summary-Mediated Prompting in LLM-Assisted Code Modification
Tang, N., Smith, E., Huang, Y., Collin, M., & Li, T. J.-J. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00017 · Cited by 5
Generating Synthetic Dialogues from Prompts to Improve Task-Oriented Dialogue Systems
Steindl, S., Schäfer, U., & Ludwig, B. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42608-7_17 · Cited by 4
Humanity Amplified: The Fusion of Deep Learning and Human Insight to Shape the Future of InnovationPreprint
Bauschard, S., & Quidwai, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4621210 · Cited by 6
Automatic generation of introductory programming exercises with large language models
Duong, T. N. B., Nguyen, H. L., & Gottipati, S. (2025) · Research and Practice in Technology Enhanced Learning · doi:10.58459/rptel.2026.21025 · Cited by 3
The Impact of Security Mindset on the Use of AI Assistants in Computing Education
Yuan, J., & Li, Y. (2025) · IEEE Transactions on Education · doi:10.1109/te.2025.3610340 · Cited by 3
RMGenie: An LLM-Based Agent Framework for Open Source Software README Generation
Cui, X., Wu, J., Li, Z., Luo, T., & Ling, X. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00052 · Cited by 4
Maisch, R., Hagel, N., & Bartel, A. (2025) · European Conference on Software Engineering Education (ECSEE) · doi:10.1145/3723010.3723019 · Cited by 4
Customizing ChatGPT to Help Computer Science Principles Students Learn Through Conversation
Frazier, M., Damevski, K., & Pollock, L. (2024) · Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 · doi:10.1145/3649217.3653570 · Cited by 2
PROMPT ENGINEERING ON THE ENGINEERING DESIGN PROCESS
Valentine, R. M. (2024) · International Conference on Engineering and Product Design Education (EPDE) · doi:10.35199/epde.2024.76 · Cited by 2
Artificial Intelligence Empowered Teaching Reform and Exploration of Python Programming Course
Liang, J. (2025) · OALib · doi:10.4236/oalib.1112859 · Cited by 4
Design Implications for Student and Educator Needs in AI-Supported Programming Learning Tools
Ma, B., Xie, Y., Li, H., Li, G., Chen, L. Y. E., Shimada, A., & Konomi, S. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2730083 · Cited by 3
Satisficing vs. Maximizing in Prompt Writing: Trait and Task Effects in Human–AI Interaction
Wyszynski, M., Weber, S., Fritzsche, R., Hofgesang, M., & Niehaves, B. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790693 · Cited by 2
From Code Generation to Conceptual Learning: Student Use of LLMs in a Web Programming Course
Isa, H.-Y., Weston, M., Wellyanto, M. R., Karna, I., Talton, J. O., & Kumar, R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3793207 · Cited by 4
AI-Generated Slides: Are They Good? Can Students Tell?
Leinonen, J., Zhang, L., & Hellas, A. (2026) · Western Canadian Conference on Computing Education · doi:10.1145/3828792.3828803 · Cited by 2
Generative AI in Introductory Programming for Non-CS STEM Students: A Two-Semester Study
de Oliveira, R. M., da Silva, E. P., Wainer, J., & Azevedo, R. (2026) · IEEE Access · doi:10.1109/access.2026.3701610 · Cited by 3
Tran, K., Harper, C., & Price, T. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811653 · Cited by 2
DoME: An Architecture for Domain Model Evolution at Runtime Using NLP
Gomes, A. M., & Maia, P. H. M. (2023) · Brazilian Symposium on Software Engineering · doi:10.1145/3613372.3613405 · Cited by 2
CONSIDERATIONS FOR UPDATING PROGRAMMING COURSES
(2023) · Issues in Information Systems · doi:10.48009/2_iis_2023_112 · Cited by 2
Creating Thorough Tests for AI-Generated Code is Hard
Singhal, S., & Kumar, V. (2023) · Bangalore Annual Compute Conference · doi:10.1145/3627217.3627238 · Cited by 2
Self-citationWhat is Skill? (and why does it matter?)
McDermott, R., & Daniels, M. (2023) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie58773.2023.10343520 · Cited by 3
Prompt Engineering, An Alternative for Video Game Development?
Rozo-Torres, A., & Sarmiento, W. J. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-57982-0_19 · Cited by 2
Faceless Adversary, Feckless Colleague: The Many Sides of ChatGPT
Bird, W. H. (2024) · Western Canadian Conference on Computing Education · doi:10.1145/3660650.3660656 · Cited by 2
Mihaljević, B., Radovan, A., & Zagar, M. (2024) · Education and New Developments 2022 – Volume I · doi:10.36315/2024v1end123 · Cited by 7
Análise dos efeitos do idioma na geração automática de respostas por aplicações de LLM
Delbone, F. R., Wiese, I., & Silva, M. A. G. (2024) · doi:10.5753/educomp_estendido.2024.238811 · Cited by 3
Using Large Language Models to Automate Annotation and Part-of-Math Tagging of Math Equations
Shan, R., & Youssef, A. S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-66997-2_1 · Cited by 7
Cisneros, A. A. A., Cisneros, R. A., Palomino, C. A., & COLQUEHUANCA, J. (2024) · Revista Digital del Doctorado en Educación de la Universidad Central de Venezuela · doi:10.55560/arete.2024.ee.10.8 · Cited by 3
GPT AI in Computer Science Education: A Systematic Mapping Study
Strik, B., Menolli, A., & Brancher, J. D. (2024) · Anais do ... Simpósio Brasileiro de Informática na Educação/Anais do Simpósio Brasileiro de Informática na Educação · doi:10.5753/sbie.2024.242103 · Cited by 2
Empowering Computing Students with Large Language Models by Developing an Escape Room Game
Giacaman, N., & Terragni, V. (2025) · ACM eBooks · doi:10.1145/3708897 · Cited by 2
Self-citationHyseni, L. N., & Dermaku, A. (2025) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.1846 · Cited by 3
Using small language models as tools for teaching in the ELSEI master's program
Aammou, S., Tagdimi, Z., & Tarik, T. (2024) · Cadernos de Educação Tecnologia e Sociedade · doi:10.14571/brajets.v17.nse4.84-95 · Cited by 2
Oppenlaender, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.14103 · Cited by 2
IIITD, A., Sethi, D., Gupta, N., Sharma, S., Jain, S., Singhal, U., & Kumar, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.16805 · Cited by 3
Can You Spot the AI? Incorporating GenAI into Technical Writing Assignments
Rajabi, P., & Kerslake, C. P. (2024) · Western Canadian Conference on Computing Education · doi:10.1145/3660650.3660673 · Cited by 2
Hare, B. K., Gladbach, J., Shah, S. J. H., & Xu, D. (2025) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3641555.3704749 · Cited by 1
Transforming Higher Education with AI: A Cognitive Infocommunications Perspective
Horváth, I. (2024) · IEEE International Conference on Cognitive Infocommunications · doi:10.1109/coginfocom63007.2024.10894731 · Cited by 1
Yu, H., Shen, B., Zhang, J. Y., Lin, S., Li, L., Liang, G., Li, Y., Wang, Q., & Xie, T. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3725213 · Cited by 3
Work in Progress: Unlocking Code Generation Through Synergistic Prompt Engineering
Ho, K.-H., Georgiades, M., Fan, T.-K. J., Hou, Y., Fong, K. C. K., & Chan, T.-T. (2025) · IEEE World Engineering Education Conference (EDUNINE) · doi:10.1109/edunine62377.2025.10980842 · Cited by 1
Terragni, V., & Giacaman, N. (2025) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst62969.2025.10989046 · Cited by 1
Self-citationCodeSCM: Causal Analysis for Multi-Modal Code Generation
Gupta, M., Bhatt, N., & Jana, S. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.345 · Cited by 3
The Impact of Artificial Intelligence on Programmer Jobs: Threat or Opportunity?
Hananto, A., Hasibuan, N. S., Olivia, S., Tukino, T., & Novalia, E. (2025) · JUSIFO (JURNAL SISTEM INFORMASI) · doi:10.19109/jusifo.v11i1.25082 · Cited by 3
AI-Powered Tutoring for Novice Programmers: Supporting Students Throughout Project Development
Sánchez, J. D. L., Ribeiro, B. M., Manrique, R., & Garcés, K. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98281-1_10 · Cited by 2
The End of Programming as We Know It: Envisioning Radical Re-Conceptualizations of Co-Coding with AI
Jönsson, M., Tholander, J., Rost, M., Sarkar, A., Fernaeus, Y., Kharrufa, A., & Ludwig, T. G. (2025) · Decennial Aarhus Conference · doi:10.1145/3737609.3747093 · Cited by 1
The use of artificial intelligence in teaching programming to students
Bolotbaev, D., & Temirov, M. (2025) · Bulletin of the Jusup Balasagyn Kyrgyz National University · doi:10.58649/1694-8033-2025-2(122)-79-86 · Cited by 1
Avaliando o Papel do GPT-4o na Aprendizagem de Engenharia de Software: Um Relato de Experiência
Branco, B. M. D. C. C., Neto, A. L. D. M., Avelino, G. A., & Machado, V. P. (2025) · Simpósio Brasileiro de Engenharia de Software (SBES) · doi:10.5753/sbes.2025.10716 · Cited by 1
Zuo, F., Qian, G., Qu, X., Rhee, J., & Fu, J. (2025) · Conference on Information Technology Education · doi:10.1145/3769694.3771150 · Cited by 1
Mittal, A., Shah, H., & Keshap, P. (2025) · Conference on Information Technology Education · doi:10.1145/3769694.3771136 · Cited by 1
AI Chatbots vs. Traditional Search: A Comparative Study on Student Information Retrieval
Pasquarelli, L., Koutcheme, C., & Hellas, A. (2025) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie63693.2025.11328491 · Cited by 2
Pombo, N. G. C. C., & Cunha, C. A. D. S. (2025) · IEEE International Conference on Teaching, Assessment, and Learning for Engineering · doi:10.1109/tale66047.2025.11346609 · Cited by 1
Cochran, K., & Piech, C. (2026) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3770762.3772663 · Cited by 3
Big Data Research on Personalized Learning in Computer Education: A Thematic Evolution Analysis
Mo, Y., Zhang, J., & Mou, Y. (2026) · Computer Applications in Engineering Education · doi:10.1002/cae.70159 · Cited by 1
Khan, K. I., Correia, A. B., Nasir, A., Dantas, R. M., & Mata, P. N. (2026) · Journal of the Knowledge Economy · doi:10.1007/s13132-026-03182-5 · Cited by 1
Kavuri, S., Delgado-Diaz, D., Ortiz, K., Chao, A., Molina, I. V., Ochoa, B., & Porter, L. (2026) · Australasian Computing Education Conference · doi:10.1145/3786228.3786250 · Cited by 1
Conversational signatures: structural patterns in human–AI interaction across models and platforms
Kraishan, O. (2026) · Interacting with Computers · doi:10.1093/iwc/iwag017 · Cited by 2
Designing a curriculum for AI prompting strategy for business decision‐makers—A qualitative approach
Mulgund, P., Singh, R. K., Venkatesan, S., Thimmanayakanapalya, S. S., & Venkatesan, S. (2026) · Decision Sciences Journal of Innovative Education · doi:10.1111/dsji.70020 · Cited by 1
Integrating AI Literacy, Secure Coding, and Critical Evaluation
Atul Kumar Dnyaan Prasad Global University, Namazov, G., Nadira, A., Rizamat, S., Bobur, M., Mohichekhra, K., & Ozoda, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8450-4.ch008 · Cited by 1
Comparing Intent Communication Modes for Instruction-based Image Editing
Wazzan, A., Qaiser, I., MacNeil, S., & Souvenir, R. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816216 · Cited by 1
Pădurean, V.-A., Riegel, K., Barbre, G., Blake, M., Denny, P. C., Gotovos, A., Leinonen, J., MacNeil, S., Prather, J. E., & Singla, A. K. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811639 · Cited by 1
Self-citationKohn, T., Antensteiner, T., Hahnel, C., Kern, B., Keuning, H., Kia, E., Kiesler, N., Komm, D., Marinus, E., Šavelka, J., et al. (2026) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3856208.3856233 · Cited by 1
Bug-eecha 2.0: An Educational Game for CS1 Students and Instructors
Joseph, A. M., Sarma, S., Shelly, & Kumar, V. (2023) · Bangalore Annual Compute Conference · doi:10.1145/3627217.3627236 · Cited by 1
Self-citationGuardRails: Automated Suggestions for Clarifying Ambiguous Purpose Statements
Pawagi, M., & Kumar, V. (2023) · Bangalore Annual Compute Conference · doi:10.1145/3627217.3627234 · Cited by 2
Self-citationTowards a Generative AI Design DialoguePreprint
Owen, A. E., & Roberts, J. C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.01283 · Cited by 2
Anti-patterns in Students' Conditional StatementsPreprint
Naude, E., Denny, P. C., & Luxton-Reilly, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.18989 · Cited by 1
Self-citationKaputa, Z. G., Rajaram, A., Feliciano, V. A., Lyu, Z., Agrawala, M., & Subramonyam, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.09664 · Cited by 1
Hao, Z., Luo, H., Chen, Y., & Zhang, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.02966 · Cited by 2
Metadata
- Authors
- Denny, P., Kumar, V., & Giacaman, N.
- Affiliations
- University of Auckland (NZ)
- Indian Institute of Science Bangalore (IN)
- Venue
- Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1
- Publisher
- ACM
- Published
- 2023-03-02
- Pages
- 1136-1142
- OpenAlex
- W4323037544
- Cited by
- 287
- References
- 22
- Citations by year
- 2026: 57 · 2025: 84 · 2024: 99 · 2023: 47
- FWCI
- 86.69
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Technical Symposium on Computer Science Education (SIGCSE) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- GitHub Copilot, prompt engineering, natural language programming, OpenAI Codex, code generation, introductory programming education, computational thinking, error resolution
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.