My AI Wants to Know if This Will Be on the Exam: Testing OpenAI’s Codex on CS2 Programming Exercises
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
The introduction of OpenAI Codex sparked a surge of interest in the impact of generative AI models on computing education practices. Codex is also the underlying model for GitHub Copilot, a plugin which makes AI-generated code accessible to students through auto-completion in popular code editors. Research in this area, particularly on the educational implications, is nascent and has focused almost exclusively on introductory programming (or CS1) questions. Very recent work has shown that Codex performs considerably better on typical CS1 exam questions than most students. It is not clear, however, what Codex’s limits are with regard to more complex programming assignments and exams. In this paper, we present results detailing how Codex performs on more advanced CS2 (data structures and algorithms) exam questions taken from past exams. We compare these results to those of students who took the same exams under normal conditions, demonstrating that Codex outscores most students. We consider the implications of such tools for the future of undergraduate computing education.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, this approach may have become obsolete given how easily most LLMs can now solve introductory computing problems with simple prompts [85,86,167,175].”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [86]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Teachers who shift away from using many small problems to create larger and more authentic problems in an attempt to reduce reliance of generative AI will lose access to the quick and easy assessment methods such as automated grading associated with many introductory programming courses [86].”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [86]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the consistent pattern is that ChatGPT and other LLMs are capable of producing content sufficient to pass undergraduate level assessment in diverse curriculum areas.”
From Bob or Bot: Exploring ChatGPT's Answers to University Computer Science Assessment (2024) · cites this paper as [20]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Hence, the strengths and limitations reported in prior work [7, 10, 11, 38, 46] still hold to a certain degree.”
From Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses (2023) · cites this paper as [11]
The cited source says what the claim attributes to it.
In the citing paper
“The impressive documented performance of large language models in solving typical CS1 and CS2 problems suggests that some rethinking of assessments is essential [86].”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [86]
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Competition-Level Code Generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.07814 · Cited by 1,043
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660
Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models
Sarsa, S., Denny, P., Hellas, A., & Leinonen, J. (2022) · Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 1 · doi:10.1145/3501385.3543957 · Cited by 497 · cited 2×
“explored the potential for Codex to generate novel programming exercises and explanations of code [23].”
BackgroundSelf-citationIntroductory programming: a systematic literature review
Luxton-Reilly, A., Simon, Albluwi, I., Becker, B. A., Giannakos, M., Kumar, A. N., Ott, L., Paterson, J., Scott, M. J., Sheard, J., et al. (2018) · Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education · doi:10.1145/3293881.3295779 · Cited by 485
Self-citationThe Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming
Finnie-Ansley, J., Denny, P., Becker, B. A., Luxton-Reilly, A., & Prather, J. (2022) · Proceedings of the 24th Australasian Computing Education Conference · doi:10.1145/3511861.3511863 · Cited by 463
Self-citationGitHub Copilot AI pair programmer: Asset or Liability?
Dakhel, A. M., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M. C., Ming, Z., & Jiang (2022) · arXiv · doi:10.48550/arxiv.2206.15331 · Cited by 417 · cited 2×
“explored Copilot’s ability to generate code for various introductory algorithms, as well as five introductory Python programming problems [6].”
BackgroundGitHub Copilot AI pair programmer: Asset or Liability?
Moradi Dakhel, A., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M. C., & Jiang, Z. M. (. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111734 · Cited by 417
An empirical evaluation of GitHub copilot's code suggestions
Nguyen, N., & Nadi, S. (2022) · Proceedings of the 19th International Conference on Mining Software Repositories · doi:10.1145/3524842.3528470 · Cited by 313 · cited 1×
“Nguyen and Nadi evaluated the performance of Copilot on 33 LeetCode questions, with varying difficulty, observing correctness rates of between 27% and 57% across four languages [21].”
BackgroundA theory of instruction for introductory programming skills
Xie, B., Loksa, D., Nelson, G. L., Davidson, M. J., Dong, D., Kwik, H., Tan, A. H., Hwa, L., Li, M., & Ko, A. J. (2019) · Computer Science Education · doi:10.1080/08993408.2019.1565235 · Cited by 191
Research works printed but not linked: 3
- Brett Becker. 2017. Artificial Intelligence in Education: What Is It, Where Is It Now, Where Is It Going. Ireland’s Yearbook of Education 2018 (2017), 42–46.
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, 2021. Evaluating Large Language Models Trained on Code. https://arxiv.org/abs/2107.03374. (2021). · arXiv
- ACM Joint Task Force on Computing Curriculaand IEEE Computer Society. 2013. Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science. ACM, NY, NY, USA.
Cited by
Items 101–200 of 279
Generative AI in Computer Science Education: Challenges and Opportunities
Franklin, D., Denny, P., Gonzalez-Maldonado, D. A., & Tran, M. (2025) · doi:10.1017/9781009581738 · Cited by 20
Self-citationIterative Student Program Planning using Transformer-Driven Feedback
Rivera, E., Steinmaurer, A., Fisler, K., & Krishnamurthi, S. (2024) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3649217.3653607 · Cited by 20
Leinonen, J., Denny, P. C., Kiljunen, O., MacNeil, S., Sarsa, S., & Hellas, A. (2025) · Australasian Computing Education Conference · doi:10.1145/3716640.3716647 · Cited by 20
Self-citation"Give Me the Code": Log Analysis of First-Year CS Students' Interactions with GPT
Alves, P., & Cipriano, B. P. (2025) · International Conference on Computer Supported Education · doi:10.5220/0013194400003932 · Cited by 20
Students' Use of GitHub Copilot for Working with Large Code Bases
Shah, A., Chernova, A., Tomson, E., Porter, L., Griswold, W. G., & Soosai Raj, A. G. (2025) · Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3641554.3701800 · Cited by 19
Fostering Responsible AI Use Through Negative Expertise: A Contextualized Autocompletion Quiz
MacNeil, S., Prather, J. E., Nabid, R. A., Gutierrez, S., Carvalho, S., Shrestha, S., Denny, P. C., Reeves, B. N., Leinonen, J., & Rossetti, R. L. (2025) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3724363.3729067 · Cited by 19
Self-citationWhat Skills Do You Need When Developing Software Using ChatGPT? (Discussion Paper)
Jeuring, J. T., Groot, R., & Keuning, H. (2023) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3631802.3631807 · Cited by 19
Outcomes, Perceptions, and Interaction Strategies of Novice Programmers Studying with ChatGPT
Penney, J., Acharya, P., Hilbert, P., Parekh, P., Sarma, A., Steinmacher, I., & Gerosa, M. A. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736625 · Cited by 17
Chang, C. I., Choi, W. C., Choi, I. C., & Lei, H. (2024) · International Conference on Education and E-Learning (ICEEL) · doi:10.1145/3719487.3719519 · Cited by 17
Generative AI in Programming Education: Evaluating ChatGPT’s Effect on Computational Thinking
Sivasakthi, M., & Meenakshi, A. (2025) · SN Computer Science · doi:10.1007/s42979-025-04051-9 · Cited by 16
“While artificial intelligence tools like GitHub Copilot shine at code generation, ChatGPT’s natural language interface uniquely supports conceptual understanding through dialogue, a gap noted in past research [3, 10].”
Feng, T. H., Denny, P., Wuensche, B., Luxton-Reilly, A., & Hooper, S. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636263 · Cited by 16
Self-citationCompiler-Integrated, Conversational AI for Debugging CS1 Programs
Renzella, J., Vassar, A., Solano, L. L., & Taylor, A. (2025) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3641554.3701827 · Cited by 16
Cipriano, B. P., & Alves, P. (2025) · International Conference on Computer Supported Education · doi:10.5220/0013212000003932 · Cited by 16
Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation
Salim, S. I., Yang, R., Cooper, A., Ray, S., Debray, S., & Rahaman, S. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.27 · Cited by 16
An HCI-Centric Survey and Taxonomy of Human-Generative-AI Interactions
Shi, J., Jain, R., Doh, H., Suzuki, R., & Ramani, K. (2023) · arXiv · doi:10.48550/arxiv.2310.07127 · Cited by 15
Kohn, T. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70043 · Cited by 15
Gutierrez, S., Hou, I., Lee, J., Angelikas, K., Man, O., Mettille, S., Prather, J. E., Denny, P. C., & MacNeil, S. (2025) · Australasian Computing Education Conference · doi:10.1145/3716640.3716654 · Cited by 15
Self-citationFeng, H., Denny, P. C., Wuensche, B. C., Luxton-Reilly, A., & Whalley, J. L. (2024) · doi:10.1145/3680533.3697064 · Cited by 15
Self-citationLi, N., Broner, S., Kim, Y., Mizuo, K., Sauder, E., To, C., Wang, A., Gila, O., & Shindler, M. (2025) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3641554.3701946 · Cited by 15
Haque, S., & Hundhausen, C. D. (2025) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3702652.3744231 · Cited by 14
Análise das Respostas do ChatGPT em Relação ao Conteúdo de Programação para Iniciantes
Filho, L. C. P., de Souza, T. D. P. C., & de Paula, L. B. (2023) · Anais do ... Simpósio Brasileiro de Informática na Educação/Anais do Simpósio Brasileiro de Informática na Educação · doi:10.5753/sbie.2023.234870 · Cited by 14
Denny, P. C., Kumar, V., MacNeil, S., Prather, J. E., & Leinonen, J. (2025) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3724363.3729093 · Cited by 14
Self-citationMailach, A., Gorgosch, D., Siegmund, N., & Siegmund, J. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10561-6 · Cited by 13
Rokkones, R. K., & Giannakos, M. N. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2564368 · Cited by 13
A Data Science Course Utilizing GenAI
Browning, J. W., Bustard, J. D., Anderson, N., & Galway, L. (2024) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10893452 · Cited by 13
Learning without Limits: Analysing the Usage of Generative AI in a Summative Assessment
Clift, L., & Petrovska, O. (2025) · Conference on Computing Education Practice (CEP) · doi:10.1145/3702212.3702214 · Cited by 13
Chang, C. I., Choi, W. C., & Choi, I. C. (2024) · Proceedings of the 2024 7th Artificial Intelligence and Cloud Computing Conference · doi:10.1145/3719384.3719439 · Cited by 11
Github Copilot's Impact on Developer Productivity : A Review of Early Evidence
Adepoju, S. A. (2023) · International Journal of Scientific Research in Science and Technology · doi:10.32628/ijsrst2221192 · Cited by 11
EXPLORING GITHUB COPILOT ASSISTANCE FOR WORKING WITH CLASSES IN A PROGRAMMING COURSE
(2023) · Issues in Information Systems · doi:10.48009/4_iis_2023_106 · Cited by 11
Generating Planning Feedback for Open-Ended Programming Exercises with LLMs
Demirtaş, M. A., Zheng, C., Fowler, M., & Cunningham, K. I. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98417-4_3 · Cited by 11
Understanding Programming Students' Help-Seeking Preferences in the Era of Generative AI
Penney, J., Acharya, P., Hilbert, P., Parekh, P., Sarma, A., Steinmacher, I., & Gerosa, M. A. (2025) · ACM Global Computing Education Conference (CompEd) · doi:10.1145/3736181.3747165 · Cited by 11
Probeable Problems for Beginner-level Programming-with-AI Contests
Pawagi, M., & Kumar, V. (2024) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3632620.3671108 · Cited by 11
Beyond Question Shuffling: Randomization Techniques in Programming Assessment
Hickman, H., McKeown, P., & Bell, T. (2023) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie58773.2023.10342976 · Cited by 11
Open, Small, Rigmarole – Evaluating Llama 3.2 3B’s Feedback for Programming Exercises
Azaiz, I., Kiesler, N., Strickroth, S., & Zhang, A. (2025) · International Journal of Engineering Pedagogy (iJEP) · doi:10.3991/ijep.v15i5.55359 · Cited by 11
Benchmarking Educational Program RepairPreprint
Koutcheme, C., Dainese, N., Sarsa, S., Leinonen, J., Hellas, A., & Denny, P. C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.05347 · Cited by 11
Self-citationThe underlying potential of NLP for microcontroller programming education
Rocha, A., Sousa, L., Alves, M., & Sousa, A. J. (2024) · Computer Applications in Engineering Education · doi:10.1002/cae.22778 · Cited by 10
Gupta, R. D., Hosain, M. T., Mridha, M. F., & Ahmed, S. U. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93864-1_5 · Cited by 10
Teaching Programming Error Message Understanding
Bouvier, D. J., Lovellette, E., Santos, E. A., Becker, B. A., Dasigi, V. G., Forden, J., Glebova, O., Joshi, S., Kurkovsky, S., & Russell, S. (2024) · doi:10.1145/3598579.3689377 · Cited by 10
Self-citationA systematic review of research on large language models for computer programming education
Zhu, M., Xu, L., & Ericson, B. J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.21818 · Cited by 10
The Robots are Here: Navigating the Generative AI Revolution in Computing Education
Prather, J., Denny, P., Leinonen, J., Becker, B. A., Albluwi, I., Craig, M., Keuning, H., Kiesler, N., Kohn, T., Luxton-Reilly, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.00658 · Cited by 9
Self-citationTackling Students' Coding Assignments with LLMs
Dingle, A., & Kruliš, M. (2024) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1145/3643795.3648389 · Cited by 9
Design of AI-Powered Tool for Self-Regulation Support in Programming EducationPreprint
Li, H., & Ma, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.03068 · Cited by 9
Cognitive Apprenticeship and Artificial Intelligence Coding Assistants
Poitras, E., Crane, B. G. C., Dempsey, D., Bragg, T. A., Siegel, A. A., & Lin, M. P.-C. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-1066-3.ch013 · Cited by 8
Exploring the Profile of University Assessments Flagged as Containing AI-Generated Material
Gooch, D., Waugh, K., Richards, M., Slaymaker, M., & Woodthorpe, J. (2024) · ACM Inroads · doi:10.1145/3656478 · Cited by 7
Thematic Analysis of Expert Opinions on the Use of Large Language Models in Software Development
Yadav, S., Kaushik, A., & Qureshi, A. M. (2025) · Applied AI Letters · doi:10.1002/ail2.127 · Cited by 7
Shaping Programming and Data Science Education: Insights from GenAI Technical Book Trends
Bakharia, A., & Abdi, S. (2024) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt61570.2024.00040 · Cited by 7
ÖZGÜL, E. (2026) · Kastamonu Eğitim Dergisi · doi:10.24106/kefdergi.1878122 · Cited by 6
Incremental Repair Feedback on Automated Assessment of Programming Assignments
Paiva, J. C., Leal, J. P., & Figueira, Á. (2025) · Electronics · doi:10.3390/electronics14040819 · Cited by 6
Aruleba, K., Oyelere, S. S., Obaido, G., & Sanusi, I. T. (2025) · Conference on United Kingdom & Ireland Computing Education Research (UKICER) · doi:10.1145/3754508.3754517 · Cited by 6
Yılmaz, F. G. K., Olpak, Y. Z., & Yılmaz, R. (2026) · Journal of Research on Technology in Education · doi:10.1080/15391523.2026.2683793 · Cited by 6
JavaLLM: A Fine-Tuned LLM for Java Programming Education
Zhang, J., & Liu, K. (2024) · International Symposium on Computer Science and Intelligent Control (ISCSIC) · doi:10.1109/iscsic64297.2024.00064 · Cited by 6
Mazzone, S. B., Forden, J., & Brylow, D. (2024) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10892816 · Cited by 6
Mandated media innovation impacts on knowledge dissemination in workplace training
Zutshi, A., Creed, A. S., Bhattacharya, A., Sohal, A. S., & Bavık, A. (2023) · Knowledge Management Research & Practice · doi:10.1080/14778238.2023.2209745 · Cited by 6
Lau, S., Boroojeni, K. G., Keeling, H. N., & Marroquin, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790682 · Cited by 6
Pasquarelli, L., Koutcheme, C., & Hellas, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.13326 · Cited by 6
From Assistance to Misconduct: Unpacking the Complex Role of Generative AI in Student Learning
Axelsson, A., Wallgren, D. T., Verma, U., Cajander, Å., Daniels, M., Eckerdal, A., & McDermott, R. (2024) · 2024 IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10893133 · Cited by 5
“Also, studies have shown that code generated by AI outscores the performance of most students, posing challenges on the educational system [19].”
The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers
Prather, J., Reeves, B., Leinonen, J., MacNeil, S., Randrianasolo, A. S., Becker, B., Kimmel, B., Wright, J., & Briggs, B. (2024) · arXiv · Cited by 196
“LLMs can solve most introductory programming exercises in both CS1 [18] and CS2 courses [19].”
Self-citationThe Recent Trends of Research on GitHub Copilot: A Systematic Review
Ani, Z. C., Hamid, Z. A., & Zhamri, N. N. (2024) · Communications in Computer and Information Science · doi:10.1007/978-981-99-9589-9_27 · Cited by 5
Santos, E. A., Salmon, A., & Hammer, K. (2025) · IEEE Access · doi:10.1109/access.2025.3613500 · Cited by 5
Self-citationLeveling up Learning: Serious Games for Computing Education - Long-Term Opportunities and Risks
Wuensche, B. C., Hooper, S., Whalley, J. L., Straand, I. J., Denny, P. C., Crow, T. V. H., Lange-Nawka, D. C., Luxton-Reilly, A., & Thompson, S. E. R. (2025) · Australasian Computing Education Conference · doi:10.1145/3716640.3716655 · Cited by 5
Self-citationFrom Code Generation to Conceptual Learning: Student Use of LLMs in a Web Programming Course
Isa, H.-Y., Weston, M., Wellyanto, M. R., Karna, I., Talton, J. O., & Kumar, R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3793207 · Cited by 4
Examining the Usage of Generative AI Models in Student Learning Activities for Software Programming
Chen, R., Jiang, S., Shen, J., Moon, A., & Wei, L. (2025) · ACM International Conference on AI-Powered Software (AIware) · doi:10.1109/aiware69974.2025.00016 · Cited by 4
PLAID: Supporting Computing Instructors to Identify Domain-Specific Programming Plans at Scale
Jain, Y., Demirtaş, M. A., & Cunningham, K. I. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713832 · Cited by 4
One Line at a Time: Scaffolding Reflective Code Evaluation through Structured AI Assistance
Qiu, V., Parker, L., Riegel, K., Giacaman, N., Denny, P. C., MacNeil, S., & Prather, J. E. (2026) · Australasian Computing Education Conference · doi:10.1145/3786228.3786243 · Cited by 4
Self-citationDarwiyanto, E., Gusnaen, R. A., & Nurtantyana, R. (2025) · IAES International Journal of Artificial Intelligence · doi:10.11591/ijai.v14.i6.pp4579-4589 · Cited by 4
Large language models in software development learning
Hernández, S. C., Cruz, M. P. T., Sánchez, J. M. E., Luna-Gómez, A. A., Orozco, A. M. P., & Medina, R. H. (2026) · Discover Education · doi:10.1007/s44217-026-01750-0 · Cited by 3
Zou, W., Goh, T., Zhu, H., Liu, M., & Yang, B. X. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2520931 · Cited by 3
Integration of AI Code-Writing Assistants in IT Higher Education
Käver, A., Leoste, J., Marmor, K., & Rodríguez-Piñero, P. T. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-07989-3_4 · Cited by 3
The Powerful AI: An Exploration of Generative Artificial Intelligence Taxonomy and Applications
Vadhil, F. A., Nanne, M. F., & Salihi, M. L. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-71429-0_18 · Cited by 3
Ugon, J., Pierce, C., Cain, A., & Laird, B. (2026) · Australasian Computing Education Conference · doi:10.1145/3786228.3786244 · Cited by 3
Design Implications for Student and Educator Needs in AI-Supported Programming Learning Tools
Ma, B., Xie, Y., Li, H., Li, G., Chen, L. Y. E., Shimada, A., & Konomi, S. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2730083 · Cited by 3
Cochran, K., & Piech, C. (2026) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3770762.3772663 · Cited by 3
Filho, L. C. P., de Souza, T. D. P. C., & de Paula, L. B. (2025) · Revista Brasileira de Informática na Educação · doi:10.5753/rbie.2025.4477 · Cited by 3
Tavasoli, R., VarastehNezhad, A., Masumi, M., & Taghiyareh, F. (2025) · International Computer Conference, Computer Society of Iran (CSICC) · doi:10.1109/csicc65765.2025.10967440 · Cited by 3
Feng, H., Wünsche, B. C., Denny, P. C., Luxton-Reilly, A., & Hooper, S. (2026) · Computers & Graphics · doi:10.1016/j.cag.2026.104589 · Cited by 3
Self-citationAutomatic generation of introductory programming exercises with large language models
Duong, T. N. B., Nguyen, H. L., & Gottipati, S. (2025) · Research and Practice in Technology Enhanced Learning · doi:10.58459/rptel.2026.21025 · Cited by 3
Generative AI in HigherEducation: The Students’Perception
Verma, U. (2025) · Uppsala University Publications (Uppsala University) · Cited by 3
Tabarsi, B. T., Yasir, T., Reichert, H., Tian, X., Gadireddy, S., DiMarco, C., Briceno, D., & Barnes, T. M. (2025) · doi:10.36227/techrxiv.176463808.80840600/v1 · Cited by 3
Salmon, A., Hammer, K., Santos, E. A., & Becker, B. A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.05706 · Cited by 3
Self-citationIIITD, A., Sethi, D., Gupta, N., Sharma, S., Jain, S., Singhal, U., & Kumar, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.16805 · Cited by 3
Gardella, N., Bolton, M. L., & Riggs, S. L. (2026) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208261467254 · Cited by 2
AI-Generated Slides: Are They Good? Can Students Tell?
Leinonen, J., Zhang, L., & Hellas, A. (2026) · Western Canadian Conference on Computing Education · doi:10.1145/3828792.3828803 · Cited by 2
A Validated Scale Measuring Student Self-Efficacy for Programming with Generative AI
Prather, J. E., Margulieux, L. E., Kharitonova, Y., Yang, Y., Reeves, B. N., Denny, P. C., Benario, J. G., Holmes, E., Spaulding, E., Barbre, G., et al. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811645 · Cited by 2
Self-citationTran, K., Harper, C., & Price, T. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811653 · Cited by 2
Integration of a neural network assistant on programming into a digital educational platform
Кушниренко, А. Г., Leonov, A. G., Мащенко, К. А., Martynov, N. S., Пчелин, К. К., & Shlyakhov, A. V. (2026) · Informatics and Education · doi:10.32517/0234-0453-2026-41-1-23-36 · Cited by 2
Digital Enhancement: Applying Generative AI in Students Decision Making Process
Cabral, C. A. A., & Pereira, R. (2025) · Studies in systems, decision and control · doi:10.1007/978-3-031-78412-5_7 · Cited by 2
Tang, K., Tong-Ho, H. N. H., Riegel, K., Denny, P. C., Giacaman, N., & Leinonen, J. (2026) · Australasian Computing Education Conference · doi:10.1145/3786228.3786249 · Cited by 2
Self-citationXingjian, Santo, W. L., Dumlao, J. M. Z., & Ericson, B. J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.16475 · Cited by 2
CHAT GPT APLICADO AO ENSINO DE REALIDADE AUMENTADA: APLICAÇÕES NO AMBIENTE EDUCACIONAL
de Abreu, G. M. S., Madeira, L. R., & Neto, C. D. S. S. (2024) · Revista Foco · doi:10.54751/revistafoco.v17n3-067 · Cited by 2
AI Chatbots vs. Traditional Search: A Comparative Study on Student Information Retrieval
Pasquarelli, L., Koutcheme, C., & Hellas, A. (2025) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie63693.2025.11328491 · Cited by 2
Can LLMs be Fair Graders? Only if You Bribe Them with the Right Rubric
Mazzone, S. B., Forden, J., & Brylow, D. (2025) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie63693.2025.11328653 · Cited by 2
AI Code Assistants in Programming Education: A Narrative Literature Review
Farooq, U., Morganti, D., & Anwar, S. (2026) · Education Sciences · doi:10.3390/educsci16060961 · Cited by 2
Kwubeghari, A. (2025) · Caliphate Journal of Science and Technology · doi:10.4314/cajost.v7i2.4 · Cited by 2
Using Large Language Models to Generate Worked Examples of CS2-Level Programming Questions
Lindvall, V., & Marcus, S. (2024) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 2
Kohn, T., Antensteiner, T., Hahnel, C., Kern, B., Keuning, H., Kia, E., Kiesler, N., Komm, D., Marinus, E., Šavelka, J., et al. (2026) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3856208.3856233 · Cited by 2
Evaluating Code Generation of LLMs in Advanced Computer Science ProblemsPreprint
Catir, E., Claesson, R., & Tsoupidi, R. M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.14964 · Cited by 2
Tamami, F. R., & Rosyidah, A. (2026) · Indonesian Journal of Progressive Pedagogy · doi:10.61987/ijpp.v2i2.1280 · Cited by 2
Generative AI in Cyber Security: New Threats and Solutions for Adversarial Attacks
AbdelLatif, R. A., Hamed, M. A., Gomaa, I. A., & AttaAlla, M. A. (2026) · IEEE International Conference on Smart Sustainable Systems for Computer and Engineering Applications (3SCEA) · doi:10.1109/3scea68071.2026.11602991 · Cited by 1
Unraveling Ambiguities: Analyzing Student Approaches to Solving Probeable Problems
Kumar, V., Denny, P. C., Leinonen, J., Prather, J. E., & MacNeil, S. (2026) · Australasian Computing Education Conference · doi:10.1145/3786228.3786255 · Cited by 1
Self-citationGenerative AI Tool in Learning an Introductory Programming: A Research Perspective
Sivasakthi, M., & Meenakshi, A. V. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-86296-0_10 · Cited by 1
Metadata
- Authors
- Finnie-Ansley, J., Denny, P., Luxton-Reilly, A., Santos, E. A., Prather, J., & Becker, B. A.
- Affiliations
- University of Auckland (NZ)
- University College Dublin (IE)
- Abilene Christian University (US)
- ORCID
- Venue
- Proceedings of the 25th Australasian Computing Education Conference
- Publisher
- ACM
- Published
- 2023-01-30
- Pages
- 97-104
- OpenAlex
- W4317502110
- Cited by
- 186
- References
- 21
- Citations by year
- 2026: 34 · 2025: 59 · 2024: 59 · 2023: 34
- FWCI
- 55.98
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Australasian Computing Education Conference (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- OpenAI Codex, generative AI in education, data structures and algorithms, AI-generated code, GitHub Copilot, student performance comparison, undergraduate computer science education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.