The DOI matches its Crossref record; 105 of the 106 references it cites were found in a registry or an index; 15 of the 30 sources its claims cite were checked for support.
“It’s Weird That it Knows What I Want”: Usability and Interactions with Copilot for Novice Programmers
Open accessAbstract
Recent developments in deep learning have resulted in code-generation models that produce source code from natural language and code-based prompts with high accuracy. This is likely to have profound effects in the classroom, where novices learning to code can now use free tools to automatically suggest solutions to programming exercises and assignments. However, little is currently known about how novices interact with these tools in practice. We present the first study that observes students at the introductory level using one such code auto-generating tool, Github Copilot, on a typical introductory programming (CS1) assignment. Through observations and interviews we explore student perceptions of the benefits and pitfalls of this technology for learning, present new observed interaction patterns, and discuss cognitive and metacognitive difficulties faced by students. We consider design implications of these findings, specifically in terms of how tools like Copilot can better support and scaffold the novice programming experience.
Overview
CoChat Summary
TL;DR
This paper reports the first observational and interview study of novice CS1 students using GitHub Copilot on a typical introductory programming assignment. It finds that novices often saw Copilot as fast and helpful, but also struggled to understand its suggestions, worried about over-reliance, and exhibited two new interaction patterns: shepherding and drifting.
Key findings
- Most students perceived Copilot as helping them write code faster and avoid syntax errors, but many also worried about not understanding generated code and becoming dependent on it; “Outsourcing” concerns appeared in 46 interview excerpts from 17 unique participants, and “Speed” in 37 excerpts from 17 unique participants.
- The study identifies two novel interaction patterns: *shepherding*, where students spent most of their time coercing Copilot toward a solution, and *drifting*, where students moved from one incorrect suggestion to the next, backtracking and deleting code without direction.
- Students showed cognitive and metacognitive difficulties: “Confused” was observed 65 times across 13 unique participants, and “Metacognitive” appeared 24 times in observations and 17 times in interviews, each across 10 unique participants.
- Some students attributed intelligence or agency to Copilot, e.g., “It like read my mind!” and “I thought it was weird that it knows what I want,” while others raised fears about future job impacts, privacy, plagiarism, and over-reliance.
Methods
- Nineteen novice programmers (5 women, 14 men, ages 18–22) enrolled in a CS1 course at a Midwestern private research university in the USA were observed one at a time in April 2022 while solving a Minesweeper-style C++ assignment in Visual Studio Code with Copilot enabled.
- Each participant had 30 minutes, followed a think-aloud protocol, and then completed a short interview about whether Copilot helped them understand the problem, whether they would use it, its advantages, and their fears or worries.
- Data from observations and interviews were analyzed together using reflexive thematic analysis following Braun and Clarke’s six phases; themes included Interactions, Cognitive, Purpose, and Speculation, with saturation treated as the point where additional data produced no new emergent themes.
Limitations
- The paper states that screens and audio were not recorded due to IRB considerations, that study conditions were somewhat lab-like rather than normal in-class assignment conditions, that only Copilot was studied, and that data collection occurred in April 2022 shortly after Copilot’s release, so novelty effects may have shaped results.
- Evident from the text: the sample is small (19 students), from a single institution and single CS1 course, with only a 30-minute exposure to Copilot, so findings may not generalize to other populations, courses, or longer-term use.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
11 Valid · 3 Not valid · 11 Not checked · 109 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Code generators like Codex have been shown to perform better than the average student on introductory-level programming problems, thus they provide an effective tool for students who might be contemplating cheating [34].”
What the sources say
- Supportedcites [34] [34] James Finnie-Ansley, Paul Denny, Brett A. Becker, Andrew Luxton-Reilly, and James Prather. 2022. The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming. In Australasian Computing Education Conference (Virtual Event, Australia) (ACE '22) . Association for Computing Machinery, New York, NY, USA, 10-19. https://doi.org/10.1145/3511861.3511863 - source
Section 4.1.1 (page not available) reports that Codex ranked 17th alongside 71 students on two introductory programming tests, placing it in the top quartile—above the average student. Section 5.3 also describes Codex as a way for students to obtain solutions with low risk of similarity detection, supporting the sentence’s academic-integrity context. (LLM, conf=0.92)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Our participants acknowledged that such over-reliance could hinder their own learning, a concern that has also been expressed by computing educators [32].”
What the sources say
- Supportedcites [32] [32] Neil A Ernst and Gabriele Bavota. 2022. AI-Driven Development Is Here: Should You Worry? IEEE Software 39, 2 (2022), 106-110. - source
The cited paper’s “Learning to Program” discussion (Possible Challenges with AIDE, p. 3) expresses the concern attributed to it: learners may accept AI-generated recommendations without fully understanding them. “Our participants” refers to the citing study’s participants, not to this source. (LLM, conf=0.91)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“The primary benefit that novices saw in using Copilot was that it accelerated their progress on the programming task, mirroring results with experienced programmers as reported by Barke et al. [8] and Vaithilingam et al. [80].”
What the sources say
- Supportedcites [8] [8] Shraddha Barke, Michael B. James, and Nadia Polikarpova. 2022. Grounded Copilot: How Programmers Interact with Code-Generating Models. arXiv preprint arXiv:2206.15000 (2022). https://doi.org/10.48550/ARXIV.2206.15000 - source
Barke et al. are cited for the parallel finding among experienced programmers, not for the citing paper’s finding about novices. The Introduction says Copilot helps programmers reach a known next step more quickly, and §4.1 describes its use to type out intended code faster. The Related Work section notes that their participants were mostly experienced developers. (LLM, conf=0.92)
- Not checked — no full text of the sourcecites [80] [80] Priyan Vaithilingam, Tianyi Zhang, and Elena L. Glassman. 2022. Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In CHI Conference on Human Factors in Computing Systems Extended Abstracts . Association for Computing Machinery, New York, NY, USA, 1-7. - source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“On introductory level problems like the one in our study, Copilot generates correct solutions most - but not all - of the time [34].”
What the sources say
- Partly supportedcites [34] [34] James Finnie-Ansley, Paul Denny, Brett A. Becker, Andrew Luxton-Reilly, and James Prather. 2022. The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming. In Australasian Computing Education Conference (Virtual Event, Australia) (ACE '22) . Association for Computing Machinery, New York, NY, USA, 10-19. https://doi.org/10.1145/3511861.3511863 - sourceExperimental — accuracy not yet measured
The Introduction says Codex powers GitHub Copilot, and §4.1 reports that Codex solved 19 of 23 introductory programming questions within ten attempts. But only about half were solved on the first attempt, so the paper does not establish that a single Copilot-generated solution is correct most of the time. Page numbers are unavailable in our copy. (LLM, conf=0.87)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Still others utilized the system like a colleague who can help them when stuck, a behavior previously documented between students co-regulating their learning in study groups [69], and which matches the idea of Copilot being 'your AI pair programmer.'”
What the sources say
- Supportedcites [69] [69] James Prather, Lauren Margulieux, Jacqueline Whalley, Paul Denny, Brent N. Reeves, Brett A. Becker, Paramvir Singh, Garrett Powell, and Nigel Bosch. 2022. Getting By With Help From My Friends: Group Study in Introductory Programming Understood as Socially Shared Regulation. In Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 1 (Lugano and Virtual Event, Switzerland) (ICER '22) . Association for Computing Machinery, New York, NY, USA, 164-176. https://doi.org/10.1145/3501385.3543970 - source
The cited part of the sentence is the peer-helping-when-stuck behaviour documented in study groups, not the Copilot framing. Prather et al. (2022) report 'help seeking -social' as the most common co-regulation theme in their study-group data (§4.2.2 Co-regulation Themes), illustrated by student statements such as 'When we try hard but still can not figure out how to solve the problem, we tend to ask each other' and 'I work with my friends when I become stuck ... and we help each other to understand the problem better and tackle it' (§5.3). Table 3 quantifies the theme (35 responses at Lab 7, 35 at Lab 12 for co-regulation), and §5.2 discusses the shift from resources to social help-seeking within groups. Thus the source does document students treating peers as a helping colleague when stuck while co-regulating in study groups. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
56 more claims · 1 with their cited sources checked
Create a free account to see all 134 claims and to verify the 2 references still open.
- Probably not a substantial claim. 46
- It may state two separate claims as one. 38
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our findings affirm that AI-generated outputs can boost productivity; they often require rigorous review and contextual interpretation to ensure software safety and integrity [14].”
From Redefining the Programmer: Human-AI Collaboration, LLMs, and Security in Modern Software Engineering (2025) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Moreover, the dependency on GenAI to complete academic tasks could reduce students' critical thinking and independent problem-solving skills [16,36].”
From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent research has shown promising possibilities for providing students with LLM partners in pair programming, given the right context and with the right scaffolding and support [44,108,159].”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [159]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Wermelinger [191] speculated that students may spend excessive time on prompt engineering in the hope of hitting on a successful solution rather than making progress towards the solution, which was indeed observed in the study by Prather et al. [159] in the 'shepherding' interaction pattern.”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [159]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“From this study, two main modalities emerged (Prather et al., 2024): in 'acceleration' mode, the programmer is aware of their next steps and utilizes the tool to speed up the coding process. In contrast, in the 'exploration' mode, the programmer is uncertain about how to proceed and employs the tool to investigate his options.”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Prather et al., 2024
Experimental — accuracy not yet measured
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Reflecting on reflexive thematic analysis
Braun, V., & Clarke, V. (2019) · Qualitative Research in Sport, Exercise and Health · doi:10.1080/2159676x.2019.1628806 · Cited by 19,677 · cited 1×
“Coding is fluid and involves combining and splitting codes in an iterative process as the researchers become more familiar with the data, and throughout the analysis [19].”
BackgroundKaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905
Extreme Programming Explained: Embrace Change
Beck, K. L., & Andres, C. (2000) · Cited by 5,902
Conceptual and design thinking for thematic analysis.
Braun, V., & Clarke, V. (2022) · Qualitative Psychology · doi:10.1037/qup0000196 · Cited by 4,502 · cited 1×
“To analyze the data we used relexive thematic analysis [18, 20] which aims to understand pattern and meaning present in qualitative data.”
MethodLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · cited 2×
“The internal architecture of the models is opaque for most users, which has led to multiple approaches for constructing functional prompts, often called “prompt engineering” [51].”
BackgroundLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · cited 2×
“The internal architecture of the models is opaque for most users, which has led to multiple approaches for constructing functional prompts, often called “prompt engineering” [51].”
BackgroundCognitive load theory, learning difficulty, and instructional design
Sweller, J. (1994) · Learning and Instruction · doi:10.1016/0959-4752(94)90003-5 · Cited by 3,801 · cited 1×
“We hypothesize that lengthy auto-generated code suggestions may lead to an increased cognitive load [98] among novices and call on researchers to explore this in future work.”
Background
Research works printed but not linked: 3
- Emily M. Bender. 2019. A Typology of Ethical Risks in Language Technology with an eye towards where transparent documentation can help. Future of Artificial Intelligence: Language, Ethics, Technology Workshop.
- 10.1109/SP46215.2023.00001 · doi:10.1109/sp46215.2023.00001
- Ashley D. Rittmayer and Margaret E. Beier. 2008. Overview: Self-efficacy in STEM. SWE-AWE CASEE Overviews 1, 3 (2008), 12.
Cited by
Items 201–247 of 247
Yazar, S. (2026) · Üniversite Araştırmaları Dergisi · doi:10.32329/uad.1890338 · Cited by 1
Integration of Artificial Intelligence Tools Into the Agroforestry Engineering Curriculum
Aragón-Rodríguez, F., Herranz-Matey, I., Rocamora, C., Almeida-Ñauñay, A. F., & Perdigones, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-1285-9.ch011
García, R. N. G., Arias, S. C. D. L., Llontop, R. F. G., Hashimoto, E. M. C., Gallegos, J. P. C., Zaquinaula, I. R. A., Vara, F. E. O., Severino, E. C. P., Ruiz, L. C. V., García, J. E. G., et al. (2026) · South Florida Journal of Development · doi:10.46932/sfjdv7n6-017
Chiang, Y.-C., & Chang, C.-H. (2026) · International Conference on Information and Education Technology (ICIET) · doi:10.1109/iciet69664.2026.11561654
“However, research by indicates that when interacting with such all-encompassing AI, novices tend to request code directly, often lacking a deep understanding of the underlying logic [4].”
RTL-SMARTIE: An AI-Assisted Tutor for RTL Design Education
Yarnell, R. C., Amoruso, P. E., & Emeterio, A. (2026) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3787109.3816047
Chang, C.-K., & Li, K.-H. (2026) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3774398.3811596
Bougault, P., Liang Ma, A., & Guey, W. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29842-3_20
Grounding Programming Chatbot in Computational Thinking: Design and Evaluation of MazeMate
Hou, C., Yu, H., Zhu, G., Anas, J. D., Liu, J., & Ong, Y. S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29744-0_35
Hassen, M. Z. (2026) · Science Journal of Education · doi:10.11648/j.sjedu.20261403.14
How Do Developers Interact with AI? An Exploratory Study on Modeling Developer Programming Behavior
Wu, Y., Li, Z. S., Stolee, K. T., & Xu, B. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808120 · Cited by 3
Human-Centered AI for e-Collaboration: Software Development, Design, and Implementation
Le, H. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8217-3.ch003
Computing Science Education on the AI Innovation Landscape
Karimi, O. B., Robinson, R., Bonjour, T., et al. (2026) · Annual Conference on Innovation and Technology in Computer Science Education · doi:10.1145/3803401.3812053
Effective Use of Large Language Models for Social Constructivism in Computer Science Education
Falkner, N., Leppänen, L., & Leinonen, J. (2026) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3803400.3809392 · Cited by 1
Self-citationHow Can We Know When We See It? A Systematic Review of Cognitive Control Skills and Behaviors
Domino, M., Thompson, J. A., Hicks, A., Jamieson, A. C., Parajuli, K., Edmison, B., Edwards, S. H., & Shaffer, C. A. (2026) · ACM Transactions on Computing Education · doi:10.1145/3821524
Yang, Y., Feng, H., Lu, B., & Sun, J. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-30826-9_26
Human-AI interactions (HAII) in STEM education: A systematic review from 2010 to 2025
Wu, M., & Ouyang, F. (2026) · Educational Research Review · doi:10.1016/j.edurev.2026.100821 · Cited by 2
Teaching Programming in the Age of AI: Pedagogical Strategies for Computational Sociology
Ramos, M. C., & Liang, F. (2026) · Teaching Sociology · doi:10.1177/0092055x261469125
Gardella, N., Bolton, M. L., & Riggs, S. L. (2026) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208261467254
Grasping AI Reliance in Program Comprehension and Coding through the AIRELI Persona Taxonomy
Alakmeh, T., Anderson, N. B., Jackson, V., Pereira, G. V., Akırmak, Ü., Estey, A., Prikladnicki, R., van der Hoek, A., Storey, M., & Fritz, T. (2026) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3794763.3794804 · Cited by 5
PUȚANU, A. (2026) · Proceedings of the ... International Conference on Business Excellence · doi:10.2478/picbe-2026-0112
“Most of this work, however, looks at programming, education, or general content creation (Bird et al., 2022; Prather et al., 2023; Lau & Guo, 2023).”
Lietz, R., Whittaker, S., & Su, N. M. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2711023
“Our data collection and analysis were guided by reflexive thematic analysis practices commonly used in qualitative HCI research (Braun and Clarke 2019; Liang, Munson, and Kientz 2021; Prather et al. 2024; Shi et al. 2023).”
Teaching Assistants as Assessors: Deepening the Narrative in Grading Practices
Ahmed, F., Bouali, N., & Gerhold, M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-21997-8_6
Pădurean, V.-A., Riegel, K., Barbre, G., Blake, M., Denny, P. C., Gotovos, A., Leinonen, J., MacNeil, S., Prather, J. E., & Singla, A. K. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811639 · Cited by 1
Self-citationScaffolding Autocomplete: Improving Guidance for Learners using Generative Code Suggestions
Prather, J. E., MacNeil, S., Luxton-Reilly, A., Margulieux, L. E., Reeves, B. N., Denny, P. C., Leinonen, J., Homer, J., Nabid, R. A., & Rossetti, R. L. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811654
Self-citationTran, K., Harper, C., & Price, T. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811653 · Cited by 2
Gardella, N., Prather, J. E., Leinonen, J., Denny, P. C., Pettit, R., & Riggs, S. L. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811664
Self-citationWhen AI Is Wrong on Purpose: How Students Respond to Buggy GenAI Code
Pădurean, V.-A., Riegel, K., Gotovos, A., Mahapatra, J., Ghosh, A., Denny, P. C., Leinonen, J., Prather, J. E., & Singla, A. K. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811667 · Cited by 4
Self-citationA Validated Scale Measuring Student Self-Efficacy for Programming with Generative AI
Prather, J. E., Margulieux, L. E., Kharitonova, Y., Yang, Y., Reeves, B. N., Denny, P. C., Benario, J. G., Holmes, E., Spaulding, E., Barbre, G., et al. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811645 · Cited by 2
Self-citationTowards Human-Centered Large Language Models for Personalized Programming Education
Fan, J. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765965.3816672
“However, LLM-generated feedback is often misaligned with learners’ needs, as it does not account for prior knowledge or learning processes [8].”
Kocak, O., İdil, Ş., & Aydoğan, M. (2026) · Research Square · doi:10.21203/rs.3.rs-9615155/v1
AI-assisted Arduino Coding in First-Year Engineering Team Projects
Tian, E., & Boback, A. (2026) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--59397
Scaling Formative Feedback in Programming Education through Mastery Learning and AI
Zhang, W., & Pastorino, J. (2026) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--60396 · Cited by 1
Pair Programming with AI: A Systematic Literature Review
Smith, J. M. (2026) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--60384
Zhong, Q., Zhong, Y., Yu, J., & Ma, M. (2026) · International Conference on Computer Science and Technologies in Education (CSTE) · doi:10.1109/cste69562.2026.11649712
“It can provide immediate, personalized feedback, reduce initial frustration associated with syntax errors, and expose students to diverse coding styles and algorithmic solutions, potentially fostering a more exploratory and confident learning environment [5].”
Exploring the Influence of Generative AI on Engineering Students' Learning Experience
Sadri, A. M., & Kittur, J. (2026) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--59827
Wang, X., Hou, Z., Lang, S., & Yu, Z. (2026) · IEEE Conference on Cybernetics and Intelligent Systems · doi:10.1109/cis-ram71614.2026.11668538
“The three interaction patterns identified in this study through the analysis of code commit behaviors are broadly consistent with the classifications of AI-assisted learning behaviors reported in the existing literature [21-22] .”
Human-Centric Integration of Large Language Models into Software Engineering Tasks
Ferino, S. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3774748.3787649
Hey, ChatGPT, Look at My Work: Using Conversational AI in Requirements Engineering Education
Badihi, S., Tegegn, M., Riddell, E., Czarnecki, K., & Rubin, J. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773112 · Cited by 1
Kumar, A., Wongsirichot, T., & Nanthaamornphong, A. (2026) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2026.100677 · Cited by 2
Beyond Coding: How Students Use Generative AI Across the Full Software Development Lifecycle
Matti, S., Szendrei, R., & Pintér, B. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-32732-1_14
Flores-Martin, D., & Berrocal, J. (2026) · Array · doi:10.1016/j.array.2026.101242
Meng, Y., Zhang, X., & Han, L. (2026) · Higher Education · doi:10.1007/s10734-026-01773-4 · Cited by 1
Deng, C., Ma, Z., Du, M., & Zheng, Y. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01277-2
Hadjichristofi, C., Gerokonstantis, D., Papadeas, P. V., & Vescoukis, V. (2026) · Annals of Computer Science and Information Systems · doi:10.15439/2026f9512
“Research on novice programmers further shows that interacting with AI-based coding assistants can be challenging and may lead to unintended learning outcomes [21].”
Li, S., Tang, J., Qiang, Z., & Wang, Z. (2026) · Journal of Intelligence · doi:10.3390/jintelligence14100232
From Black Box to Glass Box: GenAI in Non-STEM Computing Education
Gutiérrez‐Madroñal, L., Caballero-Hernández, J. A., Baena-Perez, R., & Palomo‐Duarte, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37982-5_17
Beyond Code Generation: Reassessing Software Quality in AI-Assisted Development
Cernau, L. D. (2026) · Acta Universitatis Sapientiae Informatica · doi:10.1007/s44427-026-00046-3
“Empir-icalevidencefortheserisksispresentedin[9].”
Influential
Metadata
- Authors
- Prather, J., Reeves, B. N., Denny, P., Becker, B. A., Leinonen, J., Luxton-Reilly, A., Powell, G., Finnie-Ansley, J., & Santos, E. A.
- Affiliations
- Abilene Christian University (US)
- University of Auckland (NZ)
- University College Dublin (IE)
- ORCID
- Venue
- ACM Transactions on Computer-Human Interaction
- Published
- 2023-11-29
- Volume
- 31
- Issue
- 1
- Pages
- 1-31
- ISSN
- 1073-0516 (ISSN-L)
- 1557-7325
- OpenAlex
- W4386099272
- Cited by
- 256
- References
- 91
- Citations by year
- 2026: 89 · 2025: 85 · 2024: 66 · 2023: 16
- FWCI
- 57.98
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Transactions on Computer-Human Interaction (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- GitHub Copilot, code generation, novice programmers, introductory programming, CS1, programming education, student perceptions, metacognition, cognitive load, scaffolding, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.