The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index; 6 of the 16 sources its claims cite were checked for support.
Grounded Copilot: How Programmers Interact with Code-Generating Models
Open accessAbstract
Powered by recent advances in code-generating models, AI assistants like Github Copilot promise to change the face of programming forever. But what is this new face of programming? We present the first grounded theory analysis of how programmers interact with Copilot, based on observing 20 participants—with a range of prior experience using the assistant—as they solve diverse programming tasks across four languages. Our main finding is that interactions with programming assistants are bimodal : in acceleration mode , the programmer knows what to do next and uses Copilot to get there faster; in exploration mode , the programmer is unsure how to proceed and uses Copilot to explore their options. Based on our theory, we provide recommendations for improving the usability of future AI programming assistants.
Overview
CoChat Summary
TL;DR
This paper presents a grounded theory of how programmers interact with GitHub Copilot, based on observing 20 participants across four programming languages. Its central finding is that Copilot-assisted programming is bimodal: programmers use Copilot either in acceleration mode, to execute a known plan faster, or in exploration mode, to discover how to proceed.
Key findings
- Interactions with Copilot are bimodal: acceleration mode occurs when the programmer already knows what to do next and uses Copilot to get there faster; exploration mode occurs when the programmer is unsure how to proceed and uses Copilot to explore options or get a starting point.
- Across all participants, total study time in exploration mode was 248.6 minutes, more than twice the 104.7 minutes in acceleration mode.
- Prompting strategies differed by mode: in acceleration mode, code prompts were most common (71.4%) and context prompts second (15.2%); in exploration mode, comment prompts were much more common than in acceleration mode (57.2% vs 13.1%).
- Validation also differed: exploration involved more careful examination and documentation reading, while acceleration more often used execution for rapid feedback; participants also used five livestream videos to confirm behavior consistent with the theory.
Methods
- The authors conducted a Straussian grounded theory study with 20 participants (15 from academia, 5 from industry; 9 had prior Copilot usage) who solved programming tasks with Copilot across Python, Rust, Haskell, and Java.
- Participants completed a training task, a core task lasting about 20–40 minutes while talking through their interactions, and a semi-structured interview; sessions were recorded and transcribed.
- Analysis interleaved data collection and qualitative coding, with open, axial, and selective coding, theoretical sampling, and theoretical saturation at the 20th participant; the authors then re-coded videos quantitatively and qualitatively analyzed five livestream videos from YouTube and Twitch.
Limitations
- Stated in the paper: participants worked on tasks of the authors’ design rather than their own projects, and the tasks focused only on code authorship, not refactoring, testing, debugging, or other software engineering activities.
- Stated in the paper: the 20-participant sample skews toward academia and is not representative of all programmers; 11 participants had not used Copilot before the study, and prior users may have formed usage patterns during the technical preview.
- Stated in the paper: the authors could not observe programmers over a longer period to study changing usage patterns, and rapid progress in code-generating models could render the findings obsolete.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
4 Valid · 0 Not valid · 8 Not checked · 13 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“OpenAI's Codex model [Chen et al. 2021], which contains 12 billion model parameters and is trained on 54 million software repositories on GitHub, is able to correctly solve 30-70% of novel Python problems”
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > 1 INTRODUCTION
What the sources say
- Partly supportedcites [Chen et al. 2021] sourceExperimental — accuracy not yet measured
Section 3.1 (p. 4) says the training data came from 54 million public GitHub repositories, and Section 3 (p. 4) describes Codex models with up to 12 billion parameters. The Abstract (p. 1) reports 28.8% of HumanEval problems solved with one sample and 70.2% with 100 samples per problem. The claimed 30–70% range is approximately right, but it omits that the upper end required repeated sampling and evaluation on a specific problem set. (LLM, conf=0.86)
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Copilot is powered by the OpenAI Codex family of models [Chen et al. 2021], which are derived by fine-tuning GPT-3 [Brown et al. 2020] on publicly available Github repositories.”
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > 2 COPILOT-ASSISTED PROGRAMMING, BY EXAMPLE
What the sources say
- Supportedcites [Chen et al. 2021] source
The Introduction (p. 1) says descendants of the Codex models described in this paper power GitHub Copilot. The Abstract (p. 1) identifies Codex as fine-tuned on publicly available GitHub code, and Methods (p. 4) describes fine-tuning from the GPT-3 model family. (LLM, conf=0.94)
- Supportedcites [Brown et al. 2020] source
The Brown et al. citation is attached to “GPT-3,” identifying the model used in the citing sentence. The Abstract (p. 1) states that the paper trains GPT-3. The separate claims about Copilot, Codex, and GitHub repositories are not attributed to this citation. (LLM, conf=0.93)
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“has recently been adopted to study phenomena in software engineering [Stol et al. 2016] and programming languages [Lubin and Chasins 2021].”
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > 1 INTRODUCTION
What the sources say
- Supportedcites [Stol et al. 2016] source
The citation to Stol et al. is attached to the software-engineering portion of the sentence, not the programming-languages portion. The Conclusion (p. 11) describes grounded theory as an increasingly popular research method in software engineering and reports its use in SE studies. (LLM, conf=0.94)
- Supportedcites [Lubin and Chasins 2021] source
The Lubin and Chasins citation is attached to the programming-languages portion of the sentence. Section 8.1 describes their work as a grounded theory analysis of programming sessions by expert programmers, and Section 4 presents the resulting theory. The separate software-engineering portion is attributed to Stol et al., not this source. (LLM, conf=0.94)
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“We follow the framework laid out by Strauss and Corbin [Strauss and Corbin 1990], commonly called Straussian Grounded Theory [Stol et al. 2016].”
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > Grounded Theory Process.
What the sources say
- Not matchedcites [Strauss and Corbin 1990]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [Stol et al. 2016] source
The citation to Stol et al. supports the name of the framework, not the citing authors’ statement that they follow it. Section 2.3 (p. 4) explicitly identifies Strauss and Corbin’s grounded theory as “Straussian GT.” (LLM, conf=0.95)
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our main finding is that interactions with programming assistants are bimodal: in acceleration mode, the programmer knows what to do next and uses Copilot to get there faster; in exploration mode, the programmer is unsure how to proceed and uses Copilot to explore their options.”
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > Abstract
20 more claims
Create a free account to see all 25 claims and every source verdict.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Earlier research noted the difficulties users faced in effectively communicating intent and matching their abstraction level to the model's capabilities, often relying on explicit comments (Sarkar et al., 2022a; Barke et al., 2023).”
From Vibe coding: programming through conversation with artificial intelligence (2025) · cites this paper as Barke et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While a number of researchers have looked at how current developers may use LLM-based tools [32,52,81] and how LLM-tools may be incorporated into professional software development tools [84], the participants in these studies have been programmers who learned to code initially without using LLMs.”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These two activities align with the grounded theory proposed by Barke et al. [18], which describes two developers' interaction modes with AI assistants -acceleration (supported by code generation) and exploration (supported by code summarization).”
From Using AI-based coding assistants in practice: State of affairs, perceptions, and ways forward (2025) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This focus on augmenting the content quality of completions has inadvertently overshadowed a vital aspect of the user experience: the interaction dynamics between the developers and the AI tools [2, 34, 42].”
From A Transformer-Based Approach for Smart Invocation of Automatic Code Completion (2024) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This introduces a workflow where the programmer may intentionally allow the AI to 'drive' more consistently, even for familiar tasks, creating an fluid interaction style that is distinct from the segmented modes previously identified (Barke et al., 2023).”
From Vibe coding: programming through conversation with artificial intelligence (2025) · cites this paper as Barke et al., 2023
Experimental — accuracy not yet measured
16 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > Usability of Copilot.
“only one participant in our study was an undergraduate student”
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > PCU
“P9 Rust 1 Occasional No Undergraduate Student Benford's law P10 Python Occasional No Undergraduate Student Chat Client”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > 4.2.5 Programmers Validate Suggestions Explicitly.
“All participants in the Advent of Code task ran their code to check whether they parsed the input file correctly.”
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > Static analysis.
“P17 did not run their code even once for the Advent of Code task”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > 5.1.2 Prompting Strategies across Interaction Modes.
“Fig. 6 shows the aggregate percentage of times the 20 participants invoked Copilot using the four different prompting strategies.”
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > 8 LIMITATIONS AND THREATS TO VALIDITY
“Fig. 6.Prevalence of prompting strategy as percentage of total prompting time.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > 4.1.2 Programmers Focus on Small Logical Units.
“when P1 wanted to send a message to a client connection object in the Chat Client task”
§ Grounded Copilot: How Programmers Interact with Code-Generating Models > PCU
“P1 Python Professional Yes Professor Chat Server”
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
The SAGE Handbook of Grounded Theory
Bryant, A., & Charmaz, K. (2007) · SAGE · doi:10.4135/9781848607941 · Cited by 2,798
Flow and the Foundations of Positive Psychology: The Collected Works of Mihaly Csikszentmihalyi
Csikszentmihalyi, M. (2014) · doi:10.1007/978-94-017-9088-8 · Cited by 1,653
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
Competition-Level Code Generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.07814 · Cited by 1,043
(2000) · doi:10.1145/351240.351266 · Cited by 753
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660
Research works printed but not linked: 8
- Mohammad Bavarian Heewoo Jun Nikolas Tezak John Schulman Christine McLeavey Jerry Tworek and Mark Chen. 2022. Efficient Training of Language Models to Fill in the Middle. Jul https://doi.org/10.48550/arXiv.2207.14255 arXiv:2207.14255 [cs]. · DOI · Paper page
- Daniel Fried Armen Aghajanyan Jessy Lin Sida Wang Eric Wallace Freda Shi Ruiqi Zhong Wen-tau Yih Luke Zettlemoyer and Mike Lewis. 2022. InCoder: A Generative Model for Code Infilling and Synthesis. Apr https://doi.org/10.48550/arXiv.2204.05999 arXiv:2204.05999 [cs]. · DOI · Paper page
- Daniel Kahneman . 2011. Thinking , fast and slow . Penguin Books . isbn:978-0-14-103357-0 Daniel Kahneman. 2011. Thinking, fast and slow. Penguin Books. isbn:978-0-14-103357-0
- Kite. 2020 . Kite: AI-Powered Completions for JupyterLab. https://www.kite.com/integrations/jupyter/ Kite. 2020. Kite: AI-Powered Completions for JupyterLab. https://www.kite.com/integrations/jupyter/
- Yujia Li David Choi Junyoung Chung Nate Kushman Julian Schrittwieser Rémi Leblond Tom Eccles James Keeling Felix Gimeno Agustin Dal Lago Thomas Hubert Peter Choy Cyprien de Masson d'Autume Igor Babuschkin Xinyun Chen Po-Sen Huang Johannes Welbl Sven Gowal Alexey Cherepanov James Molloy Daniel J. Mankowitz Esme Sutherland Robson Pushmeet Kohli Nando de Freitas Koray Kavukcuoglu and Oriol Vinyals. 2022. Competition-Level Code Generation with AlphaCode. https://doi.org/10.48550/ARXIV.2203.07814 · DOI · Paper page
- Anselm L. Strauss and Juliet Corbin . 1990 . Basics of Qualitative Reseach: Grounded Theory Procedures and Techniques. SAGE Publications, Inc .. isbn:0-8039-3250-2 Anselm L. Strauss and Juliet Corbin. 1990. Basics of Qualitative Reseach: Grounded Theory Procedures and Techniques. SAGE Publications, Inc.. isbn:0-8039-3250-2
- Steven J. Taylor and Robert Bogdan . 1998 . Introduction to qualitative research methods: A guidebook and resource, 3 rd ed. John Wiley and Sons Inc . isbn:978-0-471-16868-3 Steven J. Taylor and Robert Bogdan. 1998. Introduction to qualitative research methods: A guidebook and resource, 3rd ed. John Wiley and Sons Inc. isbn:978-0-471-16868-3
- Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Lukasz Kaiser and Illia Polosukhin. 2017. Attention Is All You Need. Dec https://doi.org/10.48550/arXiv.1706.03762 arXiv:1706.03762 [cs]. · DOI · Paper page
Cited by
Items 301–400 of 434
Keyframer: A Design Probe for Exploring LLM Assistance in 2D Animation Design
Tseng, T., Cheng, R., McNutt, A. M., & Nichols, J. W. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00014 · Cited by 3
Masi, A. D., Wang, C., & Moccozet, L. (2026) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3829807.3829904
Secure Code Generation and Software Engineering With Generative AI
Le, H. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-8397-1.ch007
Jung, S., & Ahn, H. (2026) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2026.24.8.231
Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability
Borg, M., Hewett, D., Hagatulah, N., Couderc, N., Söderberg, E., Graham, D., Kini, U., & Farley, D. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10889-1 · Cited by 4
Chen, H. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-33997-3_20
Understanding and Enhancing JSON-based DSL Interfaces for VisualizationPreprint
McNutt, A. M. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/fy246
Zhang, M., Jiang, W., & Zhang, Z. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6926957
Ethical Standards in the Use of AI for Computer Science Students
Le, H. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-0439-5.ch011
Toward Agentic Coding: Designing for Habit Formation in AI-Assisted Data Work
Konagai, N., Lynn, M. T., & Snee, E. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799043
Abdelsalam, Y., Peitek, N., Maurer, A.-M., Toneva, M., & Apel, S. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773219 · Cited by 7
State-RAG: A Two-Layer Retrieval System for Large Language Model-Driven Website Builders
Vengaiah, C., Reddy, P. S., Swamy, M. B. P., & Reddy, M. S. T. C. (2026) · International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV) · doi:10.1109/icicv68925.2026.11554694
Lee, Y. J., Hwang, J., Lee, J., Ha, J., Gersich, K., & Lee, K.-S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6864440
The Developer as Curator: A Canvas for Coding Agent Context
Böhme, L., Beckmann, T., Perscheid, M., & Hirschfeld, R. (2026) · ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!) · doi:10.1145/3840586.3843212
WIP: Guiding, Not Giving: Analyzing the Effectiveness of Guided Problem-Solving in CS1
Deoraj, R., Kurivella, S., & Moraes, M. C. (2025) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie63693.2025.11328186
Adapting Socio-Technical Congruence to Human-Agent Collectives
Oppenlaender, J., Visuri, A., & Hosio, S. J. (2026) · ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838753
Large language models in software development learning
Hernández, S. C., Cruz, M. P. T., Sánchez, J. M. E., Luna-Gómez, A. A., Orozco, A. M. P., & Medina, R. H. (2026) · Discover Education · doi:10.1007/s44217-026-01750-0 · Cited by 3
Jain, V., & Jain, A. K. (2026) · Qeios · doi:10.32388/ey1g5q
From Design to Code: Exploring LLM-Supported Workflows in Figma Dev Mode
Rajpal, S., & Riener, A. (2026) · Mensch und Computer (MuC) · doi:10.1145/3820253.3820266
Yüksel, A. S. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108251
Jalote, P. (2025) · Undergraduate topics in computer science · doi:10.1007/978-3-031-74318-4_6
Trust in LLM-Supported Engineering Work: The Role of Downstream DependencyPreprint
Herath, M., & Palmquist, A. (2026) · Research Square · doi:10.21203/rs.3.rs-11038705/v1
Group Conversational Agents: A Review of Designs that Support and Shape Group Interaction
Yeo, S., Zhang, T., Bateman, S., Hsieh, G., Kim, Y., Perrault, S. T., Li, J., & Tang, A. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3812934 · Cited by 1
ABC: Towards a Universal Code Styler through Model Merging
Chen, Y., Gao, Z., Shi, C., Li, B., & Gao, M. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763104 · Cited by 1
Yeşilyayla, H. (2026) · Journal of Business Innovation and Governance · doi:10.54472/jobig.1921341
Fingerprinting AI Coding Agents on GitHub
Ghaleb, T. A. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793582 · Cited by 9
Debugging into Existence with Program Synthesis
Frankel, G., Segal, S., & Peleg, H. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00021
How AI Coding Agents Communicate: A Study of Pull Request Characteristics and Human Review Responses
Watanabe, K., Tsuchida, R., Monno, T., Huang, B., Yamasaki, K., Fan, Y., Shimari, K., & Matsumoto, K. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793593 · Cited by 9
Singh, N., Sovrano, F., Hellendoorn, V. J., & Bacchelli, A. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808165 · Cited by 1
Optimizing LLM Code Suggestions: Feedback-Driven Timing with Lightweight State Bounds
Awad, M. N. A., Ivanov, S., & Tikhonova, O. B. (2025) · IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) · doi:10.1109/asew67777.2025.00049
Problem-Based Learning by Building an Incremental Web Application
Navarro-Lázaro, J., Ortega, G., Garzón, E. M., Orts, F. J., & Puertas, A. M. (2025) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016350
Comparative Analysis of Code Quality Generated by Large Language Models
Alharbi, F., & Albattah, W. (2025) · International Conference on Electrical, Computer and Energy Technologies (ICECET) · doi:10.1109/icecet63943.2025.11472555
Navigating Skill Development and Ethical Challenges in Coding Education with Generative AI
Le, H. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-2660-3.ch009
How Can Copilot Assist in Creating Accessible Websites? An Empirical Study
Teran, L. A., da Rocha, T. A., da Silva, I. M. M., de Salles, R. B., Melo, G. L. N., Mota, M. P., & de Souza, C. R. B. (2025) · Simpósio Brasileiro de Fatores Humanos em Sistemas Computacionais (IHC) · doi:10.5753/ihc.2025.10811
RoboCritics: Enabling Reliable End-to-End LLM Robot Programming through Expert-Informed Critics
Kim, C. Y., White, N. T., He, E. C. L., Sala, F., & Mutlu, B. (2026) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3757279.3785550 · Cited by 6
Li, S., Tang, J., Qiang, Z., & Wang, Z. (2026) · Journal of Intelligence · doi:10.3390/jintelligence14100232
Prause, M. (2026) · Systems · doi:10.3390/systems14081018 · Cited by 1
Zhang, T., Mao, X., Zhao, Y., Yang, K., Zhang, Z., Lu, Y., Chen, Y., & Yu, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3849809
HiLDE: Intentional Code Generation via Human-in-the-Loop Decoding
Gonzalez, E. A., Rothkopf, R., Lerner, S., & Polikarpova, N. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00032 · Cited by 5
Self-citationTowards fully autonomous network management: A survey on LLM-based Multi-Agent Systems
Kim, K.-H., Park, C., Lee, H., Park, C., Kim, T., Choi, M.-J., & Kim, E. (2026) · ICT Express · doi:10.1016/j.icte.2026.07.003 · Cited by 6
Ashif, A. (2026) · IT Professional · doi:10.1109/mitp.2026.3701936
Grasping AI Reliance in Program Comprehension and Coding through the AIRELI Persona Taxonomy
Alakmeh, T., Anderson, N. B., Jackson, V., Pereira, G. V., Akırmak, Ü., Estey, A., Prikladnicki, R., van der Hoek, A., Storey, M., & Fritz, T. (2026) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3794763.3794804 · Cited by 5
Beyond the Prompt: How Novices Engage with AI During Code Summarization
Mohamed, S., Parvin, A., Ismail, N., Amaya, K., Parra, E., & Oliver, M. D. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3786580.3786961
Who Tells the Documentation Team? How AI Agents Disrupt Documentation Feedback Loops
Bhat, A., & Guo, J. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3806708
Lee, Y. J., Hwang, J., Lee, J., Ha, J., Gersich, K., & Lee, K.-S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6329635
Colaboração com Assistente de Codificação Baseado em IA: Benefícios e Desafios
Mendes, W., Souza, S., & de Souza, C. R. B. (2024) · Simpósio Brasileiro de Sistemas Colaborativos (SBSC) · doi:10.5753/sbsc.2024.237964
Challenges Engaging Formal CBSE in Industrial Applications
Li, Y., & Sun, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-52183-6_8
On the Impact of Formal Verification on Software Development
Mugnier, E., Zhou, Y., Jhala, R., & Coblenz, M. J. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763181 · Cited by 4
AI-assisted Arduino Coding in First-Year Engineering Team Projects
Tian, E., & Boback, A. (2026) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--59397
Secure Coding and Generative AI: Building Ethical Foundations for Student Coders
Le, H. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-6063-8.ch011
Do Generative AI Tools Change How Developers Comment and Document Code?
Ebiwonjumi, O., Ahakonye, L. A. C., & Kim, D.-S. (2026) · Digital Signal Processing and Signal Processing Education Workshop · doi:10.1109/icaiic68212.2026.11454326 · Cited by 1
Enhancing Code Completion with Position Embedding of Abstract Syntax Tree
Li, J., Lu, J., & Zhu, L. (2024) · IEEE International Conference on Frontiers Technology of Information and Computer (ICFTIC) · doi:10.1109/icftic64248.2024.10913308
Prompts Engineering Challenges in Software Code Generation
Gutierrez, Y., Camacho, E., Pardo, C., & Villarreal, V. (2025) · IEEE Congreso Internacional en Inteligencia Ambiental, Ingeniería de Software y Salud Electrónica y Móvil (AmITIC) · doi:10.1109/amitic68284.2025.11214606
Превір, М., & Kyrylenko, O. I. (2025) · Problems of Modern Textbook · doi:10.32405/2411-1309-2025-35-94-106 · Cited by 2
Enhancing Code LLMs with Reinforcement Learning in Code Generation: A SurveyPreprint
He, Y., Shi, T., Li, Y., & Chen, Q. (2025) · Qeios · doi:10.32388/8g8tb2
SuperCode: Sustainability PER AI-driven Co-design
Broekema, P. C., & van Nieuwpoort, R. V. (2025) · ACM International Conference on Computing Frontiers: Workshops and Special Sessions (CF Companion) · doi:10.1145/3706594.3727576 · Cited by 2
Empirical Analysis of Generative Ai Adoption in Software DevelopmentPreprint
Shao, D. D., & Ishengoma, F. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5149425
Matejić, T., Mihailović, Đ., & Dukić, A. (2025) · Limes plus. · doi:10.69899/limes-plus25222046m
Wajid, M. A., Camacho-Zúñiga, C., Pirzado, F. A., Terashima‐Marín, H., & Wajid, M. S. (2026) · Human Behavior and Emerging Technologies · doi:10.1155/hbe2/8254212
Contreras, A., Guerra, E., & de Lara, J. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-19050-5_7
Wang, C., Zhang, Z., Xu, X., Zheng, M., Wei, G., & Zhang, X. (2026) · Foundations and Trends® in Programming Languages · doi:10.1108/ftpgl-11-2025-0068
From NL2SQL to NL2Code: Exploring A New Paradigm for Natural Language to Programming Code Generation
Tang, L., Hu, Z., & Liu, Y. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-5373-7_90 · Cited by 1
Conducting Final Programming Exams with Auto-Grading and Code Evaluation Tools
Subirats, J. L., Luque-Polo, G., & Luque‐Baena, R. M. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-75016-8_25 · Cited by 1
The Impact of Artificial Intelligence on Programmer Jobs: Threat or Opportunity?
Hananto, A., Hasibuan, N. S., Olivia, S., Tukino, T., & Novalia, E. (2025) · JUSIFO (JURNAL SISTEM INFORMASI) · doi:10.19109/jusifo.v11i1.25082 · Cited by 3
A Systematic Mapping Study on Developer Profile Modeling in Software Engineering
Sarı, A., Alptekin, G. I., & Tosun, A. (2026) · IEEE Access · doi:10.1109/access.2026.3696198
Coding Kanban: A browser workbench for supervising CLI-based AI coding agents
Xu, Z.-K., & Hu, X. (2026) · SoftwareX · doi:10.1016/j.softx.2026.102919
La revolución de la inteligencia artificial
Lumbreras, S., & Rayón, A. (2023) · Universidad Pontificia Comillas eBooks · doi:10.14422/9788484686057-02 · Cited by 2
Dawson, M. G., Deer, R., & Boguslawski, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5055559
Efecto de la IA sobre las habilidades técnicas de los desarrolladores de software
Jácome, A. J. M. (2026) · Revista Tecnológica - ESPOL · doi:10.37815/rte.v38n1.1417
Inspectable Control for Structure-Preserving Software Regeneration
Gavrilov, A., Gazzaev, A.-B., Mozikov, M., et al. (2026) · SIGSOFT FSE Companion · doi:10.1145/3803437.3807386
Design Guidelines for Human-Generative AI Interaction
Lin, L., Wang, Y., Ping, Y., Gao, J., Wang, Z., & Wang, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93718-7_15
An exploratory study on LLM-generated code and comments in code repositories
Ji, Y., Wang, J., Zhou, Y. X., Chen, F., & Yang, H. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113026 · Cited by 1
Exploring Generative AI An Analysis of Output Consistency and PerformancePreprint
Kumbhare, A., Khedkar, H., & Shirsat, K. P. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5219884
AI in Software Engineering: Overview, Tools, Limitations, Risks, and Strengths
Andrews, S. K. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-2811-7.ch005
Are LLMs Magic for Learning? The Impact of LLM-Support on the Development of Programming SkillsPreprint
YILDIRIM, D., & Keskin, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6314977
de Oliveira, L. A. F., & de Oliveira, C. C. (2026) · Research Society and Development · doi:10.33448/rsd-v15i4.50966 · Cited by 1
The use of Generative AI in technical debt management: A systematic literature review
Gosu, S., Feitosa, D., & Avgeriou, P. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108257
Human-Centered AI for e-Collaboration: Software Development, Design, and Implementation
Le, H. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8217-3.ch003
Clustering and Profiling Student Study Behaviors and Interactions with an AI Coding Assistant
Poitras, E., Durai, J. P. S., Boisvert, J., Doucette, K., Lin, M. P.-C., Kryven, M., & Sampangi, R. V. (2025) · IEEE Smart World Congress (SWC) · doi:10.1109/swc65939.2025.00067 · Cited by 1
Code Understanding Using Sherlock
Deshmukh, M., Ghadage, D. A., Rangankar, M. S., Supugade, P. D., & Isane, D. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-06677-0_15
The Prompt–Refactor–Verify (PRV) Cycle: A Human-Centered Framework for AI-Assisted Programming
Yanev, N., Getova, I. D., Mihaylova, E., Kostadinova, I. S., Bankovska, M., & Dimitrov, G. (2025) · International Conference on Communications, Information, Electronic and Energy Systems (CIEES) · doi:10.1109/ciees66347.2025.11300239 · Cited by 3
Embracing large language models in education: insights and reflections from the Caribbean context
Ildephonce, I., Hunte, A. A., & Rolle-Greenidge, G. (2026) · Caribbean Educational Research Journal · doi:10.1108/cerj-04-2026-0009
CodeWatcher: IDE Telemetry Data Extraction Tool for Understanding Coding Interactions with LLMsPreprint
Rodríguez-Pérez, G., & Basha, M. (2025) · OSF Preprints (OSF Preprints)
AI-Powered Legal Assistant: Enhancing Legal Research with Generative and Retrieval Models
S, S., Jayadharshini, P., M, V., et al. (2025) · 2025 IEEE 5th International Conference on ICT in Business Industry & Government (ICTBIG) · doi:10.1109/ictbig68706.2025.11323672
Patel, D. (2026) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.5157
HKT-SmartAudit: Distilling Lightweight Models for Smart Contract Auditing
Wei, Z., Sun, J., Zhang, Z., Hou, Z., Zhang, X., Li, M., Sun, Y., Wu, D., Liu, Y., Li, C., et al. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3683274 · Cited by 1
Multi-Agent Strategies for Bridging Programming Language Gaps in Code Generation
Moumoula, M. B., Nikiema, S. L., Djiré, A. E., Kaboré, A. K., Klein, J., & Bissyandé, T. F. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3841174 · Cited by 1
Digital nudging in software development: a review and research agenda
Pribik, I., & Felfernig, A. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1898557
Iterative vs. Plan-Driven Approaches in Human-LLM Collaborative Modeling: An Empirical Study
Reinhartz-Berger, I., & Tsoury, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-28117-3_1 · Cited by 2
Yin, W. (2026) · Creative Education · doi:10.4236/ce.2026.171007 · Cited by 2
Mudrinic, M., Radenković, M., Лабус, А., Лабус, А., Radenković, B., Hurlburt, G. F., & Reisman, S. (2026) · Computer · doi:10.1109/mc.2026.3719093
AI-Human Interaction Patterns: An Exploratory Study
Alegría, J. A. H., Ojeda, M. C. C., Álvarez, M. C. G., Villegas, M. L., & Collazos, C. A. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-31223-5_2
Zhao, J.-H., Zhang, Y.-F., Shangguan, S., Lin, Y.-T., & Yang, Q.-F. (2026) · Teaching and Teacher Education · doi:10.1016/j.tate.2026.105581 · Cited by 2
109PwnPilot: could an adversary be pair programming with our most trusted software engineers?
Horne, D. (2025) · De Gruyter eBooks · doi:10.1515/9783111436548-006
Human-AI Co-Creation: A New Interaction Paradigm for Human-AI Interaction
Davis, N., Clemens, M. P., Rezwana, J., & Browne, E. (2026) · Springer eBooks · doi:10.1007/978-981-95-3658-0_76 · Cited by 4
Adavi, K. A. K., Ghosh, P., Banks, R., Sarkar, A., & Lindley, S. E. (2026) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3808045.3808058 · Cited by 1
Scaffolding Autocomplete: Improving Guidance for Learners using Generative Code Suggestions
Prather, J. E., MacNeil, S., Luxton-Reilly, A., Margulieux, L. E., Reeves, B. N., Denny, P. C., Leinonen, J., Homer, J., Nabid, R. A., & Rossetti, R. L. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811654
Pimenov, A., & Kovalchuk, S. V. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-78459-0_25
Metadata
- Authors
- Barke, S., James, M. B., & Polikarpova, N.
- Affiliations
- University of California San Diego (US)
- Venue
- Proceedings of the ACM on Programming Languages
- Published
- 2023-04-06
- Volume
- 7
- Issue
- OOPSLA1
- Pages
- 85-111
- ISSN
- 2475-1421 (ISSN-L)
- OpenAlex
- W4362659486
- Cited by
- 436
- References
- 46
- Citations by year
- 2026: 147 · 2025: 131 · 2024: 119 · 2023: 39
- FWCI
- 102.07
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Proceedings of the ACM on Programming Languages (publishedVersion, cc-by)
- Funders
- National Science Foundation (2107397, 1955457)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Keywords
- GitHub Copilot, acceleration mode, exploration mode, usability, human-AI collaboration, programming tasks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-27 (prompt extract-v2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.