Taking Flight with Copilot: Early insights and opportunities of AI-powered pair-programming tools
Open accessAbstract
Over the next five years, AI-powered tools likely will be helping developers in many diverse tasks. For example, such models may be used to improve code review, directing reviewers to parts of a change where review is most needed or even directly providing feedback on changes. Models such as Codex may suggest fixes for defects in code, build failures, or failing tests. These models are able to write tests automatically, helping to improve code quality and downstream reliability of distributed systems. This study of Copilot shows that developers spend more time reviewing code than actually writing code. As AI-powered tools are integrated into more software development tasks, developer roles will shift so that more time is spent assessing suggestions related to the task than doing the task itself.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The more developers accept CCTs suggestions, the more they perceive themselves as productive.
“As a result, the more developers accept CCTs suggestions, the more they perceive themselves as productive (Bird et al., 2022).”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Bird et al., 2022
In “How Does Copilot Impact Productivity?”, acceptance rate has the highest positive correlation with perceived productivity (ρ = 0.24), and the authors say users perceive themselves as more productive when accepting Copilot suggestions. The citing sentence extends this finding from Copilot users to CCTs generally, which the passage does not establish. (LLM, conf=0.84)
Experimental — accuracy not yet measuredAn AI partner can provide immediate feedback and personalized assistance, playing the role of both driver and navigator as needed.
“For instance, an AI partner can provide immediate feedback and personalized assistance, effectively playing the role of both driver and navigator as needed (Bird et al., 2022).”
From The impact of AI-assisted pair programming on student motivation, programming anxiety, collaborative learning, and programming performance: a comparative study with traditional pair programming and individual approaches (2025) · cites this paper as Bird et al., 2022
In “WHAT ARE PEOPLE USING COPILOT FOR?” (page not specified), the paper says Copilot adapts recommendations to a developer’s current code and coding style, supporting personalized assistance. The Abstract (page not specified) assigns the navigator role to the developer and code-writing to the AI assistant; it does not describe the AI as taking both roles as needed. The Discussion describes direct feedback on changes as a possible future use, not an established capability. (LLM, conf=0.86)
Experimental — accuracy not yet measuredMany researchers in the domain of Artificial Intelligence for Software Engineering (AI4SE) are considering the impacts of AI on different aspects of software development, such as requirements gathering, coding, testing, and wider considerations such as individual productivity.
“Unsurprisingly, then, many researchers in the domain of Artificial Intelligence for Software Engineering (AI4SE) [76] are considering the impacts of AI on different aspects of software development, such as requirements gathering, coding, testing [36, 54], and wider considerations such as individual productivity [14].”
From The Impact of Generative AI on Creativity in Software Development: A Research Agenda (2025) · cites this paper as [14]
Many LLMs are easily available and free for students, and early reports reveal that students are already using them for assistance on their assignments.
“Many of these models are easily available and free for students, and early reports reveal that students are already using them for assistance on their assignments [44].”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [44]
Recent research has shown promising possibilities for providing students with LLM partners in pair programming, given the right context and with the right scaffolding and support.
“Recent research has shown promising possibilities for providing students with LLM partners in pair programming, given the right context and with the right scaffolding and support [44,108,159].”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [44]
GitHub Issues, GitHub Discussions, and Stack Overflow have been frequently used in empirical software engineering studies to collect developer-reported problems.
“Data from these three sources has been frequently used in empirical studies to gather various developer-reported problems in software engineering (Bird et al., 2023) (de Dieu et al., 2023) (El aoun et al., 2021).”
From Exploring the problems, their causes and solutions of AI pair programming: A study on GitHub and Stack Overflow (2025) · cites this paper as Bird et al., 2023
Recent works exploring developer-AI interaction have highlighted the need for mechanisms to facilitate verification of AI-generated code.
“In fact, several recent works exploring developer-AI interaction have highlighted the need for mechanisms to facilitate verification of AI-generated code[14], [9], such as those that allow users to use tests that disambiguate between the different code suggestions [15].”
From LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluation (2024) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660 · see citing works
An empirical evaluation of GitHub copilot's code suggestions
Nguyen, N., & Nadi, S. (2022) · Proceedings of the 19th International Conference on Mining Software Repositories · doi:10.1145/3524842.3528470 · Cited by 313 · see citing works
Productivity Assessment of Neural Code Completion
Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., & Aftandilian, E. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534864 · Cited by 200 · see citing works
Self-citationProductivity Assessment of Neural Code Completion
Ziegler, A., Kalliamvakou, E., Simister, S., Sittampalam, G., Li, A., Rice, A., Rifkin, D., & Aftandilian, E. (2022) · arXiv · doi:10.48550/arxiv.2205.06537 · Cited by 200 · see citing works
Self-citationThe SPACE of Developer Productivity: There's more to it than you think.
Forsgren, N., Storey, M.-A., Maddila, C., Zimmermann, T., Houck, B., & Butler, J. (2021) · Queue · doi:10.1145/3454122.3454124 · Cited by 164 · see citing works
Self-citationIs GitHub copilot a substitute for human pair-programming?: an empirical study
Imai, S. (2022) · Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings · doi:10.1145/3510454.3522684 · Cited by 116 · see citing works
Choose your programming copilot
Sobania, D., Briesch, M., & Rothlauf, F. (2022) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3512290.3528700 · Cited by 91 · see citing works
AI-Driven Development Is Here: Should You Worry?
Ernst, N. A., & Bavota, G. (2022) · IEEE Software · doi:10.1109/ms.2021.3133805 · Cited by 62 · see citing works
Williams, L. A. (2010) · Encyclopedia of Software Engineering · doi:10.1081/e-ese-120044181 · Cited by 23 · see citing works
Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?Preprint
Asare, O., Nagappan, M., & Asokan, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.04741 · Cited by 13 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [3]3. Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., Karri, R. 2022. Asleep at the keyboard? Assessing the security of GitHub Copilot's code contributions. IEEE Symposium on Security and Privacy, 754-768; https://www.computer.org/ csdl/proceedings-article/sp/2022/131600a980/1FlQxERjKCs.
- [14]14. Christian Bird is a senior principal researcher in the Software Analysis and Intelligence (SAINTES) group at Microsoft Research. He is primarily interested in the relationship between software design, social dynamics, and processes in large development projects and in developing tools and techniques to help software teams. His work has focused on code review, branching and merging, developer productivity, and applying AI/ML to software engineering tasks. Christian is an ACM Distinguished Scientist. He received his B.S. from Brigham Young University, and his Ph.D. from the University of California, Davis.
Cited by
Sorted by citations
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650 · see citing works
The Robots Are Here: Navigating the Generative AI Revolution in Computing Education
Prather, J., Denny, P., Leinonen, J., Becker, B. A., Albluwi, I., Craig, M., Keuning, H., Kiesler, N., Kohn, T., Luxton-Reilly, A., et al. (2023) · Proceedings of the 2023 Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3623762.3633499 · Cited by 396 · see citing works
A Survey on Large Language Models for Code Generation
Jiang, J., Wang, F., Shen, J., Kim, S., & Kim, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3747588 · Cited by 251 · see citing works
Lau, S., & Guo, P. (2023) · Proceedings of the 2023 ACM Conference on International Computing Education Research V.1 · doi:10.1145/3568813.3600138 · Cited by 241 · see citing works
Navigating the Complexity of Generative AI Adoption in Software Engineering
Russo, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3652154 · Cited by 217 · see citing works
A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges
Liang, J. T., Yang, C., & Myers, B. A. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3608128 · Cited by 203 · see citing works
Kabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642596 · Cited by 160 · see citing works
LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluation
Fakhoury, S., Naik, A., Sakkas, G., Chakraborty, S., & Lahiri, S. K. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3428972 · Cited by 120 · see citing works
Fan, G., Liu, D., Zhang, R., & Pan, L. (2025) · International Journal of STEM Education · doi:10.1186/s40594-025-00537-3 · Cited by 109 · see citing works
“For instance, an AI partner can provide immediate feedback and personalized assistance, effectively playing the role of both driver and navigator as needed (Bird et al., 2022).”
Investigating and Designing for Trust in AI-powered Code Generation Tools
Wang, R., Cheng, R., Ford, D., & Zimmermann, T. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658984 · Cited by 87 · see citing works
Self-citationIronies of Generative AI: Understanding and Mitigating Productivity Loss in Human-AI Interaction
Simkute, A., Tankelevitch, L., Kewenig, V., Scott, A. E., Sellen, A., & Rintel, S. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2405782 · Cited by 89 · see citing works
Bugs in large language models generated code: an empirical study
Tambon, F., Dakhel, A. M., Nikanjam, A., Khomh, F., Desmarais, M. C., & Antoniol, G. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10614-4 · Cited by 70 · see citing works
How Far Are We? The Triumphs and Trials of Generative AI in Learning Software Engineering
Choudhuri, R., Liu, D., Steinmacher, I., Gerosa, M. A., & Sarma, A. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639201 · Cited by 68 · see citing works
How Beginning Programmers and Code LLMs (Mis)read Each Other
Nguyen, S., Babe, H. M., Zi, Y., Guha, A., Anderson, C. J., & Feldman, M. Q. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642706 · Cited by 81 · see citing works
Ulfsnes, R., Moe, N. B., Stray, V., & Skarpen, M. (2024) · Generative AI for Effective Software Development · doi:10.1007/978-3-031-55642-5_10 · Cited by 62 · see citing works
Significant Productivity Gains through Programming with Large Language Models
Weber, T., Brandmaier, M., Schmidt, A., & Mayer, S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3661145 · Cited by 64 · see citing works
Generative Artificial Intelligence for Software Engineering—A Research Agenda
Nguyen‐Duc, A., Cabrero‐Daniel, B., Przybylek, A., Arora, C., Khanna, D., Herda, T., Rafiq, U., Melegati, J., Guerra, E., Kemell, K., et al. (2025) · Software: Practice and Experience · doi:10.1002/spe.70005 · Cited by 60 · see citing works
Copiloting the future: How generative AI transforms Software Engineering
Banh, L., Holldack, F., & Strobel, G. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107751 · Cited by 66 · see citing works
The Good and Bad of AI Tools in Novice Programming Education
Zviel-Girshin, R. (2024) · Education Sciences · doi:10.3390/educsci14101089 · Cited by 55 · see citing works
Automatic Programming: Large Language Models and Beyond
Lyu, M. R.-T., Ray, B., Roychoudhury, A., Tan, S. H., & Thongtanunam, P. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708519 · Cited by 65 · see citing works
The Next Frontier in Software Development: AI-Augmented Software Development Processes
Özkaya, İ. (2023) · IEEE Software · doi:10.1109/ms.2023.3278056 · Cited by 52 · see citing works
Cheng, R., Wang, R., Zimmermann, T., & Ford, D. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3651990 · Cited by 47 · see citing works
Self-citationIvie: Lightweight Anchored Explanations of Just-Generated Code
Yan, L., Hwang, A., Wu, Z., & Head, A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642239 · Cited by 40 · see citing works
Liu, J., & Li, S. (2024) · Journal of Educational Computing Research · doi:10.1177/07356331241240460 · Cited by 40 · see citing works
Kim, Y. W., Cha, M. C., Yoon, S. H., & Lee, S. C. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2375701 · Cited by 38 · see citing works
A comparison of human, GPT-3.5, and GPT-4 performance in a university-level coding course
Yeadon, W., Peach, A., & Testrow, C. P. (2024) · Scientific Reports · doi:10.1038/s41598-024-73634-y · Cited by 37 · see citing works
Navigating software development in the ChatGPT and GitHub copilot era
France, S. L. (2024) · Business Horizons · doi:10.1016/j.bushor.2024.05.009 · Cited by 38 · see citing works
Zhou, X., Liang, P., Zhang, B., Li, Z., Ahmad, A., Shahin, M., & Waseem, M. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112204 · Cited by 30 · see citing works
What drives the corporate payoffs of using generative artificial intelligence?
Bughin, J. (2024) · Structural Change and Economic Dynamics · doi:10.1016/j.strueco.2024.09.011 · Cited by 31 · see citing works
An Industry Case Study on Adoption of AI-based Programming Assistants
Davila, N., Wiese, I., Steinmacher, I., da Silva, L. L., Kawamoto, A. L. S., Favaro, G. J. P., & Nunes, I. (2024) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3639477.3643648 · Cited by 30 · see citing works
How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineering
Treude, C., & Gerosa, M. A. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00033 · Cited by 28 · see citing works
Validating AI-Generated Code with Live Programming
Ferdowsi, K., Huang, R. L., James, M. B., Polikarpova, N., & Lerner, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642495 · Cited by 27 · see citing works
Demystifying practices, challenges and expected features of using GitHub copilot
Zhang, B., Liang, P., Zhou, X., Ahmad, A., & Waseem, M. (2023) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194023410048 · Cited by 26 · see citing works
Can ChatGPT Support Developers? An Empirical Evaluation of Large Language Models for Code Generation
Jin, K., Wang, C.-Y., Pham, H. V., & Hemmati, H. (2024) · Proceedings of the 21st International Conference on Mining Software Repositories · doi:10.1145/3643991.3645074 · Cited by 24 · see citing works
The Impact of Generative AI on Creativity in Software Development: A Research Agenda
Jackson, V., Vasilescu, B., Russo, D., Ralph, P., Prikladnicki, R., Izadi, M., D’Angelo, S., Inman, S., Andrade, A., & van der Hoek, A. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708523 · Cited by 23 · see citing works
Non-Expert Programmers in the Generative AI Future
Feldman, M. Q., & Anderson, C. J. (2024) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3663384.3663393 · Cited by 21 · see citing works
Generative AI: Current Status and Future Directions
Leong, L.-Y., Hew, T., Ooi, K., Tan, G., & Koohang, A. (2025) · Journal of Computational Information Systems · doi:10.1080/08874417.2025.2482571 · Cited by 18 · see citing works
Kemell, K., Saarikallio, M., Nguyen‐Duc, A., & Abrahamsson, P. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107805 · Cited by 20 · see citing works
Pinto, G., De Souza, C., Rocha, T., Steinmacher, I., Souza, A., & Monteiro, E. (2024) · Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI · doi:10.1145/3644815.3644949 · Cited by 17 · see citing works
Polaris: A Safety-focused LLM Constellation Architecture for Healthcare
Mukherjee, S., Gamble, P., Ausin, M. S., Kant, N., Aggarwal, K., Manjunath, N., Datta, D., Liu, Z., Ding, J., Busacca, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.13313 · Cited by 17 · see citing works
Who’s Helping Who? When Students Use ChatGPT to Engage in Practice Lab Sessions
Brender, J., El‐Hamamsy, L., Francesco Mondada, C., & Bumbacher, E. W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-64302-6_17 · Cited by 16 · see citing works
Generative AI for Test Driven Development: Preliminary Results
Mock, M., Melegati, J., & Russo, B. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-72781-8_3 · Cited by 16 · see citing works
Insights from the Frontline: GenAI Utilization Among Software Engineering Students
Choudhuri, R., Ramakrishnan, A., Chatterjee, A., Trinkenreich, B., Steinmacher, I., Gerosa, M. A., & Sarma, A. (2025) · IEEE International Conference on Software Engineering Education and Training (CSEE&T) · doi:10.1109/cseet66350.2025.00007 · Cited by 20 · see citing works
Ahlgren, T. L., Sunde, H. F., Kemell, K., & Nguyen‐Duc, A. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107832 · Cited by 15 · see citing works
Human-AI experience in integrated development environments: a systematic literature review
Sergeyuk, A., Zakharov, I., Koshchenko, E., & Izadi, M. (2026) · Empirical Software Engineering · doi:10.1007/s10664-025-10793-0 · Cited by 26 · see citing works
Generative AI for Software Development: A Family of Studies on Code Generation
Dakhel, A. M., Nikanjam, A., Khomh, F., Desmarais, M. C., & Washizaki, H. (2024) · Springer eBooks · doi:10.1007/978-3-031-55642-5_7 · Cited by 16 · see citing works
"You're on a bicycle with a little motor": Benefits and Challenges of Using AI Code Assistants
Mendes, W., Souza, S., & de Souza, C. R. B. (2024) · International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1145/3641822.3641882 · Cited by 15 · see citing works
BISCUIT: Scaffolding LLM-Generated Code with Ephemeral UIs in Computational Notebooks
Cheng, R., Barik, T., Leung, A., Hohman, F., & Nichols, J. W. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00012 · Cited by 15 · see citing works
How Much Does AI Impact Development Speed? an Enterprise-Based Randomized Controlled Trial
Paradis, E., Grey, K., Madison, Q., Nam, D., Macvean, A., Meimand, V., Zhang, N., Ferrari-Church, B., & Chandra, S. (2025) · 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00060 · Cited by 20 · see citing works
Impacts of the Usage of Generative Artificial Intelligence on Software Development Process
Santos, P. D. O., Figueiredo, A. C., Moura, P., Diirr, B., Alvim, A. C. F., & dos Santos, R. P. (2024) · Anais do Simpósio Brasileiro de Sistemas de Informação (SBSI) · doi:10.1145/3658271.3658337 · Cited by 14 · see citing works
Multi-line AI-Assisted Code Authoring
Dunay, O., Cheng, D., Tait, A., Thakkar, P., Rigby, P. C., Chiu, A., Ahmad, I., Ganesan, A., Maddila, C., Murali, V., et al. (2024) · SIGSOFT FSE Companion · doi:10.1145/3663529.3663836 · Cited by 29 · see citing works
Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?
Kou, B., Chen, S., Wang, Z., Ma, L., & Zhang, T. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660807 · Cited by 24 · see citing works
Rethinking AI code generation: a one-shot correction approach based on user feedback
Le, K. T., & Andrzejak, A. (2024) · Automated Software Engineering · doi:10.1007/s10515-024-00451-y · Cited by 15 · see citing works
Meta-Manager: A Tool for Collecting and Exploring Meta Information about Code
Horvath, A., Macvean, A., & Myers, B. A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642676 · Cited by 14 · see citing works
Garg, A., Gupta, S., Vats, S., Handa, P., & Goel, N. (2024) · Expert Systems · doi:10.1111/exsy.13697 · Cited by 13 · see citing works
SQLucid: Grounding Natural Language Database Queries with Interactive Explanations
Tian, Y., Kummerfeld, J. K., Li, T. J.-J., & Zhang, T. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676368 · Cited by 13 · see citing works
Chatting with AI: Deciphering Developer Conversations with ChatGPT
Mohamed, S., Parvin, A., & Parra, E. (2024) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3643991.3645078 · Cited by 12 · see citing works
From Triumph to Uncertainty: The Journey of Software Engineering in the AI Era
Mastropaolo, A., Escobar‐Velásquez, C., & Linares‐Vásquez, M. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3709360 · Cited by 19 · see citing works
AI-Augmented Software Development: Boosting Efficiency and Quality
Pangavhane, S., Raktate, G., Pariane, P., Shelar, K., Wakchaure, R., & Kale, J. (2024) · International Conference on Decision Aid Sciences and Applications (DASA) · doi:10.1109/dasa63652.2024.10836523 · Cited by 11 · see citing works
Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agency
Ajmani, L., Abdelkadir, N. A., & Chancellor, S. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732071 · Cited by 11 · see citing works
Chen, T. (2024) · International Conference on Generative Artificial Intelligence and Information Security (GAIIS) · doi:10.1145/3665348.3665383 · Cited by 10 · see citing works
Pankiv, K., & Kloetzer, L. (2024) · Citizen Science Theory and Practice · doi:10.5334/cstp.733 · Cited by 10 · see citing works
LLM-assisted medical documentation: efficacy, errors, and ethical considerations in ophthalmology
Satheakeerthy, S., Jesudason, D., Pietris, J., Bacchi, S., & Chan, W. O. (2025) · Eye · doi:10.1038/s41433-025-03767-5 · Cited by 10 · see citing works
Kohn, T. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70043 · Cited by 13 · see citing works
Al Haque, E., Brown, C., LaToza, T. D., & Johnson, B. (2025) · Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering · doi:10.1145/3696630.3731665 · Cited by 9 · see citing works
Large Language Models (LLMs) in Programming Learning: The Current Research State and Agenda
Fu, Q., Zhao, Y., Jia, Z., & Zheng, Y. (2025) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2025.3622043 · Cited by 10 · see citing works
Li, L., & Yan, B. (2026) · Academy of Management Review · doi:10.5465/amr.2024.0546 · Cited by 14 · see citing works
Augmenting Virtual Labs with Artificial Intelligence for Hybrid Learning
Murali, R., Ravi, N., & Surendran, A. (2024) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon60312.2024.10578649 · Cited by 8 · see citing works
tl;dr: Chill, y’all: AI Will Not Devour SE
Kang, E., & Shaw, M. (2024) · Proceedings of the 2024 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software · doi:10.1145/3689492.3689816 · Cited by 8 · see citing works
Human-Machine Teaming and Team Effectiveness in AI Tools for Software Engineering
Rauf, I., Sharp, H., López, T., & Michel Wermelinger, B. (2025) · International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1109/chase66643.2025.00017 · Cited by 8 · see citing works
Revolutionizing Newcomers’ Onboarding Process in OSS Communities: The Future AI Mentor
Tan, X. L., Long, X., Zhu, Y., Shi, L., Lian, X., & Zhang, L. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715767 · Cited by 8 · see citing works
Steenhoek, B. J., Sivaraman, K., Gonzalez, R. S., Mohylevskyy, Y., Moghaddam, R. Z., & Le, W. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00126 · Cited by 8 · see citing works
Industrial Experience Report on AI-Assisted Coding in Professional Software Development
Ramler, R., Moser, M., Fischer, L., Nissl, M., & Heinzl, R. (2024) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1145/3643795.3648377 · Cited by 8 · see citing works
PAGURI: A User Experience Study of Creative Interaction with Text-to-Music Models
Ronchini, F., Comanducci, L., Perego, G., & Antonacci, F. (2025) · Electronics · doi:10.3390/electronics14173379 · Cited by 8 · see citing works
Assessing the Use of GitHub Copilot on Students of Engineering of Information Systems
Cirett-Galán, F., Peralta, R. T., Navarro-Hernández, R., Ochoa, J. L., Contreras-Rivera, S., Estrada-Ríos, L. A., & Machado-Encinas, G. (2024) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194024500335 · Cited by 7 · see citing works
Unveiling the Potential of Large Language Models in Generating Semantic and Cross-Language Clones
Roy, P. R., Alam, A. I., Al-omari, F., Roy, B., Roy, C. K., & Schneider, K. A. (2023) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc60764.2023.00011 · Cited by 7 · see citing works
Gustrowsky, B., Villarreal, J. L., & Alférez, G. H. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-66336-9_19 · Cited by 7 · see citing works
AI Over-Hype: A Dangerous Threat (and How to Fix It)
Johnson, B., & Menzies, T. (2024) · IEEE Software · doi:10.1109/ms.2024.3439138 · Cited by 6 · see citing works
The role of firm AI capabilities in generative AI-pair coding
Bughin, J. (2024) · Journal of Decision Systems · doi:10.1080/12460125.2024.2428187 · Cited by 6 · see citing works
Trust Dynamics in AI-Assisted Development: Definitions, Factors, and Implications
Sabouri, S., Eibl, P., Zhou, X.-Y., Ziyadi, M., Medvidović, N., Lindemann, L., & Chattopadhyay, S. (2025) · International Conference on Software Engineering · doi:10.1109/icse55347.2025.00199 · Cited by 17 · see citing works
Lau, S., & Guo, P. J. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00041 · Cited by 6 · see citing works
Creating artificial societies for policy decision support: a research agenda and call to action
Clemen, T., Tolk, A., Clemen, U. A., Glake, D., & Günther, G. (2024) · SIMULATION · doi:10.1177/00375497241295765 · Cited by 5 · see citing works
Hicks, C. M., Lee, C. S., & Foster-Marks, K. L. (2024) · doi:10.31234/osf.io/2gej5 · Cited by 6 · see citing works
Understanding and effectively mitigating code review anxiety
Lee, C. S., & Hicks, C. M. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10550-9 · Cited by 5 · see citing works
Time Warp: The Gap Between Developers' Ideal vs Actual Workweeks in an AI-Driven Era
Kumar, S., Goel, D., Zimmermann, T., Houck, B., Ashok, B., & Bansal, C. (2025) · 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00007 · Cited by 8 · see citing works
Self-citationAnnuš, N. (2025) · Education Sciences · doi:10.3390/educsci15091152 · Cited by 5 · see citing works
Meta-X design: from tools to partners in human–machine co–creation for industry 5.0
Yu, C., Yan, T., & Chen, Y. (2025) · Journal of Engineering Design · doi:10.1080/09544828.2025.2571005 · Cited by 4 · see citing works
SQUIRE: Interactive UI Authoring via Slot QUery Intermediate REpresentations
Leung, A., Cheng, R., Wu, J., Nichols, J., & Barik, T. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747672 · Cited by 4 · see citing works
Gambo, O. O., Onifade, O. J., Olanrewaju-Smart, W., Abuya, O. J., Salawu, R. E., Massenon, R., & Gambo, I. (2026) · Discover Education · doi:10.1007/s44217-026-01142-4 · Cited by 7 · see citing works
Analyst’s Perception on the Use of AI-based Tools in the Software Development Life Cycle
Giffari, R., Ridho, M. M., Sensuse, D. I., Hidayat, D. S., & Purwaningsih, E. H. (2024) · Jurnal Sistem Informasi · doi:10.21609/jsi.v20i1.1399 · Cited by 3 · see citing works
A fine-grained taxonomy of code review feedback in TypeScript projects
Davila, N., Nunes, I., & Wiese, I. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10604-y · Cited by 4 · see citing works
Leveraging AI for Productive and Trustworthy HPC Software: Challenges and Research Directions
Teranishi, K., Menon, H., Godoy, W. F., Balaprakash, P., Bau, D., Ben‐Nun, T., Bhatele, A. S., Franchetti, F., Franusich, M., Gamblin, T., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-07612-0_47 · Cited by 4 · see citing works
Early Results from a Study of GenAI Adoption in a Large Brazilian Company: The Case of Globo
Pereira, G. V., Prikladnicki, R., Jackson, V., van der Hoek, A., Fortes, L., & Macaubas, I. (2024) · Springer eBooks · doi:10.1007/978-3-031-55642-5_13 · Cited by 5 · see citing works
Exploring the Impact of GitHub Copilot on Health Informatics Education
Avramovic, S., Avramovic, I., & Wojtusiak, J. (2024) · Applied Clinical Informatics · doi:10.1055/a-2414-7790 · Cited by 4 · see citing works
He, H., & Li, X. (2025) · International Conference on Distance Education and Learning (ICDEL) · doi:10.1109/icdel65868.2025.11193454 · Cited by 3 · see citing works
Martin-Lopez, A., Tufano, R., Guglielmi, E., Sánchez, A. B., Sanz, A. L., Scalabrino, S., Oliveto, R., Segura, S., & Bavota, G. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3679627 · Cited by 4 · see citing works
Chen, N., Qiu, L. K., Wang, A. Z., Wang, Z., & Yang, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790726 · Cited by 8 · see citing works
Building Software by Rolling the Dice: A Qualitative Study of Vibe Coding
Chou, Y.-H., Jiang, B., Chen, Y. W., Weng, M., Jackson, V., Zimmermann, T., & Jones, J. A. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797105 · Cited by 3 · see citing works
Self-citationEnhancing Real-Time Object Detection With Advanced YOLOv9 and OpenCV in Python
Parjane, P., Raktate, G., Shelar, K., Wakchuare, R., More, S., & Kale, J. (2024) · International Conference on Decision Aid Sciences and Applications (DASA) · doi:10.1109/dasa63652.2024.10836635 · Cited by 3 · see citing works
- Authors
- Bird, C., Ford, D., Zimmermann, T., Forsgren, N., Kalliamvakou, E., Lowdermilk, T., & Gazit, I.
- Affiliations
- Microsoft (United States) (US)
- GitHub Next
- ORCID
- Venue
- Queue
- Published
- 2022-12-31
- Volume
- 20
- Issue
- 6
- Pages
- 35-57
- ISSN
- 1542-7730 (ISSN-L)
- 1542-7749
- OpenAlex
- W4318159335
- Semantic Scholar
- 5e8bc5f84f3a550319b0d2b54cc0062b410d2328
- Cited by
- 158
- References
- 6
- Citations by year
- 2026: 35 · 2025: 48 · 2024: 64 · 2023: 11
- FWCI
- 30.45
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Queue (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Keywords
- GitHub Copilot, AI-assisted programming, code review, developer roles, code generation, developer productivity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
