A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges
Open accessAbstract
The software engineering community recently has witnessed widespread deployment of AI programming assistants, such as GitHub Copilot. However, in practice, developers do not accept AI programming assistants' initial suggestions at a high frequency. This leaves a number of open questions related to the usability of these tools. To understand developers' practices while using these tools and the important usability challenges they face, we administered a survey to a large population of developers and received responses from a diverse set of 410 developers. Through a mix of qualitative and quantitative analyses, we found that developers are most motivated to use AI programming assistants because they help developers reduce key-strokes, finish programming tasks quickly, and recall syntax, but resonate less with using them to help brainstorm potential solutions. We also found the most important reasons why developers do not use these tools are because these tools do not output code that addresses certain functional or non-functional requirements and because developers have trouble controlling the tool to generate the desired output. Our findings have implications for both creators and users of AI programming assistants, such as designing minimal cognitive effort interactions with these tools to reduce distractions for users while they are programming.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Only 40% of CCT suggestions are accepted the first time they are proposed to developers, mainly because developers cannot control context or granularity.
“Some recent studies confirm these symptoms: only 40% of CCT suggestions are accepted the first time they are proposed to developers, mainly because developers cannot control context or granularity (Liang et al., 2024).”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Liang et al., 2024
The Introduction (page unavailable) says developers do not accept initial suggestions at a high frequency, but gives acceptance rates of 23.3%, 27.9%, and 28.8% for GitHub Copilot suggestions—not 40%. The paper also reports difficulties controlling output and requests for better code context and adjustable suggestion length, but it does not establish that inability to control context or granularity is the main reason for the claimed first-time acceptance rate. (LLM, conf=0.87)
Experimental — accuracy not yet measuredLiang et al. (2024) found that GitHub Copilot was the most widely used tool (74.6%), and its users reported a median of 30.5% of their code was generated by the AI assistant.
“GitHub Copilot was the most widely used tool (74.6%), whose users reported that a median of 30.5% of their code was now generated by the AI assistant.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Liang et al. (2024)
Section 4.1 reports that GitHub Copilot had the most users (306 of 410 survey respondents, or 74.6%) and that its users reported a median of 30.5% of their code was written with its help. It does not say that all of that code was generated by the assistant; “written with its help” is a broader measure. (LLM, conf=0.86)
Experimental — accuracy not yet measuredLiang et al. (2024) and Butler et al. (2025) found that developers generally feel productivity gains using the same assistant.
“Similarly, Liang et al. ( 2024) and Butler et al. (2025) found that developers generally feel productivity gains using the same assistant.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Liang et al. (2024)
Section 4.2 (page unavailable) reports that 76% of participants who used AI programming assistants at least monthly cited finishing tasks faster as an important reason for using them. This supports a general perception of productivity benefits, but the survey covered multiple assistants and does not establish that developers felt those gains using the same specific assistant. (LLM, conf=0.84)
Experimental — accuracy not yet measuredFrustration occurs when suggestions do not satisfy functional or non-functional requirements.
“Liang et al. [18] found that frustration occurs when the suggestions do not satisfy functional or non-functional requirements.”
From How do Copilot Suggestions Impact Developers' Frustration and Productivity? (2025) · cites this paper as Liang et al. [18]
Section 5.5, “Giving up on outputted code” (page not available in our copy), reports that participants gave up on generated code when it failed to meet functional or non-functional requirements. It does not establish that they experienced frustration, which is the response the citing sentence attributes to the study. (LLM, conf=0.84)
Experimental — accuracy not yet measuredAcceleration mode is usually preferred, as developers often use the tools to complete repetitive code that cannot be copy-pasted and for recalling syntax they don't remember.
“Acceleration mode is usually preferred, as developers often use the tools to complete repetitive code that cannot be copy-pasted (Liang et al., 2024), and for recalling syntax they don't remember (Xu et al., 2022).”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Liang et al., 2024
The literature highlights the need for explanations (such as reporting the source or linking to documentation) for additional context on the generated code.
“The literature also highlights the need for explanations (such as reporting the source or linking to documentation) for additional context on the generated code (Liang et al., 2024; Xu et al., 2022).”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Liang et al., 2024
Developers often do not use CCTs because these tools do not generate code that addresses certain functional or non-functional requirements, and because developers have trouble controlling the tool to generate the desired output.
“They also found that developers often do not use CCTs because these tools do not generate code that addresses certain functional or non-functional requirements, and because developers have trouble controlling the tool to generate the desired output.”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Liang et al., 2024
Developers are most motivated to use AI programming assistants because they help reduce keystrokes, finish programming tasks quickly, and recall syntax.
“The survey revealed that developers are most motivated to use AI programming assistants because they help developers reduce keystrokes, finish programming tasks quickly, and recall syntax.”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Liang et al., 2024
Liang et al. (2024) found that most common reasons for not using AI tools were subpar code quality and difficulty controlling AI assistants.
“On the other hand, the most common reasons for not using such tools were 1) that the code output often is of subpar quality, and 2) the difficulty of controlling the AI assistants to produce the desired output.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Liang et al. (2024)
Liang et al. (2024) found that primary motivations for adopting AI assistants were reducing keystrokes, completing tasks faster, and skipping web searches for code examples.
“The primary motivations for adopting AI assistants were 1) reducing keystrokes, 2) completing tasks faster, and 3) skipping web searches for code examples.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Liang et al. (2024)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Five misunderstandings about case-study research
Flyvbjerg, B. (2006) · Qualitative Inquiry · doi:10.1177/1077800405284363 · Cited by 14,266 · see citing works
The Coding Manual for Qualitative Researchers
Lungu, M. (2022) · American Journal of Qualitative Research · doi:10.29333/ajqr/12085 · Cited by 10,887 · see citing works
McDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206 · see citing works
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084 · see citing works
Herbsleb, J. D., & Moitra, D. (2001) · IEEE Software · doi:10.1109/52.914732 · Cited by 1,059 · see citing works
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660 · see citing works
Design lessons from the fastest q&a site in the west
Mamykina, L., Manoim, B., Mittal, M., Hripcsak, G. M., & Hartmann, B. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979366 · Cited by 554 · see citing works
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., & Hellendoorn, V. J. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534862 · Cited by 540 · see citing works
Kitchenham, B. A., & Pfleeger, S. L. (2007) · Springer eBooks · doi:10.1007/978-1-84800-044-5_3 · Cited by 515 · see citing works
GitHub Copilot AI pair programmer: Asset or Liability?
Moradi Dakhel, A., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M. C., & Jiang, Z. M. (. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111734 · Cited by 417 · see citing works
Work practices and challenges in pull-based development
Gousios, G., Storey, M., & Bacchelli, A. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884826 · Cited by 333 · see citing works
Work Practices and Challenges in Pull-Based Development: The Integrator's Perspective
Gousios, G., Zaidman, A., Storey, M., & Arie Van Deursen, A. (2015) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2015.55 · Cited by 320 · see citing works
Denny, P., Kumar, V., & Giacaman, N. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569823 · Cited by 287 · see citing works
(2015) · doi:10.1007/s10664-013-9279-3 · Cited by 273 · see citing works
Ross, S., Martinez, F., Houde, S., Muller, M. J., & Weisz, J. D. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584037 · Cited by 272 · see citing works
Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257 · see citing works
Hammer, D., & Berland, L. K. (2013) · Journal of the Learning Sciences · doi:10.1080/10508406.2013.802652 · Cited by 214 · see citing works
Productivity Assessment of Neural Code Completion
Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., & Aftandilian, E. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534864 · Cited by 200 · see citing works
Productivity Assessment of Neural Code Completion
Ziegler, A., Kalliamvakou, E., Simister, S., Sittampalam, G., Li, A., Rice, A., Rifkin, D., & Aftandilian, E. (2022) · arXiv · doi:10.48550/arxiv.2205.06537 · Cited by 200 · see citing works
The (R) Evolution of social media in software engineering
Storey, M., Singer, L., Cleary, B., Filho, F. F., & Zagalsky, A. (2014) · Future of Software Engineering (FOSE) · doi:10.1145/2593882.2593887 · Cited by 198 · see citing works
The SPACE of Developer Productivity: There's more to it than you think.
Forsgren, N., Storey, M.-A., Maddila, C., Zimmermann, T., Houck, B., & Butler, J. (2021) · Queue · doi:10.1145/3454122.3454124 · Cited by 164 · see citing works
What Makes a Great Software Engineer?
Li, P., Ko, A. J., & Zhu, J.-M. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.335 · Cited by 161 · see citing works
Taking Flight with Copilot: Early insights and opportunities of AI-powered pair-programming tools
Bird, C., Ford, D., Zimmermann, T., Forsgren, N., Kalliamvakou, E., Lowdermilk, T., & Gazit, I. (2022) · Queue · doi:10.1145/3582083 · Cited by 158 · see citing works
In-IDE Code Generation from Natural Language: Promise and Challenges
Xu, F. F., Vasilescu, B., & Neubig, G. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3487569 · Cited by 154 · cited 6× · see citing works
“Other studies report that software developers face barriers in comprehending and debugging generated code to fit their use cases, because they need to have prior knowledge of the underlying programming principles, frameworks, or APIs [12, 60].”
BackgroundResultInfluentialAssessing the quality of GitHub copilot’s code generation
Yetiştiren, B., Ozsoy, I., & Tüzün, E. (2022) · International Conference on Predictive Models in Software Engineering · doi:10.1145/3558489.3559072 · Cited by 148 · see citing works
Programmers Are Users Too: Human-Centered Methods for Improving Programming Tools
Myers, B. A., Ko, A. J., LaToza, T. D., & Yoon, Y. (2016) · Computer · doi:10.1109/mc.2016.200 · Cited by 144 · see citing works
Self-citationImproving developer participation rates in surveys
Smith, E., Loftin, R., Murphy-Hill, E. R., Bird, C., & Zimmermann, T. (2013) · International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1109/chase.2013.6614738 · Cited by 133 · see citing works
Is GitHub copilot a substitute for human pair-programming?: an empirical study
Imai, S. (2022) · Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings · doi:10.1145/3510454.3522684 · Cited by 116 · see citing works
Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming
Mozannar, H., Bansal, G., Fourney, A., & Horvitz, E. (2022) · arXiv · doi:10.48550/arxiv.2210.14306 · Cited by 114 · cited 3× · see citing works
“The most popular strategy participants reported was providing very clear and explicit explanations of what the code should do in comments, which is a major activity while using AI programming assistants [44].”
BackgroundMethodTowards a theory of software development expertise
Baltes, S., & Diehl, S. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3236061 · Cited by 102 · see citing works
Chasins, S., Mueller, M., & Bodík, R. (2018) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3242587.3242661 · Cited by 86 · see citing works
On the Design of AI-powered Code Assistants for Notebooks
McNutt, A. M., Wang, C., DeLine, R. A., & Drucker, S. M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580940 · Cited by 81 · see citing works
Jiang, E., Toh, E., Molina, A., Olson, K., Kayacik, C., Donsbach, A., Cai, C. J., & Terry, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501870 · Cited by 79 · see citing works
Decision making in software architecture
van Vliet, H., & Tang, A. (2016) · Journal of Systems and Software · doi:10.1016/j.jss.2016.01.017 · Cited by 79 · see citing works
Code, Camera, Action: How Software Developers Document and Share Program Knowledge Using YouTube
MacLeod, L., Storey, M., & Bergen, A. (2015) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2015.19 · Cited by 75 · see citing works
What distinguishes great software engineers
Li, P. L., Ko, A. J., & Begel, A. (2020) · Empirical Software Engineering · doi:10.1007/s10664-019-09773-y · Cited by 48 · see citing works
HCI Guidelines for Gender Equity and Inclusivity
Scheuerman, M. K., Spiel, K., Haimson, O. L., Hamidi, F., & Branham, S. M. (2020) · Maryland Shared Open Access Repository (USMAI Consortium) · doi:10.13016/m2nw1f-p0jx · Cited by 48 · see citing works
Small-Step Live Programming by Example
Ferdowsi, K., Ordookhanians, A., Peleg, H., Lerner, S., & Polikarpova, N. (2020) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3379337.3415869 · Cited by 42 · see citing works
Exploring the Learnability of Program Synthesizers by Novice Programmers
Jayagopal, D., Lubin, J., & Chasins, S. E. (2022) · Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3526113.3545659 · Cited by 40 · see citing works
Leaving My Fingerprints: Motivations and Challenges of Contributing to OSS for Social Good
Huang, Y., Ford, D., & Zimmermann, T. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00096 · Cited by 38 · see citing works
Characterizing Software Engineering Work with Personas Based on Knowledge Worker Actions
Ford, D., Zimmermann, T., Bird, C., & Nagappan, N. (2017) · 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) · doi:10.1109/esem.2017.54 · Cited by 37 · see citing works
Understanding skills for OSS communities on GitHub
Liang, J. T., Zimmermann, T., & Ford, D. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549082 · Cited by 32 · see citing works
Self-citationVaithilingam, P., Glassman, E. L., Groenwegen, P., Gulwani, S., Henley, A. Z., Malpani, R., Pugh, D., Radhakrishna, A., Soares, G., Wang, J., et al. (2023) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip58684.2023.00022 · Cited by 25 · see citing works
Grounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2022) · arXiv · doi:10.48550/arxiv.2206.15000 · Cited by 23 · cited 11× · see citing works
“This could assist the model in adapting to whether the developer was in acceleration mode–associated with short completions– or exploration mode–associated with long completions [12].”
BackgroundMethodResultInfluentialAnalyzing Web Search Behavior for Software Engineering Tasks
Rao, N., Bansal, C., Zimmermann, T., Awadallah, A. H., & Nagappan, N. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378083 · Cited by 19 · see citing works
Robe, P., Kuttal, S. K., AuBuchon, J., & Hart, J. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549127 · Cited by 19 · see citing works
A Qualitative Study on the Implementation Design Decisions of Developers
Liang, J. T., Arab, M., Ko, M., Ko, A. J., & LaToza, T. D. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00047 · Cited by 12 · see citing works
Self-citationLiang, J. T., Myers, B., & Yang, C. (2023) · Figshare · doi:10.6084/m9.figshare.22355017 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 5
- Ruijia Cheng, Ruotong Wang, Thomas Zimmermann, and Denae Ford. 2022. "It would work for me too": How online communities shape software developers' trust in AI-powered code generation tools. arXiv preprint arXiv:2212.03491 (2022). · arXiv
- Victor Dibia, Adam Fourney, Gagan Bansal, Forough Poursabzi-Sangdeh, Han Liu, and Saleema Amershi. 2022. Aligning offline metrics and human judgments of value of AI-pair programmers. arXiv preprint arXiv:2210.16494 (2022). · arXiv
- Ben Puryear and Gina Sprint. 2022. GitHub Copilot in the classroom: Learning to code with AI assistance. Journal of Computing Sciences in Colleges 38, 1 (2022), 37-47.
- Mohammad Tahaei and Kami Vaniea. 2022. Lessons Learned From Recruiting Participants With Programming Skills for Empirical Privacy and Security Stud- ies. In International Workshop on Recruiting Participants for Empirical Software Engineering (RoPES).
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
Cited by
Sorted by citations
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650 · see citing works
The Metacognitive Demands and Opportunities of Generative AI
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642902 · Cited by 280 · see citing works
Large Language Model for Vulnerability Detection and Repair: Literature Review and the Road Ahead
Zhou, X., Cao, S., Sun, X., & Lo, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708522 · Cited by 167 · see citing works
Using AI-based coding assistants in practice: State of affairs, perceptions, and ways forward
Sergeyuk, A., Golubev, Y., Bryksin, T., & Ahmed, I. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2024.107610 · Cited by 101 · see citing works
Prather, J. E., Leinonen, J., Kiesler, N., Benario, J. G., Lau, S., MacNeil, S., Norouzi, N., Opel, S., Pettit, V., Porter, L., et al. (2025) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3689187.3709614 · Cited by 97 · see citing works
Ironies of Generative AI: Understanding and Mitigating Productivity Loss in Human-AI Interaction
Simkute, A., Tankelevitch, L., Kewenig, V., Scott, A. E., Sellen, A., & Rintel, S. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2405782 · Cited by 89 · see citing works
"I look at it as the king of knowledge": How Blind People Use and Understand Generative AI Tools
Adnin, R., & Das, M. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675631 · Cited by 73 · see citing works
Kuhail, M. A., Mathew, S. S., Khalil, A., Berengueres, J., & Shah, S. J. H. (2024) · Science of Computer Programming · doi:10.1016/j.scico.2024.103111 · Cited by 68 · see citing works
The future of AI-driven software engineering
Terragni, V., Vella, A., Roop, P., & Blincoe, K. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715003 · Cited by 62 · see citing works
Software engineering education in the era of conversational AI: current trends and future directions
Sengul, C., Neykova, R., & Destefanis, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1436350 · Cited by 55 · see citing works
Student-AI Interaction: A Case Study of CS1 students
Amoozadeh, M., Nam, D., Prol, D., Alfageeh, A., Prather, J. E., Hilton, M. L., Ragavan, S. S., & Alipour, M. A. (2024) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3699538.3699567 · Cited by 41 · see citing works
Ivie: Lightweight Anchored Explanations of Just-Generated Code
Yan, L., Hwang, A., Wu, Z., & Head, A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642239 · Cited by 40 · see citing works
Scherr, S., Cao, B., Jiang, L., & Kobayashi, T. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108738 · Cited by 42 · see citing works
Weisz, J. D., Kumar, S. V., Muller, M., Browne, K.-E., Goldberg, A., Heintze, K. E., & Bajpai, S. (2025) · Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706599.3706670 · Cited by 37 · see citing works
Oyelere, S. S., & Aruleba, K. (2025) · Computers and Education Open · doi:10.1016/j.caeo.2025.100245 · Cited by 42 · see citing works
Pu, K., Lazaro, D., Arawjo, I., Xia, H., Xiao, Z., Grossman, T., & Chen, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713357 · Cited by 32 · see citing works
Prompts Are Programs Too! Understanding How Developers Build Software Containing Prompts
Liang, J. T., Lin, M. M., Rao, N., & Myers, B. A. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729342 · Cited by 34 · see citing works
Self-citationZhou, X., Liang, P., Zhang, B., Li, Z., Ahmad, A., Shahin, M., & Waseem, M. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112204 · Cited by 30 · see citing works
Shihab, M. I. H., Hundhausen, C., Tariq, A., Haque, S., Qiao, Y., & Mulanda, B. W. (2025) · Proceedings of the 2025 ACM Conference on International Computing Education Research V.1 · doi:10.1145/3702652.3744219 · Cited by 28 · see citing works
Bai, X., Huang, S., Wei, C., & Wang, R. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126357 · Cited by 31 · see citing works
Validating AI-Generated Code with Live Programming
Ferdowsi, K., Huang, R. L., James, M. B., Polikarpova, N., & Lerner, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642495 · Cited by 27 · see citing works
Oh, S., Lee, K., Park, S., Kim, D., & Kim, H. (2024) · 2024 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp54263.2024.00046 · Cited by 27 · see citing works
Codepori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology
Rasheed, Z., Malik, S., Kemell, K.-K., Muhammad, W., Saari, M., Systä, K., & Abrahamsson, P. (2024) · doi:10.2139/ssrn.4979510 · Cited by 27 · see citing works
Rocks Coding, Not Development: A Human-Centric, Experimental Evaluation of LLM-Supported SE Tasks
Wang, W., Ning, H., Zhang, G., Liu, L., & Wang, Y. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3643758 · Cited by 22 · see citing works
The Impact of Generative AI Coding Assistants on Developers Who Are Visually Impaired
Flores-Saviaga, C., Hanrahan, B. V., Imteyaz, K., Clarke, S., & Savage, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714008 · Cited by 24 · see citing works
What Makes a High-Quality Training Dataset for Large Language Models: A Practitioners' Perspective
Yu, X., Zhang, Z., Niu, F., Hu, X., Xia, X., & Grundy, J. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695061 · Cited by 21 · see citing works
Non-Expert Programmers in the Generative AI Future
Feldman, M. Q., & Anderson, C. J. (2024) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3663384.3663393 · Cited by 21 · see citing works
Autoethnographic Insights from Neurodivergent GAI “Power Users”
Glazko, K. S., Cha, J., Lewis, A., Kosa, B., Wimer, B. L., Zheng, A., Zheng, Y., & Mankoff, J. C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713670 · Cited by 20 · see citing works
Hao, H., Hasan, K. A., Qin, H., Macedo, M., Tian, Y., Ding, S. H. H., & Hassan, A. E. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10540-x · Cited by 19 · see citing works
Clear, T., Cajander, Å., Clear, A., McDermott, R., Daniels, M., Divitini, M., Forshaw, M., Humble, N., Kasinidou, M., Kleanthous, S., et al. (2025) · 2024 Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3689187.3709607 · Cited by 19 · see citing works
Can GPT-4 Replicate Empirical Software Engineering Research?
Liang, J. T., Badea, C., Bird, C., DeLine, R., Ford, D., Forsgren, N., & Zimmermann, T. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3660767 · Cited by 19 · see citing works
Self-citationDeveloper Behaviors in Validating and Repairing LLM-Generated Code Using IDE and Eye Tracking
Tang, N., Chen, M., Ning, Z., Bansal, A., Huang, Y., McMillan, C., & Li, T. J.-J. (2024) · 2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00015 · Cited by 18 · see citing works
Tab to Autocomplete: The Effects of AI Coding Assistants on Web Accessibility
Mowar, P., Peng, Y.-H., Steinfeld, A. M., & Bigham, J. P. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3688513 · Cited by 18 · see citing works
What Guides Our Choices? Modeling Developers' Trust and Behavioral Intentions Towards Genai
Choudhuri, R., Trinkenreich, B., Pandita, R., Kalliamvakou, E., Steinmacher, I., Gerosa, M. A., Sanchez, C. A., & Sarma, A. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00087 · Cited by 22 · see citing works
Examining Student and Teacher Perspectives on Undisclosed Use of Generative AI in Academic Work
Adnin, R., Pandkar, A., Yao, B., Wang, D., & Das, M. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713393 · Cited by 16 · see citing works
Human-AI experience in integrated development environments: a systematic literature review
Sergeyuk, A., Zakharov, I., Koshchenko, E., & Izadi, M. (2026) · Empirical Software Engineering · doi:10.1007/s10664-025-10793-0 · Cited by 26 · see citing works
ChatGPT in Data Visualization Education: A Student Perspective
Kim, N. W., Ko, H.-K., Myers, G., & Bach, B. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00022 · Cited by 21 · see citing works
Generative AI for Test Driven Development: Preliminary Results
Mock, M., Melegati, J., & Russo, B. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-72781-8_3 · Cited by 16 · see citing works
Alami, A., & Ernst, N. (2025) · 2025 IEEE/ACM 18th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1109/chase66643.2025.00016 · Cited by 15 · see citing works
How Much Does AI Impact Development Speed? an Enterprise-Based Randomized Controlled Trial
Paradis, E., Grey, K., Madison, Q., Nam, D., Macvean, A., Meimand, V., Zhang, N., Ferrari-Church, B., & Chandra, S. (2025) · 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00060 · Cited by 20 · see citing works
“Recent work has explored the potential of these tools to assist students and novice programmers [8], [23], [24], as well as program-mers’ perceptions of [25], [26], [27] and trust in [28], [29], [30] AI tools.”
How Scientists Use Large Language Models to Program
O’Brien, G. E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713668 · Cited by 18 · see citing works
Meta-Manager: A Tool for Collecting and Exploring Meta Information about Code
Horvath, A., Macvean, A., & Myers, B. A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642676 · Cited by 14 · see citing works
Self-citationMailach, A., Gorgosch, D., Siegmund, N., & Siegmund, J. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10561-6 · Cited by 13 · see citing works
Prompting in the Wild: An Empirical Study of Prompt Evolution in Software Repositories
Tafreshipour, M., Imani, A., Huang, E., de Almeida, E. S., Zimmermann, T., & Ahmed, I. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00106 · Cited by 13 · see citing works
Correctness assessment of code generated by Large Language Models using internal representations
Bui, T.-D., Vu, T. T., Nguyen, T.-T., Nguyen, S., & Vo, H. D. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112570 · Cited by 22 · see citing works
Pirzado, F. A., Ahmed, A., Alipour, M. A., & Terashima‐Marín, H. (2025) · Journal of Computers in Education · doi:10.1007/s40692-025-00375-w · Cited by 16 · see citing works
Basha, M., & Rodríguez-Pérez, G. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107804 · Cited by 14 · see citing works
Khurana, A., Su, X., Wang, A. Y., & Chilana, P. K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713431 · Cited by 10 · see citing works
Qiao, H., Vermeulen, J., Fitzmaurice, G. W., & Matejka, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714103 · Cited by 10 · see citing works
Challenges and Paths Towards AI for Software EngineeringPreprint
Gu, A., Jain, N., Li, W., Shetty, M., Shao, Y., Li, Z., Yang, D., Ellis, K., Sen, K., & Solar-Lezama, A. (2025) · Qeios · doi:10.32388/vv1661 · Cited by 24 · see citing works
Kudriavtseva, A., Hotak, N. A., & Gadyatskaya, O. (2025) · Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing · doi:10.1145/3672608.3707778 · Cited by 9 · see citing works
Al Haque, E., Brown, C., LaToza, T. D., & Johnson, B. (2025) · Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering · doi:10.1145/3696630.3731665 · Cited by 9 · see citing works
State of the Art of the Security of Code Generated by LLMs: A Systematic Literature Review
Ramírez, L. C., Limón, X., Sánchez-García, Á. J., & Pérez-Arriaga, J. C. (2024) · International Conference in Software Engineering Research and Innovation (CONISOFT) · doi:10.1109/conisoft63288.2024.00050 · Cited by 9 · see citing works
Retrieval-augmented code completion for local projects using large language models
Hostnik, M., & Robnik‐Šikonja, M. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128596 · Cited by 11 · see citing works
“Get Me in the Groove”: a Mixed Methods Study on Supporting Adhd Professional Programmers
Newman, K., Snay, S., Endres, M., Parikh, M., & Begel, A. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00242 · Cited by 9 · see citing works
Exploring Student-AI Interactions in Vibe Coding
Geng, F., Shah, A., Li, H., Mulla, N., Swanson, S., Raj, G. S., Zingaro, D., & Porter, L. (2026) · Australasian Computing Education Conference · doi:10.1145/3786228.3786236 · Cited by 9 · see citing works
Towards Effective Validation and Integration of LLM-Generated Code
Tang, N. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00051 · Cited by 8 · see citing works
Large language models and the problem of rhetorical debt
MacArthur, M. (2025) · AI & SOCIETY · doi:10.1007/s00146-025-02403-w · Cited by 8 · see citing works
Can Developers Prompt? A Controlled Experiment for Code Documentation Generation
Kruse, H.-A., Puhlfürß, T., & Maalej, W. (2024) · 2024 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme58944.2024.00058 · Cited by 8 · see citing works
What You Need is what You Get: Theory of Mind for an LLM-Based Code Understanding Assistant
Richards, J., & Wessel, M. S. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00070 · Cited by 8 · see citing works
LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions
Liao, Z., Antoniak, M., Cheong, I., Cheng, E. Y.-Y., Lee, A.-H., Lo, K., Chang, J. C., & Zhang, A. X. (2024) · arXiv · doi:10.48550/arxiv.2411.05025 · Cited by 7 · see citing works
Review on AI Assistant Systems for Programming Language Learning in Learning Environments
Senanayake, S., Karunanayaka, K. T., & Ekanayake, K. V. J. P. (2024) · SLAAI International Conference on Artificial Intelligence (SLAAI-ICAI) · doi:10.1109/slaai-icai63667.2024.10844969 · Cited by 7 · see citing works
Mining software repositories for software architecture — A systematic mapping study
Soliman, M., Albonico, M., Malavolta, I., & Wortmann, A. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107677 · Cited by 12 · see citing works
TypeForge: Synthesizing and Selecting Best-Fit Composite Data Types for Stripped Binaries
Wang, Y., Liang, R., Li, Y., Hu, P., Chen, K., & Zhang, B. (2025) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp61157.2025.00193 · Cited by 7 · see citing works
Exploring GenAI in Software Development: Insights from a Case Study in a Large Brazilian Company
Pereira, G. V., Jackson, V., Prikladnicki, R., van der Hoek, A., Fortes, L., Araújo, C., Coelho, A., Chelli, L., & Ramos, D. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00035 · Cited by 7 · see citing works
Newman, K., Snay, S., Endres, M., Parikh, M., & Begel, A. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746334 · Cited by 7 · see citing works
Desolda, G., Esposito, A., Greco, F., Tucci, C., Buono, P., & Piccinno, A. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103648 · Cited by 7 · see citing works
The Evolution of Software Usability in Developer Communities: An Empirical Study on Stack Overflow
Jalali, H., Aljedaani, W., & Ludi, S. A. (2025) · Software · doi:10.3390/software4040027 · Cited by 6 · see citing works
Passi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 6 · see citing works
Lau, S., & Guo, P. J. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00041 · Cited by 6 · see citing works
Design Principles for Collaborative Generative AI Systems in Software Development
Chen, J., & Zacharias, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61175-9_23 · Cited by 6 · see citing works
An Analysis of the Costs and Benefits of Autocomplete in IDEs
Jiang, S., & Coblenz, M. J. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660765 · Cited by 6 · see citing works
Exploring the Productivity Impacts of Generative AI in Organizations
Semujanga, B., & Mikalef, P. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72234-9_9 · Cited by 6 · see citing works
Investigating Developers' Preferences for Learning and Issue Resolution Resources in the ChatGPT Era
Tayeb, A. J., Alahmadi, M. D., Tajik, E., & Haiduc, S. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00045 · Cited by 6 · see citing works
The Transformative Influence of LLMs on Software Development & Developer Productivity
Jalil, S. (2025) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa65407.2025.11166110 · Cited by 6 · see citing works
Park, S. M., Ho, M., Lin, M. P.-C., & Ryoo, J. (2025) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016517 · Cited by 5 · see citing works
Agentic Much? Adoption of Coding Agents on GitHub
Robbes, R., Matricon, T., Degueule, T., Hora, A., & Zacchiroli, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3822180 · Cited by 5 · see citing works
AI IDEs or Autonomous Agents? Measuring the Impact of Coding Agents on Software Development
Agarwal, S., He, H., & Vasilescu, B. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793589 · Cited by 5 · see citing works
Understanding Documentation Use Through Log Analysis: A Case Study of Four Cloud Services
Nam, D., Macvean, A., Myers, B. A., & Vasilescu, B. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642721 · Cited by 6 · see citing works
Self-citationMcCabe, A. T., Bjorkman, M., Engström, J., Kuang, P., Söderberg, E., & Church, L. (2024) · International Conference on Art, Science, and Engineering of Programming (<Programming>) · doi:10.1145/3660829.3660835 · Cited by 5 · see citing works
Development of Automated Software Design Document Review Methods Using Large Language Models
Fukuda, T., Nakagawa, T., Miyazaki, K., & Tokumoto, S. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00017 · Cited by 4 · see citing works
Joshi, O., Mundra, K., Yadav, D., & Das, S. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-99261-2_17 · Cited by 4 · see citing works
Towards AI-Driven Organizations
Bosch, J., & Olsson, H. H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04207-1_19 · Cited by 4 · see citing works
Developer Confidence, Competence, and Self-Efficacy in the Age of AI
Gupta, D., Xusniddin, R., Jumaniyazova, N., Sherbek, S., Rajabova, N., Shavkat, R., & Dilorom, O. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8450-4.ch002 · Cited by 4 · see citing works
Accessibility in AI-Assisted Web Development
Mowar, P. (2024) · International Web for All Conference (W4A) · doi:10.1145/3677846.3679054 · Cited by 4 · see citing works
Constraint Representation Towards Precise Data-Driven Storytelling
Shi, Y.-Z., Li, H., Ruan, L., & Qu, H. (2024) · doi:10.1109/gen4ds63889.2024.00006 · Cited by 4 · see citing works
A fine-grained taxonomy of code review feedback in TypeScript projects
Davila, N., Nunes, I., & Wiese, I. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10604-y · Cited by 4 · see citing works
Xavier: Toward Better Coding Assistance in Authoring Tabular Data Wrangling Scripts
Zhou, Y., Cai, X., Shi, Q., Huang, Y., Li, H., Qu, H., Weng, D., & Wu, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714239 · Cited by 4 · see citing works
Leveraging AI for Productive and Trustworthy HPC Software: Challenges and Research Directions
Teranishi, K., Menon, H., Godoy, W. F., Balaprakash, P., Bau, D., Ben‐Nun, T., Bhatele, A. S., Franchetti, F., Franusich, M., Gamblin, T., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-07612-0_47 · Cited by 4 · see citing works
Sáez, Y., Garcia, L. M., Mochón, A., & Isasi, P. (2026) · Scientific Reports · doi:10.1038/s41598-026-48656-3 · Cited by 6 · see citing works
Chen, A., Wong, C., Sharif, B., & Peruma, A. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00043 · Cited by 4 · see citing works
Developers' Perspective on Today's and Tomorrow's Programming Tool Assistance: A Survey
Kuang, P., Söderberg, E., & Höst, M. (2024) · International Conference on Art, Science, and Engineering of Programming (<Programming>) · doi:10.1145/3660829.3660848 · Cited by 3 · see citing works
Stalnaker, T., Wintersgill, N., Chaparro, O., Heymann, L. A., Penta, M. D., Germán, D. M., & Poshyvanyk, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3743133 · Cited by 3 · see citing works
TableTalk: Scaffolding Spreadsheet Development with a Language Agent
Liang, J. T., Kumar, A., Bajpai, Y., Gulwani, S., Le, V., Parnin, C., Radhakrishna, A., Tiwari, A., Murphy-Hill, E. R., & Soares, G. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3765286 · Cited by 3 · see citing works
Self-citationArchitecting open, accountable, and trustworthy AI-IDEs
Contreras, A., Guerra, E., & de Lara, J. (2026) · Automated Software Engineering · doi:10.1007/s10515-026-00608-x · Cited by 4 · see citing works
Chen, N., Qiu, L. K., Wang, A. Z., Wang, Z., & Yang, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790726 · Cited by 8 · see citing works
Building Software by Rolling the Dice: A Qualitative Study of Vibe Coding
Chou, Y.-H., Jiang, B., Chen, Y. W., Weng, M., Jackson, V., Zimmermann, T., & Jones, J. A. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797105 · Cited by 3 · see citing works
Responsible AI in Agile Software Engineering - An Industry Perspective
Ulfsnes, R., Moe, N. B., Emmerhoff, J., Floryan, M., Griva, A., Gundelsby, J. H., Barbala, A., & Conboy, K. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-72781-8_4 · Cited by 3 · see citing works
Leveraging AI in Software Testing: Applying ADKAR for Effective Change Management
Laine, A., & Ojanen, V. (2024) · IEEE International Conference on Industrial Engineering and Engineering Management (IEEM) · doi:10.1109/ieem62345.2024.10857157 · Cited by 4 · see citing works
Gan, X., Liang, H., & Brown, C. (2025) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst62969.2025.10988972 · Cited by 3 · see citing works
- Authors
- Liang, J. T., Yang, C., & Myers, B. A.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the IEEE/ACM 46th International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2024-02-06
- Pages
- 1-13
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W4391558520
- Cited by
- 203
- References
- 45
- Citations by year
- 2026: 68 · 2025: 92 · 2024: 42 · 2023: 1
- FWCI
- 76.92
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (1745016, 1856641, 2140739)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- GitHub Copilot, usability, developer practices, code completion, developer motivation, non-functional requirements, cognitive effort, tool control, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
