The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges
Open accessAbstract
The software engineering community recently has witnessed widespread deployment of AI programming assistants, such as GitHub Copilot. However, in practice, developers do not accept AI programming assistants' initial suggestions at a high frequency. This leaves a number of open questions related to the usability of these tools. To understand developers' practices while using these tools and the important usability challenges they face, we administered a survey to a large population of developers and received responses from a diverse set of 410 developers. Through a mix of qualitative and quantitative analyses, we found that developers are most motivated to use AI programming assistants because they help developers reduce key-strokes, finish programming tasks quickly, and recall syntax, but resonate less with using them to help brainstorm potential solutions. We also found the most important reasons why developers do not use these tools are because these tools do not output code that addresses certain functional or non-functional requirements and because developers have trouble controlling the tool to generate the desired output. Our findings have implications for both creators and users of AI programming assistants, such as designing minimal cognitive effort interactions with these tools to reduce distractions for users while they are programming.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some recent studies confirm these symptoms: only 40% of CCT suggestions are accepted the first time they are proposed to developers, mainly because developers cannot control context or granularity (Liang et al., 2024).”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Liang et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“GitHub Copilot was the most widely used tool (74.6%), whose users reported that a median of 30.5% of their code was now generated by the AI assistant.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Liang et al. (2024)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Similarly, Liang et al. ( 2024) and Butler et al. (2025) found that developers generally feel productivity gains using the same assistant.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Liang et al. (2024)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Liang et al. [18] found that frustration occurs when the suggestions do not satisfy functional or non-functional requirements.”
From How do Copilot Suggestions Impact Developers' Frustration and Productivity? (2025) · cites this paper as Liang et al. [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Acceleration mode is usually preferred, as developers often use the tools to complete repetitive code that cannot be copy-pasted (Liang et al., 2024), and for recalling syntax they don't remember (Xu et al., 2022).”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Liang et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“The literature also highlights the need for explanations (such as reporting the source or linking to documentation) for additional context on the generated code (Liang et al., 2024; Xu et al., 2022).”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Liang et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“They also found that developers often do not use CCTs because these tools do not generate code that addresses certain functional or non-functional requirements, and because developers have trouble controlling the tool to generate the desired output.”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Liang et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“The survey revealed that developers are most motivated to use AI programming assistants because they help developers reduce keystrokes, finish programming tasks quickly, and recall syntax.”
From Understanding user mental models in AI-driven code completion tools: Insights from an elicitation study (2025) · cites this paper as Liang et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“On the other hand, the most common reasons for not using such tools were 1) that the code output often is of subpar quality, and 2) the difficulty of controlling the AI assistants to produce the desired output.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Liang et al. (2024)
The cited source says what the claim attributes to it.
In the citing paper
“The primary motivations for adopting AI assistants were 1) reducing keystrokes, 2) completing tasks faster, and 3) skipping web searches for code examples.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Liang et al. (2024)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Five misunderstandings about case-study research
Flyvbjerg, B. (2006) · Qualitative Inquiry · doi:10.1177/1077800405284363 · Cited by 14,266
The Coding Manual for Qualitative Researchers
Lungu, M. (2022) · American Journal of Qualitative Research · doi:10.29333/ajqr/12085 · Cited by 10,936
McDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
Herbsleb, J. D., & Moitra, D. (2001) · IEEE Software · doi:10.1109/52.914732 · Cited by 1,059
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660
Design lessons from the fastest q&a site in the west
Mamykina, L., Manoim, B., Mittal, M., Hripcsak, G. M., & Hartmann, B. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979366 · Cited by 574
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., & Hellendoorn, V. J. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534862 · Cited by 540
Kitchenham, B. A., & Pfleeger, S. L. (2007) · Springer eBooks · doi:10.1007/978-1-84800-044-5_3 · Cited by 515
GitHub Copilot AI pair programmer: Asset or Liability?
Moradi Dakhel, A., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M. C., & Jiang, Z. M. (. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111734 · Cited by 417
Flagged first, then by citations
Research works printed but not linked: 7
- 2023. Code faster with AI code completions | Tabnine. Retrieved March 11, 2023 from https://www.tabnine.com/.
- 2023. Prolific • Quickly find research participants you can trust. Retrieved September 2, 2023 from https://prolific.co/.
- Ruijia Cheng, Ruotong Wang, Thomas Zimmermann, and Denae Ford. 2022. "It would work for me too": How online communities shape software developers' trust in AI-powered code generation tools. arXiv preprint arXiv:2212.03491 (2022). · arXiv
- Victor Dibia, Adam Fourney, Gagan Bansal, Forough Poursabzi-Sangdeh, Han Liu, and Saleema Amershi. 2022. Aligning offline metrics and human judgments of value of AI-pair programmers. arXiv preprint arXiv:2210.16494 (2022). · arXiv
- 10.5555/3575618.3575622 · DOI
- Mohammad Tahaei and Kami Vaniea. 2022. Lessons Learned From Recruiting Participants With Programming Skills for Empirical Privacy and Security Studies. In International Workshop on Recruiting Participants for Empirical Software Engineering (RoPES).
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
Cited by
Sorted by citations
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650
Using an LLM to Help With Code Understanding
Nam, D., Macvean, A., Hellendoorn, V., Vasilescu, B., & Myers, B. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639187 · Cited by 487
Self-citationThe Metacognitive Demands and Opportunities of Generative AI
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642902 · Cited by 493
Large Language Model for Vulnerability Detection and Repair: Literature Review and the Road Ahead
Zhou, X., Cao, S., Sun, X., & Lo, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708522 · Cited by 215
Using AI-based coding assistants in practice: State of affairs, perceptions, and ways forward
Sergeyuk, A., Golubev, Y., Bryksin, T., & Ahmed, I. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2024.107610 · Cited by 101
Prather, J. E., Leinonen, J., Kiesler, N., Benario, J. G., Lau, S., MacNeil, S., Norouzi, N., Opel, S., Pettit, V., Porter, L., et al. (2025) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3689187.3709614 · Cited by 168
Ironies of Generative AI: Understanding and Mitigating Productivity Loss in Human-AI Interaction
Simkute, A., Tankelevitch, L., Kewenig, V., Scott, A. E., Sellen, A., & Rintel, S. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2405782 · Cited by 89
"I look at it as the king of knowledge": How Blind People Use and Understand Generative AI Tools
Adnin, R., & Das, M. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675631 · Cited by 73
Kuhail, M. A., Mathew, S. S., Khalil, A., Berengueres, J., & Shah, S. J. H. (2024) · Science of Computer Programming · doi:10.1016/j.scico.2024.103111 · Cited by 68
The future of AI-driven software engineering
Terragni, V., Vella, A., Roop, P., & Blincoe, K. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715003 · Cited by 62
Software engineering education in the era of conversational AI: current trends and future directions
Sengul, C., Neykova, R., & Destefanis, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1436350 · Cited by 63
Student-AI Interaction: A Case Study of CS1 students
Amoozadeh, M., Nam, D., Prol, D., Alfageeh, A., Prather, J. E., Hilton, M. L., Ragavan, S. S., & Alipour, M. A. (2024) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3699538.3699567 · Cited by 68
Ivie: Lightweight Anchored Explanations of Just-Generated Code
Yan, L., Hwang, A., Wu, Z., & Head, A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642239 · Cited by 55
Scherr, S., Cao, B., Jiang, L., & Kobayashi, T. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108738 · Cited by 45
Weisz, J. D., Kumar, S. V., Muller, M., Browne, K.-E., Goldberg, A., Heintze, K. E., & Bajpai, S. (2025) · Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706599.3706670 · Cited by 37
Oyelere, S. S., & Aruleba, K. (2025) · Computers and Education Open · doi:10.1016/j.caeo.2025.100245 · Cited by 47
Pu, K., Lazaro, D., Arawjo, I., Xia, H., Xiao, Z., Grossman, T., & Chen, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713357 · Cited by 67
Prompts Are Programs Too! Understanding How Developers Build Software Containing Prompts
Liang, J. T., Lin, M. M., Rao, N., & Myers, B. A. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729342 · Cited by 57
Self-citationZhou, X., Liang, P., Zhang, B., Li, Z., Ahmad, A., Shahin, M., & Waseem, M. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112204 · Cited by 30
Shihab, M. I. H., Hundhausen, C., Tariq, A., Haque, S., Qiao, Y., & Mulanda, B. W. (2025) · Proceedings of the 2025 ACM Conference on International Computing Education Research V.1 · doi:10.1145/3702652.3744219 · Cited by 28
Bai, X., Huang, S., Wei, C., & Wang, R. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126357 · Cited by 34
Validating AI-Generated Code with Live Programming
Ferdowsi, K., Huang, R. L., James, M. B., Polikarpova, N., & Lerner, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642495 · Cited by 51
Oh, S., Lee, K., Park, S., Kim, D., & Kim, H. (2024) · 2024 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp54263.2024.00046 · Cited by 27
Codepori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology
Rasheed, Z., Malik, S., Kemell, K.-K., Muhammad, W., Saari, M., Systä, K., & Abrahamsson, P. (2024) · doi:10.2139/ssrn.4979510 · Cited by 27
Rocks Coding, Not Development: A Human-Centric, Experimental Evaluation of LLM-Supported SE Tasks
Wang, W., Ning, H., Zhang, G., Liu, L., & Wang, Y. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3643758 · Cited by 40
The Impact of Generative AI Coding Assistants on Developers Who Are Visually Impaired
Flores-Saviaga, C., Hanrahan, B. V., Imteyaz, K., Clarke, S., & Savage, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714008 · Cited by 32
What Makes a High-Quality Training Dataset for Large Language Models: A Practitioners' Perspective
Yu, X., Zhang, Z., Niu, F., Hu, X., Xia, X., & Grundy, J. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695061 · Cited by 27
Non-Expert Programmers in the Generative AI Future
Feldman, M. Q., & Anderson, C. J. (2024) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3663384.3663393 · Cited by 34
Autoethnographic Insights from Neurodivergent GAI “Power Users”
Glazko, K. S., Cha, J., Lewis, A., Kosa, B., Wimer, B. L., Zheng, A., Zheng, Y., & Mankoff, J. C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713670 · Cited by 28
Hao, H., Hasan, K. A., Qin, H., Macedo, M., Tian, Y., Ding, S. H. H., & Hassan, A. E. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10540-x · Cited by 39
Clear, T., Cajander, Å., Clear, A., McDermott, R., Daniels, M., Divitini, M., Forshaw, M., Humble, N., Kasinidou, M., Kleanthous, S., et al. (2025) · 2024 Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3689187.3709607 · Cited by 19
Can GPT-4 Replicate Empirical Software Engineering Research?
Liang, J. T., Badea, C., Bird, C., DeLine, R., Ford, D., Forsgren, N., & Zimmermann, T. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3660767 · Cited by 19
Self-citationDeveloper Behaviors in Validating and Repairing LLM-Generated Code Using IDE and Eye Tracking
Tang, N., Chen, M., Ning, Z., Bansal, A., Huang, Y., McMillan, C., & Li, T. J.-J. (2024) · 2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00015 · Cited by 18
Tab to Autocomplete: The Effects of AI Coding Assistants on Web Accessibility
Mowar, P., Peng, Y.-H., Steinfeld, A. M., & Bigham, J. P. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3688513 · Cited by 20
What Guides Our Choices? Modeling Developers' Trust and Behavioral Intentions Towards Genai
Choudhuri, R., Trinkenreich, B., Pandita, R., Kalliamvakou, E., Steinmacher, I., Gerosa, M., Sanchez, C., & Sarma, A. (2025) · 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE) · doi:10.1109/icse55347.2025.00087 · Cited by 17
Examining Student and Teacher Perspectives on Undisclosed Use of Generative AI in Academic Work
Adnin, R., Pandkar, A., Yao, B., Wang, D., & Das, M. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713393 · Cited by 16
Human-AI experience in integrated development environments: a systematic literature review
Sergeyuk, A., Zakharov, I., Koshchenko, E., & Izadi, M. (2026) · Empirical Software Engineering · doi:10.1007/s10664-025-10793-0 · Cited by 38
ChatGPT in Data Visualization Education: A Student Perspective
Kim, N. W., Ko, H.-K., Myers, G., & Bach, B. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00022 · Cited by 28
Generative AI for Test Driven Development: Preliminary Results
Mock, M., Melegati, J., & Russo, B. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-72781-8_3 · Cited by 31
Alami, A., & Ernst, N. (2025) · 2025 IEEE/ACM 18th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1109/chase66643.2025.00016 · Cited by 15
How Much Does AI Impact Development Speed? an Enterprise-Based Randomized Controlled Trial
Paradis, E., Grey, K., Madison, Q., Nam, D., Macvean, A., Meimand, V., Zhang, N., Ferrari-Church, B., & Chandra, S. (2025) · 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00060 · Cited by 20
“Recent work has explored the potential of these tools to assist students and novice programmers [8], [23], [24], as well as program-mers’ perceptions of [25], [26], [27] and trust in [28], [29], [30] AI tools.”
How Scientists Use Large Language Models to Program
O’Brien, G. E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713668 · Cited by 33
Meta-Manager: A Tool for Collecting and Exploring Meta Information about Code
Horvath, A., Macvean, A., & Myers, B. A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642676 · Cited by 19
Self-citationMailach, A., Gorgosch, D., Siegmund, N., & Siegmund, J. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10561-6 · Cited by 13
Prompting in the Wild: An Empirical Study of Prompt Evolution in Software Repositories
Tafreshipour, M., Imani, A., Huang, E., de Almeida, E. S., Zimmermann, T., & Ahmed, I. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00106 · Cited by 37
Correctness assessment of code generated by Large Language Models using internal representations
Bui, T.-D., Vu, T. T., Nguyen, T.-T., Nguyen, S., & Vo, H. D. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112570 · Cited by 39
Pirzado, F. A., Ahmed, A., Alipour, M. A., & Terashima‐Marín, H. (2025) · Journal of Computers in Education · doi:10.1007/s40692-025-00375-w · Cited by 19
Basha, M., & Rodríguez-Pérez, G. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107804 · Cited by 15
Khurana, A., Su, X., Wang, A. Y., & Chilana, P. K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713431 · Cited by 20
Qiao, H., Vermeulen, J., Fitzmaurice, G. W., & Matejka, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714103 · Cited by 15
Challenges and Paths Towards AI for Software EngineeringPreprint
Gu, A., Jain, N., Li, W., Shetty, M., Shao, Y., Li, Z., Yang, D., Ellis, K., Sen, K., & Solar-Lezama, A. (2025) · Qeios · doi:10.32388/vv1661 · Cited by 40
Kudriavtseva, A., Hotak, N. A., & Gadyatskaya, O. (2025) · Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing · doi:10.1145/3672608.3707778 · Cited by 9
Al Haque, E., Brown, C., LaToza, T. D., & Johnson, B. (2025) · Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering · doi:10.1145/3696630.3731665 · Cited by 9
State of the Art of the Security of Code Generated by LLMs: A Systematic Literature Review
Ramírez, L. C., Limón, X., Sánchez-García, Á. J., & Pérez-Arriaga, J. C. (2024) · International Conference in Software Engineering Research and Innovation (CONISOFT) · doi:10.1109/conisoft63288.2024.00050 · Cited by 12
Retrieval-augmented code completion for local projects using large language models
Hostnik, M., & Robnik‐Šikonja, M. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128596 · Cited by 13
“Get Me in the Groove”: a Mixed Methods Study on Supporting Adhd Professional Programmers
Newman, K., Snay, S., Endres, M., Parikh, M., & Begel, A. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00242 · Cited by 13
Exploring Student-AI Interactions in Vibe Coding
Geng, F., Shah, A., Li, H., Mulla, N., Swanson, S., Raj, G. S., Zingaro, D., & Porter, L. (2026) · Australasian Computing Education Conference · doi:10.1145/3786228.3786236 · Cited by 41
Towards Effective Validation and Integration of LLM-Generated Code
Tang, N. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00051 · Cited by 8
Large language models and the problem of rhetorical debt
MacArthur, M. (2025) · AI & SOCIETY · doi:10.1007/s00146-025-02403-w · Cited by 8
Can Developers Prompt? A Controlled Experiment for Code Documentation Generation
Kruse, H.-A., Puhlfürß, T., & Maalej, W. (2024) · 2024 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme58944.2024.00058 · Cited by 8
What You Need is what You Get: Theory of Mind for an LLM-Based Code Understanding Assistant
Richards, J., & Wessel, M. S. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00070 · Cited by 23
LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions
Liao, Z., Antoniak, M., Cheong, I., Cheng, E. Y.-Y., Lee, A.-H., Lo, K., Chang, J. C., & Zhang, A. X. (2024) · arXiv · doi:10.48550/arxiv.2411.05025 · Cited by 7
Review on AI Assistant Systems for Programming Language Learning in Learning Environments
Senanayake, S., Karunanayaka, K. T., & Ekanayake, K. V. J. P. (2024) · SLAAI International Conference on Artificial Intelligence (SLAAI-ICAI) · doi:10.1109/slaai-icai63667.2024.10844969 · Cited by 10
Mining software repositories for software architecture — A systematic mapping study
Soliman, M., Albonico, M., Malavolta, I., & Wortmann, A. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107677 · Cited by 13
TypeForge: Synthesizing and Selecting Best-Fit Composite Data Types for Stripped Binaries
Wang, Y., Liang, R., Li, Y., Hu, P., Chen, K., & Zhang, B. (2025) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp61157.2025.00193 · Cited by 9
Exploring GenAI in Software Development: Insights from a Case Study in a Large Brazilian Company
Pereira, G. V., Jackson, V., Prikladnicki, R., van der Hoek, A., Fortes, L., Araújo, C., Coelho, A., Chelli, L., & Ramos, D. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00035 · Cited by 14
Newman, K., Snay, S., Endres, M., Parikh, M., & Begel, A. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746334 · Cited by 9
Desolda, G., Esposito, A., Greco, F., Tucci, C., Buono, P., & Piccinno, A. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103648 · Cited by 7
The Evolution of Software Usability in Developer Communities: An Empirical Study on Stack Overflow
Jalali, H., Aljedaani, W., & Ludi, S. A. (2025) · Software · doi:10.3390/software4040027 · Cited by 6
Passi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 9
Lau, S., & Guo, P. J. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00041 · Cited by 10
Design Principles for Collaborative Generative AI Systems in Software Development
Chen, J., & Zacharias, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61175-9_23 · Cited by 7
An Analysis of the Costs and Benefits of Autocomplete in IDEs
Jiang, S., & Coblenz, M. J. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660765 · Cited by 8
Exploring the Productivity Impacts of Generative AI in Organizations
Semujanga, B., & Mikalef, P. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72234-9_9 · Cited by 8
Investigating Developers' Preferences for Learning and Issue Resolution Resources in the ChatGPT Era
Tayeb, A. J., Alahmadi, M. D., Tajik, E., & Haiduc, S. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00045 · Cited by 15
The Transformative Influence of LLMs on Software Development & Developer Productivity
Jalil, S. (2025) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa65407.2025.11166110 · Cited by 15
Park, S. M., Ho, M., Lin, M. P.-C., & Ryoo, J. (2025) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016517 · Cited by 8
Agentic Much? Adoption of Coding Agents on GitHub
Robbes, R., Matricon, T., Degueule, T., Hora, A., & Zacchiroli, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3822180 · Cited by 56
AI IDEs or Autonomous Agents? Measuring the Impact of Coding Agents on Software Development
Agarwal, S., He, H., & Vasilescu, B. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793589 · Cited by 20
Understanding Documentation Use Through Log Analysis: A Case Study of Four Cloud Services
Nam, D., Macvean, A., Myers, B. A., & Vasilescu, B. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642721 · Cited by 8
Self-citationMcCabe, A. T., Bjorkman, M., Engström, J., Kuang, P., Söderberg, E., & Church, L. (2024) · International Conference on Art, Science, and Engineering of Programming (<Programming>) · doi:10.1145/3660829.3660835 · Cited by 5
Development of Automated Software Design Document Review Methods Using Large Language Models
Fukuda, T., Nakagawa, T., Miyazaki, K., & Tokumoto, S. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00017 · Cited by 4
Joshi, O., Mundra, K., Yadav, D., & Das, S. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-99261-2_17 · Cited by 5
Towards AI-Driven Organizations
Bosch, J., & Olsson, H. H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04207-1_19 · Cited by 5
Developer Confidence, Competence, and Self-Efficacy in the Age of AI
Gupta, D., Xusniddin, R., Jumaniyazova, N., Sherbek, S., Rajabova, N., Shavkat, R., & Dilorom, O. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8450-4.ch002 · Cited by 4
Accessibility in AI-Assisted Web Development
Mowar, P. (2024) · International Web for All Conference (W4A) · doi:10.1145/3677846.3679054 · Cited by 5
Constraint Representation Towards Precise Data-Driven Storytelling
Shi, Y.-Z., Li, H., Ruan, L., & Qu, H. (2024) · doi:10.1109/gen4ds63889.2024.00006 · Cited by 6
A fine-grained taxonomy of code review feedback in TypeScript projects
Davila, N., Nunes, I., & Wiese, I. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10604-y · Cited by 4
Xavier: Toward Better Coding Assistance in Authoring Tabular Data Wrangling Scripts
Zhou, Y., Cai, X., Shi, Q., Huang, Y., Li, H., Qu, H., Weng, D., & Wu, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714239 · Cited by 6
Leveraging AI for Productive and Trustworthy HPC Software: Challenges and Research Directions
Teranishi, K., Menon, H., Godoy, W. F., Balaprakash, P., Bau, D., Ben‐Nun, T., Bhatele, A. S., Franchetti, F., Franusich, M., Gamblin, T., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-07612-0_47 · Cited by 7
Sáez, Y., Garcia, L. M., Mochón, A., & Isasi, P. (2026) · Scientific Reports · doi:10.1038/s41598-026-48656-3 · Cited by 7
Chen, A., Wong, C., Sharif, B., & Peruma, A. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00043 · Cited by 6
Developers' Perspective on Today's and Tomorrow's Programming Tool Assistance: A Survey
Kuang, P., Söderberg, E., & Höst, M. (2024) · International Conference on Art, Science, and Engineering of Programming (<Programming>) · doi:10.1145/3660829.3660848 · Cited by 5
Stalnaker, T., Wintersgill, N., Chaparro, O., Heymann, L. A., Penta, M. D., Germán, D. M., & Poshyvanyk, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3743133 · Cited by 17
TableTalk: Scaffolding Spreadsheet Development with a Language Agent
Liang, J. T., Kumar, A., Bajpai, Y., Gulwani, S., Le, V., Parnin, C., Radhakrishna, A., Tiwari, A., Murphy-Hill, E. R., & Soares, G. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3765286 · Cited by 8
Self-citationArchitecting open, accountable, and trustworthy AI-IDEs
Contreras, A., Guerra, E., & de Lara, J. (2026) · Automated Software Engineering · doi:10.1007/s10515-026-00608-x · Cited by 5
Chen, N., Qiu, L. K., Wang, A. Z., Wang, Z., & Yang, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790726 · Cited by 14
Building Software by Rolling the Dice: A Qualitative Study of Vibe Coding
Chou, Y.-H., Jiang, B., Chen, Y. W., Weng, M., Jackson, V., Zimmermann, T., & Jones, J. A. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797105 · Cited by 13
Responsible AI in Agile Software Engineering - An Industry Perspective
Ulfsnes, R., Moe, N. B., Emmerhoff, J., Floryan, M., Griva, A., Gundelsby, J. H., Barbala, A., & Conboy, K. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-72781-8_4 · Cited by 7
Leveraging AI in Software Testing: Applying ADKAR for Effective Change Management
Laine, A., & Ojanen, V. (2024) · IEEE International Conference on Industrial Engineering and Engineering Management (IEEM) · doi:10.1109/ieem62345.2024.10857157 · Cited by 5
Metadata
- Authors
- Liang, J. T., Yang, C., & Myers, B. A.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the IEEE/ACM 46th International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2024-02-06
- Pages
- 1-13
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W4391558520
- Cited by
- 203
- References
- 45
- Citations by year
- 2026: 68 · 2025: 92 · 2024: 42
- FWCI
- 76.92
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (1745016, 1856641, 2140739)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- GitHub Copilot, usability, developer practices, code completion, developer motivation, non-functional requirements, cognitive effort, tool control, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
