Examining the Use and Impact of an AI Code Assistant on Developer Productivity and Experience in the Enterprise
Open accessAbstract
AI assistants are being created to help software engineers conduct a variety of coding-related tasks, such as writing, documenting, and testing code. We describe the use of the watsonx Code Assistant (WCA), an LLM-powered coding assistant deployed internally within IBM. Through surveys of two user cohorts (N=669) and unmoderated usability testing (N=15), we examined developers' experiences with WCA and its impact on their productivity. We learned about their motivations for using (or not using) WCA, we examined their expectations of its speed and quality, and we identified new considerations regarding ownership of and responsibility for generated code. Our case study characterizes the impact of an LLM-powered assistant on developers' perceptions of productivity and it shows that although such tools do often provide net productivity increases, these benefits may not always be experienced by all users.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Weisz et al. (2025) found that the main WCA use case was code comprehension rather than generation.
“The authors report that the main WCA use case was code comprehension rather than generation.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Weisz et al. 2025
Weisz et al. (2025) found that respondents generally felt their work was faster, easier, and of higher quality with WCA, but the magnitudes were small.
“While the respondents generally felt that their work was faster, easier, and of higher quality with WCA, the magnitudes were small.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Weisz et al. 2025
Weisz et al. (2025) found that 42.6% felt WCA made them less effective.
“Moreover, the individual differences were substantial 42.6% felt that WCA made them less effective.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Weisz et al. 2025
Weisz et al. (2025) showed that while most developers feel faster with watsonx Code Assistant, a large fraction feel less effective.
“Weisz et al. (2025) contrasts the picture somewhat for the watsonx Code Assistant, showing that while most developers feel faster with the tool, a large fraction feel less effective.”
From Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability (2026) · cites this paper as Weisz et al. (2025)
Cites
User Acceptance of Computer Technology: A Comparison of Two Theoretical Models
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989) · Management Science · doi:10.1287/mnsc.35.8.982 · Cited by 25,709 · see citing works
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1× · see citing works
“LLMs have the potential to reproduce materials contained within their training data [7]; in the domain of software engineering, there is a risk of contaminating a code-base with copyrighted material or code subject to specific licenses (e.g. GPL).”
Clarke, V., & Braun, V. (2017) · The Journal of Positive Psychology · doi:10.1080/17439760.2016.1262613 · Cited by 6,543 · see citing works
McCabe, T. J. (1976) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1976.233837 · Cited by 6,059 · see citing works
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.139 · Cited by 2,824 · see citing works
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124 · see citing works
Yilmaz, R., & Karaoglan Yilmaz, F. G. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100147 · Cited by 745 · see citing works
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660 · see citing works
Grounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2023) · Proceedings of the ACM on Programming Languages · doi:10.1145/3586030 · Cited by 436 · cited 1× · see citing works
“New agentic design patterns are enabling coding assistants to perform even more complex tasks, such as converting issues into pull requests [25] and new feature descriptions into specifications and implementation plans 7 .”
GitHub Copilot AI pair programmer: Asset or Liability?
Moradi Dakhel, A., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M. C., & Jiang, Z. M. (. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111734 · Cited by 417 · see citing works
Using an LLM to Help With Code Understanding
Nam, D., Macvean, A., Hellendoorn, V., Vasilescu, B., & Myers, B. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639187 · Cited by 345 · cited 1× · see citing works
“…converting natural language to code (e.g. [2, 16, 19, 56]), converting code to natural language documentation (e.g. [16, 31]) or explanations (e.g. [37]), and converting code to code, such as by translating it from one language to another (e.g. [2, 19, 46, 57]) or by creating unit tests (e.g.…”
Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI
Wang, D., Weisz, J. D., Muller, M., Ram, P., Geyer, W., Dugan, C., Tausczik, Y., Samulowitz, H., & Gray, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359313 · Cited by 342 · see citing works
Self-citationAn empirical evaluation of GitHub copilot's code suggestions
Nguyen, N., & Nadi, S. (2022) · Proceedings of the 19th International Conference on Mining Software Repositories · doi:10.1145/3524842.3528470 · Cited by 313 · cited 1× · see citing works
“…models that have been exposed to source code in their pre-training have demonstrated a high degree of aptitude in performing a variety of tasks: converting natural language to code (e.g. [2, 16, 19, 56]), converting code to natural language documentation (e.g. [16, 31]) or explanations (e.g.…”
Ross, S., Martinez, F., Houde, S., Muller, M. J., & Weisz, J. D. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584037 · Cited by 272 · see citing works
Self-citationSoftware developers' perceptions of productivity
Meyer, A. N., Fritz, T., Murphy, G. C., & Zimmermann, T. (2014) · Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2635868.2635892 · Cited by 224 · cited 1× · see citing works
“However, Meyer et al. [34] considers “when software developers perceive themselves to be productive and... unproductive” [34, p.1] as an important aspect of productivity.”
Function points in the estimation and evaluation of the software process
Low, G. C., & Jeffery, D. R. (1990) · IEEE Transactions on Software Engineering · doi:10.1109/32.44364 · Cited by 209 · see citing works
A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges
Liang, J. T., Yang, C., & Myers, B. A. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3608128 · Cited by 203 · see citing works
Productivity Assessment of Neural Code Completion
Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., & Aftandilian, E. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534864 · Cited by 200 · see citing works
Productivity Assessment of Neural Code Completion
Ziegler, A., Kalliamvakou, E., Simister, S., Sittampalam, G., Li, A., Rice, A., Rifkin, D., & Aftandilian, E. (2022) · arXiv · doi:10.48550/arxiv.2205.06537 · Cited by 200 · see citing works
A framework and methodology for studying the causes of software errors in programming systems
Ko, A. J., & Myers, B. A. (2004) · Journal of Visual Languages & Computing · doi:10.1016/j.jvlc.2004.08.003 · Cited by 195 · see citing works
Using GitHub Copilot to Solve Simple Programming Problems
Wermelinger, M. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569830 · Cited by 191 · see citing works
AI Creativity and the Human-AI Co-creation Model
Wu, Z., Ji, D., Yu, K., Zeng, X., Wu, D., & Shidujaman, M. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-78462-1_13 · Cited by 186 · see citing works
The SPACE of Developer Productivity: There's more to it than you think.
Forsgren, N., Storey, M.-A., Maddila, C., Zimmermann, T., Houck, B., & Butler, J. (2021) · Queue · doi:10.1145/3454122.3454124 · Cited by 164 · see citing works
In-IDE Code Generation from Natural Language: Promise and Challenges
Xu, F. F., Vasilescu, B., & Neubig, G. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3487569 · Cited by 154 · see citing works
Assessing the quality of GitHub copilot’s code generation
Yetiştiren, B., Ozsoy, I., & Tüzün, E. (2022) · International Conference on Predictive Models in Software Engineering · doi:10.1145/3558489.3559072 · Cited by 148 · see citing works
Perfection Not Required? Human-AI Partnerships in Code Translation
Weisz, J. D., Muller, M., Houde, S., Richards, J., Ross, S. I., Martinez, F., Agarwal, M., & Talamadupula, K. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450656 · Cited by 129 · see citing works
Self-citationMeasuring GitHub Copilot's Impact on Productivity
Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., & Aftandilian, E. (2024) · Communications of the ACM · doi:10.1145/3633453 · Cited by 124 · cited 4× · see citing works
“In addition, researchers have found correlations between subjective and objective productivity metrics, such as the acceptance rate of suggested code [62] and the number of source code files owned by a developer [40] being correlated with perceived productivity.”
InfluentialSource-code Similarity Detection and Detection Tools Used in Academia: A Systematic Review
Novak, M., Joy, M., & Kermek, D. (2019) · ACM Transactions on Computing Education · doi:10.1145/3313290 · Cited by 117 · see citing works
Is GitHub copilot a substitute for human pair-programming?: an empirical study
Imai, S. (2022) · Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings · doi:10.1145/3510454.3522684 · Cited by 116 · cited 2× · see citing works
“…language models that have been exposed to source code in their pre-training have demonstrated a high degree of aptitude in performing a variety of tasks: converting natural language to code (e.g. [2, 16, 19, 56]), converting code to natural language documentation (e.g. [16, 31]) or explanations…”
The Art and Science of Prompt Engineering: A New Literacy in the Information Age
Lo, L. S. (2023) · Internet Reference Services Quarterly · doi:10.1080/10875301.2023.2227621 · Cited by 114 · see citing works
Exploring and Unleashing the Power of Large Language Models in Automated Code Translation
Yang, Z., Liu, F., Yu, Z., Keung, J. W., Li, J., Liu, S., Hong, Y., Ma, X., Jin, Z., & Li, G. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3660778 · Cited by 104 · see citing works
Rethinking Productivity in Software Engineering
Sadowski, C., & Zimmermann, T. (2019) · Apress eBooks · doi:10.1007/978-1-4842-4221-6 · Cited by 100 · see citing works
Analytical and empirical evaluation of software reuse metrics
Dévanbu, P., Karstu, S., Melo, W. L., & Thomas, W. M. (1996) · University Libraries (University of Maryland) · doi:10.5555/227726.227760 · Cited by 99 · see citing works
Deskilling, Upskilling, and Reskilling: a Case for Hybrid Intelligence
Rafner, J. F., Dellermann, D., Hjorth, A., Verasztó, D., Kampf, C. E., Mackay, W. E., & Sherson, J. F. (2021) · Morals & Machines · doi:10.5771/2747-5174-2021-2-24 · Cited by 89 · see citing works
Practices and Challenges of Using GitHub Copilot: An Empirical Study
Zhang, B., Liang, P., Zhou, X., Ahmad, A., & Waseem, M. (2023) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2023-077 · Cited by 75 · see citing works
Better Together? An Evaluation of AI-Supported Code Translation
Weisz, J. D., Muller, M., Ross, S. I., Martinez, F., Houde, S., Agarwal, M., Talamadupula, K., & Richards, J. T. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511157 · Cited by 69 · see citing works
Self-citationA User-centered Framework for Human-AI Co-creativity
Moruzzi, C., & Margarido, S. (2024) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613905.3650929 · Cited by 68 · see citing works
User Perspectives on Ethical Challenges in Human-AI Co-Creativity: A Design Fiction Study
Rezwana, J., & Maher, M. L. (2023) · Creativity and Cognition · doi:10.1145/3591196.3593364 · Cited by 60 · see citing works
AI-Assisted Human Labeling: Batching for Efficiency without Overreliance
Ashktorab, Z., Desmond, M., Andres, J., Muller, M., Joshi, N. N., Brachman, M., Sharma, A., Brimijoin, K., Pan, Q., Wolf, C. T., et al. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449163 · Cited by 45 · see citing works
Self-citationKuttal, S. K., Ong, B., Kwasny, K., & Robe, P. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445659 · Cited by 44 · see citing works
Code and commit metrics of developer productivity: a study on team leaders perceptions
de Oliveira, E. C. C., Fernandes, E., Steinmacher, I., de Cristo, M. A. P., Conte, T. U., & Garcia, A. (2020) · Empirical Software Engineering · doi:10.1007/s10664-020-09820-z · Cited by 43 · see citing works
An Evaluation of Khanmigo, a Generative AI Tool, as a Computer-Assisted Language Learning App
Shetye, S. (2024) · Studies in Applied Linguistics and TESOL · doi:10.52214/salt.v24i1.12869 · Cited by 41 · see citing works
RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation
Luo, Q., Ye, Y., Liang, S., Zhang, Z., Qin, Y., Lu, Y., Wu, Y., Cong, X., Lin, Y., Zhang, Y., et al. (2024) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2024.emnlp-demo.46 · Cited by 36 · see citing works
What improves developer productivity at google? code quality
Lan, C., Murphy-Hill, E., Canning, M., Jaspan, C., Green, C., Knight, A., Zhang, N., & Kammer, E. (2022) · ESEC/SIGSOFT FSE · doi:10.1145/3540250.3558940 · Cited by 29 · see citing works
Grounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2022) · arXiv · doi:10.48550/arxiv.2206.15000 · Cited by 23 · cited 1× · see citing works
“New agentic design patterns are enabling coding assistants to perform even more complex tasks, such as converting issues into pull requests [25] and new feature descriptions into specifications and implementation plans 7 .”
Flagged first, then by citations
Cited by
Sorted by citations
GenAI and the psychology of work
Hermann, E., Puntoni, S., & Morewedge, C. K. (2025) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2025.04.009 · Cited by 39 · see citing works
Challenges and Paths Towards AI for Software EngineeringPreprint
Gu, A., Jain, N., Li, W., Shetty, M., Shao, Y., Li, Z., Yang, D., Ellis, K., Sen, K., & Solar-Lezama, A. (2025) · Qeios · doi:10.32388/vv1661 · Cited by 24 · see citing works
AI IDEs or Autonomous Agents? Measuring the Impact of Coding Agents on Software Development
Agarwal, S., He, H., & Vasilescu, B. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793589 · Cited by 5 · see citing works
Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows
Chen, V., Talwalkar, A., Brennan, R., & Neubig, G. (2025) · International Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790850 · Cited by 47 · see citing works
Architecting open, accountable, and trustworthy AI-IDEs
Contreras, A., Guerra, E., & de Lara, J. (2026) · Automated Software Engineering · doi:10.1007/s10515-026-00608-x · Cited by 4 · see citing works
Martin-Lopez, A., Tufano, R., Guglielmi, E., Sánchez, A. B., Sanz, A. L., Scalabrino, S., Oliveto, R., Segura, S., & Bavota, G. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3679627 · Cited by 4 · see citing works
Chen, N., Qiu, L. K., Wang, A. Z., Wang, Z., & Yang, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790726 · Cited by 8 · see citing works
Building Software by Rolling the Dice: A Qualitative Study of Vibe Coding
Chou, Y.-H., Jiang, B., Chen, Y. W., Weng, M., Jackson, V., Zimmermann, T., & Jones, J. A. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797105 · Cited by 3 · see citing works
Si, J. J., Li, D., Wang, Q., Liu, A. W., Lin, X., Zhu, Y., Zhou, X., Wang, A. Y., & Wang, K. Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790366 · Cited by 3 · see citing works
Eyes on the Code: Mapping Critical Thinking Through Eye-Tracking for Student-LLM Coding Interactions
Sqalli, M. T. (2025) · Biannual Conference of the Italian SIGCHI Chapter (CHItaly) · doi:10.1145/3750069.3750397 · Cited by 1 · see citing works
Awad, M. N. A., Ivanov, S. V., & Tikhonova, O. B. (2025) · IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) · doi:10.1109/asew67777.2025.00032 · Cited by 1 · see citing works
A Multi-Year Survey of Use and Perceptions of Generative AI in Higher Education
Crescenzi, A., Choi, B., Huang, P.-P., Gautier, E., Little, R., & Pandya, S. (2026) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3786304.3787867 · Cited by 2 · see citing works
Jang, Y., Sakashita, M., Niinuma, K., & Gupta, A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791260 · Cited by 1 · see citing works
Kobiella, C., Breidenstein, D., & Schmidt, A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790757 · Cited by 1 · see citing works
Daryanto, T., Ding, X., Ping, K., Wilhelm, L. A., Chen, Y., Brown, C., & Rho, E. H. R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791773 · Cited by 2 · see citing works
AI and the democratization of knowledge work
Daepp, M. I. G., Tomlinson, K., Counts, S., & Suri, S. (2026) · Nature Computational Science · doi:10.1038/s43588-026-00985-z · Cited by 1 · see citing works
A Longitudinal Analysis of Software Development Evolution in a Financial Organization
Chen, W., Zhang, Z., Yu, H., Fan, G., Wang, H., & Zhi, K. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834515 · Cited by 1 · see citing works
Beyond the Commit: Developer Perspectives on Productivity with AI Coding Assistants
Chen, V., He, J., Williams, B., Valentino, J., & Talwalkar, A. (2026) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3786583.3786848 · Cited by 1 · see citing works
Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability
Borg, M., Hewett, D., Hagatulah, N., Couderc, N., Söderberg, E., Graham, D., Kini, U., & Farley, D. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10889-1 · Cited by 4 · see citing works
STEP-LINK: STEP-by-Step Tutorial Editing with Programmable LINKages
Li, T., Ke, J., Wen, Z., Wu, Y., lu, J., Zhu, B., Zhu, M., & Chen, W. (2025) · Visual Informatics · doi:10.1016/j.visinf.2025.100244
(2025) · Computer Science Engineering and Technology · doi:10.46632/cset/3/3/12
Hyun, S., Kim, H., Jang, J., Choi, H., & Babar, M. A. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3844949
Lumen: Developer Agency Through Transparent Context Control in AI-Assisted Programming
Goel, N., & Melo, G. (2025) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1109/cascon66301.2025.00024
Seo, J., Deldari, E., & Mentis, H. M. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789121 · Cited by 1 · see citing works
GenAI-assisted software development: Is it killing the creativity?
Sınav, A., Peker, V., Yıldız, H., & Tokdemir, G. (2026) · International Informatics and Software Engineering Conference (IISEC) · doi:10.1109/iisec69317.2026.11418493
Informed Skepticism: Navigating The Code Hallucination Dilemma
Faker, O., & Toker, Ş. (2026) · International Informatics and Software Engineering Conference (IISEC) · doi:10.1109/iisec69317.2026.11418436
Cha, S., Jo, S., Shin, J., & Seo, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798616
Yeman, R. J. (2026) · Women in engineering and science · doi:10.1007/978-3-032-17466-6_3
Efecto de la IA sobre las habilidades técnicas de los desarrolladores de software
Jácome, A. J. M. (2026) · Revista Tecnológica - ESPOL · doi:10.37815/rte.v38n1.1417
HookLens: Visual Analytics for Understanding React Hooks Structures
Hwang, S., Kweon, M., Rhee, J., Lee, S., Park, S., Jung, S., Jeon, H., & Seo, J. (2026) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis68791.2026.00036
A comprehensive analysis of security risks and productivity impacts of AI-generated code
Zardari, S., Osama, M., Ammar, S. M., Reshamwala, R. I., & Siddiqui, A. (2026) · AI and Ethics · doi:10.1007/s43681-026-01203-2
Do Language Models Prefer Vulnerable Code? A Probabilistic Study of Insecure Code Preference
Melo, R., Reis, S., Catarino, A., & Abreu, R. (2026) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst69053.2026.00075
Mockus, A., Banga, A. S., Bhuptani, P., Jazayeri, S., Ge, J., Grewal, G., Lai, E., Li, J., Liang, H., Mangla, R., et al. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3805211
From Syntactic Mimicry to Semantic Correctness: Behavioral Understanding in LLM Code Generation
Alam, A. I. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3804873
When AI Joins the Team: Understanding Human-Agent Collaboration in Pull Requests
Romero-Arjona, M., Barakat, S., Martin-Lopez, A., Sánchez, A. B., Bavota, G., & Segura, S. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00313
Yao, X., Xu, F., & Shao, Y. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2746841
Das, P., Reddy, Y. R., Potdar, S., & Warudkar, H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36590-3_28
Wittig, A., Mailach, A., & Siegmund, N. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834386
- Authors
- Weisz, J. D., Kumar, S. V., Muller, M., Browne, K.-E., Goldberg, A., Heintze, K. E., & Bajpai, S.
- Affiliations
- IBM (United States) (US)
- Cisco Systems, Inc., Bangalore, India
- IBM Research - Cambridge (United States) (US)
- IBM (Ireland) (IE)
- IBM (Germany) (DE)
- IBM (India) (IN)
- ORCID
- Venue
- Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2025-04-25
- Pages
- 1-13
- OpenAlex
- W4409720354
- Cited by
- 37
- References
- 43
- Citations by year
- 2026: 30 · 2025: 7
- FWCI
- 44.40
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Keywords
- developer productivity, developer experience, enterprise software development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
