The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index, and 1 are not checked yet.
AutoCodeRover: Autonomous Program Improvement
Open accessAbstract
Researchers have made significant progress in automating the software development process in the past decades. Automated techniques for issue summarization, bug reproduction, fault localization, and program repair have been built to ease the workload of developers. Recent progress in Large Language Models (LLMs) has significantly impacted the development process, where developers can use LLM-based programming assistants to achieve automated coding. Nevertheless, software engineering involves the process of program improvement apart from coding, specifically to enable software maintenance (e.g. program repair to fix bugs) and software evolution (e.g. feature additions). In this paper, we propose an automated approach for solving Github issues to autonomously achieve program improvement. In our approach called AutoCodeRover, LLMs are combined with sophisticated code search capabilities, ultimately leading to a program modification or patch. In contrast to recent LLM agent approaches from AI researchers and practitioners, our outlook is more software engineering oriented. We work on a program representation (abstract syntax tree) as opposed to viewing a software project as a mere collection of files. Our code search exploits the program structure in the form of classes/methods to enhance LLM’s understanding of the issue’s root cause, and effectively retrieve a context via iterative search. The use of spectrum-based fault localization using tests, further sharpens the context, as long as a test-suite is available. Experiments on the recently proposed SWE-bench-lite (300 real-life Github issues) show increased efficacy in solving Github issues (19% on SWE-bench-lite), which is higher than the efficacy of the recently reported Swe-agent. Interestingly, our approach resolved 57 GitHub issues in about 4 minutes each (pass@1), whereas developers spent more than 2.68 days on average. In addition, AutoCodeRover achieved this efficacy with significantly lower cost (on average, $0.43 USD), compared to other baselines. We posit that our workflow enables autonomous software engineering, where, in future, auto-generated code from LLMs can be autonomously improved.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“memory methods that maintain working state, retrieve repository evidence, store reusable experience, and support shared interaction histories [45, 46, 47, 48]”
From Code as Agent Harness (2026) · cites this paper as [46]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Mechanisms such as AST-based structured chunking, iterative query rewriting, and retrieval strategies conditioned on current localization clues can substantially improve the utility of retrieved context for downstream generation.”
From Code as Agent Harness (2026) · cites this paper as [46]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Representative works such as AutoCodeRover [46] and RepoCoder [47] show that repository-level coding tasks benefit not simply from retrieving more content, but from retrieving evidence aligned with program structure.”
From Code as Agent Harness (2026) · cites this paper as [46]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Survey on Software Fault Localization
Wong, W. E., Gao, R., Li, Y., Abreu, R., & Wotawa, F. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2521368 · Cited by 1,110
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Competition-Level Code Generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.07814 · Cited by 1,043
On the Accuracy of Spectrum-based Fault Localization
Abreu, R., Zoeteweij, P., & van Gemund, A. V. (2007) · Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007) · doi:10.1109/taic.part.2007.13 · Cited by 939
Visualization of test information to assist fault localization
Jones, J. A., Harrold, M. J., & Stasko, J. (2002) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/581339.581397 · Cited by 925
Symbolic execution for software testing
Cadar, C., & Sen, K. (2013) · Communications of the ACM · doi:10.1145/2408776.2408795 · Cited by 785
Automatically finding patches using genetic programming
Weimer, W. R., Nguyen, T. H., Goues, C. L., & Forrest, S. (2009) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2009.5070536 · Cited by 768
Böhme, M., Pham, V.-T., Nguyen, M.-D., & Roychoudhury, A. (2017) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3133956.3134020 · Cited by 759
Self-citationMechtaev, S., Yi, J., & Roychoudhury, A. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884807 · Cited by 515
Self-citationAutomated Program Repair in the Era of Large Pre-trained Language Models
Xia, C. S., Wei, Y., & Zhang, L. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00129 · Cited by 433
Research works printed but not linked: 12
- 2024. AutoCodeRover, Autonomous Software Engineering. https://autocoderover.dev/ Retrieved July 10, 2024 from
- Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé de Oliveira Pinto Jared Kaplan Harrison Edwards Yuri Burda Nicholas Joseph Greg Brockman Alex Ray Raul Puri Gretchen Krueger Michael Petrov Heidy Khlaaf Girish Sastry Pamela Mishkin Brooke Chan Scott Gray Nick Ryder Mikhail Pavlov Alethea Power Lukasz Kaiser Mohammad Bavarian Clemens Winter Philippe Tillet Felipe Petroski Such Dave Cummings Matthias Plappert Fotios Chantzis Elizabeth Barnes Ariel Herbert-Voss William Hebgen Guss Alex Nichol Alex Paino Nikolas Tezak Jie Tang Igor Babuschkin Suchir Balaji Shantanu Jain William Saunders Christopher Hesse Andrew N. Carr Jan Leike Joshua Achiam Vedant Misra Evan Morikawa Alec Radford Matthew Knight Miles Brundage Mira Murati Katie Mayer Peter Welinder Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. arXiv.org abs/2107.03374 (7 2021). issn:2331-8422 arxiv:2107.03374 https://arxiv.org/abs/2107.03374 · arXiv
- Claire Le Goues, Michael Pradel, and Abhik Roychoudhury. 2019. Automated program repair. Commun. ACM 62 (11 2019), 56–65. Issue 12. issn:0001-0782
- Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2024. Leaderboard results on SWE-bench. https://www.swebench.com/ Retrieved April 8, 2024 from
- Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench: Can Language Models Resolve Real-world Github Issues?. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=VTF8yNQM66
- Wuxia Jin et al. 2024. PyAnalyzer: An Effective and Practical Approach for Dependency Extraction from Python Code. In International Conference on Software Engineering (ICSE).
- Cognition Labs. 2024. Devin, AI software engineer. https://www.cognition-labs.com/introducing-devin Retrieved April 12, 2024 from
- Thibaud Lutellier, Hung Viet Pham, Lawrence Pang, Yitong Li, Moshi Wei, and Lin Tan. 2020. CoCoNuT: combining context-aware neural translation models using ensemble for program repair., In ISSTA ’20: 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, Virtual Event, USA, July 18-22, 2020, Sarfraz Khurshid and Corina S. Pasareanu (Eds.). International Symposium on Software Testing and Analysis, 101–114.
- Yannic Noller, Ridwan Shariffdeen, Xiang Gao, and Abhik Roychoudhury. 2022. Trust Enhancement Issues in Program Repair. In IEEE/ACM 44th International Conference on Software Engineering (ICSE).
- David Williams, James Callan, Serkan Kirbas, Sergey Mechtaev, Justyna Petke, Thomas Prideaux-Ghee, and Federica Sarro. 2023. User-Centric Deployment of Automated Program Repair at Bloomberg. arXiv preprint arXiv:2311.10516 (2023). · arXiv
- 10.1109/TSE.2016.2521368 · DOI · Paper page
- John Yang Carlos E. Jimenez Alexander Wettig Kilian Lieret Shunyu Yao Karthik Narasimhan and Ofir Press. 2024. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. arxiv:2405.15793 [cs.SE] · arXiv
Cited by
Sorted by citations
A Survey on Large Language Models for Code Generation
Jiang, J., Wang, F., Shen, J., Kim, S., & Kim, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3747588 · Cited by 251
He, J., Treude, C., & Lo, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3712003 · Cited by 246
LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead
He, J., Treude, C., & Lo, D. (2024) · arXiv · doi:10.48550/arxiv.2404.04834 · Cited by 246
Automated Program Repair via Conversation: Fixing 162 out of 337 Bugs for $0.42 Each using ChatGPT
Xia, C. S., & Zhang, L. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680323 · Cited by 468
(2025) · doi:10.1109/icse55347.2025.00157 · Cited by 157
Demystifying LLM-Based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715754 · Cited by 849
“More recently, agent-based APR techniques have also been proposed [Bouzenia et al. 2024; Chen 2024; Hidvégi et al. 2024; Xia and Zhang 2023; Zhang et al. 2024b].”
InfluentialEvaluation and Benchmarking of LLM Agents: A Survey
Mohammadi, M., Li, Y., Lo, J., & Yip, W. (2025) · Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 · doi:10.1145/3711896.3736570 · Cited by 74
RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair
Silva, A., Fang, S., & Monperrus, M. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3581062 · Cited by 128
Can LLMs Replace Manual Annotation of Software Engineering Artifacts?
Ahmed, T., Dévanbu, P., Treude, C., & Pradel, M. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00086 · Cited by 105
Software Engineering by and for Humans in an AI Era
Abrahão, S., Grundy, J., Pezzè, M., Storey, M., & Tamburri, D. A. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715111 · Cited by 60
You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects
Bouzenia, I., & Pradel, M. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3728922 · Cited by 35
Multimodal Large Language Models for Phishing Webpage Detection and Identification
Lee, J.-H., Lim, P., Hooi, B., & Divakaran, D. M. (2024) · APWG Symposium on Electronic Crime Research (eCrime) · doi:10.1109/ecrime66200.2024.00007 · Cited by 34
SoapFL: A Standard Operating Procedure for LLM-Based Method-Level Fault Localization
Qin, Y., Wang, S., Lou, Y., Dong, J., Wang, K., Li, X., & Mao, X. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3543187 · Cited by 32
CellVoyager: AI CompBio agent generates new insights by autonomously analyzing biological data
Alber, S., Chen, B., Sun, E., Isakova, A., Wilk, A. J., & Zou, J. (2026) · Nature Methods · doi:10.1038/s41592-026-03029-6 · Cited by 60
SpecRover: Code Intent Extraction via LLMs
Ruan, H., Zhang, Y., & Roychoudhury, A. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00080 · Cited by 45
Self-citationSpecRover: Code Intent Extraction via LLMs
Ruan, H., Zhang, Y., & Roychoudhury, A. (2024) · arXiv · doi:10.48550/arxiv.2408.02232 · Cited by 28
Self-citationCodepori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology
Rasheed, Z., Malik, S., Kemell, K.-K., Muhammad, W., Saari, M., Systä, K., & Abrahamsson, P. (2024) · doi:10.2139/ssrn.4979510 · Cited by 27
Human-In-The-Loop Software Development Agents
Takerngsaksiri, W., Pasuksmit, J., Thongtanunam, P., Tantithamthavorn, C. K., Zhang, R., Jiang, F., Li, J., Cook, E., Chen, K., & Wu, M. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00036 · Cited by 24
Human-In-the-Loop Software Development Agents
Takerngsaksiri, W., Pasuksmit, J., Thongtanunam, P., Tantithamthavorn, C., Zhang, R., Jiang, F., Li, J., Cook, E., Chen, K., & Wu, M. (2024) · arXiv · doi:10.48550/arxiv.2411.12924 · Cited by 24
How Effective are Large Language Models in Generating Software Specifications?
Xie, D., Yoo, B., Jiang, N., Kim, M., Tan, L., Zhang, X., & Lee, J. S. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00014 · Cited by 77
Towards Understanding the Characteristics of Code Generation Errors Made by Large Language Models
Wang, Z., Zhou, Z., Song, D., Huang, Y., Chen, S., Ma, L., & Zhang, T. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00180 · Cited by 87
Template-Guided Program Repair in the Era of Large Language Models
Huang, K., Zhang, J., Meng, X., & Liu, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00030 · Cited by 30
Alhanahnah, M., Hasan, M. R., Xu, L., & Bagheri, H. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10687-1 · Cited by 20
Evaluating Agent-Based Program Repair at Google
Rondon, P., Wei, R., Cambronero, J., Cito, J., Sun, A., Sanyam, S., Tufano, M., & Chandra, S. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00038 · Cited by 19
Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories
Bouzenia, I., & Pradel, M. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00234 · Cited by 93
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Bajpai, Y., Chopra, B., Biyani, P., Aslan, C., Coleman, D., Gulwani, S., Parnin, C., Radhakrishna, A., & Soares, G. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00011 · Cited by 18
ChatDBG: Augmenting Debugging with Large Language Models
Levin, K. H., van Kempen, N., Berger, E. D., & Freund, S. N. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729355 · Cited by 16
OpenHands: An Open Platform for AI Software Developers as Generalist Agents
Wang, X., Li, B., Song, Y., Xu, F. F., Tang, X., Zhuge, M., Pan, J., Song, Y., Li, B., Singh, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.16741 · Cited by 24
Self-citationCOFFE: A Code Efficiency Benchmark for Code Generation
Peng, Y., Wan, J., Li, Y., & Ren, X. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715727 · Cited by 15
PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback
Peng, Y. F., Gotmare, A. D., Lyu, M. R.-T., Xiong, C., Savarese, S., & Sahoo, D. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00008 · Cited by 56
VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation
Wang, Z., Zhou, Z., Song, J., Huang, Y., Shu, Z., & Ma, L. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3729343 · Cited by 14
A 2030 Roadmap for Software Engineering
Pezzè, M., Abrahão, S., Penzenstadler, B., Poshyvanyk, D., Roychoudhury, A., & Yue, T. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3731559 · Cited by 13
Self-citationValidating Network Protocol Parsers with Traceable RFC Document Interpretation
Zheng, M., Xie, D., Shi, Q., Wang, C., & Zhang, X. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728955 · Cited by 18
Penggunaan artificial intelligence dalam pembelajaran bahasa Indonesia
Apriliani, D. (2024) · DIKBASTRA Jurnal Pendidikan Bahasa dan Sastra · doi:10.22437/dikbastra.v7i1.33262 · Cited by 12
Cloud Infrastructure Management in the Age of AI Agents
Yang, Z., Bhatnagar, A., Qiu, Y., Miao, T., Kon, P. T. J., Xiao, Y., Huang, Y., Casado, M., & Chen, A. (2025) · ACM SIGOPS Operating Systems Review · doi:10.1145/3759441.3759443 · Cited by 20
Software Security Analysis in 2030 and Beyond: A Research Roadmap
Böhme, M., Bodden, E., Bultan, T., Cadar, C., Liu, Y., & Scanniello, G. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708533 · Cited by 21
Luo, Z., Zhao, H., Wolff, D., Cadar, C., & Roychoudhury, A. (2026) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp63933.2026.00003 · Cited by 17
Self-citationStatically Contextualizing Large Language Models with Typed Holes
Blinn, A., Li, X., Kim, J. H., & Omar, C. (2024) · Proceedings of the ACM on Programming Languages · doi:10.1145/3689728 · Cited by 10
Challenges and Paths Towards AI for Software EngineeringPreprint
Gu, A., Jain, N., Li, W., Shetty, M., Shao, Y., Li, Z., Yang, D., Ellis, K., Sen, K., & Solar-Lezama, A. (2025) · Qeios · doi:10.32388/vv1661 · Cited by 40
AdverIntent-Agent: Adversarial Reasoning for Repair Based on Inferred Program Intent
Ye, H., Yang, A. Z. H., Hu, C., Wang, Y., Zhang, T., & Goues, C. L. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728939 · Cited by 11
CodingGenie: A Proactive LLM-Powered Programming Assistant
Zhao, S., Zhu, A., Mozannar, H., Sontag, D. A., Talwalkar, A., & Chen, V. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728603 · Cited by 9
From Bugs to Benchmarks: A Comprehensive Survey of Software Defect Datasets
Zhu, H., Furth, R. M., Pradel, M., & Rubio-González, C. (2026) · ACM Computing Surveys · doi:10.1145/3797033 · Cited by 25
RepairBench: Leaderboard of Frontier Models for Program Repair
Silva, A., & Monperrus, M. (2025) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1109/llm4code66737.2025.00006 · Cited by 23
RunBugRun: An executable dataset for automated program repair
Prenner, J. A., & Robbes, R. (2026) · Empirical Software Engineering · doi:10.1007/s10664-025-10790-3 · Cited by 33
Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
Lei, F., Chen, J., Ye, Y., Cao, R., Shin, D., Su, H., Suo, Z., Gao, H., Hu, W., Yin, P., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.07763 · Cited by 7
Lau, S., & Guo, P. J. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00041 · Cited by 10
Watson: A Cognitive Observability Framework for the Reasoning of LLM-Powered Agents
Rombaut, B., Masoumzadeh, S., Vasilevski, K., Lin, D., & Hassan, A. E. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00067 · Cited by 17
Enhancing Automated Program Repair with Solution Design
Zhao, J., Yang, D., Zhang, L., Lian, X., Yang, Z., & Liu, F. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695537 · Cited by 6
Towards Detecting Prompt Knowledge Gaps for Improved LLM-guided Issue Resolution
Ehsani, R., Pathak, S., & Chatterjee, P. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00107 · Cited by 21
Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
Zhang, J., Hu, S., Lu, C., Lange, R., & Clune, J. (2025) · arXiv · doi:10.48550/arxiv.2505.22954 · Cited by 344
ROSE: An IDE-Based Interactive Repair Framework for Debugging
Reiss, S. P., Wei, X., Yuan, J. P., & Xin, Q. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3705306 · Cited by 6
Large Language Models for Validating Network Protocol Parsers
Zheng, M., Xie, D., & Zhang, X. (2025) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw67851.2025.00009 · Cited by 6
Hierarchical Knowledge Injection for Improving LLM-based Program Repair
Ehsani, R., Parra, E., Haiduc, S., & Chatterjee, P. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00122 · Cited by 11
Ehsani, R., Pathak, S., Parra, E., Haiduc, S., & Chatterjee, P. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10745-8 · Cited by 5
ELT-Bench: An End-to-End Benchmark for Evaluating AI Agents on ELT Pipelines
Jin, T., Zhu, Y., & Kang, D. D. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3773749.3773750 · Cited by 20
CoDocBench: A Dataset for Code-Documentation Alignment in Software Maintenance
Pai, K. S., Dévanbu, P., & Ahmed, T. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00077 · Cited by 5
Optimization for threat classification of various data types-based on ML model and LLM
Hong, C., & Oh, T. (2025) · Scientific Reports · doi:10.1038/s41598-025-05182-y · Cited by 6
Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving
Zan, D., Huang, Z., Liu, W., Chen, H., Zhang, L., Xin, S., Chen, L., Liu, Q., Zhong, X., Li, A., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.02605 · Cited by 5
“Following prior work [20, 49, 39], we report the following primary evaluation metrics for end-to-end performance: (1) Resolved Rate (%): the percentage of issues resolved.”
Self-citationSWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
Deng, X., Da, J., Pan, E., He, Y. Y., Ide, C., Garg, K., Lauffer, N., Park, A., Pasari, N., Rane, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2509.16941 · Cited by 4
“Zhang et al. [27] developed AutoCodeRover, which combines LLMs with sophisticated AST-based code search capabilities, achieving 19% on SWE-bench-lite while maintaining low operational costs.”
OmniGIRL: A Multilingual and Multimodal Benchmark for GitHub Issue Resolution
Guo, L., Tao, W., Jiang, R., Wang, Y., Chen, J., Liu, X., Ma, Y., Mao, M., Zhang, H., & Zheng, Z. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728871 · Cited by 14
SWE-GPT: A Process-Centric Language Model for Automated Software Improvement
Ma, Y., Cao, R., Cao, Y., Zhang, Y., Chen, J., Liu, Y., Liu, Y., Li, B., Huang, F., & Li, Y. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728981 · Cited by 83
Self-citationIssue Localization via LLM-Driven Iterative Code Graph Searching
Jiang, Z., Ren, X., Yan, M. L., Jiang, W., Li, Y., & Liu, Z. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00249 · Cited by 4
Chang, J., Zhou, X., Wang, L., Lo, D., & Li, B. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3668601 · Cited by 25
Agentic AI Software Engineers: Programming with Trust
Roychoudhury, A., Pasareanu, C. S., Pradel, M., & Ray, B. (2026) · Communications of the ACM · doi:10.1145/3769314 · Cited by 4
Self-citationOskooei, A. R., Yukcu, S. S., Bozoglan, M. C., & Aktaş, M. S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97576-9_6 · Cited by 4
Munson, A., Gómez, J., & Cárdenas, Á. A. (2025) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1109/llm4code66737.2025.00008 · Cited by 9
NIODebugger: A Novel Approach to Repair Non-Idempotent-Outcome Tests with LLM-Based Agent
Ke, K. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00226 · Cited by 4
SBEST: Spectrum-based fault localization without fault-triggering tests
Rafi, M. N., Pacheco, L. B. S., Chen, A. R., Yang, J., & Chen, T.-H. P. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10761-8 · Cited by 4
Are “Solved Issues” in SWE-bench Really Solved Correctly? An Empirical Study
Wang, Y., Pradel, M., & Liu, Z. (2026) · Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering · doi:10.1145/3744916.3764576 · Cited by 3
SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement
Antoniades, A., Örwall, A., Zhang, K., Xie, Y., Goyal, A., & Wang, W. (2024) · arXiv · doi:10.48550/arxiv.2410.20285 · Cited by 3
“Current large language model (LLM)-based software agents (Xia et al., 2024; Zhang et al., 2024d), while powerful, often struggle with complex, long-horizon tasks that require adaptive strategies and flexible reassessment over time.”
Hydra-Reviewer: A Holistic Multi-Agent System for Automatic Code Review Comment Generation
Ren, X., Dai, C., Huang, Q., Wang, Y., Liu, C., & Jiang, B. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3621462 · Cited by 3
Automated Generation of Issue-Reproducing Tests by Combining LLMs and Search-Based Testing
Kitsios, K., Castelluccio, M., & Bacchelli, A. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00165 · Cited by 3
Let the Code Speak: Incorporating Program Dynamic State for Better Method-Level Fault Localization
Qin, Y., Wang, S., Lin, B., Peng, X. L., Ouyang, S., Chen, L., & Mao, X. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00072 · Cited by 3
Treefix: Enabling Execution with a Tree of Prefixes
Souza, B., & Pradel, M. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00215 · Cited by 2
TableTalk: Scaffolding Spreadsheet Development with a Language Agent
Liang, J. T., Kumar, A., Bajpai, Y., Gulwani, S., Le, V., Parnin, C., Radhakrishna, A., Tiwari, A., Murphy-Hill, E. R., & Soares, G. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3765286 · Cited by 8
Reference-Based Retrieval-Augmented Unit Test Generation
Zhang, Z., Liu, X., Lin, Y., Gao, X., Sun, H., & Yuan, Y. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3765758 · Cited by 2
Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Repair
Huang, K., Zhang, J., Xie, X., & Chen, C. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00100 · Cited by 2
Liu, C., Lin, Y., Huang, Y., Chang, J., Qi, B., Jiang, B., Huang, Z., & Dong, J. S. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00117 · Cited by 2
FlakyGuard: Automatically Fixing Flaky Tests at Industry Scale
Li, C., Behrang, F., Shi, A., & Liu, P. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00179 · Cited by 2
FURINA: A Multi-Agent Framework for Automated Patch Correctness Assessment
Li, Y., Ji, Y., Chen, X., Shen, Z., Ge, J., & Luo, B. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3811825 · Cited by 2
BackportBench: A Multilingual Benchmark for Automated Patch Backporting
Zhong, Z., Huang, J., & He, P. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797127 · Cited by 5
Kang, S., Yun, S., Hong, J., Yoo, S., & An, G. (2026) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3786583.3786874 · Cited by 2
Are Coding Agents Generating Over-Mocked Tests? An Empirical Study
Hora, A., & Robbes, R. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793362 · Cited by 2
RepoMod-Bench: A Benchmark for Code Repository Modernization via Implementation-Agnostic Testing
Li, X., Ben-Israel, N. I., Raz, Y., Ahmed, B., Serebro, D., & Raux, A. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817463 · Cited by 2
Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection
Ahmed, T., Ganhotra, J., Shinnar, A., & Hirzel, M. J. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3787837 · Cited by 2
DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks
Huang, W., Lee, C., Tng, L., & Ge, S. (2026) · arXiv · doi:10.48550/arxiv.2607.07946 · Cited by 2
Failure as a Process: An Anatomy of CLI Coding Agent Trajectories
Zhao, X., Li, H., Li, S., Zhao, T., Barr, E. T., Sarro, F., & Ye, H. (2026) · arXiv · doi:10.48550/arxiv.2607.09510 · Cited by 17
R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
Jain, N., Singh, J., Shetty, M., Zheng, L., Sen, K., & Stoica, I. (2025) · arXiv · doi:10.48550/arxiv.2504.07164 · Cited by 1
MultiMend: multilingual program repair with context augmentation and multi-hunk patch generation
Gharibi, R., Sadreddini, M. H., & Fakhrahmad, S. M. (2026) · Automated Software Engineering · doi:10.1007/s10515-026-00611-2 · Cited by 7
FuseApplyBench: Multilingual Benchmark for Trustworthy Code Edit Applying Task
Liang, M., Zhang, Q., Zuo, Z., Zheng, S., Chen, D., Jiang, W., & Li, Y. (2025) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3713081.3732929 · Cited by 1
Extracting Fix Ingredients using Language Models
Prenner, J. A., & Robbes, R. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00028 · Cited by 1
An end-to-end approach for fixing concurrency bugs via SHB-based context extractor
Li, Z., Yi, Q., Xiao, K., Ji, Z., & Liang, H. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.112888 · Cited by 1
Yang, Y., Li, C., Han, Z., Li, R., Xu, K., Li, Q., Zhong, W., Shen, Z., Fei, Z., Ge, J., et al. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3764584 · Cited by 7
Ramirez, O., & Ekler, P. (2025) · IEEE International Conference on Cognitive Infocommunications · doi:10.1109/coginfocom66819.2025.11200869 · Cited by 1
Wang, P., Liu, R., Huang, K., & Du, X. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3656819 · Cited by 1
Wired for Reuse: Automating Context-Aware Code Adaptation in IDEs via LLM-Based Agent
Wang, T., Jiang, Y., Dong, C., Zhang, Y., & Liu, H. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00023 · Cited by 2
Self-citationRepoMasterEval: Evaluating Code Completion via Real-World Repositories
Wu, Q., Peng, C., Gao, P., Hu, R., Gan, H., Jiang, B., Tang, J., Deng, Z., Guan, Z., Gao, C., et al. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00304 · Cited by 6
Coverage-Based Harmfulness Testing for LLM Code Transformation
Tan, H., Wang, H., Pressato, D., Xu, Y., & Tan, S. H. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00086 · Cited by 2
Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute
Ma, Y., Li, Y., Dong, Y., Jiang, X., Li, Y., Liu, Y., Cao, R., Chen, J., Huang, F., & Li, B. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00309 · Cited by 24
Madani, M., Neumann, K., Nahhas, A., Chernigovskaya, M., Walia, D. S., & Turowski, K. (2025) · International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm67465.2025.11390990 · Cited by 1
Metadata
- Authors
- Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A.
- Affiliations
- National University of Singapore (SG)
- Venue
- Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis
- Publisher
- ACM
- Published
- 2024-09-11
- Pages
- 1592-1604
- OpenAlex
- W4402442868
- Cited by
- 202
- References
- 34
- Citations by year
- 2026: 123 · 2025: 70 · 2024: 9
- FWCI
- 76.83
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Ministry of Education - Singapore (MOE-MOET32021-0001)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- program improvement, program repair, large language models, code search, spectrum-based fault localization, abstract syntax tree, software maintenance, software evolution, LLM agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
