The DOI matches its DataCite record; 34 of the 35 references it cites were found in a registry or an index, and 1 are not checked yet.
AutoCodeRover: Autonomous Program Improvement
Open accessAbstract
Researchers have made significant progress in automating the software development process in the past decades. Recent progress in Large Language Models (LLMs) has significantly impacted the development process, where developers can use LLM-based programming assistants to achieve automated coding. Nevertheless, software engineering involves the process of program improvement apart from coding, specifically to enable software maintenance (e.g. bug fixing) and software evolution (e.g. feature additions). In this paper, we propose an automated approach for solving GitHub issues to autonomously achieve program improvement. In our approach called AutoCodeRover, LLMs are combined with sophisticated code search capabilities, ultimately leading to a program modification or patch. In contrast to recent LLM agent approaches from AI researchers and practitioners, our outlook is more software engineering oriented. We work on a program representation (abstract syntax tree) as opposed to viewing a software project as a mere collection of files. Our code search exploits the program structure in the form of classes/methods to enhance LLM's understanding of the issue's root cause, and effectively retrieve a context via iterative search. The use of spectrum-based fault localization using tests, further sharpens the context, as long as a test-suite is available. Experiments on SWE-bench-lite (300 real-life GitHub issues) show increased efficacy in solving GitHub issues (19% on SWE-bench-lite), which is higher than the efficacy of the recently reported SWE-agent. In addition, AutoCodeRover achieved this efficacy with significantly lower cost (on average, $0.43 USD), compared to other baselines. We posit that our workflow enables autonomous software engineering, where, in future, auto-generated code from LLMs can be autonomously improved.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Single-harness systems such as SWE-agent [30], AutoCodeRover [38], OpenHands [29], and mini-SWE-agent [32] report per-system numbers, but their scaffolds, prompts, budgets, and termination policies vary with the system, making cross-system differences hard to attribute to harness design.”
From Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks (2026) · cites this paper as [38]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Each of these reports per-harness resolved rates but does not vary the harness as an experimental axis: prompt, scaffold, runtime, and termination policy are bundled together with each release.”
From Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks (2026) · cites this paper as [38]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Despite its challenging nature, performance on SWE-bench has been steadily increasing (Zhang et al., 2024; factory.ai,2024).”
From MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering (2024) · cites this paper as Zhang et al., 2024
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“AutoCodeRover [38] adds code-aware retrieval to the agent loop.”
From Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks (2026) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Survey on Software Fault Localization
Wong, W. E., Gao, R., Li, Y., Abreu, R., & Wotawa, F. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2521368 · Cited by 1,110
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Competition-Level Code Generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.07814 · Cited by 1,043
On the Accuracy of Spectrum-based Fault Localization
Abreu, R., Zoeteweij, P., & van Gemund, A. V. (2007) · Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007) · doi:10.1109/taic.part.2007.13 · Cited by 939
Visualization of test information to assist fault localization
Jones, J. A., Harrold, M. J., & Stasko, J. (2002) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/581339.581397 · Cited by 925
Symbolic execution for software testing
Cadar, C., & Sen, K. (2013) · Communications of the ACM · doi:10.1145/2408776.2408795 · Cited by 785
Automatically finding patches using genetic programming
Weimer, W. R., Nguyen, T. H., Goues, C. L., & Forrest, S. (2009) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2009.5070536 · Cited by 768
Böhme, M., Pham, V.-T., Nguyen, M.-D., & Roychoudhury, A. (2017) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3133956.3134020 · Cited by 759
Self-citationMechtaev, S., Yi, J., & Roychoudhury, A. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884807 · Cited by 515
Self-citationAutomated Program Repair in the Era of Large Pre-trained Language Models
Xia, C. S., Wei, Y., & Zhang, L. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00129 · Cited by 433
SemFix: Program repair via semantic analysis
Nguyen, H. D. T., Qi, D., Roychoudhury, A., & Chandra, S. (2013) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2013.6606623 · Cited by 397
Self-citationIs the cure worse than the disease? overfitting in automated program repair
Smith, E. K., Barr, E. T., Le Goues, C., & Brun, Y. (2015) · Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2786805.2786825 · Cited by 323
Constructing the Call Graph of a Program
Ryder, B. G. (1979) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1979.234183 · Cited by 305
VulRepair: a T5-based automated software vulnerability repair
Fu, M. C., Tantithamthavorn, C. K., Le, T., Nguyen, V., & Phung, D. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549098 · Cited by 278
Automated Repair of Programs from Large Language Models
Fan, Z., Gao, X., Mirchev, M., Roychoudhury, A., & Tan, S. H. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00128 · Cited by 256
Self-citationGetafix: learning to fix bugs automatically
Bader, J., Scott, A., Pradel, M., & Chandra, S. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3360585 · Cited by 238
A syntax-guided edit decoder for neural program repair
Zhu, Q., Sun, Z., Xiao, Y.-A., Zhang, W., Yuan, K. F., Xiong, Y., & Zhang, L. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468544 · Cited by 222
Examining Zero-Shot Vulnerability Repair with Large Language Models
Pearce, H. A., Tan, B., Ahmad, B., Karri, R., & Dolan-Gavitt, B. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179420 · Cited by 206
Impact of Code Language Models on Automated Program Repair
Jiang, N., Liu, K., Lutellier, T., & Tan, L. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00125 · Cited by 202
InferFix: End-to-End Program Repair with LLMs
Jin, M., Shahriar, S., Tufano, M., Shi, X., Lu, S., Sundaresan, N., & Svyatkovskiy, A. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3613892 · Cited by 191
Fuzzing: Challenges and Reflections
Böhme, M., Cadar, C., & Roychoudhury, A. (2020) · IEEE Software · doi:10.1109/ms.2020.3016773 · Cited by 161
Self-citationRAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair
Wang, W.-S., Wang, Y., Joty, S. R., & Hoi, S. (2023) · ESEC/SIGSOFT FSE · doi:10.1145/3611643.3616256 · Cited by 151
SapFix: Automated End-to-End Repair at Scale
Marginean, A., Bader, J., Chandra, S., Harman, M., Jia, Y., Mao, K., Mols, A., & Scott, A. (2019) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00039 · Cited by 144
BugRedux: Reproducing field failures for in-house debugging
Jin, W., & Orso, A. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227168 · Cited by 122
iFixR: bug report driven program repair
Koyuncu, A., Liu, K., Bissyandé, T. F., Kim, D., Monperrus, M., Klein, J., & Traon, Y. L. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3338935 · Cited by 100
Can OpenAI's codex fix bugs?: an evaluation on QuixBugs
Prenner, J. A., Babii, H., & Robbes, R. (2022) · Proceedings of the Third International Workshop on Automated Program Repair · doi:10.1145/3524459.3527351 · Cited by 99
PyCG: Practical Call Graph Generation in Python
Salis, V., Sotiropoulos, T., Λουρίδας, Π., Spinellis, D. D., & Mitropoulos, D. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00146 · Cited by 93
Keller, F., Grunske, L., Heiden, S., Filieri, A., van Hoorn, A., & Lo, D. (2017) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs.2017.22 · Cited by 83
Gao, X. Y., Mechtaev, S., & Roychoudhury, A. (2019) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3293882.3330558 · Cited by 61
Self-citation2024 IEEE Symposium on Security and Privacy (SP)
(2024) · doi:10.1109/sp54263.2024 · Cited by 61
Better Automatic Program Repair by Using Bug Reports and Tests Together
Motwani, M., & Brun, Y. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00109 · Cited by 37
PyTy: Repairing Static Type Errors in Python
Chow, Y. W., Grazia, L. D., & Pradel, M. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639184 · Cited by 33
Language Server Protocol and Implementation
Gunasinghe, N., & Marcus, N. (2021) · Apress eBooks · doi:10.1007/978-1-4842-7792-8 · Cited by 15
CrossFix: Resolution of GitHub issues via similar bugs recommendation
Tan, S. H., Li, Z., & Yan, L. (2023) · Journal of Software Evolution and Process · doi:10.1002/smr.2554 · Cited by 5
Ovalle, B. S. T. (2022) · Encuentro Internacional de Educación en Ingeniería · doi:10.26507/paper.2300 · Cited by 4
Research works printed but not linked: 12
- Marcel Böhme, Cristian Cadar, and Abhik Roychoudhury. 2021. Fuzzing: Chal- lenges and Reflections. IEEE Software 38, 3 (2021).
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluat- ing Large Language Models Trained on Code. arXiv.org abs/2107.03374 (7 2021). arXiv:2107.03374 https://arxiv.org/abs/2107.03374 · arXiv
- Claire Le Goues, Michael Pradel, and Abhik Roychoudhury. 2019. Automated program repair. Commun. ACM 62 (11 2019), 56-65. Issue 12.
- Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2024. Leaderboard results on SWE-bench. Retrieved April 8, 2024 from https://www.swebench.com/
- Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench: Can Language Models Resolve Real-world Github Issues?. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=VTF8yNQM66
- Wuxia Jin et al. 2024. PyAnalyzer: An Effective and Practical Approach for De- pendency Extraction from Python Code. In International Conference on Software Engineering (ICSE).
- Thibaud Lutellier, Hung Viet Pham, Lawrence Pang, Yitong Li, Moshi Wei, and Lin Tan. 2020. CoCoNuT: combining context-aware neural translation models us- ing ensemble for program repair., In ISSTA '20: 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, Virtual Event, USA, July 18-22, 2020, Sarfraz Khurshid and Corina S. Pasareanu (Eds.). International Symposium on Software Testing and Analysis, 101-114.
- Sergey Mechtaev, Jooyong Yi, and Abhik Roychoudhury. 2016. Angelix: scalable multiline program patch synthesis via symbolic analysis., In Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14-22, 2016, Laura K. Dillon, Willem Visser, and Laurie Williams (Eds.). International Conference on Software Engineering, 691-701.
- Yannic Noller, Ridwan Shariffdeen, Xiang Gao, and Abhik Roychoudhury. 2022. Trust Enhancement Issues in Program Repair. In IEEE/ACM 44th International Conference on Software Engineering (ICSE).
- David Williams, James Callan, Serkan Kirbas, Sergey Mechtaev, Justyna Petke, Thomas Prideaux-Ghee, and Federica Sarro. 2023. User-Centric Deployment of Automated Program Repair at Bloomberg. arXiv preprint arXiv:2311.10516 (2023). · arXiv
- W Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A survey on software fault localization. IEEE Transactions on Software Engineering 42, 8 (2016), 707-740.
- John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent: Agent-Computer In- terfaces Enable Automated Software Engineering. arXiv:2405.15793 [cs.SE] · arXiv
Cited by
Sorted by citations
A Survey on Large Language Models for Code Generation
Jiang, J., Wang, F., Shen, J., Kim, S., & Kim, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3747588 · Cited by 251
He, J., Treude, C., & Lo, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3712003 · Cited by 246
Automated Program Repair via Conversation: Fixing 162 out of 337 Bugs for $0.42 Each using ChatGPT
Xia, C. S., & Zhang, L. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680323 · Cited by 468
(2025) · doi:10.1109/icse55347.2025.00157 · Cited by 157
Demystifying LLM-Based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715754 · Cited by 849
Metadata
- Authors
- Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A.
- Affiliations
- National University of Singapore (SG)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- Pages
- 1592-1604
- arXiv
- 2404.05427
- OpenAlex
- W4402442868
- Cited by
- 202
- References
- 34
- Citations by year
- 2026: 123 · 2025: 70 · 2024: 9
- FWCI
- 76.83
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Ministry of Education, India
- Bộ Giáo dục và Ðào tạo
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- program improvement, program repair, large language models, code search, spectrum-based fault localization, abstract syntax tree, software maintenance, software evolution, LLM agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
