The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair
Open accessAbstract
During Automated Program Repair (APR), it can be challenging to synthesize correct patches for real-world systems in general-purpose programming languages. Recent Large Language Models (LLMs) have been shown to be helpful “copilots” in assisting developers with various coding tasks, and have also been directly applied for patch synthesis. However, most LLMs treat programs as sequences of tokens, meaning that they are ignorant of the underlying semantics constraints of the target programming language. This results in plenty of statically invalid generated patches, impeding the practicality of the technique. Therefore, we propose Repilot, a framework to further copilot the AI “copilots” (i.e., LLMs) by synthesizing more valid patches during the repair process. Our key insight is that many LLMs produce outputs autoregressively (i.e., token by token), resembling human writing programs, which can be significantly boosted and guided through a Completion Engine. Repilot synergistically synthesizes a candidate patch through the interaction between an LLM and a Completion Engine, which 1) prunes away infeasible tokens suggested by the LLM and 2) proactively completes the token based on the suggestions provided by the Completion Engine. Our evaluation on a subset of the widely-used Defects4j 1.2 and 2.0 datasets shows that Repilot fixes 66 and 50 bugs, respectively, surpassing the best-performing baseline by 14 and 16 bugs fixed. More importantly, Repilot is capable of producing more valid and correct patches than the base LLM when given the same generation budget.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“This approach enhances the compilation rate and correctness of patches while reducing the number of invalid attempts in the generation process.”
From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Wei et al. (2023)
The cited source says what the claim attributes to it.
In the citing paper
“Large Language Models (LLMS) trained on code (LLMCS) have demonstrated significant effectiveness in generating program fixes (Joshi et al. 2023; Xia, Ding, and Zhang 2023; Wei, Xia, and Zhang 2023; Fan et al. 2023; Xia, Wei, and Zhang 2023; Brancas, Manquinho, and Martins 2024).”
From Counterexample Guided Program Repair Using Zero-Shot Learning and MaxSAT-based Fault Localization (2025) · cites this paper as Wei, Xia, and Zhang 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.139 · Cited by 2,824
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495
Defects4J: a database of existing faults to enable controlled testing studies for Java programs
Just, R., Jalali, D., & Ernst, M. D. (2014) · Proceedings of the 2014 International Symposium on Software Testing and Analysis · doi:10.1145/2610384.2628055 · Cited by 1,447
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
GenProg: A Generic Method for Automatic Software Repair
Goues, C. L., Nguyen, T. H., Forrest, S., & Weimer, W. R. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.104 · Cited by 1,084
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Unified Pre-training for Program Understanding and GenerationPreprint
Ahmad, W. U., Chakraborty, S., Ray, B., & Chang, K.-W. (2021) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2103.06333 · Cited by 1,000
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974
Research works printed but not linked: 11
- 10.18653/v1 · DOI
- Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Scott Yih, Luke Zettlemoyer, and Mike Lewis. 2023. InCoder: A Generative Model for Code Infilling and Synthesis. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=hQwb-lbM6EL
- Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie LIU, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2021. GraphCodeBERT: Pre-training Code Representations with Data Flow. In International Conference on Learning Representations. https://openreview.net/forum?id=jLoC4ez43PZ
- Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The Curious Case of Neural Text Degeneration. In International Conference on Learning Representations. https://openreview.net/forum?id=rygGQyrFvH
- Sophia D Kolak, Ruben Martins, Claire Le Goues, and Vincent Josua Hellendoorn. 2022. Patch Generation with Language Models: Feasibility and Scaling Behavior. In Deep Learning for Code Workshop. https://openreview.net/forum?id=rHlzJh_b1-5
- Gabriel Poesia, Alex Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christopher Meek, and Sumit Gulwani. 2022. Synchromesh: Reliable Code Generation from Pre-trained Language Models. In International Conference on Learning Representations. https://openreview.net/forum?id=KmtVD97J43e
- 10.18653/v1 · DOI
- Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (Eds.). 27, Curran Associates, Inc.. https://proceedings.neurips.cc/paper_files/paper/2014/file/a14ac55a4f27472c5d894ec1c3c743d2-Paper.pdf
- 10.5555/3295222.3295349 · DOI
- 10.18653/v1 · DOI
- Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. Curran Associates Inc., Red Hook, NY, USA.
Cited by
Sorted by citations
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029
Software Testing With Large Language Models: Survey, Landscape, and Vision
Wang, J., Huang, Y., Chen, C., Liu, Z., Wang, S., & Wang, Q. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3368208 · Cited by 573
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650
When LLMs meet cybersecurity: a systematic literature review
Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 383
A Survey of Learning-based Automated Program Repair
Zhang, Q., Fang, C., Ma, Y., Sun, W., & Chen, Z. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3631974 · Cited by 200
When Software Security Meets Large Language Models: A Survey
Zhu, X., Zhou, W., Han, Q., Ma, W., Wen, S., & Xiang, Y. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124971 · Cited by 110
Generative AI for self-adaptive systems: State of the art and research roadmap
Li, J., Zhang, M., Li, N., Weyns, D., Jin, Z., & Tei, K. (2024) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3686803 · Cited by 77
Look Before You Leap: An Exploratory Study of Uncertainty Analysis for Large Language Models
Huang, Y., Song, J., Wang, Z., Zhao, S., Chen, H., Juefei-Xu, F., & Ma, L. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3519464 · Cited by 275
Jaffal, N. O., Alkhanafseh, M., & Mohaisen, D. (2025) · AI · doi:10.3390/ai6090216 · Cited by 69
SpecGen: Automated Generation of Formal Program Specifications via Large Language Models
Ma, L., Liu, S., Li, Y., Xie, X., & Bu, L. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00129 · Cited by 179
Prompt Engineering or Fine-Tuning: An Empirical Assessment of LLMs for Code
Shin, J., Tang, C. J., Mohati, T., Nayebi, M., Wang, S., & Hemmati, H. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00082 · Cited by 165
How secure is AI-generated code: a large-scale comparison of large language models
Tihanyi, N., Bisztray, T., Ferrag, M. A., Jain, R., & Cordeiro, L. C. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10590-1 · Cited by 95
RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair
Silva, A., Fang, S., & Monperrus, M. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3581062 · Cited by 128
Shi, J., Yang, Z., & Lo, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708525 · Cited by 43
Software engineering education in the era of conversational AI: current trends and future directions
Sengul, C., Neykova, R., & Destefanis, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1436350 · Cited by 63
Teaching Code LLMs to Use Autocompletion Tools in Repository-Level Code Generation
Wang, C., Zhang, J., Feng, Y., Li, T., Sun, W., Liu, Y., & Peng, X. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3714462 · Cited by 40
“Additionally, ToolFormer and ToolCoder are unable to handle scenarios where the tools return multiple candidates.”
Agents in software engineering: survey, landscape, and vision
Wang, Y., Zhong, W., Huang, Y., Shi, E., Yang, M., Chen, J., Li, H., Ma, Y., Wang, Q., & Zheng, Z. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00544-2 · Cited by 84
Can Language Models Employ the Socratic Method? Experiments with Code Debugging
Al-Hossami, E., Bunescu, R., Smith, J., & Teehan, R. (2024) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3626252.3630799 · Cited by 46
Comprehensive Fine-Tuning Large Language Models of Code for Automated Program Repair
Huang, K., Zhang, J., Bao, X., Wang, X., & Liu, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3532759 · Cited by 30
Reasoning Runtime Behavior of a Program with LLM: How Far are We?
Chen, J., Pan, Z., Hu, X., Li, Z., Li, G., & Xia, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00012 · Cited by 87
Zhang, L., Li, K., Sun, K., Wu, D., Liu, Y., Tian, H., & Liu, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3590108 · Cited by 61
Self-citationThe use of large language models for program repair
Zubair, F., Al-Hitmi, M., & Catal, C. (2024) · Computer Standards & Interfaces · doi:10.1016/j.csi.2024.103951 · Cited by 23
Template-Guided Program Repair in the Era of Large Language Models
Huang, K., Zhang, J., Meng, X., & Liu, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00030 · Cited by 30
One Size Does Not Fit All: Multi-granularity Patch Generation for Better Automated Program Repair
Lin, B., Wang, S., Wen, M., Chen, L., & Mao, X. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680381 · Cited by 20
LPR: Large Language Models-Aided Program Reduction
Zhang, M., Tian, Y., Xu, Z., Dong, Y., Tan, S. H., & Sun, C. P. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3652126 · Cited by 21
A Survey on Failure Analysis and Fault Injection in AI Systems
Yu, G., Tan, G., Huang, H., Zhang, Z., Chen, P., Natella, R., Zheng, Z., & Lyu, M. R.-T. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3732777 · Cited by 38
Licoeval: Evaluating LLMs on License Compliance in Code Generation
Xu, W., Gao, K., He, H., & Zhou, M. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00052 · Cited by 39
Cheng, Y., Bajaber, O., Tsegai, S. A., Song, D. X., & Gao, P. (2025) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp63326.2025.00057 · Cited by 45
An Empirical Study on the Code Refactoring Capability of Large Language Models
Cordeiro, J., Noei, S., & Zou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3801158 · Cited by 77
Aligning the Objective of LLM-Based Program Repair
Xu, J., Fu, Y., Tan, S. H., & He, P. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00169 · Cited by 47
Evolving Paradigms in Automated Program Repair: Taxonomy, Challenges, and Opportunities
Huang, K., Xu, Z., Yang, S., Sun, H., Li, X., Yan, Z., & Zhang, Y. (2024) · ACM Computing Surveys · doi:10.1145/3696450 · Cited by 16
Type-Constrained Code Generation with Language Models
Mündler-Sasahara, N., He, J.-X., Wang, H., Sen, K., Song, D., & Vechev, M. T. (2025) · Proc. ACM Program. Lang. · doi:10.1145/3729274 · Cited by 77
bjCnet: A contrastive learning-based framework for software defect prediction
Han, J., Huang, C., & Liu, J. (2024) · Computers & Security · doi:10.1016/j.cose.2024.104024 · Cited by 12
Clozemaster: Fuzzing Rust Compiler by Harnessing Llms for Infilling Masked Real Programs
Gao, H., Yang, Y., Sun, M., Wu, J., Zhou, Y., & Xu, B. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00175 · Cited by 20
LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-Based Code Completion
Wang, C., Huang, K., Zhang, J., Feng, Y., Zhang, L., Liu, Y., & Peng, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00245 · Cited by 17
Self-citationBenchmarking Automated Program Repair: An Extensive Study on Both Real-World and Artificial Bugs
Ouyang, Y.-C., Yang, J., & Zhang, L. (2024) · International Symposium on Software Testing and Analysis · doi:10.1145/3650212.3652140 · Cited by 31
Self-citationStatically Contextualizing Large Language Models with Typed Holes
Blinn, A., Li, X., Kim, J. H., & Omar, C. (2024) · Proceedings of the ACM on Programming Languages · doi:10.1145/3689728 · Cited by 10
Challenges and Paths Towards AI for Software EngineeringPreprint
Gu, A., Jain, N., Li, W., Shetty, M., Shao, Y., Li, Z., Yang, D., Ellis, K., Sen, K., & Solar-Lezama, A. (2025) · Qeios · doi:10.32388/vv1661 · Cited by 40
History-Driven Fuzzing for Deep Learning Libraries
Harzevili, N. S., Mohajer, M. M., Wei, M., Pham, H. V., & Wang, S. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3688838 · Cited by 11
Unearthing Gas-Wasting Code Smells in Smart Contracts With Large Language Models
Jiang, J., Li, Z., Qin, H., Jiang, M., Luo, X., Wu, X., Wang, H., Tang, Y., Qian, C., & Chen, T. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3491578 · Cited by 10
AdverIntent-Agent: Adversarial Reasoning for Repair Based on Inferred Program Intent
Ye, H., Yang, A. Z. H., Hu, C., Wang, Y., Zhang, T., & Goues, C. L. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728939 · Cited by 11
LLM-enhanced evolutionary test generation for untyped languages
Yang, R., Xu, X., & Wang, R. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00496-7 · Cited by 9
Your Fix Is My Exploit: Enabling Comprehensive DL Library API Fuzzing with Large Language Models
Zhang, K., Wang, S., Han, J., Zhu, X., Li, X., Wang, S., & Wen, S. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00041 · Cited by 17
Towards Secure Code Generation With LLMs: A Study on Common Weakness Enumeration
Zhao, J., Sun, Y., Huang, C., Liu, C., Guan, Y., Zeng, Y., & Liu, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3619281 · Cited by 17
Position: Vibe Coding Needs Vibe Reasoning: Improving Vibe Coding with Formal Verification
Mitchell, J., & Shaaban, Y. (2025) · Proceedings of the 1st ACM SIGPLAN International Workshop on Language Models and Programming Languages · doi:10.1145/3759425.3763390 · Cited by 8
ZS4C: Zero-Shot Synthesis of Compilable Code for Incomplete Code Snippets Using LLMs
Kabir, A., Wang, S., Tian, Y., Chen, T.-H. P., Asaduzzaman, M., & Zhang, W. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3702979 · Cited by 13
ReAPR: Automatic program repair via retrieval-augmented large language models
Liu, Z., Du, X., & Liu, H. (2025) · Software Quality Journal · doi:10.1007/s11219-025-09728-1 · Cited by 10
Can LLMs Generate User Stories and Assess Their Quality?
Quattrocchi, G., Pasquale, L., Spoletini, P., & Baresi, L. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3670612 · Cited by 7
Towards Neural Synthesis for SMT-Assisted Proof-Oriented Programming
Chakraborty, S., Ebner, G., Bhat, S., Fakhoury, S., Fatima, S., Lahiri, S. K., & Swamy, N. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00002 · Cited by 29
Enhancing AI-based Generation of Software Exploits with Contextual Information
Liguori, P., Improta, C., Natella, R., Čukić, B., & Cotroneo, D. (2024) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre62328.2024.00027 · Cited by 6
Exploring Large Language Models in Information Systems: A Survey
Bianchini, F., & Marinacci, M. (2025) · Springer eBooks · doi:10.1007/978-3-031-92285-5_5 · Cited by 7
Detecting, Creating, Repairing, and Understanding Indivisible Multi-Hunk Bugs
Xin, Q., Wu, H., Tang, J., Liu, X., Reiss, S. P., & Xuan, J. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660828 · Cited by 6
ROSE: An IDE-Based Interactive Repair Framework for Debugging
Reiss, S. P., Wei, X., Yuan, J. P., & Xin, Q. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3705306 · Cited by 6
RetypeR: Integrated Retrieval-based Automatic Program Repair for Python Type Errors
Hao, S., Shi, X., & Liu, H. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00028 · Cited by 5
Zhang, Y., Jin, Z., Xing, Y., Li, G., Liu, F., Zhu, J., Dou, W., & Wei, J. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3718739 · Cited by 28
Learning-Based Automated Program Repair: A Systematic Literature Review
Chen, Q., Li, D., Zhao, M., Wong, W. E., & Li, H. (2025) · Complex System Modeling and Simulation · doi:10.23919/csms.2025.0004 · Cited by 4
RefactorGPT: a ChatGPT-based multi-agent framework for automated code refactoring
Karabıyık, M. A. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.3257 · Cited by 4
Lin, X., Xie, Q., Zhao, B., Tian, Y., Zonouz, S., Ruan, N., Li, J., Beyah, R., & Ji, S. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00091 · Cited by 4
Integrating Various Software Artifacts for Better LLM-based Bug Localization and Program Repair
Feng, Q., Ma, X., Sheng, J., Feng, Z., Song, W., & Liang, P. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3770581 · Cited by 4
Zhong, R., Li, Y., Yu, G., Gu, W., Kuang, J., Huo, Y., & Lyu, M. R.-T. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3773287 · Cited by 11
Counterexample Guided Program Repair Using Zero-Shot Learning and MaxSAT-based Fault Localization
Orvalho, P., Janota, M., & Manquinho, V. (2025) · arXiv · doi:10.48550/arxiv.2502.07786 · Cited by 4
DR.FIX: Automatically Fixing Data Races at Industry Scale
Behrang, F., Zhang, Z., Saioc, G.-V., Liu, P., & Chabbi, M. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3729265 · Cited by 6
Boosting Redundancy-Based Automated Program Repair by Fine-Grained Pattern Mining
Jiang, J., Li, F., Zhao, Z., Ye, Z., Liu, M., Wang, B., Zhang, H., & Chen, J. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00018 · Cited by 3
Code Generation of Smart Contracts with LLMs: A Case Study on Hyperledger Fabric
Olivieri, L., Beste, D., Negrini, L., Schönherr, L., Ciná, A. E., & Ferrara, P. (2025) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre66568.2025.00034 · Cited by 3
AI-driven self-healing across the edge–cloud continuum: A systematic literature review
Bonilla, L., Díaz-de-Arcaya, J., Aguirre-Usandizaga, J., & Almeida, A. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108232 · Cited by 4
Rethinking the Evaluation of Secure Code Generation
Dai, S.-C., Xu, J. L., & Tao, G. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773217 · Cited by 36
SOLAR: Illuminating LLM performance in API discovery and service ranking for edge AI and IoT
Al‐Masri, E., & Subramanian, I. N. (2025) · Internet of Things · doi:10.1016/j.iot.2025.101630 · Cited by 3
Can Large Language Models Discover Metamorphic Relations? A Large-Scale Empirical Study
Zhang, J., Sun, C., Liu, H., & Dong, S. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00011 · Cited by 5
Automated Refactoring of Non-Idiomatic Python Code: A Differentiated Replication with LLMS
Midolo, A., & Penta, M. D. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00020 · Cited by 3
When Automated Program Repair Meets Regression Testing—An Extensive Study on Two Million Patches
Lou, Y., Yang, J., Benton, S., Hao, D., Tan, L., Chen, Z., Zhang, L. H., & Zhang, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3672450 · Cited by 2
Self-citationA Comprehensive Study on Code Completion for Large Language Models
Cai, Y. (2025) · International Conference on Advanced Algorithms and Control Engineering (ICAACE) · doi:10.1109/icaace65325.2025.11020128 · Cited by 2
Element-Based Automated DNN Repair with Fine-Tuned Masked Language Model
Wang, X., Zhang, M., Meng, X., Zhang, J., Liu, Y., & Hu, C. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715716 · Cited by 2
Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Repair
Huang, K., Zhang, J., Xie, X., & Chen, C. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00100 · Cited by 2
Robustness Analysis of Watermarking Techniques for LLM-Generated Code
Cao, D., Liu, S., Zhang, J., & Xiang, Y. (2025) · IEEE Networking Letters · doi:10.1109/lnet.2025.3616325 · Cited by 2
Exploring Direct Instruction and Summary-Mediated Prompting in LLM-Assisted Code Modification
Tang, N., Smith, E., Huang, Y., Collin, M., & Li, T. J.-J. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00017 · Cited by 4
Reading between the Lines: Context-Aware AI-based generation of software exploits
Improta, C., Liguori, P., Natella, R., Čukić, B., & Cotroneo, D. (2026) · Empirical Software Engineering · doi:10.1007/s10664-025-10796-x · Cited by 2
Clean Code, Better Models: Enhancing LLM Performance with Smell-Cleaned Dataset
Xue, Z., Zhang, X., Gao, Z., Hu, X., Gao, S., Xia, X., & Li, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3793252 · Cited by 3
Unveiling the power of code pre-trained models in automated program repair: A systematic review
Zhan, S., Wang, X. J., Wei, D., Xiang, X., Lin, Y., Wu, H., & Lu, J. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113015 · Cited by 2
MoonBit: Explore the Design of an AI-Friendly Programming Language
Fei, H., Zhang, Y., Zhang, H., Wang, Y., & Liu, Q. (2024) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1145/3643795.3648376 · Cited by 2
Leveraging Search-Based and Pre-Trained Code Language Models for Automated Program Repair
Lijzenga, O., Moghadam, I. H., & Zaytsev, V. (2025) · ACM Symposium on Applied Computing · doi:10.1145/3672608.3707774 · Cited by 2
Porting Software Libraries to OpenHarmony: Transitioning from TypeScript or JavaScript to ArkTS
Zhou, B., Shi, J., Wang, Y., Li, L., Li, T. O., Yu, H., & Zhu, Z. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728941 · Cited by 2
MultiMend: multilingual program repair with context augmentation and multi-hunk patch generation
Gharibi, R., Sadreddini, M. H., & Fakhrahmad, S. M. (2026) · Automated Software Engineering · doi:10.1007/s10515-026-00611-2 · Cited by 7
Yang, Y., Li, C., Han, Z., Li, R., Xu, K., Li, Q., Zhong, W., Shen, Z., Fei, Z., Ge, J., et al. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3764584 · Cited by 7
Empirical Evaluation of LLMs for Automated Program Fault Localisation
Liu, Y., Wang, X., Liu, H., Huang, R., & Wu, Y. (2025) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c65679.2025.00063 · Cited by 1
Fu, A., Xu, P., Li, J., Kuang, B., & Gao, Y. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3618407 · Cited by 4
Multi-Location Software Model Completion
Welter, A., Tinnes, C., & Apel, S. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3787798 · Cited by 1
RepoMasterEval: Evaluating Code Completion via Real-World Repositories
Wu, Q., Peng, C., Gao, P., Hu, R., Gan, H., Jiang, B., Tang, J., Deng, Z., Guan, Z., Gao, C., et al. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00304 · Cited by 6
A Systematic Literature Review of Parameter-Efficient Fine-Tuning for Large Code Models
Afrin, S., Haque, M. Z., & Mastropaolo, A. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3796522 · Cited by 12
Iter-T: ITERative Test Suite Generation for Automated Program Repair
Godio, A., Brida, S. G., Regis, G., Bagheri, H., Nguyen, T. H., Aguirre, N., & Frias, M. F. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3671416 · Cited by 1
TLR: Codebase-Level C Memory Management Error Repair with Large Language Models
Cheng, X. Y., Guo, Z., Huo, H., & Sui, Y. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797085 · Cited by 1
Evaluating Spectrum-Based Fault Localization on Deep Learning Libraries
Yan, M., Chen, J., Jiang, T., Jiang, J., & Wang, Z. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3552622 · Cited by 1
PReMM: LLM-Based Program Repair for Multi-method Bugs via Divide and Conquer
Xie, L., Li, Z., Pei, Y., Wen, Z., Liu, K., Zhang, T., & Li, X. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763097 · Cited by 7
A Survey on Code Pre-Trained Models for Neural Program RepairPreprint
Zhan, S., Wang, X., Wei, D., & Cao, X. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4831589
Exploration Study About LLM with Proper Prompt in Automated Program Repair
Liang, H., Gao, C., & Jia, Y. F. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4509-1_28
A Comprehensive Study of Bug Characteristics on Foundation Language Models
Han, J., Wang, G., Liu, J., Bao, L., Hu, X., Wei, J., & Deng, S. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00037
Zhan, S., Wang, X., Wei, D., Cao, X., & Chen, B. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112590
Zhang, H., Yan, Q., Min, W., Zhang, C., Kuang, L., & Xia, Y. (2025) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws67624.2025.00123
Visual Sketchbook: Enhancing Chart-to-Code Generation via Reflective Refinement
Wang, J., Xi, Z., He, W., Zhu, D., Dou, S., Gui, T., & Zhang, Q. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3346-6_12
FlowRepair: Search-based automated program repair of CPS controllers modeled in Simulink-Stateflow
Arrieta, A., Valle, P., & Ali, S. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2025.108010 · Cited by 6
Developer Prompts in Practice: An Empirical Study of Bias, Security, and Optimization
Rzig, D. E., Paul, D. J., Pister, K. S., Henkel, J. M., & Hassan, F. (2025) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem64174.2025.00067
Metadata
- Authors
- Wei, Y., Xia, C. S., & Zhang, L.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Venue
- Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2023-11-30
- Pages
- 172-184
- OpenAlex
- W4386436496
- Cited by
- 115
- References
- 45
- Citations by year
- 2026: 27 · 2025: 62 · 2024: 24 · 2023: 2
- FWCI
- 22.26
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- NSF (National Science Foundation) (CCF-2131943,CCF-2141474)
- Kwai Inc.
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Keywords
- automated program repair, large language models, patch synthesis, token pruning, autoregressive generation, Defects4J, bug fixes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
