Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair
Open accessAbstract
During Automated Program Repair (APR), it can be challenging to synthesize correct patches for real-world systems in general-purpose programming languages. Recent Large Language Models (LLMs) have been shown to be helpful “copilots” in assisting developers with various coding tasks, and have also been directly applied for patch synthesis. However, most LLMs treat programs as sequences of tokens, meaning that they are ignorant of the underlying semantics constraints of the target programming language. This results in plenty of statically invalid generated patches, impeding the practicality of the technique. Therefore, we propose Repilot, a framework to further copilot the AI “copilots” (i.e., LLMs) by synthesizing more valid patches during the repair process. Our key insight is that many LLMs produce outputs autoregressively (i.e., token by token), resembling human writing programs, which can be significantly boosted and guided through a Completion Engine. Repilot synergistically synthesizes a candidate patch through the interaction between an LLM and a Completion Engine, which 1) prunes away infeasible tokens suggested by the LLM and 2) proactively completes the token based on the suggestions provided by the Completion Engine. Our evaluation on a subset of the widely-used Defects4j 1.2 and 2.0 datasets shows that Repilot fixes 66 and 50 bugs, respectively, surpassing the best-performing baseline by 14 and 16 bugs fixed. More importantly, Repilot is capable of producing more valid and correct patches than the base LLM when given the same generation budget.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Repilot enhances the compilation rate and correctness of patches while reducing the number of invalid attempts in the generation process.
“This approach enhances the compilation rate and correctness of patches while reducing the number of invalid attempts in the generation process.”
From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Wei et al. (2023)
Large Language Models trained on code have demonstrated significant effectiveness in generating program fixes.
“Large Language Models (LLMS) trained on code (LLMCS) have demonstrated significant effectiveness in generating program fixes (Joshi et al. 2023; Xia, Ding, and Zhang 2023; Wei, Xia, and Zhang 2023; Fan et al. 2023; Xia, Wei, and Zhang 2023; Brancas, Manquinho, and Martins 2024).”
From Counterexample Guided Program Repair Using Zero-Shot Learning and MaxSAT-based Fault Localization (2025) · cites this paper as Wei, Xia, and Zhang 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.139 · Cited by 2,824 · see citing works
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495 · see citing works
Defects4J: a database of existing faults to enable controlled testing studies for Java programs
Just, R., Jalali, D., & Ernst, M. D. (2014) · Proceedings of the 2014 International Symposium on Software Testing and Analysis · doi:10.1145/2610384.2628055 · Cited by 1,447 · see citing works
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · see citing works
GenProg: A Generic Method for Automatic Software Repair
Goues, C. L., Nguyen, T. H., Forrest, S., & Weimer, W. R. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.104 · Cited by 1,084 · see citing works
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043 · see citing works
Unified Pre-training for Program Understanding and GenerationPreprint
Ahmad, W. U., Chakraborty, S., Ray, B., & Chang, K.-W. (2021) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2103.06333 · Cited by 1,000 · see citing works
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974 · see citing works
Unified Pre-training for Program Understanding and Generation
Ahmad, W., Chakraborty, S., Ray, B., & Chang, K.-W. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.211 · Cited by 642 · see citing works
SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair
Chen, Z.-M., Kommrusch, S., Tufano, M., Pouchet, L., Poshyvanyk, D., & Martin, M. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2940179 · Cited by 585 · see citing works
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., & Hellendoorn, V. J. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534862 · Cited by 540 · see citing works
Mechtaev, S., Yi, J., & Roychoudhury, A. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884807 · Cited by 515 · see citing works
CodeSearchNet Challenge: Evaluating the State of Semantic Code Search
Husain, H., Wu, H.-H., Gazit, T., Allamanis, M., & Brockschmidt, M. (2019) · arXiv · doi:10.48550/arxiv.1909.09436 · Cited by 483 · see citing works
CoCoNuT: combining context-aware neural translation models using ensemble for program repair
Lutellier, T., Pham, H., Pang, L., Li, Y., Wei, M., & Tan, L. (2020) · International Symposium on Software Testing and Analysis · doi:10.1145/3395363.3397369 · Cited by 463 · see citing works
CodeT5+: Open Code Large Language Models for Code Understanding and Generation
Wang, Y., Le, H., Gotmare, A. D., Bui, N. D. Q., Li, J., & Hoi, S. C. H. (2023) · arXiv · doi:10.48550/arxiv.2305.07922 · Cited by 459
Grounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2023) · Proceedings of the ACM on Programming Languages · doi:10.1145/3586030 · Cited by 436 · cited 1× · see citing works
“Recent advances in Natural Language Processing (NLP) have empowered the idea of using Large Language Models (LLMs) that are pre-trained on enormous corpora of natural language and code for various code-related tasks [4, 5, 10, 38, 71].”
BackgroundAutomated Program Repair in the Era of Large Pre-trained Language Models
Xia, C. S., Wei, Y., & Zhang, L. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00129 · Cited by 433 · see citing works
Self-citationStaged program repair with condition synthesis
Long, F., & Rinard, M. (2015) · ESEC/SIGSOFT FSE · doi:10.1145/2786805.2786811 · Cited by 416 · see citing works
Liu, J., Xia, C. S., Wang, Y., & Zhang, L. (2023) · arXiv · doi:10.48550/arxiv.2305.01210 · Cited by 390 · see citing works
Self-citationProgram Synthesis with Large Language Models
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07732 · Cited by 388 · see citing works
An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation
Tufano, M., Watson, C., Bavota, G., Penta, M. D., White, M., & Poshyvanyk, D. (2019) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3340544 · Cited by 370 · see citing works
Le, X.-B. D., Lo, D., & Goues, C. L. (2016) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2016.76 · Cited by 341 · see citing works
An Analysis of the Automatic Bug Fixing Performance of ChatGPT
Sobania, D., Briesch, M., Hanna, C., & Petke, J. (2023) · 2023 IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1109/apr59189.2023.00012 · Cited by 338 · see citing works
Context-aware patch generation for better automated program repair
Wen, M., Chen, J., Wu, R., Hao, D., & Cheung, S.-C. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180233 · Cited by 337 · see citing works
CURE: Code-Aware Neural Machine Translation for Automatic Program Repair
Jiang, N., Lutellier, T., & Tan, L. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00107 · Cited by 336 · see citing works
Automatic Software Repair: A Survey
Gazzola, L., Micucci, D., & Mariani, L. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2755013 · Cited by 336 · see citing works
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
Nijkamp, E., Pang, B., Hayashi, H., Tu, L., Wang, H., Zhou, Y., Savarese, S., & Xiong, C. (2022) · arXiv · doi:10.48550/arxiv.2203.13474 · Cited by 330 · see citing works
Shaping program repair space with existing patches and similar code
Jiang, J., Xiong, Y., Zhang, H., Gao, Q., & Chen, X. (2018) · Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3213846.3213871 · Cited by 329 · see citing works
TBar: revisiting template-based automated program repair
Liu, K., Koyuncu, A., Kim, D., & Bissyandé, T. F. (2019) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3293882.3330577 · Cited by 326 · see citing works
Deng, Y., Xia, C. S., Peng, H., Yang, C., & Zhang, L. (2023) · Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3597926.3598067 · Cited by 326 · see citing works
Self-citationCodeGen2: Lessons for Training LLMs on Programming and Natural LanguagesPreprint
Nijkamp, E., Hayashi, H., Xiong, C.-M., Savarese, S., & Zhou, Y.-B. (2023) · arXiv.org · doi:10.48550/arxiv.2305.02309 · Cited by 253 · see citing works
Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset
Martínez, M., Durieux, T., Sommerard, R., Xuan, J., & Monperrus, M. (2016) · Empirical Software Engineering · doi:10.1007/s10664-016-9470-4 · Cited by 249 · see citing works
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · arXiv · doi:10.48550/arxiv.2002.08155 · Cited by 243 · see citing works
ASTOR: a program repair library for Java (demo)
Martínez, M., & Monperrus, M. (2016) · International Symposium on Software Testing and Analysis · doi:10.1145/2931037.2948705 · Cited by 240 · see citing works
Less training, more repairing please: revisiting automated program repair via zero-shot learning
Xia, C. S., & Zhang, L. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549101 · Cited by 231 · see citing works
Self-citationA syntax-guided edit decoder for neural program repair
Zhu, Q., Sun, Z., Xiao, Y.-A., Zhang, W., Yuan, K. F., Xiong, Y., & Zhang, L. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468544 · Cited by 222 · see citing works
Self-citationS3: syntax- and semantic-guided repair synthesis via programming by examples
Le, X.-B. D., Chu, D.-H., Lo, D., Goues, C. L., & Visser, W. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106309 · Cited by 219 · see citing works
The plastic surgery hypothesis
Barr, E. T., Brun, Y., Dévanbu, P., Harman, M., & Sarro, F. (2014) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2635868.2635898 · Cited by 217 · see citing works
Li, Y., Wang, S., & Nguyen, T. N. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380345 · Cited by 211 · see citing works
FixMiner: Mining relevant fix patterns for automated program repair
Koyuncu, A., Liu, K., Bissyandé, T. F., Kim, D., Klein, J., Monperrus, M., & Traon, Y. L. (2020) · Empirical Software Engineering · doi:10.1007/s10664-019-09780-z · Cited by 198 · see citing works
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Zhang, F., Chen, B., Zhang, Y., Keung, J., Liu, J., Zan, D., Mao, Y., Lou, J.-G., & Chen, W. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.151 · Cited by 173 · see citing works
Practical program repair via bytecode mutation
Ghanbari, A., Benton, S., & Zhang, L. (2019) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3293882.3330559 · Cited by 167 · see citing works
Self-citationNeural program repair with execution-based backpropagation
Ye, H., Martínez, M., & Monperrus, M. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510222 · Cited by 161 · see citing works
Automatic repair of buggy if conditions and missing preconditions with SMT
DeMarco, F., Xuan, J., Berre, D. L., & Monperrus, M. (2014) · doi:10.1145/2593735.2593740 · Cited by 141 · see citing works
Repair Is Nearly Generation: Multilingual Program Repair with LLMs
Joshi, H., Cambronero Sanchez, J., Gulwani, S., Le, V., Verbruggen, G., & Radiček, I. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25642 · Cited by 126 · see citing works
Contract-based program repair without the contracts
Chen, L., Pei, Y., & Furia, C. A. (2017) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115674 · Cited by 119 · see citing works
Conversational Automated Program RepairPreprint
Xia, C., & Zhang, L.-M. (2023) · arXiv.org · doi:10.48550/arxiv.2301.13246 · Cited by 103 · see citing works
Self-citationCan OpenAI's codex fix bugs?: an evaluation on QuixBugs
Prenner, J. A., Babii, H., & Robbes, R. (2022) · Proceedings of the Third International Workshop on Automated Program Repair · doi:10.1145/3524459.3527351 · Cited by 99 · see citing works
A study on prompt design, advantages and limitations of ChatGPT for deep learning program repair
Cao, J., Li, M., Wen, M., & Cheung, S.-C. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00492-x · Cited by 97 · see citing works
CM3: A Causal Masked Multimodal Model of the Internet
Aghajanyan, A., Huang, B., Ross, C., Karpukhin, V., Xu, H., Goyal, N., Okhonko, D., Joshi, M., Ghosh, G., Lewis, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.07520 · Cited by 42 · see citing works
Extracting Concise Bug-Fixing Patches from Human-Written Patches in Version Control Systems
Jiang, Y., Liu, H., Niu, N., Zhang, L., & Hu, Y. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00069 · Cited by 41 · see citing works
Self-citationSketchFix: a tool for automated program repair approach using lazy candidate generation
Hua, J., Zhang, M., Wang, K., & Sarfraz Khurshid, D. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3264600 · Cited by 37 · see citing works
CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion
Ding, Y., Wang, Z., Ahmad, W. U., Ding, H., Tan, M., Jain, N., Ramanathan, M. K., Nallapati, R., Bhatia, P., Roth, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.11248 · Cited by 37
A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program RepairPreprint
Cao, J., Li, M., Wen, M., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.08191 · Cited by 33 · see citing works
Grounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2022) · arXiv · doi:10.48550/arxiv.2206.15000 · Cited by 23 · cited 1× · see citing works
“Recent advances in Natural Language Processing (NLP) have empowered the idea of using Large Language Models (LLMs) that are pre-trained on enormous corpora of natural language and code for various code-related tasks [4, 5, 10, 38, 71].”
BackgroundReliable Fix Patterns Inferred from Static Checkers for Automated Program Repair
Liu, K., Zhang, J., Li, L., Koyuncu, A., Kim, D., Ge, C., Liu, Z., Klein, J., & Bissyandé, T. F. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3579637 · Cited by 13 · see citing works
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Zhang, F., Chen, B., Zhang, Y., Keung, J., Liu, J., Zan, D., Mao, Y., Lou, J.-G., & Chen, W. (2023) · arXiv · Cited by 10 · see citing works
Wei, Y., Xia, C. S., & Zhang, L. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.8280747 · Cited by 1 · see citing works
Self-citationBricon, F., Snjeza, Grunberg, R., Chen, S., Wang, J., Zhang, Y., Chen, S., Zheng, Y., Ercan, G., Aeschlimann, M., et al. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.8278008 · Cited by 1 · see citing works
Self-citationFuzz4All: Universal Fuzzing with Large Language Models
Xia, C. S., Paltenghi, M., Tian, J. L., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.04748
Self-citationKeep the Conversation Going: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT
Xia, C. S., & Zhang, L. (2023) · arXiv · doi:10.48550/arxiv.2304.00385
Self-citationWei, Y., Xia, C. S., & Zhang, L. (2023) · Zenodo · doi:10.5281/zenodo.8281250
Self-citationUniverseFly/eclipse.jdt.ls: Modified Eclipse JDT LS 1.0.3
Wei, Y. (2023) · Zenodo · doi:10.5281/zenodo.8278193
Self-citationRevisiting the Plastic Surgery Hypothesis via Large Language Models
Xia, C. S., Ding, Y., & Zhang, L. (2023) · arXiv · doi:10.48550/arxiv.2303.10494
Self-citation
Flagged first, then by citations
Research works printed but not linked: 7
- Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Scott Yih, Luke Zettlemoyer, and Mike Lewis. 2023. InCoder: A Generative Model for Code Infilling and Synthesis. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=hQwb- lbM6EL
- Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie LIU, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2021. GraphCodeBERT: Pre-training Code Representations with Data Flow. In International Conference on Learning Representations. https: //openreview.net/forum?id=jLoC4ez43PZ
- Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The Curious Case of Neural Text Degeneration. In International Conference on Learning Representations. https://openreview.net/forum?id=rygGQyrFvH
- Sophia D Kolak, Ruben Martins, Claire Le Goues, and Vincent Josua Hellendoorn. 2022. Patch Generation with Language Models: Feasibility and Scaling Behavior. In Deep Learning for Code Workshop. https://openreview.net/forum?id=rHlzJh_ b1-5
- Gabriel Poesia, Alex Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christopher Meek, and Sumit Gulwani. 2022. Synchromesh: Reliable Code Generation from Pre-trained Language Models. In International Conference on Learning Represen- tations. https://openreview.net/forum?id=KmtVD97J43e
- Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (Eds.), Vol. 27. Curran Associates, Inc. https://proceedings.neurips.cc/paper_ files/paper/2014/file/a14ac55a4f27472c5d894ec1c3c743d2-Paper.pdf
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS'17).
Cited by
Sorted by citations
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029 · see citing works
Software Testing With Large Language Models: Survey, Landscape, and Vision
Wang, J., Huang, Y., Chen, C., Liu, Z., Wang, S., & Wang, Q. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3368208 · Cited by 573 · see citing works
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650 · see citing works
When LLMs meet cybersecurity: a systematic literature review
Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 220 · see citing works
A Survey of Learning-based Automated Program Repair
Zhang, Q., Fang, C., Ma, Y., Sun, W., & Chen, Z. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3631974 · Cited by 153 · see citing works
When Software Security Meets Large Language Models: A Survey
Zhu, X., Zhou, W., Han, Q., Ma, W., Wen, S., & Xiang, Y. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124971 · Cited by 105 · see citing works
Generative AI for self-adaptive systems: State of the art and research roadmap
Li, J., Zhang, M., Li, N., Weyns, D., Jin, Z., & Tei, K. (2024) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3686803 · Cited by 77 · see citing works
Look Before You Leap: An Exploratory Study of Uncertainty Analysis for Large Language Models
Huang, Y., Song, J., Wang, Z., Zhao, S., Chen, H., Juefei-Xu, F., & Ma, L. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3519464 · Cited by 132 · see citing works
Jaffal, N. O., Alkhanafseh, M., & Mohaisen, D. (2025) · AI · doi:10.3390/ai6090216 · Cited by 69 · see citing works
SpecGen: Automated Generation of Formal Program Specifications via Large Language Models
Ma, L., Liu, S., Li, Y., Xie, X., & Bu, L. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00129 · Cited by 98 · see citing works
Prompt Engineering or Fine-Tuning: An Empirical Assessment of LLMs for Code
Shin, J., Tang, C. J., Mohati, T., Nayebi, M., Wang, S., & Hemmati, H. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00082 · Cited by 101 · see citing works
How secure is AI-generated code: a large-scale comparison of large language models
Tihanyi, N., Bisztray, T., Ferrag, M. A., Jain, R., & Cordeiro, L. C. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10590-1 · Cited by 63 · see citing works
RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair
Silva, A., Fang, S., & Monperrus, M. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3581062 · Cited by 49 · see citing works
Shi, J., Yang, Z., & Lo, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708525 · Cited by 43 · see citing works
Software engineering education in the era of conversational AI: current trends and future directions
Sengul, C., Neykova, R., & Destefanis, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1436350 · Cited by 55 · see citing works
Teaching Code LLMs to Use Autocompletion Tools in Repository-Level Code Generation
Wang, C., Zhang, J., Feng, Y., Li, T., Sun, W., Liu, Y., & Peng, X. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3714462 · Cited by 40 · see citing works
“Additionally, ToolFormer and ToolCoder are unable to handle scenarios where the tools return multiple candidates.”
Agents in software engineering: survey, landscape, and vision
Wang, Y., Zhong, W., Huang, Y., Shi, E., Yang, M., Chen, J., Li, H., Ma, Y., Wang, Q., & Zheng, Z. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00544-2 · Cited by 50 · see citing works
Can Language Models Employ the Socratic Method? Experiments with Code Debugging
Al-Hossami, E., Bunescu, R., Smith, J., & Teehan, R. (2024) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3626252.3630799 · Cited by 35 · see citing works
Comprehensive Fine-Tuning Large Language Models of Code for Automated Program Repair
Huang, K., Zhang, J., Bao, X., Wang, X., & Liu, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3532759 · Cited by 30 · see citing works
Reasoning Runtime Behavior of a Program with LLM: How Far are We?
Chen, J., Pan, Z., Hu, X., Li, Z., Li, G., & Xia, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00012 · Cited by 40 · see citing works
Zhang, L., Li, K., Sun, K., Wu, D., Liu, Y., Tian, H., & Liu, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3590108 · Cited by 23 · see citing works
Self-citationThe use of large language models for program repair
Zubair, F., Al-Hitmi, M., & Catal, C. (2024) · Computer Standards & Interfaces · doi:10.1016/j.csi.2024.103951 · Cited by 23 · see citing works
Template-Guided Program Repair in the Era of Large Language Models
Huang, K., Zhang, J., Meng, X., & Liu, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00030 · Cited by 21 · see citing works
One Size Does Not Fit All: Multi-granularity Patch Generation for Better Automated Program Repair
Lin, B., Wang, S., Wen, M., Chen, L., & Mao, X. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680381 · Cited by 20 · see citing works
LPR: Large Language Models-Aided Program Reduction
Zhang, M., Tian, Y., Xu, Z., Dong, Y., Tan, S. H., & Sun, C. P. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3652126 · Cited by 21 · see citing works
A Survey on Failure Analysis and Fault Injection in AI Systems
Yu, G., Tan, G., Huang, H., Zhang, Z., Chen, P., Natella, R., Zheng, Z., & Lyu, M. R.-T. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3732777 · Cited by 18 · see citing works
Licoeval: Evaluating LLMs on License Compliance in Code Generation
Xu, W., Gao, K., He, H., & Zhou, M. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00052 · Cited by 22 · see citing works
Cheng, Y., Bajaber, O., Tsegai, S. A., Song, D. X., & Gao, P. (2025) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp63326.2025.00057 · Cited by 25 · see citing works
An Empirical Study on the Code Refactoring Capability of Large Language Models
Cordeiro, J., Noei, S., & Zou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3801158 · Cited by 17 · see citing works
Aligning the Objective of LLM-Based Program Repair
Xu, J., Fu, Y., Tan, S. H., & He, P. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00169 · Cited by 27 · see citing works
Evolving Paradigms in Automated Program Repair: Taxonomy, Challenges, and Opportunities
Huang, K., Xu, Z., Yang, S., Sun, H., Li, X., Yan, Z., & Zhang, Y. (2024) · ACM Computing Surveys · doi:10.1145/3696450 · Cited by 16 · see citing works
Type-Constrained Code Generation with Language Models
Mündler-Sasahara, N., He, J.-X., Wang, H., Sen, K., Song, D., & Vechev, M. T. (2025) · Proc. ACM Program. Lang. · doi:10.1145/3729274 · Cited by 77 · see citing works
bjCnet: A contrastive learning-based framework for software defect prediction
Han, J., Huang, C., & Liu, J. (2024) · Computers & Security · doi:10.1016/j.cose.2024.104024 · Cited by 12 · see citing works
Clozemaster: Fuzzing Rust Compiler by Harnessing Llms for Infilling Masked Real Programs
Gao, H., Yang, Y., Sun, M., Wu, J., Zhou, Y., & Xu, B. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00175 · Cited by 12 · see citing works
LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-Based Code Completion
Wang, C., Huang, K., Zhang, J., Feng, Y., Zhang, L., Liu, Y., & Peng, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00245 · Cited by 17 · see citing works
Self-citationBenchmarking Automated Program Repair: An Extensive Study on Both Real-World and Artificial Bugs
Ouyang, Y., Yang, J., & Zhang, L. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3652140 · Cited by 11 · see citing works
Self-citationStatically Contextualizing Large Language Models with Typed Holes
Blinn, A., Li, X., Kim, J. H., & Omar, C. (2024) · Proceedings of the ACM on Programming Languages · doi:10.1145/3689728 · Cited by 10 · see citing works
Challenges and Paths Towards AI for Software EngineeringPreprint
Gu, A., Jain, N., Li, W., Shetty, M., Shao, Y., Li, Z., Yang, D., Ellis, K., Sen, K., & Solar-Lezama, A. (2025) · Qeios · doi:10.32388/vv1661 · Cited by 24 · see citing works
History-Driven Fuzzing for Deep Learning Libraries
Harzevili, N. S., Mohajer, M. M., Wei, M., Pham, H. V., & Wang, S. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3688838 · Cited by 11 · see citing works
Unearthing Gas-Wasting Code Smells in Smart Contracts With Large Language Models
Jiang, J., Li, Z., Qin, H., Jiang, M., Luo, X., Wu, X., Wang, H., Tang, Y., Qian, C., & Chen, T. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3491578 · Cited by 9 · see citing works
AdverIntent-Agent: Adversarial Reasoning for Repair Based on Inferred Program Intent
Ye, H., Yang, A. Z. H., Hu, C., Wang, Y., Zhang, T., & Goues, C. L. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728939 · Cited by 11 · see citing works
LLM-enhanced evolutionary test generation for untyped languages
Yang, R., Xu, X., & Wang, R. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00496-7 · Cited by 9 · see citing works
Your Fix Is My Exploit: Enabling Comprehensive DL Library API Fuzzing with Large Language Models
Zhang, K., Wang, S., Han, J., Zhu, X., Li, X., Wang, S., & Wen, S. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00041 · Cited by 10 · see citing works
Towards Secure Code Generation With LLMs: A Study on Common Weakness Enumeration
Zhao, J., Sun, Y., Huang, C., Liu, C., Guan, Y., Zeng, Y., & Liu, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3619281 · Cited by 10 · see citing works
Position: Vibe Coding Needs Vibe Reasoning: Improving Vibe Coding with Formal Verification
Mitchell, J., & Shaaban, Y. (2025) · Proceedings of the 1st ACM SIGPLAN International Workshop on Language Models and Programming Languages · doi:10.1145/3759425.3763390 · Cited by 8 · see citing works
ZS4C: Zero-Shot Synthesis of Compilable Code for Incomplete Code Snippets Using LLMs
Kabir, A., Wang, S., Tian, Y., Chen, T.-H. P., Asaduzzaman, M., & Zhang, W. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3702979 · Cited by 9 · see citing works
ReAPR: Automatic program repair via retrieval-augmented large language models
Liu, Z., Du, X., & Liu, H. (2025) · Software Quality Journal · doi:10.1007/s11219-025-09728-1 · Cited by 8 · see citing works
Can LLMs Generate User Stories and Assess Their Quality?
Quattrocchi, G., Pasquale, L., Spoletini, P., & Baresi, L. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3670612 · Cited by 7 · see citing works
Towards Neural Synthesis for SMT-Assisted Proof-Oriented Programming
Chakraborty, S., Ebner, G., Bhat, S., Fakhoury, S., Fatima, S., Lahiri, S. K., & Swamy, N. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00002 · Cited by 7 · see citing works
Enhancing AI-based Generation of Software Exploits with Contextual Information
Liguori, P., Improta, C., Natella, R., Čukić, B., & Cotroneo, D. (2024) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre62328.2024.00027 · Cited by 6 · see citing works
Exploring Large Language Models in Information Systems: A Survey
Bianchini, F., & Marinacci, M. (2025) · Springer eBooks · doi:10.1007/978-3-031-92285-5_5 · Cited by 6 · see citing works
Detecting, Creating, Repairing, and Understanding Indivisible Multi-Hunk Bugs
Xin, Q., Wu, H., Tang, J., Liu, X., Reiss, S. P., & Xuan, J. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660828 · Cited by 6 · see citing works
ROSE: An IDE-Based Interactive Repair Framework for Debugging
Reiss, S. P., Wei, X., Yuan, J. P., & Xin, Q. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3705306 · Cited by 5 · see citing works
RetypeR: Integrated Retrieval-based Automatic Program Repair for Python Type Errors
Hao, S., Shi, X., & Liu, H. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00028 · Cited by 5 · see citing works
Zhang, Y., Jin, Z., Xing, Y., Li, G., Liu, F., Zhu, J., Dou, W., & Wei, J. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3718739 · Cited by 28 · see citing works
Learning-Based Automated Program Repair: A Systematic Literature Review
Chen, Q., Li, D., Zhao, M., Wong, W. E., & Li, H. (2025) · Complex System Modeling and Simulation · doi:10.23919/csms.2025.0004 · Cited by 4 · see citing works
RefactorGPT: a ChatGPT-based multi-agent framework for automated code refactoring
Karabıyık, M. A. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.3257 · Cited by 4 · see citing works
Lin, X., Xie, Q., Zhao, B., Tian, Y., Zonouz, S., Ruan, N., Li, J., Beyah, R., & Ji, S. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00091 · Cited by 4 · see citing works
Integrating Various Software Artifacts for Better LLM-based Bug Localization and Program Repair
Feng, Q., Ma, X., Sheng, J., Feng, Z., Song, W., & Liang, P. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3770581 · Cited by 4 · see citing works
Zhong, R., Li, Y., Yu, G., Gu, W., Kuang, J., Huo, Y., & Lyu, M. R.-T. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3773287 · Cited by 6 · see citing works
Counterexample Guided Program Repair Using Zero-Shot Learning and MaxSAT-based Fault Localization
Orvalho, P., Janota, M., & Manquinho, V. (2025) · arXiv · doi:10.48550/arxiv.2502.07786 · Cited by 4 · see citing works
DR.FIX: Automatically Fixing Data Races at Industry Scale
Behrang, F., Zhang, Z., Saioc, G.-V., Liu, P., & Chabbi, M. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3729265 · Cited by 3 · see citing works
Boosting Redundancy-Based Automated Program Repair by Fine-Grained Pattern Mining
Jiang, J., Li, F., Zhao, Z., Ye, Z., Liu, M., Wang, B., Zhang, H., & Chen, J. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00018 · Cited by 3 · see citing works
Code Generation of Smart Contracts with LLMs: A Case Study on Hyperledger Fabric
Olivieri, L., Beste, D., Negrini, L., Schönherr, L., Ciná, A. E., & Ferrara, P. (2025) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre66568.2025.00034 · Cited by 3 · see citing works
AI-driven self-healing across the edge–cloud continuum: A systematic literature review
Bonilla, L., Díaz-de-Arcaya, J., Aguirre-Usandizaga, J., & Almeida, A. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108232 · Cited by 3 · see citing works
Rethinking the Evaluation of Secure Code Generation
Dai, S.-C., Xu, J. L., & Tao, G. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773217 · Cited by 12 · see citing works
SOLAR: Illuminating LLM performance in API discovery and service ranking for edge AI and IoT
Al‐Masri, E., & Subramanian, I. N. (2025) · Internet of Things · doi:10.1016/j.iot.2025.101630 · Cited by 3 · see citing works
Can Large Language Models Discover Metamorphic Relations? A Large-Scale Empirical Study
Zhang, J., Sun, C., Liu, H., & Dong, S. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00011 · Cited by 3 · see citing works
Automated Refactoring of Non-Idiomatic Python Code: A Differentiated Replication with LLMS
Midolo, A., & Penta, M. D. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00020 · Cited by 3 · see citing works
When Automated Program Repair Meets Regression Testing—An Extensive Study on Two Million Patches
Lou, Y., Yang, J., Benton, S., Hao, D., Tan, L., Chen, Z., Zhang, L. H., & Zhang, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3672450 · Cited by 2 · see citing works
Self-citationA Comprehensive Study on Code Completion for Large Language Models
Cai, Y. (2025) · International Conference on Advanced Algorithms and Control Engineering (ICAACE) · doi:10.1109/icaace65325.2025.11020128 · Cited by 2 · see citing works
Element-Based Automated DNN Repair with Fine-Tuned Masked Language Model
Wang, X., Zhang, M., Meng, X., Zhang, J., Liu, Y., & Hu, C. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715716 · Cited by 2 · see citing works
Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Repair
Huang, K., Zhang, J., Xie, X., & Chen, C. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00100 · Cited by 2 · see citing works
Robustness Analysis of Watermarking Techniques for LLM-Generated Code
Cao, D., Liu, S., Zhang, J., & Xiang, Y. (2025) · IEEE Networking Letters · doi:10.1109/lnet.2025.3616325 · Cited by 2 · see citing works
Exploring Direct Instruction and Summary-Mediated Prompting in LLM-Assisted Code Modification
Tang, N., Smith, E., Huang, Y., Collin, M., & Li, T. J.-J. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00017 · Cited by 2 · see citing works
Reading between the Lines: Context-Aware AI-based generation of software exploits
Improta, C., Liguori, P., Natella, R., Čukić, B., & Cotroneo, D. (2026) · Empirical Software Engineering · doi:10.1007/s10664-025-10796-x · Cited by 2 · see citing works
Clean Code, Better Models: Enhancing LLM Performance with Smell-Cleaned Dataset
Xue, Z., Zhang, X., Gao, Z., Hu, X., Gao, S., Xia, X., & Li, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3793252 · Cited by 3 · see citing works
Unveiling the power of code pre-trained models in automated program repair: A systematic review
Zhan, S., Wang, X. J., Wei, D., Xiang, X., Lin, Y., Wu, H., & Lu, J. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113015 · Cited by 2 · see citing works
MoonBit: Explore the Design of an AI-Friendly Programming Language
Fei, H., Zhang, Y., Zhang, H., Wang, Y., & Liu, Q. (2024) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1145/3643795.3648376 · Cited by 2 · see citing works
Leveraging Search-Based and Pre-Trained Code Language Models for Automated Program Repair
Lijzenga, O., Moghadam, I. H., & Zaytsev, V. (2025) · ACM Symposium on Applied Computing · doi:10.1145/3672608.3707774 · Cited by 2 · see citing works
Porting Software Libraries to OpenHarmony: Transitioning from TypeScript or JavaScript to ArkTS
Zhou, B., Shi, J., Wang, Y., Li, L., Li, T. O., Yu, H., & Zhu, Z. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728941 · Cited by 2 · see citing works
MultiMend: multilingual program repair with context augmentation and multi-hunk patch generation
Gharibi, R., Sadreddini, M. H., & Fakhrahmad, S. M. (2026) · Automated Software Engineering · doi:10.1007/s10515-026-00611-2 · Cited by 3 · see citing works
Yang, Y., Li, C., Han, Z., Li, R., Xu, K., Li, Q., Zhong, W., Shen, Z., Fei, Z., Ge, J., et al. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3764584 · Cited by 1 · see citing works
Empirical Evaluation of LLMs for Automated Program Fault Localisation
Liu, Y., Wang, X., Liu, H., Huang, R., & Wu, Y. (2025) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c65679.2025.00063 · Cited by 1 · see citing works
Fu, A., Xu, P., Li, J., Kuang, B., & Gao, Y. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3618407 · Cited by 1 · see citing works
Multi-Location Software Model Completion
Welter, A., Tinnes, C., & Apel, S. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3787798 · Cited by 1 · see citing works
RepoMasterEval: Evaluating Code Completion via Real-World Repositories
Wu, Q., Peng, C., Gao, P., Hu, R., Gan, H., Jiang, B., Tang, J., Deng, Z., Guan, Z., Gao, C., et al. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00304 · Cited by 6 · see citing works
A Systematic Literature Review of Parameter-Efficient Fine-Tuning for Large Code Models
Afrin, S., Haque, M. Z., & Mastropaolo, A. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3796522 · Cited by 6 · see citing works
Iter-T: ITERative Test Suite Generation for Automated Program Repair
Godio, A., Brida, S. G., Regis, G., Bagheri, H., Nguyen, T. H., Aguirre, N., & Frias, M. F. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3671416 · Cited by 1 · see citing works
TLR: Codebase-Level C Memory Management Error Repair with Large Language Models
Cheng, X. Y., Guo, Z., Huo, H., & Sui, Y. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797085 · Cited by 1 · see citing works
Evaluating Spectrum-Based Fault Localization on Deep Learning Libraries
Yan, M., Chen, J., Jiang, T., Jiang, J., & Wang, Z. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3552622 · Cited by 1 · see citing works
PReMM: LLM-Based Program Repair for Multi-method Bugs via Divide and Conquer
Xie, L., Li, Z., Pei, Y., Wen, Z., Liu, K., Zhang, T., & Li, X. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763097 · Cited by 2 · see citing works
A Survey on Code Pre-Trained Models for Neural Program RepairPreprint
Zhan, S., Wang, X., Wei, D., & Cao, X. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4831589
Exploration Study About LLM with Proper Prompt in Automated Program Repair
Liang, H., Gao, C., & Jia, Y. F. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4509-1_28
A Comprehensive Study of Bug Characteristics on Foundation Language Models
Han, J., Wang, G., Liu, J., Bao, L., Hu, X., Wei, J., & Deng, S. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00037
Zhan, S., Wang, X., Wei, D., Cao, X., & Chen, B. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112590
Zhang, H., Yan, Q., Min, W., Zhang, C., Kuang, L., & Xia, Y. (2025) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws67624.2025.00123
Visual Sketchbook: Enhancing Chart-to-Code Generation via Reflective Refinement
Wang, J., Xi, Z., He, W., Zhu, D., Dou, S., Gui, T., & Zhang, Q. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3346-6_12
FlowRepair: Search-based automated program repair of CPS controllers modeled in Simulink-Stateflow
Arrieta, A., Valle, P., & Ali, S. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2025.108010
Developer Prompts in Practice: An Empirical Study of Bias, Security, and Optimization
Rzig, D. E., Paul, D. J., Pister, K. S., Henkel, J. M., & Hassan, F. (2025) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem64174.2025.00067
- Authors
- Wei, Y., Xia, C. S., & Zhang, L.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Venue
- Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2023-11-30
- Pages
- 172-184
- OpenAlex
- W4386436496
- Cited by
- 115
- References
- 45
- Citations by year
- 2026: 27 · 2025: 62 · 2024: 24 · 2023: 2
- FWCI
- 22.26
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- NSF (National Science Foundation) (CCF-2131943,CCF-2141474)
- Kwai Inc.
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Keywords
- automated program repair, large language models, patch synthesis, token pruning, autoregressive generation, Defects4J, bug fixes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
