The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair
Open accessAbstract
During Automated Program Repair (APR), it can be challenging to synthesize correct patches for real-world systems in general-purpose programming languages. Recent Large Language Models (LLMs) have been shown to be helpful “copilots” in assisting developers with various coding tasks, and have also been directly applied for patch synthesis. However, most LLMs treat programs as sequences of tokens, meaning that they are ignorant of the underlying semantics constraints of the target programming language. This results in plenty of statically invalid generated patches, impeding the practicality of the technique. Therefore, we propose Repilot, a framework to further copilot the AI “copilots” (i.e., LLMs) by synthesizing more valid patches during the repair process. Our key insight is that many LLMs produce outputs autoregressively (i.e., token by token), resembling human writing programs, which can be significantly boosted and guided through a Completion Engine. Repilot synergistically synthesizes a candidate patch through the interaction between an LLM and a Completion Engine, which 1) prunes away infeasible tokens suggested by the LLM and 2) proactively completes the token based on the suggestions provided by the Completion Engine. Our evaluation on a subset of the widely-used Defects4j 1.2 and 2.0 datasets shows that Repilot fixes 66 and 50 bugs, respectively, surpassing the best-performing baseline by 14 and 16 bugs fixed. More importantly, Repilot is capable of producing more valid and correct patches than the base LLM when given the same generation budget.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“This approach enhances the compilation rate and correctness of patches while reducing the number of invalid attempts in the generation process.”
From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Wei et al. (2023)
The cited source says what the claim attributes to it.
In the citing paper
“Large Language Models (LLMS) trained on code (LLMCS) have demonstrated significant effectiveness in generating program fixes (Joshi et al. 2023; Xia, Ding, and Zhang 2023; Wei, Xia, and Zhang 2023; Fan et al. 2023; Xia, Wei, and Zhang 2023; Brancas, Manquinho, and Martins 2024).”
From Counterexample Guided Program Repair Using Zero-Shot Learning and MaxSAT-based Fault Localization (2025) · cites this paper as Wei, Xia, and Zhang 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.139 · Cited by 2,824
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495
Defects4J: a database of existing faults to enable controlled testing studies for Java programs
Just, R., Jalali, D., & Ernst, M. D. (2014) · Proceedings of the 2014 International Symposium on Software Testing and Analysis · doi:10.1145/2610384.2628055 · Cited by 1,447
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
GenProg: A Generic Method for Automatic Software Repair
Goues, C. L., Nguyen, T. H., Forrest, S., & Weimer, W. R. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.104 · Cited by 1,084
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Unified Pre-training for Program Understanding and GenerationPreprint
Ahmad, W. U., Chakraborty, S., Ray, B., & Chang, K.-W. (2021) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2103.06333 · Cited by 1,000
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974
Unified Pre-training for Program Understanding and Generation
Ahmad, W., Chakraborty, S., Ray, B., & Chang, K.-W. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.211 · Cited by 642
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., & Hellendoorn, V. J. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534862 · Cited by 540
Mechtaev, S., Yi, J., & Roychoudhury, A. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884807 · Cited by 515
CodeSearchNet Challenge: Evaluating the State of Semantic Code Search
Husain, H., Wu, H.-H., Gazit, T., Allamanis, M., & Brockschmidt, M. (2019) · arXiv · doi:10.48550/arxiv.1909.09436 · Cited by 483
CoCoNuT: combining context-aware neural translation models using ensemble for program repair
Lutellier, T., Pham, H., Pang, L., Li, Y., Wei, M., & Tan, L. (2020) · International Symposium on Software Testing and Analysis · doi:10.1145/3395363.3397369 · Cited by 463
CodeT5+: Open Code Large Language Models for Code Understanding and Generation
Wang, Y., Le, H., Gotmare, A. D., Bui, N. D. Q., Li, J., & Hoi, S. C. H. (2023) · arXiv · doi:10.48550/arxiv.2305.07922 · Cited by 459
SEQUENCER: Sequence-to-Sequence Learning for End-to-End Program Repair
Chen, Z., Kommrusch, S., Tufano, M., Pouchet, L.-N., Poshyvanyk, D., & Monperrus, M. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2940179 · Cited by 448
Grounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2023) · Proceedings of the ACM on Programming Languages · doi:10.1145/3586030 · Cited by 436 · cited 1×
“Recent advances in Natural Language Processing (NLP) have empowered the idea of using Large Language Models (LLMs) that are pre-trained on enormous corpora of natural language and code for various code-related tasks [4, 5, 10, 38, 71].”
BackgroundAutomated Program Repair in the Era of Large Pre-trained Language Models
Xia, C. S., Wei, Y., & Zhang, L. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00129 · Cited by 433
Self-citationStaged program repair with condition synthesis
Long, F., & Rinard, M. (2015) · ESEC/SIGSOFT FSE · doi:10.1145/2786805.2786811 · Cited by 416
Liu, J., Xia, C. S., Wang, Y., & Zhang, L. (2023) · arXiv · doi:10.48550/arxiv.2305.01210 · Cited by 390
Self-citationProgram Synthesis with Large Language Models
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07732 · Cited by 388
TBar: revisiting template-based automated program repair
Liu, K., Koyuncu, A., Kim, D., & Bissyandé, T. F. (2019) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3293882.3330577 · Cited by 378
CURE: Code-Aware Neural Machine Translation for Automatic Program Repair
Jiang, N., Lutellier, T., & Tan, L. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00107 · Cited by 371
An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation
Tufano, M., Watson, C., Bavota, G., Penta, M. D., White, M., & Poshyvanyk, D. (2019) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3340544 · Cited by 370
Context-aware patch generation for better automated program repair
Wen, M., Chen, J., Wu, R., Hao, D., & Cheung, S.-C. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180233 · Cited by 356
Le, X.-B. D., Lo, D., & Goues, C. L. (2016) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2016.76 · Cited by 341
An Analysis of the Automatic Bug Fixing Performance of ChatGPT
Sobania, D., Briesch, M., Hanna, C., & Petke, J. (2023) · 2023 IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1109/apr59189.2023.00012 · Cited by 338
Automatic Software Repair: A Survey
Gazzola, L., Micucci, D., & Mariani, L. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2755013 · Cited by 336
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
Nijkamp, E., Pang, B., Hayashi, H., Tu, L., Wang, H., Zhou, Y., Savarese, S., & Xiong, C. (2022) · arXiv · doi:10.48550/arxiv.2203.13474 · Cited by 330
Shaping program repair space with existing patches and similar code
Jiang, J., Xiong, Y., Zhang, H., Gao, Q., & Chen, X. (2018) · Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3213846.3213871 · Cited by 329
Deng, Y., Xia, C. S., Peng, H., Yang, C., & Zhang, L. (2023) · Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3597926.3598067 · Cited by 326
Self-citationAutomatic repair of real bugs in java: a large-scale experiment on the defects4j dataset
Martínez, M., Durieux, T., Sommerard, R., Xuan, J., & Monperrus, M. (2016) · Empirical Software Engineering · doi:10.1007/s10664-016-9470-4 · Cited by 249
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · arXiv · doi:10.48550/arxiv.2002.08155 · Cited by 243
ASTOR: a program repair library for Java (demo)
Martínez, M., & Monperrus, M. (2016) · International Symposium on Software Testing and Analysis · doi:10.1145/2931037.2948705 · Cited by 240
Li, Y., Wang, S., & Nguyen, T. N. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380345 · Cited by 239
CodeGen2: Lessons for Training LLMs on Programming and Natural LanguagesPreprint
Nijkamp, E., Hayashi, H., Xiong, C., Savarese, S., & Zhou, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.02309 · Cited by 234
Less training, more repairing please: revisiting automated program repair via zero-shot learning
Xia, C. S., & Zhang, L. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549101 · Cited by 231
Self-citationA syntax-guided edit decoder for neural program repair
Zhu, Q., Sun, Z., Xiao, Y.-A., Zhang, W., Yuan, K. F., Xiong, Y., & Zhang, L. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468544 · Cited by 222
Self-citationS3: syntax- and semantic-guided repair synthesis via programming by examples
Le, X.-B. D., Chu, D.-H., Lo, D., Goues, C. L., & Visser, W. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106309 · Cited by 219
The plastic surgery hypothesis
Barr, E. T., Brun, Y., Dévanbu, P., Harman, M., & Sarro, F. (2014) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2635868.2635898 · Cited by 217
FixMiner: Mining relevant fix patterns for automated program repair
Koyuncu, A., Liu, K., Bissyandé, T. F., Kim, D., Klein, J., Monperrus, M., & Traon, Y. L. (2020) · Empirical Software Engineering · doi:10.1007/s10664-019-09780-z · Cited by 198
Practical program repair via bytecode mutation
Ghanbari, A., Benton, S., & Zhang, L. (2019) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3293882.3330559 · Cited by 174
Self-citationRepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Zhang, F., Chen, B., Zhang, Y., Keung, J., Liu, J., Zan, D., Mao, Y., Lou, J.-G., & Chen, W. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.151 · Cited by 173
Neural program repair with execution-based backpropagation
Ye, H., Martínez, M., & Monperrus, M. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510222 · Cited by 161
Conversational Automated Program RepairPreprint
Xia, C. S., & Zhang, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.13246 · Cited by 152
Self-citationAutomatic repair of buggy if conditions and missing preconditions with SMT
DeMarco, F., Xuan, J., Berre, D. L., & Monperrus, M. (2014) · doi:10.1145/2593735.2593740 · Cited by 141
A study on prompt design, advantages and limitations of ChatGPT for deep learning program repair
Cao, J., Li, M., Wen, M., & Cheung, S.-C. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00492-x · Cited by 136
Repair Is Nearly Generation: Multilingual Program Repair with LLMs
Joshi, H., Cambronero Sanchez, J., Gulwani, S., Le, V., Verbruggen, G., & Radiček, I. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25642 · Cited by 126
Contract-based program repair without the contracts
Chen, L., Pei, Y., & Furia, C. A. (2017) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115674 · Cited by 119
Can OpenAI's codex fix bugs?: an evaluation on QuixBugs
Prenner, J. A., Babii, H., & Robbes, R. (2022) · Proceedings of the Third International Workshop on Automated Program Repair · doi:10.1145/3524459.3527351 · Cited by 99
An Analysis of the Automatic Bug Fixing Performance of ChatGPT
(2023) · doi:10.48550/arxiv.2301.08653 · Cited by 49
Extracting Concise Bug-Fixing Patches from Human-Written Patches in Version Control Systems
Jiang, Y., Liu, H., Niu, N., Zhang, L., & Hu, Y. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00069 · Cited by 45
Self-citationCM3: A Causal Masked Multimodal Model of the Internet
Aghajanyan, A., Huang, B., Ross, C., Karpukhin, V., Xu, H., Goyal, N., Okhonko, D., Joshi, M., Ghosh, G., Lewis, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.07520 · Cited by 42
SketchFix: a tool for automated program repair approach using lazy candidate generation
Hua, J., Zhang, M., Wang, K., & Sarfraz Khurshid, D. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3264600 · Cited by 37
CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion
Ding, Y., Wang, Z., Ahmad, W. U., Ding, H., Tan, M., Jain, N., Ramanathan, M. K., Nallapati, R., Bhatia, P., Roth, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.11248 · Cited by 37
A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program RepairPreprint
Cao, J., Li, M., Wen, M., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.08191 · Cited by 33
Grounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2022) · arXiv · doi:10.48550/arxiv.2206.15000 · Cited by 23 · cited 1×
“Recent advances in Natural Language Processing (NLP) have empowered the idea of using Large Language Models (LLMs) that are pre-trained on enormous corpora of natural language and code for various code-related tasks [4, 5, 10, 38, 71].”
BackgroundReliable Fix Patterns Inferred from Static Checkers for Automated Program Repair
Liu, K., Zhang, J., Li, L., Koyuncu, A., Kim, D., Ge, C., Liu, Z., Klein, J., & Bissyandé, T. F. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3579637 · Cited by 13
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Zhang, F., Chen, B., Zhang, Y., Keung, J., Liu, J., Zan, D., Mao, Y., Lou, J.-G., & Chen, W. (2023) · arXiv · Cited by 10
Wei, Y., Xia, C. S., & Zhang, L. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.8280747 · Cited by 1
Self-citationBricon, F., Snjeza, Grunberg, R., Chen, S., Wang, J., Zhang, Y., Chen, S., Zheng, Y., Ercan, G., Aeschlimann, M., et al. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.8278008 · Cited by 1
Self-citationFuzz4All: Universal Fuzzing with Large Language Models
Xia, C. S., Paltenghi, M., Tian, J. L., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.04748
Self-citationKeep the Conversation Going: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT
Xia, C. S., & Zhang, L. (2023) · arXiv · doi:10.48550/arxiv.2304.00385
Self-citationWei, Y., Xia, C. S., & Zhang, L. (2023) · Zenodo · doi:10.5281/zenodo.8281250
Self-citationUniverseFly/eclipse.jdt.ls: Modified Eclipse JDT LS 1.0.3
Wei, Y. (2023) · Zenodo · doi:10.5281/zenodo.8278193
Self-citationRevisiting the Plastic Surgery Hypothesis via Large Language Models
Xia, C. S., Ding, Y., & Zhang, L. (2023) · arXiv · doi:10.48550/arxiv.2303.10494
Self-citation
Flagged first, then by citations
Research works printed but not linked: 11
- 10.18653/v1 · DOI
- Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Scott Yih, Luke Zettlemoyer, and Mike Lewis. 2023. InCoder: A Generative Model for Code Infilling and Synthesis. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=hQwb-lbM6EL
- Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie LIU, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2021. GraphCodeBERT: Pre-training Code Representations with Data Flow. In International Conference on Learning Representations. https://openreview.net/forum?id=jLoC4ez43PZ
- Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The Curious Case of Neural Text Degeneration. In International Conference on Learning Representations. https://openreview.net/forum?id=rygGQyrFvH
- Sophia D Kolak, Ruben Martins, Claire Le Goues, and Vincent Josua Hellendoorn. 2022. Patch Generation with Language Models: Feasibility and Scaling Behavior. In Deep Learning for Code Workshop. https://openreview.net/forum?id=rHlzJh_b1-5
- Gabriel Poesia, Alex Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christopher Meek, and Sumit Gulwani. 2022. Synchromesh: Reliable Code Generation from Pre-trained Language Models. In International Conference on Learning Representations. https://openreview.net/forum?id=KmtVD97J43e
- 10.18653/v1 · DOI
- Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (Eds.). 27, Curran Associates, Inc.. https://proceedings.neurips.cc/paper_files/paper/2014/file/a14ac55a4f27472c5d894ec1c3c743d2-Paper.pdf
- 10.5555/3295222.3295349 · DOI
- 10.18653/v1 · DOI
- Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. Curran Associates Inc., Red Hook, NY, USA.
Cited by
Sorted by citations
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029
Software Testing With Large Language Models: Survey, Landscape, and Vision
Wang, J., Huang, Y., Chen, C., Liu, Z., Wang, S., & Wang, Q. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3368208 · Cited by 573
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650
When LLMs meet cybersecurity: a systematic literature review
Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 383
A Survey of Learning-based Automated Program Repair
Zhang, Q., Fang, C., Ma, Y., Sun, W., & Chen, Z. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3631974 · Cited by 200
Metadata
- Authors
- Wei, Y., Xia, C. S., & Zhang, L.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Venue
- Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2023-11-30
- Pages
- 172-184
- OpenAlex
- W4386436496
- Cited by
- 115
- References
- 45
- Citations by year
- 2026: 27 · 2025: 62 · 2024: 24 · 2023: 2
- FWCI
- 22.26
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- NSF (National Science Foundation) (CCF-2131943,CCF-2141474)
- Kwai Inc.
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Keywords
- automated program repair, large language models, patch synthesis, token pruning, autoregressive generation, Defects4J, bug fixes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
