Teaching Code LLMs to Use Autocompletion Tools in Repository-Level Code Generation
Open accessAbstract
Recent code large language models (LLMs) have shown promising performance in generating standalone functions. However, they face limitations in repository-level code generation due to their lack of awareness of repository-level dependencies ( e.g., user-defined attributes), resulting in dependency errors such as undefined-variable and no-member errors. In this work, we introduce ToolGen , an approach that integrates autocompletion tools into the code LLM generation process to address these dependencies. ToolGen comprises two main phases: Trigger Insertion and Model Fine-tuning (Offline), and Tool-integrated Code Generation (Online). During the offline phase, ToolGen augments functions within a given code corpus with a special mark token, indicating positions to trigger autocompletion tools. These augmented functions, along with their corresponding descriptions, are then used to fine-tune a selected code LLM. In the online phase, ToolGen iteratively generates functions by predicting tokens step-by-step using the fine-tuned LLM. Whenever a mark token is encountered, ToolGen invokes the autocompletion tool to suggest code completions and selects the most appropriate one through constrained greedy search. We conduct comprehensive experiments to evaluate ToolGen ’s effectiveness in repository-level code generation across three distinct code LLMs: CodeGPT, CodeT5, and CodeLlama. To facilitate this evaluation, we create a benchmark comprising 671 real-world code repositories and introduce two new dependency-based metrics: Dependency Coverage and Static Validity Rate . The results demonstrate that ToolGen significantly improves Dependency Coverage by 31.4% to 39.1% and Static Validity Rate by 44.9% to 57.7% across the three LLMs, while maintaining competitive or improved performance in widely recognized similarity metrics such as BLEU-4, CodeBLEU, Edit Similarity, and Exact Match. On the CoderEval dataset, ToolGen achieves improvements of 40.0% and 25.0% in test pass rate (Pass@1) for CodeT5 and CodeLlama, respectively, while maintaining the same pass rate for CodeGPT. ToolGen also demonstrates high efficiency in repository-level code generation, with latency ranging from 0.63 to 2.34 seconds for generating each function. Furthermore, our generalizability evaluation confirms ToolGen ’s consistent performance when applied to diverse code LLMs, encompassing various model architectures and scales.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An agent harness turns a stateless model into a functional agent capable of long-running task execution.
“An agent harness refers to the software layer that surrounds an LLM with tools, APIs, sandboxes, memory, validators, permission boundaries, execution loops, and feedback channels, thereby turning a stateless model into a functional agent capable of long-running task execution [17, 18, 19, 20, 21, 22, 23].”
From Code as Agent Harness (2026) · cites this paper as [20]
The Abstract describes TOOLGEN invoking an autocompletion tool during iterative code generation, and the Efficiency section describes a cache for repeated suggestions. These passages support tool integration and a generation loop, but do not describe an agent harness that turns a stateless model into an agent capable of long-running tasks. Page numbers are unavailable in our copy. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3× · see citing works
“Recent advancements have introduced a variety of code large language models (LLMs) [7, 12, 13, 15, 17, 21, 28, 29, 33, 37, 46, 53, 54] constructed upon the Transformer model architecture [45], achieving promising performance in code-related applications [19, 20, 47—50, 52, 57, 59, 60].”
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · see citing works
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 2,523 · cited 1× · see citing works
“191:9”
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Bengio, S., Vinyals, O., Jaitly, N., & Shazeer, N. (2015) · arXiv · doi:10.48550/arxiv.1506.03099 · Cited by 1,621 · cited 11× · see citing works
“We define two new repository-level metrics, namely Dependency Coverage and Static Validity Rate .”
InfluentialWang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495 · cited 6× · see citing works
“Recent developments have introduced a range of LLMs for code (code LLMs), including Codex [15], CodeT5 [54], CodeT5 + [53], InCoder [21], AlphaCode [29], CodeGen [33], and CodeLlama [37], built upon the Transformer model architecture [45].”
InfluentialEvaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · cited 4× · see citing works
“Recent advancements have introduced a variety of code large language models (LLMs) [7, 12, 13, 15, 17, 21, 28, 29, 33, 37, 46, 53, 54] constructed upon the Transformer model architecture [45], achieving promising performance in code-related applications [19, 20, 47—50, 52, 57, 59, 60].”
InfluentialA Deep Reinforced Model for Abstractive Summarization
Paulus, R., Xiong, C., & Socher, R. (2017) · arXiv · doi:10.48550/arxiv.1705.04304 · Cited by 1,182 · cited 12× · see citing works
“However, our generalizability evaluation (RQ5) provides evidence of consistent performance across the three ToolGen variants, suggesting that the improvements achieved by ToolGen-llama in repository-level code generation are not attributed to data leakage.”
InfluentialToolformer: Language Models Can Teach Themselves to Use Tools
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2302.04761 · Cited by 1,150 · see citing works
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043 · cited 2× · see citing works
“Recent advancements have introduced a variety of code large language models (LLMs) [7, 12, 13, 15, 17, 21, 28, 29, 33, 37, 46, 53, 54] constructed upon the Transformer model architecture [45], achieving promising performance in code-related applications [19, 20, 47—50, 52, 57, 59, 60].”
LaMDA: Language Models for Dialog Applications
Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778 · see citing works
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660 · cited 1× · see citing works
“This limitation often leads to the generation of code with dependency errors , including undefined - variable and no - member errors.”
Training Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575 · see citing works
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536 · cited 4× · see citing works
“Building upon ToolFormer’s inspiration, Zhang et al. [61] introduce Teaching Code LLMs to Use Autocompletion Tools 191:3 ToolCoder, an approach designed to teach LLMs how to utilize information - retrieval-based (IR-based) API search tools during the code generation process.”
InfluentialCode Llama: Open Foundation Models for Code
Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12950 · Cited by 531 · cited 7× · see citing works
“Recent developments have introduced a range of LLMs for code (code LLMs), including Codex [15], CodeT5 [54], CodeT5 + [53], InCoder [21], AlphaCode [29], CodeGen [33], and CodeLlama [37], built upon the Transformer model architecture [45].”
InfluentialCodeSearchNet Challenge: Evaluating the State of Semantic Code Search
Husain, H., Wu, H.-H., Gazit, T., Allamanis, M., & Brockschmidt, M. (2019) · arXiv · doi:10.48550/arxiv.1909.09436 · Cited by 483 · see citing works
IntelliCode compose: code generation using transformer
Svyatkovskiy, A., Deng, S. K., Fu, S., & Sundaresan, N. (2020) · Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3368089.3417058 · Cited by 474 · cited 1× · see citing works
“— Edit Similarity (EditSim) [41].”
CodeT5+: Open Code Large Language Models for Code Understanding and Generation
Wang, Y., Le, H., Gotmare, A. D., Bui, N. D. Q., Li, J., & Hoi, S. C. H. (2023) · arXiv · doi:10.48550/arxiv.2305.07922 · Cited by 459 · cited 3×
“…2 and 3, our tool-integrated generation approach consistently enhances performance in dependency-based metrics while maintaining comparable similarity-based metrics across various model architectures (decoder-only and encoder–decoder) and parameter scales (ranging from 124 million to 7 billion).”
Serrano, M., & Findler, R. (2020) · Artifact Digital Object Group · doi:10.1145/3373123 · Cited by 407 · cited 1×
“All code and data can be found at our replication package [4].”
Program Synthesis with Large Language Models
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07732 · Cited by 388 · see citing works
Internet-Augmented Dialogue GenerationPreprint
Komeili, M., Shuster, K., & Weston, J. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2107.07566 · Cited by 333 · see citing works
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
Nijkamp, E., Pang, B., Hayashi, H., Tu, L., Wang, H., Zhou, Y., Savarese, S., & Xiong, C. (2022) · arXiv · doi:10.48550/arxiv.2203.13474 · Cited by 330 · cited 3× · see citing works
“Recent advancements have introduced a variety of code large language models (LLMs) [7, 12, 13, 15, 17, 21, 28, 29, 33, 37, 46, 53, 54] constructed upon the Transformer model architecture [45], achieving promising performance in code-related applications [19, 20, 47—50, 52, 57, 59, 60].”
GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow
Black, S., Leo, G., Wang, P., Leahy, C., & Biderman, S. (2021) · Zenodo · doi:10.5281/zenodo.5297715 · Cited by 324 · cited 1× · see citing works
“Recent advancements have introduced a variety of code large language models (LLMs) [7, 12, 13, 15, 17, 21, 28, 29, 33, 37, 46, 53, 54] constructed upon the Transformer model architecture [45], achieving promising performance in code-related applications [19, 20, 47—50, 52, 57, 59, 60].”
WebGPT: Browser-assisted question-answering with human feedback
Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.09332 · Cited by 272 · see citing works
StarCoder: may the source be with you!
Li, R., Allal, L. B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.06161 · Cited by 261 · see citing works
No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test GenerationPreprint
Yuan, Z.-Q., Lou, Y., Liu, M.-W., Ding, S., Wang, K.-X., Chen, Y.-X., & Peng, X. (2023) · arXiv.org · doi:10.48550/arxiv.2305.04207 · Cited by 239 · see citing works
Self-citationART: Automatic multi-step reasoning and tool-use for large language modelsPreprint
Paranjape, B., Lundberg, S. M., Singh, S., Hajishirzi, H., Zettlemoyer, L., & Ribeiro, M. T. (2023) · arXiv.org · doi:10.48550/arxiv.2303.09014 · Cited by 239 · cited 1× · see citing works
“Static Autocompletion Tools for Dynamically Typed Programming Languages: Currently, our implementation of ToolGen is specific to Python, a dynamically typed programming language.”
Internet-Augmented Dialogue Generation
Komeili, M., Shuster, K., & Weston, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.579 · Cited by 192 · see citing works
vul-rag: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG
Du, X.-Y., Zheng, G., Wang, K.-X., Zou, Y., Wang, Y.-J., Deng, W.-T., Feng, J.-Y., Liu, M.-W., Chen, B., Peng, X., et al. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3797277 · Cited by 188 · see citing works
Self-citationInCoder: A Generative Model for Code Infilling and Synthesis
Fried, D., Aghajanyan, A., Lin, J., Wang, S., Wallace, E., Shi, F., Zhong, R., Yih, W.-T., Zettlemoyer, L., & Lewis, M. (2022) · arXiv · doi:10.48550/arxiv.2204.05999 · Cited by 187 · cited 2× · see citing works
“Recent advancements have introduced a variety of code large language models (LLMs) [7, 12, 13, 15, 17, 21, 28, 29, 33, 37, 46, 53, 54] constructed upon the Transformer model architecture [45], achieving promising performance in code-related applications [19, 20, 47—50, 52, 57, 59, 60].”
Evaluating Large Language Models in Class-Level Code Generation
Du, X., Liu, M., Wang, K., Wang, H., Jun-wei, L., Chen, Y., Feng, J., Sha, C., Peng, X., & Lou, Y. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639219 · Cited by 180 · cited 1× · see citing works
“Recent advancements have introduced a variety of code large language models (LLMs) [7, 12, 13, 15, 17, 21, 28, 29, 33, 37, 46, 53, 54] constructed upon the Transformer model architecture [45], achieving promising performance in code-related applications [19, 20, 47—50, 52, 57, 59, 60].”
Self-citationRepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Zhang, F., Chen, B., Zhang, Y., Keung, J., Liu, J., Zan, D., Mao, Y., Lou, J.-G., & Chen, W. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.151 · Cited by 173 · see citing works
PAL: Program-aided Language Models
Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., & Neubig, G. (2022) · arXiv · doi:10.48550/arxiv.2211.10435 · Cited by 155 · see citing works
Arora, K., Asri, L. E., Bahuleyan, H., & Cheung, J. (2022) · Findings · doi:10.48550/arxiv.2204.01171 · Cited by 155 · cited 1×
“However, ToolGen-llama exhibits a lower pass rate than RepoCoder-llama (15 vs. 19).”
Shuster, K., Komeili, M., Adolphs, L., Roller, S., Szlam, A., & Weston, J. (2022) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2203.13224 · Cited by 154
Chen, W., Ma, X., Wang, X., & Cohen, W. W. (2022) · arXiv · doi:10.48550/arxiv.2211.12588 · Cited by 125 · cited 1× · see citing works
“Recent research [16, 18, 22, 26, 27, 32, 35, 38, 40, 42, 61] has explored the integration of external tools (e.g., search engines, web browsers, calculators, and python interpreters) into the LLM generation process, aiming to address their limitations in certain generation scenarios.”
Wei, Y., Xia, C. S., & Zhang, L. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616271 · Cited by 115 · cited 1× · see citing works
“Additionally, ToolFormer and ToolCoder are unable to handle scenarios where the tools return multiple candidates.”
Compilable Neural Code Generation with Compiler Feedback
Wang, X., Wang, Y., Wan, Y., Mi, F., Li, Y., Zhou, P., Liu, J., Wu, H., Jiang, X., & Liu, Q. (2022) · Findings · doi:10.48550/arxiv.2203.05132 · Cited by 102 · cited 1×
“These code LLMs demonstrate impressive performance in code generation and have found extensive utilization in prior studies [31, 37, 51, 53, 54] for fine-tuning and evaluation.”
CodePlan: Repository-level Coding using LLMs and Planning
Bairi, R., Sonwane, A., Kanade, A., C, V. D., Iyer, A., Parthasarathy, S., Rajamani, S., Ashok, B., & Shet, S. (2023) · arXiv · doi:10.48550/arxiv.2309.12499 · Cited by 92 · see citing works
CodePlan: Repository-Level Coding using LLMs and Planning
Bairi, R., Sonwane, A., Kanade, A., C., V. D., Iyer, A., Parthasarathy, S., Rajamani, S., Ashok, B., & Shet, S. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3643757 · Cited by 92 · see citing works
PanGu-Coder: Program Synthesis with Function-Level Language ModelingPreprint
Christopoulou, F., Lampouras, G., Gritta, M., Zhang, G., Guo, Y., Li, Z.-Q., Zhang, Q., Xiao, M., Shen, B., Li, L., et al. (2022) · arXiv.org · doi:10.48550/arxiv.2207.11280 · Cited by 88 · cited 1× · see citing works
“Recent advancements have introduced a variety of code large language models (LLMs) [7, 12, 13, 15, 17, 21, 28, 29, 33, 37, 46, 53, 54] constructed upon the Transformer model architecture [45], achieving promising performance in code-related applications [19, 20, 47—50, 52, 57, 59, 60].”
Zhang, J.-N., Huang, K.-F., Huang, Y.-H., Chen, B., Wang, R., Wang, C., & Peng, X. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3705304 · Cited by 69 · see citing works
Self-citationGPT-NeoX-20B: An Open-Source Autoregressive Language Model
Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.06745 · Cited by 67 · cited 1× · see citing works
“Recent advancements have introduced a variety of code large language models (LLMs) [7, 12, 13, 15, 17, 21, 28, 29, 33, 37, 46, 53, 54] constructed upon the Transformer model architecture [45], achieving promising performance in code-related applications [19, 20, 47—50, 52, 57, 59, 60].”
Internet-augmented language models through few-shot prompting for open-domain question answering
Lazaridou, A., Gribovskaya, E., Stokowiec, W., & Grigorev, N. (2022) · arXiv · doi:10.48550/arxiv.2203.05115 · Cited by 64 · see citing works
SantaCoder: don't reach for the stars!
Allal, L. B., Li, R., Kocetkov, D., Mou, C., Akiki, C., Ferrandis, C. M., Muennighoff, N., Mishra, M., Gu, A., Dey, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.03988 · Cited by 61 · see citing works
Guiding Language Models of Code with Global Context using MonitorsPreprint
Agrawal, L. A., Kanade, A., Goyal, N., Lahiri, S. K., & Rajamani, S. (2023) · arXiv.org · doi:10.48550/arxiv.2306.10763 · Cited by 37 · cited 1× · see citing works
“More relevant examples are Repilot [58], STALL + [30], and MGD [6], which utilize code completion tools to filter out impractical suggestions made by LLMs, focusing on generating API/line-level code completions and valid bug-fix patches rather than entire functions.”
Automatically Detecting the Scopes of Source Code Comments
Chen, H., Huang, Y., Liu, Z., Chen, X.-P., Zhou, F., & Luo, X. (2018) · Annual International Computer Software and Applications Conference · doi:10.1109/compsac.2018.00029 · Cited by 35 · cited 1×
“Extracting descriptions for code blocks outside functions for model training and evaluation is challenging, due to the difficulty in determining the scope of line comments [14, 24].”
Repository-Level Prompt Generation for Large Language Models of Code
Shrivastava, D., Larochelle, H., & Tarlow, D. (2022) · arXiv · doi:10.48550/arxiv.2206.12839 · Cited by 27 · see citing works
Evaluating Large Language Models for Line-Level Vulnerability Localization
Zhang, J., Wang, C., Li, A., Sun, W.-S., Zhang, C., Ma, W., & Liu, Y. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3649250 · Cited by 27 · see citing works
Self-citationBoosting Static Resource Leak Detection via LLM-based Resource-Oriented Intention Inference
Wang, C., Liu, J., Peng, X., Liu, Y., & Lou, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00131 · Cited by 25 · see citing works
Self-citationA Comparative Study on Method Comment and Inline Comment
Huang, Y., Guo, H., Ding, X., Shu, J., Chen, X.-P., Luo, X.-P., Zheng, Z.-B., & Zhou, X.-C. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3582570 · Cited by 24 · cited 1× · see citing works
“Extracting descriptions for code blocks outside functions for model training and evaluation is challenging, due to the difficulty in determining the scope of line comments [14, 24].”
Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models
Wang, Y., Jiang, T., Liu, M., Chen, J., Mao, M., Liu, X., Ma, Y., & Zheng, Z. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715749 · Cited by 21 · cited 1× · see citing works
“Recent advancements have introduced a variety of code large language models (LLMs) [7, 12, 13, 15, 17, 21, 28, 29, 33, 37, 46, 53, 54] constructed upon the Transformer model architecture [45], achieving promising performance in code-related applications [19, 20, 47—50, 52, 57, 59, 60].”
How and Why LLMs Use Deprecated APIs in Code Completion? An Empirical Study
Wang, C., Huang, K.-F., Zhang, J., Feng, Y., Zhang, L., Liu, Y., & Peng, X. (2024) · arXiv.org · doi:10.48550/arxiv.2406.09834 · Cited by 17
Self-citationSTALL + : Boosting LLM-based Repository-level Code Completion with Static Analysis
Jun-wei, L., Chen, Y., Liu, M., Peng, X., & Lou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3846180 · Cited by 12 · cited 1× · see citing works
“More relevant examples are Repilot [58], STALL + [30], and MGD [6], which utilize code completion tools to filter out impractical suggestions made by LLMs, focusing on generating API/line-level code completions and valid bug-fix patches rather than entire functions.”
Self-citationRepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Zhang, F., Chen, B., Zhang, Y., Keung, J., Liu, J., Zan, D., Mao, Y., Lou, J.-G., & Chen, W. (2023) · arXiv · Cited by 10 · see citing works
TIGER: A Generating-Then-Ranking Framework for Practical Python Type Inference
Wang, C., Zhang, J., Lou, Y., Liu, M., Sun, W., Liu, Y., & Peng, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00019 · Cited by 9 · cited 1× · see citing works
“Recent advancements have introduced a variety of code large language models (LLMs) [7, 12, 13, 15, 17, 21, 28, 29, 33, 37, 46, 53, 54] constructed upon the Transformer model architecture [45], achieving promising performance in code-related applications [19, 20, 47—50, 52, 57, 59, 60].”
Self-citationGenerating Variable Explanations via Zero-shot Prompt Learning
Wang, C., Lou, Y., Liu, J., & Peng, X. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00130 · Cited by 8 · see citing works
Self-citation
Flagged first, then by citations
8 of the 65 references couldn’t be matched to a known paper.
Research works printed but not linked: 7
- Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, Eds., 2017, pp. 5998-6008. [Online]. Available: https://proceedings.neurips.cc/paper/ 2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
- J. Zhang, C. Wang, A. Li, W. Sun, C. Zhang, W. Ma, and Y. Liu, "An empirical study of automated vulnerability localization with large language models," arXiv preprint arXiv:2404.00287, 2024. · arXiv
- J. Zhang, K. Huang, B. Chen, C. Wang, Z. Tian, and X. Peng, "Malicious package detection in npm and pypi using a single model of malicious behavior sequence," arXiv preprint arXiv:2309.02637, 2023. · arXiv
- H. Yu, B. Shen, D. Ran, J. Zhang, Q. Zhang, Y. Ma, G. Liang, Y. Li, T. Xie, and Q. Wang, "Codereval: A benchmark of pragmatic code generation with generative pre-trained models," arXiv preprint arXiv:2302.00288, 2023. · arXiv
- K. Zhang, G. Li, J. Li, Z. Li, and Z. Jin, "Toolcoder: Teach code generation models to use apis with search tools," arXiv preprint arXiv:2305.04032, 2023. · arXiv
- Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, Eds., vol. 202. PMLR, 2023, pp. 31 693-31 715. [Online]. Available: https://proceedings.mlr.press/v202/shrivastava23a.html
- Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, Eds., vol. 202. PMLR, 2023, pp. 10 764-10 799. [Online]. Available: https://proceedings.mlr.press/v202/gao23f.html
Cited by
Sorted by citations
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029 · see citing works
Self-citationEvaluating Large Language Models in Class-Level Code Generation
Du, X., Liu, M., Wang, K., Wang, H., Jun-wei, L., Chen, Y., Feng, J., Sha, C., Peng, X., & Lou, Y. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639219 · Cited by 180 · see citing works
“These include code generation [46, 54, 62, 72], code summarization [12, 51] and various software maintenance tasks, including vulnerability detection [58, 61, 73], test generation [68, 69], and program repair [26, 36, 37, 63].”
MethodSelf-citationLLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation
Zhang, Z., Wang, C., Wang, Y., Shi, E., Ma, Y., Zhong, W., Chen, J., Mao, M., & Zheng, Z. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728894 · Cited by 131 · see citing works
“Recent works attempt to integrate static analysis tools [42] or apply retrieval-augmented generation (RAG) based on repository-level retrieval corpora [49] to address such context awareness issues.”
Self-citationA Survey on Large Language Models for Code Generation
Jiang, J., Wang, F., Shen, J., Kim, S., & Kim, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3747588 · Cited by 251 · see citing works
“GitHub Copilot [62] Code Llama [63] ToolGen [64] B. Advanced Code”
Source Code Summarization in the Era of Large Language ModelsPreprint
Sun, W.-S., Miao, Y., Li, Y., Zhang, H.-Y., Fang, C.-R., Liu, Y., Deng, G.-L., Liu, Y., & Chen, Z.-Y. (2024) · International Conference on Software Engineering · doi:10.48550/arxiv.2407.07959 · Cited by 97 · see citing works
“…processing (NLP) [13], [14], an increasing number of software engineering (SE) researchers have started integrating them into the resolution process of various SE tasks [15]–[18], such as code generation [19], [20], program repair [21], [22], and vulnerability detection/localization [23], [24].”
Self-citationAgents in software engineering: survey, landscape, and vision
Wang, Y., Zhong, W., Huang, Y., Shi, E., Yang, M., Chen, J., Li, H., Ma, Y., Wang, Q., & Zheng, Z. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00544-2 · Cited by 50 · see citing works
The Current Challenges of Software Engineering in the Era of Large Language Models
Gao, C., Hu, X., Gao, S., Xia, X., & Jin, Z. G. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3712005 · Cited by 65 · see citing works
“ToolGen [151] integrates auto-completion tools to resolve the dependency issues in repository-level code generation.”
RLCoder: Reinforcement Learning for Repository-Level Code Completion
Wang, Y., Wang, Y., Guo, D., Chen, J., Zhang, R., Ma, Y., & Zheng, Z. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00014 · Cited by 43 · see citing works
“Repository-level code completion, which leverages the broader context of an entire code repository, has become a focal point for research in the field of code completion and many studies have attempted to improve repository-level code completion performance [12]–[17], [19], [21], [48].”
BackgroundMethodLLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-Based Code Completion
Wang, C., Huang, K., Zhang, J., Feng, Y., Zhang, L., Liu, Y., & Peng, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00245 · Cited by 17 · see citing works
“Large language models (LLMs) [1], [2], [3], [4], [5], [6] have significantly advanced various aspects of software engineering [7], including code completion [8], [9], [10], code understanding [11], [12], defect detection [13], [14], and program repair [15], [16], [17].”
Self-citationHuynh, N., & Lin, B. (2025) · arXiv.org · doi:10.48550/arxiv.2503.01245 · Cited by 62 · see citing works
“GitHub Copilot [62] Code Llama [63] ToolGen [64] B. Advanced Code”
Large Language Model-Based Agents for Software Engineering: A Survey
Liu, J., Wang, K., Chen, Y., Peng, X., Chen, Z., Zhang, L., & Lou, Y. (2024) · arXiv · doi:10.48550/arxiv.2409.02977 · Cited by 49 · see citing works
Self-citationSTALL + : Boosting LLM-based Repository-level Code Completion with Static Analysis
Jun-wei, L., Chen, Y., Liu, M., Peng, X., & Lou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3846180 · Cited by 12 · see citing works
“For static analysis-integrated techniques [9, 22, 63, 74], researchers focus on integrating static analysis into different phases ( e.g., prompting phase or decoding phase) of LLM-based repository-level code completion.”
InfluentialSelf-citationBoosting Static Resource Leak Detection via LLM-based Resource-Oriented Intention Inference
Wang, C., Liu, J., Peng, X., Liu, Y., & Lou, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00131 · Cited by 25 · see citing works
Self-citationRetrieval-Augmented Code Generation: A Survey with Focus on Repository-Level ApproachesPreprint
Tao, Y.-C., Li, Y.-T., Qin, Y., & Liu, Y. (2025) · arXiv.org · doi:10.48550/arxiv.2510.04905 · Cited by 41 · see citing works
“ToolGen [21] fine-tunes models with augmented functions annotated by a <COMP> token, enabling the model to learn trigger points for invoking external completion tools.”
Self-citationUncertainty-Guided Chain-of-Thought for Code Generation with LLMs
Zhu, Y.-Q., Li, G., Jiang, X., Li, J., Mei, H., Jin, Z., & Dong, Y. (2025) · arXiv.org · doi:10.48550/arxiv.2503.15341 · Cited by 40
“Second, targeted fine-tuning optimizes LLMs using domain-specific datasets, sharp-ening their ability to generate contextually relevant code [19, 36, 40].”
Repository-level Code Translation Benchmark Targeting Rust
Ou, G.-S., Liu, M.-W., Chen, Y., Peng, X., & Zheng, Z.-B. (2024) · arXiv.org · doi:10.48550/arxiv.2411.13990 · Cited by 35
“Many of these stem from using non-existent, unimplemented, unresolved, or undeclared elements, highlighting the hallucination phenomenon also seen in other repository-level code generation tasks [59], [60].”
Self-citationEsale: Enhancing Code-Summary Alignment Learning for Source Code Summarization
Fang, C.-R., Sun, W.-S., Chen, Y., Chen, X., Wei, Z., Zhang, Q.-J., You, Y., Luo, B., Liu, Y., & Chen, Z.-Y. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3422274 · Cited by 34 · see citing works
Self-citationBeyond pip Install: Evaluating LLM Agents for the Automated Installation of Python Projects
Milliken, L., Kang, S., & Yoo, S. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00009 · Cited by 5 · see citing works
“…self-consistency [12]) and LLM-based architectures (such as agent [13] and multi-agent architectures [14], [15]) have been adopted to perform “repository-level” tasks [16], [17], which we define in this paper to be tasks that require reading and/or writing multiple files in a given repository.”
Prompt-Based Code Completion via Multi-Retrieval Augmented Generation
Tan, H., Luo, Q., Jiang, L., Zhan, Z.-Z., Li, J., Zhang, H.-T., & Zhang, Y. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3725812 · Cited by 30 · see citing works
Hallucination by Code Generation LLMs: Taxonomy, Benchmarks, Mitigation, and Challenges
Lee, Y., Song, J. Y., Kim, D., Kim, J., Kim, M., & Nam, J. (2025) · arXiv.org · doi:10.48550/arxiv.2504.20799 · Cited by 24
“…under the Benchmarking category (20 papers [3, 6–9, 11, 17, 23, 25, 27–30, 35, 38, 49, 50, 60, 66, 67]) and the Mitigation category (16 papers [12, 13, 21, 26, 32, 36, 39, 40, 43, 45, 48, 51, 54, 55, 62, 63]), while fewer studies are categorized under Taxonomy (five papers [15, 24, 42, 52, 57]).”
Marking Code Without Breaking It: Code Watermarking for Detecting LLM-Generated Code
Kim, J., Park, S., & Han, Y.-S. (2025) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2502.18851 · Cited by 18
“These advancements have unlocked new possibilities, including software development and automated code generation (Nam et al., 2024; Wang et al., 2024; Guo et al., 2024a).”
RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories
Wang, Y.-L., Zhang, Z., Wang, C., Xu, X.-Y., Liu, M.-W., Wang, Y., Chen, J.-C., & Zheng, Z.-B. (2026) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2601.22706 · Cited by 17
“Ultimately, prompt-based security guidance lacks uniformity and fails to reliably enhance composite performance on average.”
Self-citationExploring the Potential of Diffusion Large Language Models in Code Generation
Li, C.-Z., Zhang, Y.-T., Li, J., Cai, L.-Y., & Li, G. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3848514 · Cited by 17
Chang, J., Zhou, X., Wang, L.-L., Lo, D., & Li, B.-X. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3668601 · Cited by 17 · see citing works
“For parameter-efficient fine-tuning, we inserted LoRA adapters into all linear layers, with lora_rank = 8, lora_alpha = 16, and lora_dropout = 0 [47], [48].”
How Far Are AI Scientists from Changing the World?Preprint
Xie, Q.-J., Weng, Y.-X., Zhu, M.-J., Shen, F.-C., Huang, S.-L., Lin, Z., Zhou, J.-H., Mao, Z.-L., Yang, Z.-J., Yang, L.-Y., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2507.23276 · Cited by 16 · see citing works
“CodeAgent (Zhang et al., 2024a) integrates five programming tools, enabling interaction with software artifacts for information retrieval, code symbol navigation, and code testing.”
A Systematic Literature Review of Parameter-Efficient Fine-Tuning for Large Code Models
Afrin, S., Haque, M. Z., & Mastropaolo, A. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3796522 · Cited by 6 · see citing works
TIGER: A Generating-Then-Ranking Framework for Practical Python Type Inference
Wang, C., Zhang, J., Lou, Y., Liu, M., Sun, W., Liu, Y., & Peng, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00019 · Cited by 9 · see citing works
“This challenge manifests as a limitation in their coverage, hindering their ability to provide comprehensive type inference scenarios.”
Self-citationTowards an Understanding of Context Utilization in Code Intelligence
Wang, Y., Duan, K., Zheng, D., Shi, E., Zhang, F., Wang, Y., Chen, J., Liu, X., Ma, Y., Zhang, H., et al. (2026) · ACM Computing Surveys · doi:10.1145/3797261 · Cited by 2 · see citing works
“To assess the evaluation and reuse of resources in context-relevant studies, we examine the metrics used to measure model performance, the benchmarks developed to validate context-aware models, and the open-source status of the papers.”
Enhancing Project-Specific Code Completion by Inferring Internal API Information
Deng, L., Ren, X., Ni, C., Liang, M., Lo, D., & Liu, Z.-X. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3592823 · Cited by 13 · see citing works
“Consequently, the study of project-specific code completion has gained attention, with many efforts [12], [13], [14], [15], [16], [17], [18], [44], [45], [46] aimed at capturing and combining project context information.”
The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates
Russo, G., Horta Ribeiro, M., Davidson, T. R., Veselovsky, V., & West, R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757667 · Cited by 12 · see citing works
“The final classification was done on 26 April 2024 to ensure that a single API checkpoint (dated 4 April 2024) can be used to reproduce our results.”
AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion
Jiang, T., Wang, Y., Wang, Y., Guo, D., Shi, E., Ma, Y., Chen, T., & Zheng, Z. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00085 · Cited by 1 · see citing works
“Tool-augmented methods such as CodeAgent [58] and ToolGen [59] investigate external tool invocation, while RLCoder [26] employs reinforcement learning for repository-specific retrieval optimization.”
DroidCoder: Enhanced Android Code Completion with Context-Enriched Retrieval-Augmented Generation
Yu, X., Li, C., Pan, M., & Li, X. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695063 · Cited by 5 · see citing works
“Furthermore, following the previous works [19, 46], we also experiment to evaluate the quality of the code generated by DroidCoder.”
Beyond PEFT: Layer-Wise Optimization for More Effective and Efficient Large Code Model Tuning
Wang, C., Feng, J., Gao, S., Gao, C., Li, Z., Peng, T., Huang, H., Deng, Y., & Lyu, M. R.-T. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729341 · Cited by 4 · see citing works
Self-citationLLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities
Tang, Y.-J., & Runkler, T. A. (2026) · arXiv.org · doi:10.18420/se2026-ws_15 · Cited by 9
Zheng, D., Wang, Y., Shi, E., Zhang, R., Ma, Y., Zhang, H., & Zheng, Z. (2025) · 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE) · doi:10.1109/icse55347.2025.00228 · Cited by 11 · see citing works
“Many technical studies have been proposed to better tackle repository-level code generation tasks [2], [4], [6], [48]–[50], [65]–[69], [77], [78].”
RustRepoTrans: Repository-level Context Code Translation Benchmark Targeting Rust
Ou, G.-S., Liu, M.-W., Chen, Y., Wang, Y.-L., Peng, X., & Zheng, Z.-B. (2024) · International Conference on Automated Software Engineering · doi:10.1109/ase63991.2025.00057 · Cited by 8 · see citing works
“Many of these stem from using non-existent, unimplemented, unresolved, or undeclared elements, highlighting the hallucination phenomenon also seen in other repository-level code generation tasks [59], [60].”
Self-citationTowards Trustworthy LLMs for Code: A Data-Centric Synergistic Auditing Framework
Wang, C., Chen, Z.-P., Li, T.-L., Zhang, Y.-L., & Liu, Y. (2024) · 2025 IEEE/ACM 47th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier66352.2025.00017 · Cited by 8 · see citing works
Self-citationImproving Parallel Program Performance Through DSL-Driven Code Generation with LLM OptimizersPreprint
Wei, A., Nie, A., Teixeira, T. S. F. X., Yadav, R., Lee, W., Wang, K., & Aiken, A. (2024) · arXiv.org · doi:10.48550/arxiv.2410.15625 · Cited by 8 · see citing works
“While LLMs have shown success in generating isolated function-level Python code (Chen et al., 2021; Austin et al., 2021), they face challenges when generating system-level C++ code in large repositories due to complex APIs and broader contextual dependencies (Du et al., 2023; Wang et al., 2024).”
BackgroundRepoReasoner: Evaluating Repository-Level Code Reasoning Ability of Long-Context Language Models
Wang, Y., Wang, S., Wang, Y., Zhang, B., Guo, D., Chen, J., & Zheng, Z. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808131
“Tool-integrated approaches such as CodeAgent [62] and context-aware frameworks [33, 46] further extend these capabilities by incorporating external tools and autocompletion into the generation loop.”
ATGen: Adversarial Reinforcement Learning for Test Case GenerationPreprint
Li, Q., Dai, X.-Y., Liu, W.-W., Li, X., Wang, Y.-S., Tang, R.-M., Yu, Y., & Zhang, W.-N. (2025) · arXiv.org · doi:10.48550/arxiv.2510.14635 · Cited by 7 · see citing works
“Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation (Wang & Chen, 2023; Jiang et al., 2024; Wang et al., 2025; Huang et al., 2024), tackling a wide range of programming tasks (Etsenake & Nagappan, 2024; Nijkamp et al., 2023; Li et al., 2024; 2025).”
A Deep Dive into Retrieval-Augmented Generation for Code Completion: Experience on WeChat
Yang, Z.-Z., Peng, T., Gao, C.-Y., Wang, C.-Z., Huang, H.-L., & Deng, Y. (2025) · IEEE International Conference on Software Maintenance and Evolution · doi:10.1109/icsme64153.2025.00062 · Cited by 7 · see citing works
“Code completion, which automatically predicts and suggests code fragments based on the surrounding programming context, has evolved from simple token-level suggestions to generating entire code blocks [1], [2].”
Self-citationWang, B.-Y., Wang, X.-L., Li, G.-C., Zhi, C., Han, J.-X., Zhao, X.-K., Wang, N., Deng, S.-G., & Yin, J.-W. (2026) · Proceedings of the ACM on Software Engineering · doi:10.1145/3832136 · Cited by 6
Envisioning Intelligent Requirements Engineering via Knowledge-Guided Multi-Agent Collaboration
Huang, J.-P., Jin, D.-M., Sun, W.-S., Liu, Y., & Jin, Z. (2025) · International Conference on Automated Software Engineering · doi:10.1109/ase63991.2025.00334 · Cited by 6 · see citing works
“The emergence of Large Language Models (LLMs) has reshaped many software engineering tasks, from code generation [5] to requirements modeling [6].”
Self-citationSALE : Low-bit Estimation for Efficient Sparse Attention in Long-context LLM PrefillingPreprint
Ji, X.-D., Zhang, H., Fu, F.-C., & Cui, B. (2025) · arXiv.org · doi:10.48550/arxiv.2505.24179 · Cited by 5 · see citing works
“…context understanding in many complex applications such as long book summarization [1–3], long document question-answering [4–6], and repository-level code completion [7, 8], state-of-the-art Large Language Models (LLM) are now capable of supporting increasingly longer context window [9–12].”
Anchor Attention, Small Cache: Code Generation With Large Language Models
Zhang, X., Zhou, Y., Yang, G., Gall, H. C., & Chen, T. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3570680 · Cited by 6 · see citing works
“For instance, in repository-level code generation [23], [24], [25], the relevant context that needs to be considered during generation comes from not only the current, but also externals, files, e.g., imported packages, source code files in the same directory, configuration files and even API…”
Advancing Binary Code Similarity Detection via Context-Content Fusion and LLM Verification
Dong, C.-P., Guo, J., Yang, S.-G., Li, Y., Fang, D.-L., Xiao, Y., Chen, Y.-L., & Sun, L.-M. (2025) · International Conference on Automated Software Engineering · doi:10.1109/ase63991.2025.00033 · Cited by 4
“In the era of large language models (LLMs), numerous researchers have integrated LLMs into a wide array of software engineering tasks, including code generation [21], [22], code summarization [23], [24], and program repair [25], [26].”
Han, J.-X., Yu, Z., Bao, L.-F., Liu, J., Wan, Y., Yin, J., Deng, S.-G., & Han, S. (2025) · arXiv.org · doi:10.48550/arxiv.2511.08060 · Cited by 4
“In recent years, Large Language Models (LLMs) have achieved remarkable success across a wide range of domains [18], including code generation [19]–[21], code repair [22]– [24], and vulnerability detection [25]–[27].”
Code Digital Twin: Empowering LLMs with Tacit Knowledge for Complex Software Development
Peng, X., & Wang, C. (2025) · arXiv.org · doi:10.48550/arxiv.2510.16395 · Cited by 4
“Our proposed methodology integrates LLMs, static analysis, and human expertise to construct and continuously evolve this representation in alignment with software development.”
Self-citationHelloBench: Evaluating Long Text Generation Capabilities of Large Language Models
Que, H., Duan, F., He, L., Mou, Y., Zhou, W., Liu, J., Rong, W., Wang, Z. M., Yang, J., Zhang, G., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.16191 · Cited by 3 · see citing works
“…applications, long text generation tasks correlate with various real-world uses of LLMs, such as story generation (Venkatraman et al., 2024; Bai et al., 2024; Zhou et al., 2023), repository-level code completion (Liu et al., 2024b; Wang et al., 2024a), document generation (Luo et al., 2024), etc.”
BackgroundBeyond Translation Accuracy: Addressing False Failures in LLM-Based Code Translation
Rabbi, F., Saha, S. K., & Yang, J.-Q. (2026) · AIware · doi:10.1145/3805760.3814890 · Cited by 3
BabelCoder: Agentic Code Translation with Specification Alignment
Rabbi, F., Saha, S. K., Pham, T. M. T., Wang, S., & Yang, J.-Q. (2025) · arXiv.org · doi:10.48550/arxiv.2512.06902 · Cited by 3
“In contrast, TOOLGEN [36] integrates auto-completion tools into LLM-based code generation to better mimic developer-tool interaction.”
Beyond Sequences: Two-dimensional Representation and Dependency Encoding for Code Generation
Zhang, X., Zhou, Y. L., Yang, G., Cheng, W., & Chen, T. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.308 · Cited by 1 · see citing works
Beyond Repository Boundaries: Cross-Repository Graph Retrieval for Code Generation
Le-Anh, M., Hai, N. L., Tran, Q., Hoang, A. N., Van, L. N., Le, B., & Bui, N. (2026) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/2026.findings-emnlp.433 · Cited by 2
“…models for code (CodeLLMs) has expanded automatic code generation from isolated programming tasks toward implementations that must be integrated into existing projects (Nguyen et al., 2023; Wang et al., 2025b; Tao et al., 2026; Hai et al., 2025; Zhang et al., 2024; Athale and Vaddina, 2025).”
How to Ask the AI: A User Perspective Survey for Large Language Model Prompting
Zhang, Y.-Q., Zhang, Y.-F., Zhao, M.-J., Feng, S., & Cheung, Y.-M. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2026.3700136 · Cited by 2
“…structures and contextual information [8], thus enhancing their ability to generalize across tasks and domains, e.g., knowledge question and answering (Q&A) [9], general conversation [10], code generation [11], and research assistance [12] for general users, scientists, and AI researchers.”
Chen, Z., Li, Z., Song, Z., Shi, Z.-Q., & Sun, L.-M. (2025) · International Conference on Parallel and Distributed Systems · doi:10.1109/icpads67057.2025.11322923 · Cited by 2 · see citing works
Design-Specification Tiling for ICL-based CAD Code Generation
Du, Y., Xi, S.-Z., Sun, H., & Li, M. (2026) · arXiv.org · doi:10.48550/arxiv.2603.12712 · Cited by 1
“Recent advances in large language models (LLMs) have demonstrated remarkable capabilities in code generation [Liu et al. , 2025a; Yang et al. , 2024; Lyu et al. , 2025; Wang et al. , 2025a], motivating their application to CAD code generation [Guan et al. , 2025; Niu et al. , 2025].”
Hanif, T. H., Kusumo, D. S., & Selviandro, N. (2025) · 2025 International Conference on Artificial Intelligence, Blockchain, Cloud Computing, and Data Analytics (ICoABCD) · doi:10.1109/icoabcd67551.2025.11470978 · Cited by 1 · see citing works
“Wang et al. [14] measured the latency for code generation per function, reporting times between 0.63 and 2.34 minutes.”
InfluentialGuo, H.-Y., Xie, X.-H., Dai, H.-N., Di, P., Zhang, Y., Tao, B., & Zheng, Z. (2025) · arXiv.org · doi:10.48550/arxiv.2507.10103 · Cited by 1
Leveraging Generative AI for Accelerating Enterprise Application Development: Insights from ChatGPT
Rajbhoj, A., Sant, T., Somase, A., & Kulkarni, V. (2024) · Asia-Pacific Software Engineering Conference · doi:10.1109/apsec65559.2024.00052 · Cited by 1 · see citing works
Demonstration Attack against In-Context Learning for Code Intelligence
Ge, Y., Sun, W., Lou, Y., Fang, C., Zhang, Y., Li, Y., Zhang, X., Liu, Y., Zhao, Z., & Chen, Z. (2024) · arXiv · doi:10.48550/arxiv.2410.02841
“LLMs trained on large-scale code repositories, such as Star-Chat [42] and CodeLLama [34], have demonstrated exceptional performance in code intelligence tasks like code generation [8, 45], code summarization [1, 10, 38, 39] and code translation [22, 44].”
MethodSelf-citationNing, X., Tieu, K., Fu, D., Wei, T., Li, Z., Bei, Y., Zou, J., Ai, M., Liu, Z., Li, T.-W., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.18747
Self-citationCode Digital Twin: A Knowledge Infrastructure for AI-Assisted Complex Software DevelopmentPreprint
Peng, X., & Wang, C. (2025) · arXiv.org · doi:10.48550/arxiv.2503.07967
“…ahead, key challenges include scaling twin construction and updates to large, fast-evolving projects, governing knowledge quality and provenance, expanding from code-only in-stantiations to broader artifact and history traceability, and establishing evaluation protocols that measure how such…”
Self-citationRevisiting LLMs on New Feature Implementation in Real-World Software Development Practices
Li, J., Huang, T.-X., Zhao, T., Wang, Y.-M., Zhang, Y.-R., Hu, A.-S., Zhang, K., Wang, L., Sheng, Z.-N., Liu, K., et al. (2026) · Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3832783.3837511
Formula Completion Suggestions for Alloy Models with Selection Guidance
Patwary, M. N., & Sullivan, A. (2026) · Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3832783.3837517
Zhang, T., Mao, X., Zhao, Y., Yang, K., Zhang, Z., Lu, Y., Chen, Y., & Yu, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3849809
“Wang et al. propose ToolGen [50], which fine-tunes LLMs to trigger autocompletion tools.”
Framework for Safety Analysis of LLM-Generated Code in Multi-Agent Systems
Avagian, D., & Ayrapetyants, K. A. (2026) · Automatic Documentation and Mathematical Linguistics · doi:10.3103/s000510552670072x
RepoOMP: Repository-Aware Hotspot OpenMP Parallelization via Dependency-Aware Context Reduction
Qian, Y., Gao, K., Zhang, Z., Peng, S., & Li, L. (2026) · arXiv.org · doi:10.48550/arxiv.2608.05855
Hallucinations in LLM-Based Code Summarization: Unveiling, Detection, and Mitigation
Wan, G.-X., Feng, Y.-N., Wan, Y., Chu, Z.-Y., Bi, Z., Han, J.-X., Zhao, Z., Zhang, H.-Y., Yuan, P.-P., Shi, X.-H., et al. (2026) · Proc. ACM Softw. Eng. · doi:10.1145/3808139
Self-citationLLM-Assisted Model-Based GUI Testing for Vue.js Web Applications
Li, T., Cui, C.-H., Huang, R.-B., Towey, D., Guo, S.-K., & Ma, L. (2026) · arXiv.org · doi:10.48550/arxiv.2606.27665
“Large Language Models (LLMs) [52, 53] have been applied to a broad range of software engineering tasks [19, 44], including software testing [33, 39].”
Self-citationNan, S.-Y., Luo, Y., Wang, J., Zhang, N., & Li, B. (2026) · arXiv.org · doi:10.48550/arxiv.2606.08135
“This task is particularly challenging due to complex dependencies and the limited context window of LLMs [23, 25, 29, 30].”
Chiseling Out Efficiency: Structured Skeleton Supervision for Efficient Code Generation
Yu, Y., Sun, Z., Li, J., Wan, Y., Li, C.-Y., Zhang, H.-Y., Wang, R., Huang, T., Jin, Z., Li, G., et al. (2026) · Proc. ACM Softw. Eng. · doi:10.1145/3808194
EvoCoGen: Collaborative Code Generation Guided by Evolutionary Coupling
Sun, Z. X., Guan, B., Zhang, Y., Li, J., & Wang, Y. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11581560
“Context enrichment methods such as ATHENA [5] and TOOLGEN [6] incorporate retrieved static structures (e.g., call graphs) into prompts [7], [8].”
MATRIX: Multi-Layer Code Watermarking via Dual-Channel Constrained Parity-Check Encoding
Nie, Y.-Q., Wang, C., Xu, G.-S., Xu, G.-A., Wang, C.-Y., Wang, H., & Wang, K.-L. (2026) · arXiv.org · doi:10.48550/arxiv.2604.16001
“WhileCode LLMs have revolutionized software development by providing unprecedented assistance to programmers [9]–[12] and democratizing code creation for non-expert developers [8], [13], their widespread adoption has simultaneously raised critical legal, ethical, and security concerns.”
Self-citationAdverMCTS: Combating Pseudo-Correctness in Code Generation via Adversarial Monte Carlo Tree Search
Li, Q., Liu, W.-W., Zhang, W.-N., Yu, Y., & An, B. (2026) · arXiv.org · doi:10.48550/arxiv.2604.10449
SkelDPO: A Skeleton-Guided Direct Preference Optimization Framework for Efficient Code Generation
Yu, Y., & Lyu, C. (2026) · IEEE International Conference on Program Comprehension · doi:10.1145/3794763.3794824
SimulatorCoder: DNN Accelerator Simulator Code Generation and Optimization via Large Language Models
Xia, Y., Li, T., Zhou, H., Zhou, X., Chen, C., & Zhang, R. (2026) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.48550/arxiv.2602.17169
“LLMs can be used to generate code based on natural language descriptions, and many approaches have been proposed to improve the accuracy of LLM-based code generation, including prompt engineering, fine-tuning [6], and program self-repair [7].”
Self-citationAn Empirical Study on the Effects of System Prompts in Instruction-Tuned Models for Code GenerationPreprint
Cheng, Z., & Mastropaolo, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.15228 · Cited by 1 · see citing works
Evaluating LLMs in the Context of a Functional Programming Course: A Comprehensive Study
Zhang, Y., Pientka, B., & Si, X. (2026) · The Art, Science, and Engineering of Programming · doi:10.22152/programming-journal.org/2026/11/5
“Some studies address this by evaluating larger code generation tasks [13], competition-level code [22], or GitHub repository code [22, 50, 52], finding lower performance on more complex problems.”
Automated Power Dispatch Architecture Based on Multi-Agent Collaboration
Fan, J.-Y., Xiao, H.-B., Tan, P.-X., Zeng, J., Song, J.-H., Xiang, R., Zheng, K.-D., Song, J., & He, G.-N. (2026) · 2026 IEEE PES International Meeting (PES IM) · doi:10.1109/pesim67009.2026.11438861
Repository-level code completion with adaptive segmentation and fused retrieval
Shen, Y., Qin, P., Zhao, K., Zhang, F., Zhang, F., & Li, G. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10762-7
“Wang et al. (2025a) and Huang et al. (2024) used agents to accomplish automated code completion, including invoking multiple tools or components to collaborate with each other.”
InfluentialWhat a diff makes: automating code migration with large language models
Rosenfeld, K., Kerr, C. C., & Lundin, J. M. (2025) · arXiv.org · doi:10.48550/arxiv.2511.00160
“Relatedly, there have been significant results from tool calling to code generation and repair Patil et al. (2023); Tang et al. (2024); Wang et al. (2024); Zheng et al. (2024).”
Empowering LLM-based Software Defect Prediction with Chain-of-Thought and In-context Learning
Duan, X., Zhang, Y., Cui, Z.-Q., & Chen, J. (2025) · IEEE International Conference on Systems, Man and Cybernetics · doi:10.1109/smc58881.2025.11343579
“In recent years, Large Language Models (LLMs) have demonstrated excellent performance in tasks such as code generation and code completion, owing to their powerful semantic understanding and contextual reasoning capabilities [9]–[11].”
EduBot - Can LLMs Solve Personalized Learning and Programming Assignments?
Wang, Y., Xie, J.-X., & Subramanian, L. (2025) · arXiv.org · doi:10.48550/arxiv.2504.17824
“Moving beyond independent function completions and generations, TOOLGEN (Wang et al. 2024) focuses on repository-level completion tasks, demonstrating strong performance in resolving dependency coverage issues.”
AI-Powered Code Autocompletion and Bug Detection for Developers
Polu, O. R. (2025) · International Journal of Science and Research (IJSR) · doi:10.21275/sr25027114413
How efficient is your clone? Evaluating LLM capabilities for runtime and memory-aware code selection
Ramaswamy, P., Akbar, M. A., Rao, M., & Razzaq, A. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.112973
- Authors
- Wang, C., Zhang, J., Feng, Y., Li, T., Sun, W., Liu, Y., & Peng, X.
- Affiliations
- Nanyang Technological University (SG)
- Fudan University (CN)
- Venue
- ACM Transactions on Software Engineering and Methodology
- Published
- 2025-01-27
- ISSN
- 1049-331X (ISSN-L)
- 1557-7392
- OpenAlex
- W4406866655
- Cited by
- 40
- References
- 65
- Citations by year
- 2026: 14 · 2025: 21 · 2024: 5
- FWCI
- 36.69
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- ACM Transactions on Software Engineering and Methodology (publishedVersion)
- DR-NTU (Nanyang Technological University) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- repository-level code generation, code large language models, software repositories
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
