The DOI matches its DataCite record; 72 of the 81 references it cites were found in a registry or an index, and 1 are not checked yet.
InCoder: A Generative Model for Code Infilling and Synthesis
Open accessAbstract
Code is seldom written in a single left-to-right pass and is instead repeatedly edited and refined. We introduce InCoder, a unified generative model that can perform program synthesis (via left-to-right generation) as well as editing (via infilling). InCoder is trained to generate code files from a large corpus of permissively licensed code, where regions of code have been randomly masked and moved to the end of each file, allowing code infilling with bidirectional context. Our model is the first generative model that is able to directly perform zero-shot code infilling, which we evaluate on challenging tasks such as type inference, comment generation, and variable re-naming. We find that the ability to condition on bidirectional context substantially improves performance on these tasks, while still performing comparably on standard program synthesis benchmarks in comparison to left-to-right only models pretrained at similar scale. The InCoder models and code are publicly released. https://sites.google.com/view/incoder-code-models
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“However, we lack many models capable of generating code (Black et al., 2021;Fried et al., 2022;Black et al., 2022;Wang & Komatsuzaki, 2021;Biderman et al., 2023b).”
From OctoPack: Instruction Tuning Code Large Language Models (2023) · cites this paper as Fried et al., 2022
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Following existing works evaluating LLM code generation [12, 23] and common evaluation practices over LiveCodeBench and DevEval, we use Pass@1 as the evaluation metric for all RQs rather than code similarity scores, as it is more appropriate for problem-solving and code generation tasks.”
From What Makes In-Context Examples Effective for Code Generation? (2026) · cites this paper as [23]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Longer contexts enable us to move forward to”
From Context Composing for Full Line Code Completion (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Simultaneously, pretrained large language models (LLMs) have demonstrated strong capabilities at solving a wide range of programming tasks.”
From Learning Performance-Improving Code Edits (2023) · cites this paper as Fried et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“Unfortunately, standard left-to-right language models cannot directly infill text, and popular masked language models are mainly trained to infill very short spans [17, 22, 52, 54].”
BackgroundMethodPyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 2×
“For our learning rate scheduler, we use the built-in polynomial decay learning rate scheduler available in [49] with 1500 warmup updates.”
MethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Neural models for code generation have either utilized a left-to-right (causal) autoregressive language modeling objective [16, 18] or, as BERT does, a masked language modeling objective [22, 24].”
MethodExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“Unfortunately, standard left-to-right language models cannot directly infill text, and popular masked language models are mainly trained to infill very short spans [17, 22, 52, 54].”
BackgroundScaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“Looking forward, we expect our model performance to continue to increase with more parameters, data, and training steps [39, 32].”
BackgroundLewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816
Self-citationSpace/time trade-offs in hash coding with allowable errors
Bloom, B. (1970) · CACM · doi:10.1145/362686.362692 · Cited by 8,424 · cited 1×
“(6)We implement match checking using a Bloom filter [15] whose keys are: the file extension, number of tokens in the file, and an MD5 hash [53] of the sequence of tokens, which is highly accurate at identifying files with exactly matching token sequences.”
MethodTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Research works printed but not linked: 4
- Uri Alon, Roy Sadaka, Omer Levy, and Eran Yahav. Structural language models of code. In ICML , pp. 245-256. PMLR, 2020.
- Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. Learning and evaluating contextual embedding of source code. In ICML , 2020.
- Lewis Tunstall, Leandro von Werra, and Thomas Wolf. Natural Language Processing with Transformers . O'Reilly Media, Inc., 2022.
- John M Zelle and Raymond J Mooney. Learning to parse database queries using inductive logic programming. In AAAI , 1996.
Cited by
Sorted by citations
Items 101–200 of 219
Deep learning for code generation: a survey
Zhang, H., Zhang, K., Li, Z., Li, J., Li, J., Li, Y., Zhao, Y., Zhu, Y. S., Liu, F., Li, G., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-3956-3 · Cited by 20
Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers
Shi, Y., Zhang, H., Wan, C., & Gu, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00005 · Cited by 38
“We selected Incoder [12], Phi-1 [15], StarCoder [25], Wizard-Coder [30], CodeGen2 [34] and CodeLlama [43] from Huggingface 3 .”
Aligning the Objective of LLM-Based Program Repair
Xu, J., Fu, Y., Tan, S. H., & He, P. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00169 · Cited by 47
“However, except for very few models ( e.g. , InCoder [43]), decoder-only LLMs barely incorporate denoising objective during pre-training.”
Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering
Si, C., Zhang, Y., Li, R., Yang, Z., Liu, R., & Yang, D. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.199 · Cited by 17
“LLMs trained on code, such as Codex (Chen et al., 2021), Star-Coder (Li et al., 2023), InCoder (Fried et al., 2022), CodeLlama (Rozière et al., 2023), and DeepSeek-Coder (Guo et al., 2024), enable a wave of programming support applications such as automatic code completion and infilling, and allowing users to chat wit…”
The bewitching AI: The Illusion of Communication with Large Language Models
Bottazzi Grifoni, E., & Ferrario, R. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00893-6 · Cited by 15
“Wang et al.’s (2023) technique of modifying logical word embeddings before encoder input shows progress but remains early-stage with an incomplete list of addressed logical words.”
Enriching Source Code with Contextual Data for Code Completion Models: An Empirical Study
van Dam, T., Izadi, M., & Arie Van Deursen, A. (2023) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr59073.2023.00035 · Cited by 21
“InCoder is a Transformer-based decoder-only model which is able to infill code based on both left and right contexts [7] using causal modeling [35].”
BackgroundMethodAPIGen: Generative API Method Recommendation
Chen, Y., Gao, C., Zhu, M., Liao, Q., Wang, Y., & Xu, G. (2024) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner60148.2024.00025 · Cited by 25
“However, it’s worth noting that there are other LLMs available, including Incoder [54] and Code-Gen [28].”
BackgroundLin, Y., Li, C., Ding, M., Tomizuka, M., Zhan, W., & Althoff, M. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3441493 · Cited by 22
Rome was Not Built in a Single Step: Hierarchical Prompting for LLM-based Chip Design
Nakkab, A., Zhang, S. Q., Karri, R., & Garg, S. (2024) · ACM/IEEE Workshop on Machine Learning for CAD (MLCAD) · doi:10.1109/mlcad62225.2024.10740247 · Cited by 14
Pytester: Deep reinforcement learning for text-to-testcase generation
Takerngsaksiri, W., Charakorn, R., Tantithamthavorn, C. K., & Li, Y.-F. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112381 · Cited by 27
Exploring Continual Learning for Code Generation Models
Yadav, P., Sun, Q., Ding, H., Li, X., Zhang, D., Tan, M., Bhatia, P., Ma, X., Nallapati, R. M., Ramanathan, M. K., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.68 · Cited by 15
Who is Smarter? An Empirical Study of AI-Based Smart Contract Creation
Karanjai, R., Li, E., Xu, L., & Shi, W. L. (2023) · Conference on Blockchain Research & Applications for Innovative Networks and Services (BRAINS) · doi:10.1109/brains59668.2023.10316829 · Cited by 17
“These models can be used for zero-shot and few-shots program generation tasks, such as code completion, code infilling, and text-to-code conversion [25], [26], [31].”
MethodFranc: A Lightweight Framework for High-Quality Code Generation
Siddiq, M. L., Casey, B., & Santos, J. C. S. (2024) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam63643.2024.00020 · Cited by 38
“Examples of this type of model include CodeBERT [32], CodeT5 [34], and Codex [2].”
InfluentialQiu, J., Han, W., Zhu, J., Xu, M., Rosenberg, M., Liu, E., Weber, D. J., & Zhao, D. X. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.33 · Cited by 21
Clozemaster: Fuzzing Rust Compiler by Harnessing Llms for Infilling Masked Real Programs
Gao, H., Yang, Y., Sun, M., Wu, J., Zhou, Y., & Xu, B. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00175 · Cited by 20
“This is because Incoder employs a strategy of random masking between code snippets during the model training phase [33], which enhances its ability to fill in code within the middle of existing code.”
InfluentialLLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-Based Code Completion
Wang, C., Huang, K., Zhang, J., Feng, Y., Zhang, L., Liu, Y., & Peng, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00245 · Cited by 17
“Large language models (LLMs) [1], [2], [3], [4], [5], [6] have significantly advanced various aspects of software engineering [7], including code completion [8], [9], [10], code understanding [11], [12], defect detection [13], [14], and program repair [15], [16], [17].”
Improving Retrieval-Augmented Deep Assertion Generation via Joint Training
Zhang, Q., Fang, C., Zheng, Y., Qian, R., Yu, S., Zhao, Y., Zhou, J., Yang, Y., Zheng, T., & Chen, Z. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3545970 · Cited by 11
“It is noteworthy that we do not consider decoder-only PLMs, like CodeGPT [52] and InCoder [69], and even recent LLMs, such as Code Llama [70] and StarCoder [71], as they are built without an encoder, thus failing to provide meaningful code representations for assertion retrieving.”
Correctness assessment of code generated by Large Language Models using internal representations
Bui, T.-D., Vu, T. T., Nguyen, T.-T., Nguyen, S., & Vo, H. D. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112570 · Cited by 39
Diverse title generation for Stack Overflow posts with multiple-sampling-enhanced transformer
Zhang, F., Liu, J., Wan, Y., Yu, X., Liu, X., & Keung, J. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111672 · Cited by 12
“Furthermore, some other studies [31, 10, 2, 48, 14, 11] proposed to utilize the large-scale unlabelled data and self-supervised learning to pre-train models through self-supervised objectives and achieved state-of-the-art results on code summarization benchmarks.”
BackgroundMethodToward Automatic Tutoring of Math Word Problems in Intelligent Tutoring Systems
Arnau‐González, P., Serrano-Mamolar, A., Katsigiannis, S., Althobaiti, T., & Arevalillo‐Herráez, M. (2023) · IEEE Access · doi:10.1109/access.2023.3290478 · Cited by 11
“The pre-trained LLMs used in this work are (i) Saleforce’s CodeGen [21] and (ii) Facebook’s InCoder [9].”
MethodAligning Offline Metrics and Human Judgments of Value for Code Generation Models
Dibia, V. C., Fourney, A., Bansal, G., Poursabzi-Sangdeh, F., Liu, H., & Amershi, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.540 · Cited by 24
“Much of the current research evaluating code generation models aims to approximate overall value via some notion of correctness (Chen et al., 2021; Fried et al., 2022; Austin et al., 2021; Chowdhery et al., 2022; Nijkamp et al., 2022; Hendrycks et al., 2021; Kulal et al., 2019).”
BackgroundMethodInfluentialA search-and-fill strategy to code generation for complex software requirements
Dong, Y., Kong, L., Zhang, L., Wang, S., Liu, X., Liu, S., & Chen, M.-C. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107584 · Cited by 10
Self-citationUser Centric Evaluation of Code Generation Tools (Invited Paper)
Miah, T., & Zhu, H. (2024) · IEEE International Conference on Artificial Intelligence Testing (AITest) · doi:10.1109/aitest62860.2024.00022 · Cited by 18
“For example, AlphaCode [2], InCoder [3], StarCoder [4], PolyCoder [5], Replit’s Ghost-write 3 and Tabnine 4 have been developed for code completion, code generation, code comment summary generation, and debugging (code correction) tasks.”
MethodAnalyzing the dependability of Large Language Models for code clone generation
Eagal, A., Stolee, K. T., & Ore, J. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112548 · Cited by 13
INSPECT: Intrinsic and Systematic Probing Evaluation for Code Transformers
Karmakar, A., & Robbes, R. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3341624 · Cited by 9
“We probe 8 pre-trained source code models, as well as a natural language model ( BERT ) as our baseline.”
MethodLow-cost language models: Survey and performance evaluation on Python code generation
Espejel, J. L., Alassan, M. S. Y., Bouhandi, M., Dahhane, W., & Ettifouri, E. H. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109490 · Cited by 15
A Static Evaluation of Code Completion by Large Language Models
Ding, H., Kumar, V., Tian, Y., Wang, Z., Kwiatkowski, R., Li, X., Ramanathan, M. K., Ray, B., Bhatia, P., & Sengupta, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-industry.34 · Cited by 14
On the Impact of Language Selection for Training and Evaluating Programming Language Models
Katzy, J., Izadi, M., & Arie Van Deursen, A. (2023) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam59687.2023.00038 · Cited by 8
“In this work, we used an encoder-based architecture that used the masked language modeling task [4], we want to extend the work to other tasks such as code infilling [10], [14], [17] and architectures such as the T5 models [15].”
BackgroundMethodInfluentialProgram Repair with Minimal Edits Using CodeT5
Shirafuji, A., Rahman, M. M., Amin, M. F. I., & Watanobe, Y. (2023) · International Conference on Awareness Science and Technology (iCAST) · doi:10.1109/icast57874.2023.10359288 · Cited by 10
Liu, C., Cai, Y., Lin, Y., Huang, Y.-H., Pei, Y., Jiang, B., Yang, P., Dong, J. S., & Mei, H. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3652142 · Cited by 10
Automated Program Repair Using Generative Models for Code Infilling
Koutcheme, C., Sarsa, S., Leinonen, J., Hellas, A., & Denny, P. C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36272-9_74 · Cited by 11
Do large language models know chemistry?Preprint
White, A. D., Hocky, G. M., Gandhi, H. A., Ansari, M., Cox, S., Wellawatte, G. P., Sasmal, S., Yang, Z., Liu, K., Singh, Y., et al. (2022) · ChemRxiv · doi:10.26434/chemrxiv-2022-3md3n · Cited by 8
A Survey of Post-Training Scaling in Large Language Models
Lai, H., Liu, X., Gao, J., Cheng, J., Qi, Z., Xu, Y., Yao, S., Zhang, D., Du, J., Hou, Z., et al. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.140 · Cited by 8
A Transformer-Based Approach for Smart Invocation of Automatic Code Completion
de Moor, A., van Deursen, A., & Izadi, M. (2024) · Proceedings of the 1st ACM International Conference on AI-Powered Software · doi:10.1145/3664646.3664760 · Cited by 7
“Our line-completions are generated by smaller language models than the industry-standard, allowing the user to pick one of three completions provided by InCoder [11], UniXCoder [14], and CodeGPT [28].”
MethodUnseen Horizons: Unveiling the Real Capability of LLM Code Generation Beyond the Familiar
Zhang, Y., Xie, Y., Lit, S., Liu, P., Wang, C., Jia, Z., Huang, X., Song, J., Luo, C., Zheng, Z., et al. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00082 · Cited by 7
Durán, F., Martínez, M., Lago, P., & Martínez‐Fernández, S. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112574 · Cited by 15
“Our input dataset was derived utilizing a methodology similar to those outlined by Fried et al. and Bavarian et al. [66, 67].”
AskIt: Unified Programming Interface for Programming with Large Language Models
Okuda, K., & Amarasinghe, S. P. (2024) · Proceedings of the CGO/Proceedings of the ... CGO · doi:10.1109/cgo57630.2024.10444830 · Cited by 10
Promoting open science in test-driven software experiments
Kessel, M., & Atkinson, C. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.111971 · Cited by 7
TIGER: A Generating-Then-Ranking Framework for Practical Python Type Inference
Wang, C., Zhang, J., Lou, Y., Liu, M., Sun, W., Liu, Y., & Peng, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00019 · Cited by 11
“3) Generation-based : Utilize generative models to produce types based on given inputs [3], [14]–[16].”
InfluentialImpact of Large Language Models of Code on Fault Localization
Ji, S., Lee, S.-H., Lee, C.-S., Han, Y.-S., & Im, H. (2025) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst62969.2025.10989036 · Cited by 12
“Recently, with rapidly advancing computing resources, large language models (LLMs) based on the Transformers [62] with attention mechanisms have been utilized in various domains.”
A Simple, Yet Effective Approach to Finding Biases in Code Generation
Mouselinos, S., Malinowski, M., & Michalewski, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.718 · Cited by 13
The Effectiveness of Security Interventions on GitHub
Fischer, F., Höbenreich, J., & Großklags, J. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3623174 · Cited by 6
“have recently developed an LLM that can perform infilling tasks on source code [15].”
MethodFLAME: A Small Language Model for Spreadsheet Formulas
Joshi, H., Ebenezer, A., Cambronero, J., Gulwani, S., Kanade, A., Le, V., Radiček, I., & Verbruggen, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29197 · Cited by 12
Iaso: Enhancing Syntax Error Correction using Deep Learning TechniquesPreprint
Li, W., & Wong, W. (2023) · Research Square · doi:10.21203/rs.3.rs-3190830/v1 · Cited by 6
Anchor Attention, Small Cache: Code Generation With Large Language Models
Zhang, X., Zhou, Y., Yang, G., Gall, H. C., & Chen, T. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3570680 · Cited by 6
“Similarly, Meta introduced models such as InCoder [3] and CodeLlama [9], while Salesforce developed CodeRL [55] and CodeGen [8].”
Fuzzy-Assisted Contrastive Decoding Improving Code Generation of Large Language Models
Wang, S., Ding, L., Zhan, Y., Luo, Y., Liu, S., & Ding, W. (2025) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2025.3575060 · Cited by 5
Self-citationA survey of coverage-guided greybox fuzzing with deep neural models
Qiu, J., Jiang, Y., Miao, Y., Luo, W., Pan, L., & Zheng, X. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107797 · Cited by 7
Introducing bidirectional attention for autoregressive models in abstractive summarization
Zhao, J., Sun, X., & Feng, C. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121497 · Cited by 5
Utilizing generative AI for crack detection in the marble industry
Vrochidou, Ε., Sidiropoulos, G. K., Ouzounis, A. G., Tsimperidis, I., Kalpakis, V., Stamkos, A., & Papakostas, G. A. (2025) · Engineering Research Express · doi:10.1088/2631-8695/adaca7 · Cited by 6
DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Models
Yuan, M., Chen, J., Xing, Z., Quigley, A. J., Luo, Y., Luo, T., Mohammadi, G., Lu, Q., & Zhu, L. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00109 · Cited by 8
“Research on performance enhancement in program repair has also progressed, with models like PLBART [43], CodeT5 [44], CodeGen [45], and InCoder [46] showing competitive or superior performance compared to deep learning-based techniques.”
FormaT5: Abstention and Examples for Conditional Table Formatting with Natural Language
Singh, M. P., Cambronero, J., Gulwani, S., Le, V., Negreanu, C., Nouri, E., Raza, M., & Verbruggen, G. (2023) · Proceedings of the VLDB Endowment · doi:10.14778/3632093.3632111 · Cited by 5
A Semantics-Based Approach on Binary Function Similarity Detection
Zhang, Y., Fang, B., Xiong, Z., Wang, Y., Liu, Y., Zheng, C., & Zhang, Q. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3389014 · Cited by 5
“For ChatGPT, we apply a temperature of 1, along with default settings of top p = 0.95, as suggested in previous studies [57].”
MethodInfluentialRobustness evaluation of code generation systems via concretizing instructions
Yan, M. X., Chen, J., Zhang, J. M., Cao, X., Yang, C., & Harman, M. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107645 · Cited by 18
Model-Agnostic Syntactical Information for Pre-Trained Programming Language Models
Saberi, I., & Fard, F. H. (2023) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr59073.2023.00036 · Cited by 6
Follow-Up Attention: An Empirical Study of Developer and Neural Model Code Exploration
Paltenghi, M., Pandita, R., Henley, A. Z., & Ziegler, A. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3445338 · Cited by 11
“Then we query three LLMs of code, namely CodeGen [2], InCoder [20] and GPT-J [21] on the same sensemaking task and compare their attention signal 1 to the attention of developers.”
MethodInfluentialFundamental Approaches to Software Engineering
Beyer, D., & Cavalcanti, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-57259-3 · Cited by 7
Evaluating the Test Adequacy of Benchmarks for LLMs on Code Generation
Liu, X., Sun, X., Bo, L., Hu, Y., Liu, X., & Ye, Z. (2025) · Journal of Software Evolution and Process · doi:10.1002/smr.70034 · Cited by 4
“We evaluate the coding ability of six recent and popular LLMs: two models of CodeGen [6] with different sizes, two models of InCoder [19] with different sizes, as well as SantaCoder [20] and GPT-NEO [21].”
InfluentialA Software Bug Fixing Approach Based on Knowledge-Enhanced Large Language Models
Bo, L., He, Y., Sun, X., Ji, W., & Wu, X. (2024) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs62785.2024.00026 · Cited by 6
“This RQ is to evaluate the effectiveness of our our approach by comparing with the state-of-the-art approaches (SequenceR [7], Review4Repair [11], Incoder [26]).”
BackgroundMethodUnderstanding Defects in Generated Codes by Language Models
Esfahani, A. M., Kahani, N., & Ajila, S. A. (2024) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1109/cascon62161.2024.10837857 · Cited by 6
Zhou, X., Weyssow, M., Widyasari, R., Zhang, T., He, J., Lyu, Y., Chang, J., Zhang, B., Huang, D., & Lo, D. (2025) · arXiv · doi:10.48550/arxiv.2502.06215 · Cited by 3
SeTemAPR: Incorporating Semantic Knowledge in Template-based Neural Program Repair
Zhang, Y., & Wang, Y. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650123 · Cited by 4
Fine-Tuning LLMs for Code Mutation: A New Era of Cyber Threats
Setak, M., & Madani, P. (2024) · IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tps-isa62245.2024.00043 · Cited by 5
A Path Less Traveled: Reimagining Software Engineering Automation via a Neurosymbolic Paradigm
Mastropaolo, A., & Poshyvanyk, D. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728720 · Cited by 3
“Notable examples include CodeLlama [19], StarCoder [16], Incoder [10] and DeepSeek-Coder [15].”
An exploratory study on fine-tuning large language models for secure code generation
Li, J., Rabbi, F., Cheng, C., Sangalay, A., Tian, Y., & Yang, J. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10803-9 · Cited by 9
“…tested along axes developers care about: non-functional requirements (e.g., performance, memory, security) expose brittleness that functional tests miss [29], and behavior varies across programming languages, complicating generalization claims and motivating cross-language reporting [42].”
Code Generation Using Self-Interactive Assistant
Zhao, Z., Sun, J., Cai, C., & Wei, Z. (2024) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac61105.2024.00377 · Cited by 3
Assessing GitHub Copilot in Solidity Development: Capabilities, Testing, and Bug Fixing
Baralla, G., Ibba, G., & Tonelli, R. (2024) · IEEE Access · doi:10.1109/access.2024.3486365 · Cited by 4
“…asking the model to guess entire blocks, up to 58%, reached in the simpler scenario of a few tokens masked from the same code statement Fried et al. [15] introduced InCoder, a unified generative model for both program synthesis and code editing, mainly focused on the Python programming language.”
MethodAdvFusion: Adapter-based Knowledge Transfer for Code Summarization on Code Language Models
Saberi, I., Esmaeili, A., Fard, F. H., & Chen, F. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00059 · Cited by 9
Seed-Coder: Let the Code Model Curate Data for Itself
Seed, B., Zhang, Y., Su, J., Sun, Y., Xi, C., Xiao, X., Zheng, S., Zhang, A., Liu, K., Zan, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.03524 · Cited by 3
Full Line Code Completion: Bringing AI to Desktop
Semenkin, A., Bibaev, V., Sokolov, Y., Krylov, K., Kalina, A., Khannanova, A., Savenkov, D., Rovdo, D., Davidenko, I., Karnaukhov, K., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.08704 · Cited by 8
“The numerator of the metric includes code from all code completion sources: standard and inline completions, so it allows us to assess the entire code completion experience.”
Towards Better Answers: Automated Stack Overflow Post Updating
Mai, Y., Gao, Z., Wang, H., Bi, T., Hu, X., Xia, X., & Sun, J. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00024 · Cited by 7
“…by the great advancements and potential of Large Language Models (LLMs) [9], [10], [11], [12], [13], [14], [15], [16], [17] in code generation [18], [19], [20], [21], [22], [23], [24], in this work, we proposed a novel LLM-based framework, named S OUP ( S tack O verflow U pdator for P ost),…”
Improving Transformer-based Program Repair Model through False Behavior Diagnosis
Kim, Y., Kim, M., & Lee, E. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.865 · Cited by 2
“We followed previous studies (Chen et al., 2023; Alexandr et al., 2021; Khan-delwal et al., 2019) to train the CodeGen model and fine-tuned it for one epoch because large language models (LLMs) such as CodeGen are susceptible to overfitting (Jiang et al., 2023; Fried et al., 2022).”
How Well Can Masked Language Models Spot Identifiers That Violate Naming Guidelines?
Villmow, J., Campos, V., Petry, J., Andaloussi, A. A., Ulges, A., & Weber, B. (2023) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam59687.2023.00023 · Cited by 2
“…language models pre-trained on source code, we compare our approaches with two other transformer models from current research, namely I N C ODER [35] as a pre-trained decoder model for the generative rating, and G RAPH C ODE BERT [53] as a pre-trained encoder for the discriminative rating. a)…”
BackgroundMethodInfluentialA Retrospective of ChangeDistiller: Tree Differencing for Fine-Grained Source Code Change Extraction
Fluri, B., Würsch, M., Pinzger, M., & Gall, H. C. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3538326 · Cited by 2
“For instance, Jiang et al. [43] evaluated ten code language models (CLMs), such as CodeT5 and inCoder [44], on four Java benchmarks and showed that their fine-tuned models fix significantly more bugs than other approaches.”
The Sustainability Face of Automated Program Repair Tools
Martínez, M., Martínez‐Fernández, S., & Franch, X. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3744900 · Cited by 2
“Uses 10 open-source code LLMs from four families of large models: InCoder from Meta [29], CodeGen and CodeT5 [95] from SalesForce [30], and PLBART [96].”
Software Engineering for Data Intensive Scalable Computing and Heterogeneous Computing
Kim, M. (2023) · IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00006 · Cited by 2
Context Composing for Full Line Code Completion
Semenkin, A., Sokolov, Y., & Vu, E. (2024) · arXiv (Cornell University) · doi:10.1145/3643796.3648446 · Cited by 3
“Longer contexts enable us to move forward to “smart” context composition techniques, which include fill-in-the-middle [3] and retrieval-augmented generation [2] methods.”
BackgroundDoes Few-Shot Learning Help LLM Performance in Code Synthesis?
Xu, D., Xie, T., Xia, B., Li, H., Bai, Y., Sun, Y., & Wang, W. (2024) · arXiv · doi:10.48550/arxiv.2412.02906 · Cited by 3
“More specialized code LLMs, such as CodeT5 (Wang et al., 2021), In-Coder (Fried et al., 2022), DeepSeek-Coder (Guo et al., 2024), StarCoder2 (Lozhkov et al., 2024), Qwen2.”
McEval: Massively Multilingual Code Evaluation
Chai, L., Liu, S., Yang, J., Yin, Y., Jin, K., Liu, J., Sun, T., Zhang, G., Ren, C., Guo, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.07436 · Cited by 73
“For the field of soft engineering, code LLMs (Feng et al., 2020; Chen et al., 2021; Scao et al., 2022; Li et al., 2022; Allal et al., 2023; Fried et al., 2022; Wang et al., 2021; Zheng et al., 2024; Guo et al., 2024) pre-trained on billions of code snippets, such as StarCoder (Li et al., 2023;…”
MethodA Survey of Security Testing Techniques for Deep Learning Frameworks
Li, H., Li, X., Nie, Y., & Tian, J. (2024) · doi:10.1109/icsip61881.2024.10671492 · Cited by 1
“After selecting a mutation rule, it uses an infilling LLM (e.g., Incoder [36]) to automatically mutate seed programs and generate new seeds.”
MethodOn the Applicability of Code Language Models to Scientific Computing Programs
Zhao, Q., Liu, F., Long, X., Wu, C., & Zhang, L. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3564599 · Cited by 2
“InCoder [15], proposed by Meta, supports arbitrary regions of code infilling with up to 6.7B parameters.”
InfluentialEnhancing Code Generation Through Retrieval of Cross-Lingual Semantic Graphs
Jiang, Z., Shi, Z., Gao, X., & Xiong, Y. (2024) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec65559.2024.00026 · Cited by 1
Is More or Less Automation Better? An Investigation into the LLM4TDD Process
Piya, S., Samadi, A., & Sullivan, A. (2025) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1109/llm4code66737.2025.00025 · Cited by 1
“In addition, several LLMs have been developed specifically for generating code, including Codex [3], CodeGen [14], InCoder [5] and PolyCoder [17], which could be used as the backend LLM in the LLM4TDD process.”
An end-to-end approach for fixing concurrency bugs via SHB-based context extractor
Li, Z., Yi, Q., Xiao, K., Ji, Z., & Liang, H. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.112888 · Cited by 1
“All other parameters, including frequency penalty and presence penalty , are set to the model’s default values, which aligns with common practice [36]–[38], [40], [58], [59].”
InfluentialMa, H., Ji, Z., Al-Hashimy, T., Allen, A., Cen, N., Chen, O., Chen, Y., Chen, Y., Cheng, T., Gu, Y., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2521062122 · Cited by 1
Constructing Multilingual Code Search Dataset Using Neural Machine Translation
Sekizawa, R., Duan, N., Lü, S., & Yanaka, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-srw.10 · Cited by 2
Enhancing intelligent IoT services development by integrated multi-token code completion
Xia, Y., Liang, T., Min, W., Kuang, L., & Gao, H. (2023) · Computer Communications · doi:10.1016/j.comcom.2023.10.014 · Cited by 1
Wang, R., Todd, G., Yuan, X., Xiao, Z., Côté, M.-A., & Jansen, P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.830 · Cited by 17
Towards Safe, Secure, and Usable LLMs4Code
Al-Kaswan, A. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3639478.3639803 · Cited by 2
“As a result, they are highly proficient in tasks such as generating [3], summarising [4], and completing code [5].”
BackgroundAutoTest: Evolutionary Code Solution Selection with Test Cases
Duan, Z., & Wang, J. (2024) · IEEE International Conference on Cyber Security and Cloud Computing (CSCloud) · doi:10.1109/cscloud62866.2024.00030 · Cited by 2
“For instance, models such as AlphaCode [6], Codex [7], CodeT5 [8], CodeGen [3], InCoder [5], CodeGeeX [9] and StarCoder [10] have achieved significant advancements in code generation applications.”
MethodCIPAC: A framework of automated software construction based on collective intelligence
Liu, J., Zhang, Y., Li, Y., Ma, T., & Dong, W. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112335 · Cited by 2
Deduplicating and Ranking Solution Programs for Suggesting Reference Solutions
Shirafuji, A., & Watanobe, Y. (2023) · Asia Service Sciences and Software Engineering Conference (ASSE) · doi:10.1145/3634814.3634828 · Cited by 1
“This task is usually performed in creating largescale source code datasets [10, 18] or training machine learning models on source code [1, 6] to avoid redundancy, overfitting, and memorization [2, 9, 11].”
Predicting typeScript type annotations and definitions with machine learning
Yee, M. (2024) · doi:10.17760/d20653005 · Cited by 1
Cao, J., & Lu, Y. (2025) · Qeios · doi:10.32388/mlaotg · Cited by 1
Measuring The Impact Of Programming Language Distribution
Orlanski, G., Xiao, K., Garcia, X., Hui, J., Howland, J., Malmaud, J., Austin, J., Singh, R., & Catasta, M. (2023) · arXiv · doi:10.48550/arxiv.2302.01973 · Cited by 21
“, 2022), and other works (Nijkamp et al., 2022; Fried et al., 2022; Allal et al., 2023; Christopoulou et al., 2022) have shown that decoder-only code language models achieve exceptional performance on a wide range of tasks.”
BackgroundWhat Makes In-Context Examples Effective for Code Generation?
Li, D., Chen, S., Cao, J., & Cheung, S.-C. (2026) · Proceedings of the ACM on Software Engineering · doi:10.1145/3832275
VADER-SC: A Model Agnostic Tool for Large Scale, AI Driven Automated Source Code Summarization
Horne, D., Pierson, A., Hedary, E., Freddo, G., Trejo, L., Matis, M., & Mask, L. (2023) · Congress in Computer Science, Computer Engineering, and Applied Computing (CSCE) · doi:10.1109/csce60160.2023.00416
A Survey on Code Pre-Trained Models for Neural Program RepairPreprint
Zhan, S., Wang, X., Wei, D., & Cao, X. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4831589
Metadata
- Authors
- Fried, D., Aghajanyan, A., Lin, J., Wang, S., Wallace, E., Shi, F., Zhong, R., Yih, W.-T., Zettlemoyer, L., & Lewis, M.
- Affiliations
- Meta (United States) (US)
- Carnegie Mellon University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2204.05999
- OpenAlex
- W4224060952
- Cited by
- 187
- References
- 81
- Citations by year
- 2026: 4 · 2025: 52 · 2024: 68 · 2023: 59 · 2022: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Keywords
- program synthesis, bidirectional context, code editing, type inference, comment generation, variable renaming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
