The DOI matches its DataCite record; 72 of the 81 references it cites were found in a registry or an index, and 1 are not checked yet.
InCoder: A Generative Model for Code Infilling and Synthesis
Open accessAbstract
Code is seldom written in a single left-to-right pass and is instead repeatedly edited and refined. We introduce InCoder, a unified generative model that can perform program synthesis (via left-to-right generation) as well as editing (via infilling). InCoder is trained to generate code files from a large corpus of permissively licensed code, where regions of code have been randomly masked and moved to the end of each file, allowing code infilling with bidirectional context. Our model is the first generative model that is able to directly perform zero-shot code infilling, which we evaluate on challenging tasks such as type inference, comment generation, and variable re-naming. We find that the ability to condition on bidirectional context substantially improves performance on these tasks, while still performing comparably on standard program synthesis benchmarks in comparison to left-to-right only models pretrained at similar scale. The InCoder models and code are publicly released. https://sites.google.com/view/incoder-code-models
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“However, we lack many models capable of generating code (Black et al., 2021;Fried et al., 2022;Black et al., 2022;Wang & Komatsuzaki, 2021;Biderman et al., 2023b).”
From OctoPack: Instruction Tuning Code Large Language Models (2023) · cites this paper as Fried et al., 2022
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Following existing works evaluating LLM code generation [12, 23] and common evaluation practices over LiveCodeBench and DevEval, we use Pass@1 as the evaluation metric for all RQs rather than code similarity scores, as it is more appropriate for problem-solving and code generation tasks.”
From What Makes In-Context Examples Effective for Code Generation? (2026) · cites this paper as [23]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Longer contexts enable us to move forward to”
From Context Composing for Full Line Code Completion (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Simultaneously, pretrained large language models (LLMs) have demonstrated strong capabilities at solving a wide range of programming tasks.”
From Learning Performance-Improving Code Edits (2023) · cites this paper as Fried et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“Unfortunately, standard left-to-right language models cannot directly infill text, and popular masked language models are mainly trained to infill very short spans [17, 22, 52, 54].”
BackgroundMethodPyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 2×
“For our learning rate scheduler, we use the built-in polynomial decay learning rate scheduler available in [49] with 1500 warmup updates.”
MethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Neural models for code generation have either utilized a left-to-right (causal) autoregressive language modeling objective [16, 18] or, as BERT does, a masked language modeling objective [22, 24].”
MethodExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“Unfortunately, standard left-to-right language models cannot directly infill text, and popular masked language models are mainly trained to infill very short spans [17, 22, 52, 54].”
BackgroundScaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“Looking forward, we expect our model performance to continue to increase with more parameters, data, and training steps [39, 32].”
BackgroundLewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816
Self-citationSpace/time trade-offs in hash coding with allowable errors
Bloom, B. (1970) · CACM · doi:10.1145/362686.362692 · Cited by 8,424 · cited 1×
“(6)We implement match checking using a Bloom filter [15] whose keys are: the file extension, number of tokens in the file, and an MD5 hash [53] of the sequence of tokens, which is highly accurate at identifying files with exactly matching token sequences.”
MethodTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Research works printed but not linked: 4
- Uri Alon, Roy Sadaka, Omer Levy, and Eran Yahav. Structural language models of code. In ICML , pp. 245-256. PMLR, 2020.
- Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. Learning and evaluating contextual embedding of source code. In ICML , 2020.
- Lewis Tunstall, Leandro von Werra, and Thomas Wolf. Natural Language Processing with Transformers . O'Reilly Media, Inc., 2022.
- John M Zelle and Raymond J Mooney. Learning to parse database queries using inductive logic programming. In AAAI , 1996.
Cited by
Sorted by citations
Items 201–219 of 219
Automatic Pseudocode Extraction at Scale
Toksoz, L., Tan, G., & Giles, C. L. (2024) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri62200.2024.00061
“…influenced by variety of models, as explored by Feng et al. (2020) [20], Guo et al. (2020) [21], Xu, Alon, Neubig, and Hellendoorn (2022) [22], Fried et al. (2023) [23], Rozi`ere et al. (2021) [24], Lu et al. (2021) [25] and covers tasks such as code synthesis, infilling, and docstring generation.”
BackgroundCode Generation Based on Quality-Aware Reordering
Yuan, S., & Yu, L. (2024) · International Conference on Intelligent Computing and Signal Processing (ICSP) · doi:10.1109/icsp62122.2024.10743331
Improving the CodeGeeX Model Based on the Relative Convolutional Multi-Head Attention Method
Yang, D., & Xu, A. (2024) · International Conference on Artificial Intelligence, Internet of Things and Cloud Computing Technology (AIoTC) · doi:10.1109/aiotc63215.2024.10748258
On the Generalizability of Transformer Models to Code Completions of Different Lengths
Cooper, N., Tufano, R., Bavota, G., & Poshyvanyk, D. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00042
“Fried et al. [16] investigated a novel infilling pretraining scheme for decoder-only Transformer architectures that allow them to use bidirectional context to complete code.”
InfluentialLLM-based method for generating vulnerable code equivalents
Chen, J., Liu, J., Wang, J., & Li, D. (2025) · doi:10.1117/12.3058052 · Cited by 1
LLM-Based MaSE, A Software Development Framework for Developing Multi-agent Systems
Hajjarzadeh, S., Abad, Z. S. H., & Far, B. H. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8889-0_37
PTC: Prefix Tuning CodeT5 for High-Quality Secure Network Measurement Script Generation
Wang, Y., & Han, X. (2025) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn64029.2025.00024
“Recent progress in large language models (LLMs) has introduced a range of models specifically tailored for code generation, including Codex [36], CodeT5, AlphaCode [37], CodeGen, among others [38]–[40].”
Yonathan, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5367078
Zhan, S., Wang, X., Wei, D., Cao, X., & Chen, B. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112590
EFIM: Efficient Serving of LLMs for Infilling Tasks with Improved KV Cache Reuse
Guo, T., Dong, H., Leng, Y., Liu, F., Lin, C., Xiao, N., & Zhang, X. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-99857-7_24
Reasoning Distillation and Structural Alignment for Improved Code GenerationPreprint
Jalilifard, A., Raimundo, M. M., & Rocha, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5403116
Enhancing LlaMA With Personalized User Data Integration for Next-Generation AI
Lalitha, S. D., Dharaneedharan, G. K., Dhevesh, K. R., Sashank, K., & Darshan, P. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-1419-8.ch004
A Novel Deep Learning-Based Technique for Automatic Source Code Summarization
Shruthi, D., Chethan, H. K., & Ikechukwu, A. V. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-06674-9_51
PlayCoder: Making LLM-Generated GUI Code Playable
Peng, Z., Tao, W., Yin, X., Ying, C., Luo, Y., & Guo, Y. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808097 · Cited by 10
Chen, X., Wang, Q., Wang, S. Q., Zhang, W., Liu, Y., Liu, B. F., & Jiang, Z. H. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808099
Self-citationLLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluation
Fakhoury, S., Naik, A., Sakkas, G., Chakraborty, S., & Lahiri, S. K. (2024) · arXiv · doi:10.48550/arxiv.2404.10100
“2b: assert find_First_Missing([1,2,4,6]) ==0 .”
BackgroundDo Users Write More Insecure Code with AI Assistants?
Perry, N., Srivastava, M., Kumar, D., & Boneh, D. (2022) · arXiv
A Transformer-Based Approach for Smart Invocation of Automatic Code Completion
de Moor, A., van Deursen, A., & Izadi, M. (2024) · arXiv
“Our line-completions are generated by smaller language models than the industry-standard, allowing the user to pick one of three completions provided by InCoder [11], UniXCoder [14], and CodeGPT [28].”
MethodLanguage Models for Code Completion: A Practical Evaluation
Izadi, M., Katzy, J., van Dam, T., Otten, M., Popescu, R. M., & van Deursen, A. (2024) · arXiv · doi:10.48550/arxiv.2402.16197 · Cited by 57
“Thegoalofthisstudyistosystematicallyevaluatethreepub-liccompletionmodels,namelyInCoder[2],UniXcoder[3],and CodeGPT[19]inareal-worldcodingsetup.”
BackgroundMethodInfluential
Metadata
- Authors
- Fried, D., Aghajanyan, A., Lin, J., Wang, S., Wallace, E., Shi, F., Zhong, R., Yih, W.-T., Zettlemoyer, L., & Lewis, M.
- Affiliations
- Meta (United States) (US)
- Carnegie Mellon University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2204.05999
- OpenAlex
- W4224060952
- Cited by
- 187
- References
- 81
- Citations by year
- 2026: 4 · 2025: 52 · 2024: 68 · 2023: 59 · 2022: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Keywords
- program synthesis, bidirectional context, code editing, type inference, comment generation, variable renaming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
