The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
IntelliCode compose: code generation using transformer
Open accessAbstract
In software development through integrated development environments (IDEs), code completion is one of the most widely used features. Nevertheless, majority of integrated development environments only support completion of methods and APIs, or arguments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Several methods have been proposed to incorporate repository context into the training of language models, including n-grams (Tu et al., 2014), LSTMs (Hellendoorn and Devanbu, 2017), and Transformers (Svyatkovskiy et al., 2020;Liu et al., 2022;Ding et al., 2022).”
From RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation (2023) · cites this paper as Svyatkovskiy et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The IntelliCode Compose system uses the term CTR (Click Through Rate) for this and reports a value of 10% in online trials [11].”
From Productivity Assessment of Neural Code Completion (2022) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“act syntax tree (AST) or concrete syntax tree (CST), and dataflow graph. A transformer block will typically consist of a multi-head self-attention, followed by a two-layer multi-layer perceptron (MLP) [17], optionally containing residual connections and layer normalization [23]. Recent neural architecture searches of t…”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 2×
“Several transformer models such as GPT-2, BERT, XLNet, and RoBERTa [19–22] have demonstrated the ability to learn effectively from unlabeled data to perform a wide variety of downstream tasks given supervised discriminative fine-tuning on each specific task.”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“Several transformer models such as GPT-2, BERT, XLNet, and RoBERTa [19–22] have demonstrated the ability to learn effectively from unlabeled data to perform a wide variety of downstream tasks given supervised discriminative fine-tuning on each specific task.”
BackgroundRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 2×
“We refer the interested reader to the numerous publications about transformer models [16, 20–22], and focus on code completion for the remainder of this section.”
BackgroundDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“lue and are most likely lower than the actual user sentiment. Over 150K requests, we have seen a surfacing rate of 9.2% and an acceptance rate of 10%. 11 Knowledge Distillation Knowledge distillation [38] is the model compression technique in which a smaller model – the student network – is trained to reproduce the re…”
InfluentialExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“(3) Language specific control codes, is an approach introduced in [29, 30] that uses target prefixes to facilitate constrained language modeling or multitask learning.”
Background(2004) · Cited by 9,718
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · cited 3×
“Transformers [18–21] are a family of neural networks designed to process ordered sequential data.”
BackgroundDistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653 · cited 1×
“n [38] is the model compression technique in which a smaller model – the student network – is trained to reproduce the results of a larger model – the teacher network. It has been shown in literature [39, 40], that it is possible to reach comparable performance on various tasks using distilled neural networks, resulti…”
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
Flagged first, then by citations
Research works printed but not linked: 7
- Alexey Svyatkovskiy , Sebastian Lee , Anna Hadjitofi , Maik Riechert , Juliana Franco , and Miltiadis Allamanis . Fast and memory-eficient neural code completion , 2020 . Alexey Svyatkovskiy, Sebastian Lee, Anna Hadjitofi, Maik Riechert, Juliana Franco, and Miltiadis Allamanis. Fast and memory-eficient neural code completion, 2020.
- Alec Radford , Jef Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . Language models are unsupervised multitask learners . 2019 . Alec Radford, Jef Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
- Microsoft Corporation . Ai-assisted development. https://marketplace. visualstudio.com/items?itemName=VisualStudioExptTeam.VSIntelliCode. Visited Jan 2020 . Microsoft Corporation. Ai-assisted development. https://marketplace. visualstudio.com/items?itemName=VisualStudioExptTeam.VSIntelliCode. Visited Jan 2020.
- Alec Radford . Improving language understanding by generative pre-training . 2018 . Alec Radford. Improving language understanding by generative pre-training. 2018.
- Jimmy Lei Ba , Jamie Ryan Kiros, and Geofrey E. Hinton. Layer normalization , 2016 . Jimmy Lei Ba, Jamie Ryan Kiros, and Geofrey E. Hinton. Layer normalization, 2016.
- Eclipse Foundation . Code Recommenders . www.eclipse.org/recommenders. Visited June 2017 . Eclipse Foundation. Code Recommenders. www.eclipse.org/recommenders. Visited June 2017.
- Eclipse Recommenders . Eclipse SnipMatch . http://www.eclipse.org/ recommenders/manual/#snipmatch , 2014 . Visited Jun 2017. Eclipse Recommenders. Eclipse SnipMatch. http://www.eclipse.org/ recommenders/manual/#snipmatch, 2014. Visited Jun 2017.
Cited by
Sorted by citations
Items 301–400 of 480
Demystifying API misuses in deep learning applications
Yang, D., Liu, K., Lei, Y., Li, L., Xie, H., Liu, C., Wang, Z., Mao, X., & Bissyandé, T. F. (2024) · Empirical Software Engineering · doi:10.1007/s10664-023-10413-9 · Cited by 3
“…in various application areas CEO Nvidia (2023), ranging from futuristic (e.g., code generation) Yang et al. (2020); Cambronero et al. (2019); Svyatkovskiy et al. (2020), safety-critical (e.g., self-driving cars Kuutti et al. 2020 or medical diagnosis Liu et al. 2019) to simple leisure…”
Salvi, R. C., & Bosch, N. (2025) · ACM Journal on Responsible Computing · doi:10.1145/3737882 · Cited by 4
“Large language models ( LLMs ) are artificial intelligence models designed to process and generate a wide range of text content, such as human-like language [19] and computer code [79].”
Wen, R., Assalif, A., Lee, A. S.-H., Kamadod, R., Behdinan, A., Carshon-Marsh, R., Meh, C., Ng, T. K. S., Brown, P., Jha, P. N., et al. (2025) · BMC Medicine · doi:10.1186/s12916-025-04584-z · Cited by 4
Generative AI in Software Engineering: Revolutionizing Code Generation and Debugging
Saravanan, V., Kavitha, S., Ravi, S., Seetha, A., Rambabu, C., & Kanth, T. V. R. (2025) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.1718 · Cited by 7
“By allowing developers to understand why a particular code suggestion was made, XAI enhances transparency and builds trust in AI-assisted programming environments [7].”
A Framework for On the Fly Input Refinement for Deep Learning Models
Rathnasuriya, R. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion66252.2025.00041 · Cited by 3
Guo, X., Li, C. C., & Tsuchiya, T. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3459-3_32 · Cited by 6
Question Generation from Code Snippets and Programming Error Messages
Yao, B., Chen, W., Gong, Y., Zhou, B., Xie, J., Wei, Z., Cheng, B., & Duan, N. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88480-2_32 · Cited by 3
Detecting Compiler Bugs Via a Deep Learning-Based Framework
Tang, Y., Ren, Z., Jiang, H., Qiao, L., Liu, D., Zhou, Z., & Kong, W. (2022) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194022500206 · Cited by 3
Pseudocode to Code Based on Adaptive Global and Local Information
Yu, Q., Huang, Z., & Gu, N. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00016 · Cited by 3
Exploring the Impact of Vocabulary Techniques on Code Completion: A Comparative Approach
Hussain, Y., Huang, Z., Zhou, Y., & Khan, I. A. (2023) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194023500687 · Cited by 3
DAnTE: A Taxonomy for the Automation Degree of Software Engineering Tasks
Melegati, J., & Guerra, E. (2024) · Springer eBooks · doi:10.1007/978-3-031-55642-5_3 · Cited by 3
Code Generation Using Self-Interactive Assistant
Zhao, Z., Sun, J., Cai, C., & Wei, Z. (2024) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac61105.2024.00377 · Cited by 3
“The Trans-former [10] architecture improved versatility, enabling models like Intellicode [11] to provide multi-language suggestions and CodeBERT [3] to support natural language (NL) to programming language (PL) tasks like code search.”
MethodAPT: Adaptive Prefix-Tuning on Pretrained Models for Code Intelligence
Wu, Y., Yu, Y., Yuan, Z., Huang, S., & Cai, B. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651243 · Cited by 4
“To evaluate the quality of generated code, we adopt the Levenshtein Edit Similarity metric [29].”
MethodE-code: Mastering efficient code generation through pretrained models and expert encoder group
Pan, Y., Lyu, C., Yang, Z., Li, L., Liu, Q., & Shao, X. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107602 · Cited by 4
Using Large Language Models for Aerospace Code Generation: Methods, Benchmarks, and Potential Values
He, R., Zhang, L., Lyu, M., Lyu, L., & Xue, C. (2025) · Aerospace · doi:10.3390/aerospace12060498 · Cited by 3
Multi-source Analysis of AI Vulnerabilities: Methodology and Algorithms of Data Collection
Neretin, O., Kharchenko, V. S., & Fesenko, H. V. (2023) · IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS) · doi:10.1109/idaacs58523.2023.10348671 · Cited by 3
“AI is already widely used as a code augmentation tool to improve developer productivity [3]-[4].”
MethodOn improvements of robustness of obfuscated JavaScript code detection
Пономаренко, Г. С., & Klyucharev, P. G. (2022) · Journal of Computer Virology and Hacking Techniques · doi:10.1007/s11416-022-00450-1 · Cited by 3
“One of the most prominent approaches for the feature work could be using large pretrained language model (e.g. GPT-C [28]).”
MethodCLSEBERT: Contrastive Learning for Syntax Enhanced Code Pre-Trained Model.Preprint
Wang, X., Wang, Y., Zhou, P., Mi, F., Xiao, M., Wang, Y., Li, L., Liu, X., Wu, H., Liu, J., et al. (2021) · arXiv (Cornell University) · Cited by 3
Assessing Small Language Models for Code Generation: An Empirical Study with BenchmarksPreprint
Hasan, M. M., Waseem, M., Kemell, K., Rasku, J., Ala-Rantala, J., & Abrahamsson, P. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5472927 · Cited by 3
PPM: Automated Generation of Diverse Programming Problems for Benchmarking Code Generation Models
Chen, S. H., Feng, X., Han, X., Liu, C., & Yang, W. M. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3643780 · Cited by 2
SEART Data Hub: Streamlining Large-Scale Source Code Mining and Pre-Processing
Dabić, O., Tufano, R., & Bavota, G. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00097 · Cited by 2
“To teach DL models how to deal with code-related tasks ( e.g., code generation [4], code summarization [5], code completion [6], test case generation [7]), they need millions of examples showcasing the task ( e.g., pairs of ⟨ code , textual summary ⟩ to train for code summarization).”
MethodAutomated Code Summarization by Training Large Language Models with Crowdsourced Knowledge
Xia, M., Maharjan, S., & Song, M. (2025) · IEEE/ACIS International Conference on Software Engineering Research, Management and Applications (SERA) · doi:10.1109/sera65747.2025.11154515 · Cited by 4
Ciubotaru, B.-I. (2025) · Computer Science in Cars Symposium · doi:10.1109/cscs66924.2025.00063 · Cited by 1
“They assist with debugging by identifying potential issues in code, suggesting fixes, and explaining the underlying problems [47].”
A Federated Adaptive Large Language Model Fine-Tuning Framework for Software Development
Chen, J., Cai, Z., Chen, W., Wang, W., Zheng, Z., & Yu, P. S. (2025) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2025.3623626 · Cited by 3
“Restrictions apply. language summary that accurately conveys the program’s functionality and logic [31].”
Zhou, X., Cui, T., Zhu, X., Wang, J., & Lai, X. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107942 · Cited by 2
Leveraging LLMs for Automatic Feature Extraction in Embedded Systems to Support Software Reuse
Talukder, A. A. T., Alam, O., & Azim, A. (2025) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri66576.2025.00009 · Cited by 2
“CodeGPT-small: CodeGPT-small [16], [17] is a GPT-2-based decoder-only model trained on code.”
Identifying and Mitigating API Misuse in Large Language Models
Zhuo, T. Y., He, J., Sun, J., Xing, Z., Lo, D., Grundy, J., & Du, X. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3651566 · Cited by 5
“In practice, decoder-only models are preferred in IDE environments, where developers rely on them for code completion and comment generation [39].”
Clean Code, Better Models: Enhancing LLM Performance with Smell-Cleaned Dataset
Xue, Z., Zhang, X., Gao, Z., Hu, X., Gao, S., Xia, X., & Li, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3793252 · Cited by 3
“Although previous studies have applied LLMs to solve various code-related tasks, such as code generation [8, 51, 69], test generation [59, 61], program repair [30, 34, 79], their application in eliminating code smells has been rare or still unexplored.”
FACE: Fully Overlapped PD Scheduling and Multi-Level Architecture Co-Exploration on Wafer
Xu, Z., Kong, D., Liu, J., Jiang, D., Dai, X., Deng, J., Hu, Y., & Yin, S. (2026) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca68181.2026.11408518 · Cited by 2
“Large language models (LLMs) [6], [7], [16], [68], [69] have recently emerged as pivotal enablers of artificial intelligence (AI), powering a wide spectrum of applications including chatbots [8], [11], [57], code generation [14], [19], [65], and agent-based systems [12], [50], [64].”
Software defect detection using large language models: a literature review
Chen, Y., Shen, Y., Wang, T., Ou, S., Wang, R., Li, Y., & Pan, Z. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-025-40672-2 · Cited by 3
Unveiling the power of code pre-trained models in automated program repair: A systematic review
Zhan, S., Wang, X. J., Wei, D., Xiang, X., Lin, Y., Wu, H., & Lu, J. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113015 · Cited by 2
Digital Rubber Duck: Leveraging Large Language Models for Extreme Programming
Elvira, T., Procko, T., Couder, J. O., & Ochoa, O. (2023) · Congress in Computer Science, Computer Engineering, and Applied Computing (CSCE) · doi:10.1109/csce60160.2023.00051 · Cited by 5
“Based on transformers, the code generation tool Intellicode Compose is an add-on for Integrated Development Environments (IDEs), e.g., Visual Studio Code, used by developers as a programming assistant [32].”
Ranked Syntax Completion With LR Parsing
Choi, K., Hwang, S., Moon, H., & Sasano, I. (2024) · ACM Symposium on Applied Computing · doi:10.1145/3605098.3635944 · Cited by 2
Improving AST-Level Code Completion with Graph Retrieval and Multi-Field Attention
Xia, Y., Liang, T., Min, W., & Kuang, L. (2024) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3643916.3644420 · Cited by 2
“Token-level code completion sequentially generates code tokens [7, 8, 11, 14, 17, 21, 22, 25, 26, 29, 31, 36].”
BackgroundGou, Q., Dong, Y., & Ke, Q. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112140 · Cited by 2
Smart Contract Vulnerability Detection with Self-Ensemble Pre-Trained Language Models
Dai, C., Ding, H., Ma, W., & Wu, Y. (2024) · International Conference on Computer, Information and Telecommunication Systems · doi:10.1109/cits61189.2024.10607982 · Cited by 3
Qin, Z., Bai, Y., Sohrabizadeh, A., Ding, Z., Hu, Z., Sun, Y., & Cong, J. (2024) · ACM/IEEE Workshop on Machine Learning for CAD (MLCAD) · doi:10.1145/3670474.3685952 · Cited by 11
“Source-code-based methods [10, 19, 42, 47] employ language models [9, 11, 14, 23, 32, 33, 37, 59] on source code to perform various types of tasks.”
iiPCS: Intent-Based In-Context Learning for Project-Specific Code Summarization
Wang, Y., Liu, X., Lu, X., & Zhou, A. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650776 · Cited by 2
“Wei et al. [1] investigate the duality between the code summarization task and the code generation task [18].”
BackgroundAn Exploration Into the Mechanisms and Evolution of GPT Models
AlShannaq, F. B., Shehab, M., Assaf, A. H. A., Alhenawi, E., & Awawdeh, S. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-0884-4.ch018 · Cited by 3
AI Copilot for the Modern Developer : Leveraging GenAI in Software Development
Nagpal, A. (2024) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit241051056 · Cited by 2
Generative AI for Smart Contracts in Real Estate Business
Zhao, W., Pobbathi, L., Ramprasath, S., & Patibandla, M. (2024) · International Conference on Dependable Systems and Their Applications (DSA) · doi:10.1109/dsa63982.2024.00018 · Cited by 2
“For example, Svyatkovskiy et al. [20] developed an AI-powered code completion tool that can assist developers in writing smart contracts more efficiently.”
You Don’t Have to Say Where to Edit! jLED—Joint Learning to Localize and Edit Source Code
Pian, W., Li, Y., Tian, H., Sun, T., Song, Y., Tang, X., Habib, A., Klein, J., & Bissyandé, T. F. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3712187 · Cited by 2
“The Transformer’s ability to reason about long-range dependencies has proven to be highly advantageous for various source code processing tasks, including code generation [58] and code summarization [2].”
LLM-based Interactive Code Generation: Empirical Evaluation
Shaikhelislamov, D., Drobyshevskiy, M., & Belevantsev, A. A. (2024) · Ivannikov ISPRAS Open Conference (ISPRAS) · doi:10.1109/ispras64596.2024.10899123 · Cited by 2
A Study of Prompt Engineering Techniques for Code Generation: Focusing on Data Science Applications
Saint-Jean, D., Smadi, B. A., Raza, S., Linton, S., & Igweagu, U. (2025) · Advances in intelligent systems and computing · doi:10.1007/978-3-031-89063-5_38 · Cited by 2
SpeedGen: Enhancing Code Efficiency through Large Language Model-Based Performance Optimization
Purschke, N., Kirchner, S., & Knoll, A. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00045 · Cited by 3
“Apart from that, large language models can be used for many different related tasks or subcategories, such as automated program repair [17], automated unit test generation [18], automated coding comments generation [19] and assisting the programmer with coding suggestions [20].”
Breaking Barriers in Mobile Accessibility: A Study of LLM-Generated Native Android Interfaces
Rabelo, D. M. F., Martins, R. D. S., Santos, I., Silva, P. H. G. D., da Gama, K. S., & Viana, W. (2025) · IEEE/ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft) · doi:10.1109/mobilesoft66462.2025.00010 · Cited by 7
An Empirical Study of Code Clones from Commercial AI Code Generators
Wu, W., Hu, H., Fan, Z., Qiao, Y., Huang, Y., Li, Y., Zheng, Z., & Lyu, M. R.-T. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729397 · Cited by 2
“Deep learning (DL) has showcased its transformative potential to automate or improve a gamut of software engineering tasks [39, 55].”
Hallucination Detection on Code Generation with SelfCheckGPT
Ito, W., Obara, Y., Sato, M., & Kuramitsu, K. (2025) · Journal of Information Processing · doi:10.2197/ipsjjip.33.487 · Cited by 4
An Empirical Study of Exploring the Capabilities of Large Language Models in Code Learning
Liu, S., Guo, D., Zhang, J., Ma, W., Li, Y., & Liu, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3609876 · Cited by 3
“Specifically, we use MAP@R for POJ-104, F1 for BigCloneBench, MRR for code search, smoothed BLEU-4 [48] for code summarization, exact match (i.e., EM) and BLEU-4 for code generation, EM and Levenshtein Edit Similarity [49] (i.e., Edit Sim) for code completion.”
AdaptivePaste: Intelligent Copy-Paste in IDE
Liu, X., Jang, J., Sundaresan, N., Allamanis, M., & Svyatkovskiy, A. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3613895 · Cited by 9
Self-citationVale, L. D. N., & Maia, M. D. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.09423 · Cited by 2
“IntelliCode [11] it is a general purpose multilingual code completion tool which is capable of predicting sequences of code tokens of arbitrary types, generating up to entire lines of syntactically correct code.”
BackgroundSchuster, T., Kalyan, A., Polozov, O., & Kalai, A. T. (2021) · arXiv · doi:10.48550/arxiv.2106.05784 · Cited by 2
“[54] Alexey Svyatkovskiy, Shao Kun Deng, Shengyu Fu, and Neel Sundaresan.”
BackgroundLearning to Generate Code Sketches.Preprint
Guo, D., Svyatkovskiy, A., Yin, J., Duan, N., Brockschmidt, M., & Allamanis, M. (2021) · arXiv (Cornell University) · Cited by 3
Self-citationMergeBERT: Program Merge Conflict Resolution via Neural Transformers.Preprint
Svyatkovskiy, A., Mytcowicz, T., Ghorbani, N., Fakhoury, S., Dinella, E., Bird, C., Sundaresan, N., & Lahiri, S. K. (2021) · arXiv (Cornell University) · Cited by 2
Self-citationA Comparison of Code Embeddings and BeyondPreprint
Han, S., Wang, D., Li, W., & Lu, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.07173 · Cited by 5
“The learnt representations, or the program embeddings, can be further used for downstream tasks in programming language processing and software engineering [10, 13, 24, 26, 58, 63, 67].”
MethodLong-Range Modeling of Source Code Files with eWASH: Extended Window Access by Syntax HierarchyPreprint
Clement, C. B., Lu, S., Liu, X., Tufano, M., Drain, D., Duan, N., Sundaresan, N., & Svyatkovskiy, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.08780 · Cited by 11
“Namely, the generative pre-trained transformer model for code (Svyatkovskiy et al., 2020) (GPT-C) and the Python method text-to-text transfer trans-former model (Clement et al., 2020) (PyMT5).”
BackgroundMethodInfluentialSelf-citationCan large language models detect PII in code?Preprint
Gurjar, A., Camp, L. J., Ringenberg, T. R., Ma, X., & Chaora, A. C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4619112 · Cited by 5
A Combinatorial Strategy for API Completion: Deep Learning and Heuristics
Liu, Y., Yin, Y., Deng, J., Li, W., & Peng, Z. (2024) · Electronics · doi:10.3390/electronics13183669 · Cited by 1
Improving Source Code Pre-Training via Type-Specific Masking
Zou, W., Li, Q., Li, C., Ge, J., Chen, X., Huang, L., & Luo, B. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3699599 · Cited by 2
“GPT-C [92] can predict sequences of code tokens of arbitrary types, and generate up to entire lines of syntactically correct code.”
De-fine: De composing and Re fin ing Visual Programs with Auto-Feedback
Gao, M., Li, J., Fei, H., Pang, L., Ji, W., Wang, G., Lv, Z., Zhang, W., Tang, S., & Zhuang, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681082 · Cited by 10
“This could potentially lead to sub-optimal or hard-to-maintain code and contradict the original design intention for the compositional task.”
Deep Learning-Based Code Completion: On the Impact on Performance of Contextual Information
Ciniselli, M., Pascarella, L., & Bavota, G. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00029 · Cited by 1
On the Applicability of Code Language Models to Scientific Computing Programs
Zhao, Q., Liu, F., Long, X., Wu, C., & Zhang, L. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3564599 · Cited by 2
“GPT-C [57] belongs to the earliest pre-trained Transformer decoder and is used in Visual Studio Code IDE. OpenAI’s Codex [11], a descendant of GPT-3 [6], is fine-tuned on publicly available GitHub code and powers GitHub Copilot, but its source code and training data are accessible only through their…”
Improving LLM-based code completion using LR parsing
Atique, M. M. A. B., Moon, H., Sasano, I., & Choi, K. (2025) · Journal of Computer Languages · doi:10.1016/j.cola.2025.101352 · Cited by 1
Veeramreddygari, U. (2023) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit23906195 · Cited by 1
Generative AI tools for accelerated software engineering
Kanmani, P., Prabha, K. C., Veerandeswari, J., & Manjula, M. (2025) · Advances in computers · doi:10.1016/bs.adcom.2025.06.008 · Cited by 3
OpCodeBERT: A Method for Python Code Representation Learning by BERT With Opcode
Qiu, C., Liu, J., Xiao, X., & Xiao, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3610244 · Cited by 2
“For example, existing pre-trained models [4], [5], [6], [7], [8], [9] mostly model source code as token sequences.”
Literate Programming With LLMs? — A Study on Rosetta Code and CodeNet
Sun, S., & Staron, M. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3629828 · Cited by 1
“Svyatkovskiy et al. [30] proposed IntelliCode Compose, a multilingual code completion tool.”
DockerFill: Automatically Completing Dockerfile Code With Syntax-Aware Multi-Task Learning
Wu, Y., Zhang, Y., Wang, T., Ding, B., & Wang, H. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3632074 · Cited by 1
“Inspired by the success of Transformer-based language models on regular code’s completion/generation tasks [16], [35], [45], [46], we choose to adopt the Transformer architecture for constructing D OCKER F ILL .”
Leathrum, J., Berardi, A. S., & Shen, Y.-Z. (2025) · Online World Conference on Soft Computing in Industrial Applications · doi:10.1109/wsc68292.2025.11339012 · Cited by 1
“In the context of SSE, LLMs offer a unique opportunity to bridge the gap between domain-specific knowledge and software engineering expertise, helping scientists and engineers produce more maintainable and efficient simulation code (Svyatkovskiy et al. 2020).”
RepoMasterEval: Evaluating Code Completion via Real-World Repositories
Wu, Q., Peng, C., Gao, P., Hu, R., Gan, H., Jiang, B., Tang, J., Deng, Z., Guan, Z., Gao, C., et al. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00304 · Cited by 6
“For instance, HumanEval [7] and ClassEval [9] use test cases and calculate the pass rate of models’ generated code, while RepoBench [10] compares the generated code with the ground truth based on code similarity metrics [11].”
Zhang, Y., Xia, W., Zhao, B., Yuan, T., & Yu, X. (2026) · Symmetry · doi:10.3390/sym18020248 · Cited by 2
From Function to Repository: Towards Repository-Level Evaluation of Software Vulnerability Detection
Wen, X.-C., Wang, X., Chen, Y., Hu, R., Lo, D., & Gao, C. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3662145 · Cited by 17
“We evaluate the relevance of vulnerability dependencies using two primary metrics as baselines: Jaccard Similarity and Edit Similarity [55].”
Vibe Coding: An AI-Augmented Flow Paradigm for Software Development
Bussari, S. (2025) · International Conference on Computer and Applications (ICCA) · doi:10.1109/icca66035.2025.11430768 · Cited by 1
“Empirical studies on tools like GitHub Copilot show measurable gains in developer productivity and flow [7], while IntelliCode Compose demonstrates transformer-based code generation [6].”
When Help Hurts: Verification Load and Fatigue with AI Coding Assistants
Fan, G. F., Liu, D., Pan, L., & Zhang, R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791176 · Cited by 6
Kobiella, C., Breidenstein, D., & Schmidt, A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790757 · Cited by 1
From Where to What: The Geo-Intervention Modelling Framework
Wen, R., & Li, S. (2026) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi15070292
Can AI Agents Generate Microservices? How Far are We?
Adnan, B., Esposito, M., Taibi, D., & Vaidhyanathan, K. (2026) · IEEE International Conference on Software Architecture (ICSA) · doi:10.1109/icsa66085.2026.00008 · Cited by 1
Evaluating package-level scoping strategies for repository-level code completion in Pharo
AbedelKader, O., Ducasse, S., Polito, G., Zaitsev, O., & Robbes, R. (2026) · Journal of Computer Languages · doi:10.1016/j.cola.2026.101412 · Cited by 1
“More recently, Bibaev et al., [6] proposed learning-to-rank models trained on anonymous IDE usage logs, validating that user interaction data can serve as a powerful implicit supervision source for improving completion accuracy in production settings.”
An Eye for AI: Eye-Tracking the Micro-Interruptions of GenAI Code Suggestions
Alakmeh, T., D’Angelo, S., & Fritz, T. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773132 · Cited by 1
Multi-location cryptographic code repair with neural-network-based methodologies
Xiao, Y. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3473102 · Cited by 1
Seq2Code: Transformer-Based Encoder-Decoder Model for Python Source Code Generation
Laskari, N. K., Reddy, K. A. N., & Reddy, M. I. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-19-9225-4_23 · Cited by 1
Who Are All The Stochastic Parrots Imitating? They Should Tell Us!
Shaier, S., Hunter, L., & Kann, K. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-short.13 · Cited by 2
Analyzing Generative Models for Realistic Data Augmentation across Modalities and Applications
Patel, S., Patel, U., Nanavati, J., & Patel, A. M. (2024) · International Conference on Computing for Sustainable Global Development · doi:10.23919/indiacom61295.2024.10498685 · Cited by 1
“Relational and Tabular Data Generator [18], Automatic Item Generation [19], synthetic patient notes generation [20], code generation [21], Music Transformer [22] StyleGAN [23] It uses Baseline Progressive GAN.”
MethodSG-Coder: Code completion algorithm applied on large software development platform
Li, S., Liu, Z., Liu, X., Chen, G., Gao, Y., & Liu, Y. (2023) · International Academic Exchange Conference on Science and Technology Innovation (IAECST) · doi:10.1109/iaecst60924.2023.10503165 · Cited by 1
“The application of large-scale NLP models to code has resulted in the development of code models such as CodeBERT[10], GPT-C [11], UnixCoder [12], graphCodeBert [13] and CodeX [14] which aim to improve various aspects of code tasks.”
MethodCoSTV: Accelerating Code Search with Two-Stage Paradigm and Vector Retrieval
Zheng, D., Wang, Y., Chen, W., Chen, J., & Zheng, Z. (2024) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec65559.2024.00049 · Cited by 1
“GPT-C [27] is pre-trained on massive source code to facilitate more code intelligence tasks.”
Learning code better through structural information of data flow
Zhang, Z., Chen, Y., Jiao, T., Bai, L., Guo, C., & Song, J. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07348-x · Cited by 1
“Svyatkovskiy et al. [23] propose CodePT, a model specifically tailored for programming tasks, enhancing the model’s understanding and processing capabilities for these tasks by utilizing self-supervised learning methods on extensive code repositories.”
He, R., Zhang, L., Lyu, L., & Xue, C. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00545-1 · Cited by 4
“Svyatkovskiy et al. (2020) proposed a Transformer-based approach to provide on-the-fly code suggestions in IDEs.”
SYNC: SYnergistic aNnotation Collaboration between Humans and LLMs for Enhanced Model Training
Le, T., Taylor, W., Maharjan, S., Xia, M., & Song, M. (2025) · IEEE/ACIS International Conference on Software Engineering Research, Management and Applications (SERA) · doi:10.1109/sera65747.2025.11154541 · Cited by 1
Code Autocomplete Using Transformers
Meyrer, G. T., de Araújo, D. A., & Rigo, S. J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-91699-2_15 · Cited by 1
Lee, J., Moon, H., Park, C., Seo, J., Eo, S., & Lim, H. (2022) · Applied Sciences · doi:10.3390/app12136686 · Cited by 1
Cross-Domain Evaluation of a Deep Learning-Based Type Inference System
Gruner, B., Sonnekalb, T., Heinze, T. S., & Brust, C.-A. (2023) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr59073.2023.00034 · Cited by 1
“Moreover, the dataset may also be processed further to enable its usage for related downstream tasks, e.g. code completion [53].”
BackgroundPersonalized action suggestions in low-code automation platforms
Gupta, S., Verbruggen, G., Singh, M. P., Gulwani, S., & Le, V. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion58688.2023.00100 · Cited by 1
Toward Less Hidden Cost of Code Completion with Acceptance and Ranking Models
lijingxuan, L., Huang, R., Li, W., Yao, K., & Tan, W. (2021) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.26226/morressier.613b5419842293c031b5b638 · Cited by 16
Distilling Transformers for Neural Cross-Domain SearchPreprint
Clement, C. B., Wu, C. H., Drain, D., & Sundaresan, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.03322 · Cited by 1
“Pre-trained transformers (Vaswani et al., 2017) have pushed the limits of natural language processing (NLP) tasks in both the domains of nat-ural language and software engineering (Clement et al., 2020; Svyatkovskiy et al., 2020; Tu-fano et al., 2020; Guo et al., 2020).”
BackgroundSelf-citationCross-language binary-to-source code matching approach using graph neural networks
Chen, H. (2023) · doi:10.31274/td-20240329-185 · Cited by 1
Prompt-Based Building Code Information Transformation Method for Automated Compliance CheckingPreprint
Yang, F., & Zhang, J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4819834 · Cited by 1
Predicting the Objective and Priority of Issue Reports in a Cross project Context.Preprint
Izadi, M., Akbari, K., & Heydarnoori, A. (2020) · arXiv (Cornell University)
PyART: Python API Recommendation in Real-TimePreprint
He, X., Xu, L., Zhang, X., Hao, R., Feng, Y., & Xu, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.04706
“API recommendation approaches based on Machine Learning tend to have better performance for Python than most traditional methods [21]–[24].”
MethodLearning to Extend Program Graphs to Work-in-Progress CodePreprint
Li, X., Maddison, C. J., & Tarlow, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.14038 · Cited by 4
Metadata
- Authors
- Svyatkovskiy, A., Deng, S. K., Fu, S., & Sundaresan, N.
- Affiliations
- Microsoft (United States) (US)
- Microsoft, USA
- ORCID
- Sundaresan, Neel
- Venue
- Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2020-11-08
- Pages
- 1433-1443
- OpenAlex
- W3025993830
- Cited by
- 474
- References
- 38
- Citations by year
- 2026: 42 · 2025: 97 · 2024: 117 · 2023: 103 · 2022: 52 · 2021: 56 · 2020: 7
- FWCI
- 41.33
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- code generation, code completion, transformer models, integrated development environment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
