The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
IntelliCode compose: code generation using transformer
Open accessAbstract
In software development through integrated development environments (IDEs), code completion is one of the most widely used features. Nevertheless, majority of integrated development environments only support completion of methods and APIs, or arguments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Several methods have been proposed to incorporate repository context into the training of language models, including n-grams (Tu et al., 2014), LSTMs (Hellendoorn and Devanbu, 2017), and Transformers (Svyatkovskiy et al., 2020;Liu et al., 2022;Ding et al., 2022).”
From RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation (2023) · cites this paper as Svyatkovskiy et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The IntelliCode Compose system uses the term CTR (Click Through Rate) for this and reports a value of 10% in online trials [11].”
From Productivity Assessment of Neural Code Completion (2022) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“act syntax tree (AST) or concrete syntax tree (CST), and dataflow graph. A transformer block will typically consist of a multi-head self-attention, followed by a two-layer multi-layer perceptron (MLP) [17], optionally containing residual connections and layer normalization [23]. Recent neural architecture searches of t…”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 2×
“Several transformer models such as GPT-2, BERT, XLNet, and RoBERTa [19–22] have demonstrated the ability to learn effectively from unlabeled data to perform a wide variety of downstream tasks given supervised discriminative fine-tuning on each specific task.”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“Several transformer models such as GPT-2, BERT, XLNet, and RoBERTa [19–22] have demonstrated the ability to learn effectively from unlabeled data to perform a wide variety of downstream tasks given supervised discriminative fine-tuning on each specific task.”
BackgroundRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 2×
“We refer the interested reader to the numerous publications about transformer models [16, 20–22], and focus on code completion for the remainder of this section.”
BackgroundDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“lue and are most likely lower than the actual user sentiment. Over 150K requests, we have seen a surfacing rate of 9.2% and an acceptance rate of 10%. 11 Knowledge Distillation Knowledge distillation [38] is the model compression technique in which a smaller model – the student network – is trained to reproduce the re…”
InfluentialExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“(3) Language specific control codes, is an approach introduced in [29, 30] that uses target prefixes to facilitate constrained language modeling or multitask learning.”
Background(2004) · Cited by 9,718
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · cited 3×
“Transformers [18–21] are a family of neural networks designed to process ordered sequential data.”
BackgroundDistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653 · cited 1×
“n [38] is the model compression technique in which a smaller model – the student network – is trained to reproduce the results of a larger model – the teacher network. It has been shown in literature [39, 40], that it is possible to reach comparable performance on various tasks using distilled neural networks, resulti…”
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
Flagged first, then by citations
Research works printed but not linked: 7
- Alexey Svyatkovskiy , Sebastian Lee , Anna Hadjitofi , Maik Riechert , Juliana Franco , and Miltiadis Allamanis . Fast and memory-eficient neural code completion , 2020 . Alexey Svyatkovskiy, Sebastian Lee, Anna Hadjitofi, Maik Riechert, Juliana Franco, and Miltiadis Allamanis. Fast and memory-eficient neural code completion, 2020.
- Alec Radford , Jef Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . Language models are unsupervised multitask learners . 2019 . Alec Radford, Jef Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
- Microsoft Corporation . Ai-assisted development. https://marketplace. visualstudio.com/items?itemName=VisualStudioExptTeam.VSIntelliCode. Visited Jan 2020 . Microsoft Corporation. Ai-assisted development. https://marketplace. visualstudio.com/items?itemName=VisualStudioExptTeam.VSIntelliCode. Visited Jan 2020.
- Alec Radford . Improving language understanding by generative pre-training . 2018 . Alec Radford. Improving language understanding by generative pre-training. 2018.
- Jimmy Lei Ba , Jamie Ryan Kiros, and Geofrey E. Hinton. Layer normalization , 2016 . Jimmy Lei Ba, Jamie Ryan Kiros, and Geofrey E. Hinton. Layer normalization, 2016.
- Eclipse Foundation . Code Recommenders . www.eclipse.org/recommenders. Visited June 2017 . Eclipse Foundation. Code Recommenders. www.eclipse.org/recommenders. Visited June 2017.
- Eclipse Recommenders . Eclipse SnipMatch . http://www.eclipse.org/ recommenders/manual/#snipmatch , 2014 . Visited Jun 2017. Eclipse Recommenders. Eclipse SnipMatch. http://www.eclipse.org/ recommenders/manual/#snipmatch, 2014. Visited Jun 2017.
Cited by
Sorted by citations
Items 201–300 of 480
Private-library-oriented code generation with large language models
Zan, D., Chen, B., Gong, Y., Cao, J., Zhang, F., Wu, B., Guan, B., Yin, Y., & Wang, Y. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113934 · Cited by 42
ReACC: A Retrieval-Augmented Code Completion Framework
Lu, S., Duan, N., Han, H., Guo, D., Hwang, S.-W., & Svyatkovskiy, A. (2022) · arXiv · doi:10.48550/arxiv.2203.07722 · Cited by 96
“, 2020) and pretraining approaches (Liu et al., 2020; Svyatkovskiy et al., 2020) are later frequently utilized to accomplish this.”
BackgroundMethodSelf-citationNeshaei, S. P., Rietsche, R., Su, X., & Wambsganss, T. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645169 · Cited by 9
“As a result, GPT and Transformer models can generate texts in various domains, such as medical texts [6], text summaries [58], stories [34], patent claims [62], codes [27, 91], and research papers [94].”
BackgroundMethodAutomated Commit Intelligence by Pre-training
Liu, S., Li, Y., Xie, X., Ma, W., Meng, G., & Liu, Y. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3674731 · Cited by 9
Prompt Alchemy: Automatic Prompt Refinement for Enhancing Code Generation
Ye, S.-X., Sun, Z., Wang, G.-Q., Guo, L.-W., Liang, Q.-L., Li, Z., & Liu, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3589634 · Cited by 19
“C ODE generation has become a transformative application in software engineering, allowing the automatic generation of code from high-level descriptions or specifications [1], [2], [3].”
On the effectiveness of pretrained models for API learning
Hadi, M. A., Yusuf, I. N. B., Thung, F., Luong, K., Jiang, L., Fard, F. H., & Lo, D. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527886 · Cited by 13
“[42] has introduced IntelliCode Compose - a general-purpose multilingual code completion tool that can generate up to entire lines of syntactically correct code from sequences of code tokens of arbitrary types.”
MethodImproving domain-specific neural code generation with few-shot meta-learning
Yang, Z., Keung, J., Sun, Z., Zhao, Y., Li, G., Jin, Z. G., Liu, S., & Li, Y. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107365 · Cited by 11
INSPECT: Intrinsic and Systematic Probing Evaluation for Code Transformers
Karmakar, A., & Robbes, R. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3341624 · Cited by 9
Large Language Model vs. Stack Overflow in Addressing Android Permission Related Challenges
Oishwee, S. J., Stakhanova, N., & Codabux, Z. (2024) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3643991.3644933 · Cited by 9
Natural Language to Python Source Code using Transformers
Shah, M. K., Shenoy, R., & Shankarmani, R. (2021) · International Conference on Intelligent Technologies (CONIT) · doi:10.1109/conit51480.2021.9498268 · Cited by 9
Code4ML: a large-scale dataset of annotated Machine Learning code
Drozdova, A., Трофимова, Е. В., Guseva, P., Scherbakova, A., & Ustyuzhanin, A. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1230 · Cited by 9
“ML systems are capable of analyzing (Alsolai and Roper (2020); Bilgin et al. (2020)), manipulating (Goues et al. (2019); Liu et al. (2020)), and synthesizing (Svyatkovskiy et al. (2020); Austin et al. (2021)) code.”
BackgroundLeveraging Large Language Models to Support Authoring Gamified Programming Exercises
Montella, R., Vita, C. G. D., Mellone, G., Ciricillo, T., Caramiello, D., Luccio, D. D., Kosta, S., Damaševičius, R., Maskeliūnas, R., Queirós, R., et al. (2024) · Applied Sciences · doi:10.3390/app14188344 · Cited by 8
Promises and perils of using Transformer-based models for SE research
Xiao, Y., Zuo, X., Lu, X., Dong, J. S., Cao, X., & Beschastnikh, I. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.107067 · Cited by 27
Assessing ChatGPT’s Code Generation Capabilities with Short vs Long Context Programming Problems
Shuvo, U. A., Dip, S. A., Vaskar, N. R., & Islam, A. B. M. A. A. (2024) · International Conference on Networking, Systems and Security (NSysS) · doi:10.1145/3704522.3704535 · Cited by 10
AutoPAC: Exploring LLMs for Automating Policy to Code Conversion in Business Organizations
Chowdhary, N., Dutta, T., Chattopadhyay, S., & Chakraborty, S. (2025) · International Conference on Communication Systems & Networks (COMSNETS) · doi:10.1109/comsnets63942.2025.10885751 · Cited by 10
“IntelliCode Compose [36], a multilingual code completion tool, utilizes GPT for Code ( GPT-C ), which is a variant of the GPT-2 model trained from scratch on various source codes written in programming languages like Python, C#, JavaScript and TypeScript.”
Shi, E., Wang, Y., Du, L., Zhang, H., Han, S., Zhang, D., & Sun, H. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10378-9 · Cited by 9
“…(Guo et al. 2021; Feng et al. 2020; Wang et al. 2021; Jiang et al. 2021; Liu et al. 2020; Kanade et al. 2020; Wang et al. 2021; Ahmad et al. 2021; Svyatkovskiy et al. 2020) which can utilize a large corpus of unlabeled data, improve the code understanding and achieve substantial improvements in…”
BackgroundSource-Code Generation Using Deep Learning: A Survey
Ahmed, A., Azab, S. S., & Abdelhamid, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-49011-8_37 · Cited by 10
On the Impact of Language Selection for Training and Evaluating Programming Language Models
Katzy, J., Izadi, M., & Arie Van Deursen, A. (2023) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam59687.2023.00038 · Cited by 8
“Furthermore, research has shown that models perform better on certain programming languages and worse on others [10], [11], [13].”
BackgroundProgram Repair with Minimal Edits Using CodeT5
Shirafuji, A., Rahman, M. M., Amin, M. F. I., & Watanobe, Y. (2023) · International Conference on Awareness Science and Technology (iCAST) · doi:10.1109/icast57874.2023.10359288 · Cited by 10
“…(BiLSTM), LSTM with an attention mechanism (LST-MAttn), and Transformer, are used to correct erroneous source code and provide various learning supports, such as predicting the next tokens [4], [5], fixing b ugs i n w rong c ode [6]–[9], and improving the performance and readability [10], [11].”
MethodSoftware Defect Prediction via Generative Adversarial Networks and Pre-Trained Model
Song, W., Gan, L., & Bao, T. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.01503119 · Cited by 8
van Dam, T., van der Heijden, F., de Bekker, P., Nieuwschepen, B., Otten, M., & Izadi, M. (2024) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1145/3650105.3652289 · Cited by 15
PyART: Python API Recommendation in Real-Time
He, X., Xu, L., Zhang, X., Hao, R., Feng, Y., & Xu, B. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion52605.2021.00114 · Cited by 8
A Neural Network Architecture for Program Understanding Inspired by Human Behaviors
Zhu, R., Yuan, L., Li, X., Gao, M., & Cai, W. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.353 · Cited by 8
“For example, some methods apply pre-training techniques in NLP to boost comprehension, such as Code-BERT (Feng et al., 2020), GPT-C (Svyatkovskiy et al., 2020) and PLBART (Ahmad et al., 2021).”
MethodAlexander, F., Abdiwijaya, E. A., Pherry, F., Gunawan, A. A. S., & Anderies (2022) · International Conference on Software Engineering and Information Technology (ICoSEIT) · doi:10.1109/icoseit55604.2022.10029949 · Cited by 7
“IntelliCode uses one of the NLP domains, namely N-Gram so that it can provide advice to developers and help developers Autocomplete several lines of code directly at the same time [9].”
MethodYang, Y., Liu, Y., Bao, T., Wang, W., Niu, N., & Yin, Y. (2023) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12207 · Cited by 8
Soft-Labeled Contrastive Pre-Training for Function-Level Code Representation
Li, X., Guo, D., Gong, Y., Lin, Y., Shen, Y., Qiu, X., Jiang, D., Chen, W., & Duan, N. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.9 · Cited by 18
Vale, L. D. N., & Maia, M. D. A. (2021) · IEEE/ACM International Workshop on Bots in Software Engineering (BotSE) · doi:10.1109/botse52550.2021.00016 · Cited by 10
Cascaded Fast and Slow Models for Efficient Semantic Code SearchPreprint
Gotmare, A. D., Li, J., Joty, S., & Hoi, S. C. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07811 · Cited by 10
“Recent work on code generation like Chen et al. (2021)’s 12B parameter CodeX and Austin et al. (2021)’s 137B parameter LM use large scale autoregressive language models to demonstrate impressive capabilities of generating multiple lines of code from natural language descriptions, well beyond what previous generation m…”
Liu, C. C., Zhang, X., Zhang, H., Wan, Z., Huang, Z., & Yan, M. L. (2024) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3663529.3663848 · Cited by 7
An enhanced transformer-based framework for interpretable code clone detection
Nashaat, M., Amin, R., Eid, A. H., & Abdel-Kader, R. F. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112347 · Cited by 9
Optimizing Pre-Trained Code Embeddings With Triplet Loss for Code Smell Detection
Nizam, A., İslamoğlu, E., Adali, Ö. K., & Aydın, M. (2025) · IEEE Access · doi:10.1109/access.2025.3542566 · Cited by 10
Unseen Horizons: Unveiling the Real Capability of LLM Code Generation Beyond the Familiar
Zhang, Y., Xie, Y., Lit, S., Liu, P., Wang, C., Jia, Z., Huang, X., Song, J., Luo, C., Zheng, Z., et al. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00082 · Cited by 7
NLPerturbator: Studying the Robustness of Code LLMs to Natural Language Variations
Chen, J., Li, Z.-H., Hu, X., & Xia, X. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3745764 · Cited by 45
“We summarize them into approaches based on deep neural networks [21], [22], [23], pre-trained language models [23], [24], [25], and LLMs [5], [26], [27], [28].”
MethodAn Empirical Investigation on the Performance of Domain Adaptation for T5 Code Completion
Fukumoto, D., Kashiwa, Y., HIRAO, T., Fujiwara, K., & Iida, H. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00073 · Cited by 7
“Alexey et al. [10] have also developed a multiple tokens code completion model using GPT-2 [7] and show that the model outperforms n-gram language model approaches.”
BackgroundMethodImam, A. T. (2023) · IEEE Access · doi:10.1109/access.2023.3310394 · Cited by 7
“software engineering, it generates high-quality code [46].”
Code-aware fault localization with pre-training and interpretable machine learning
Zhang, Z., Li, Y., Yang, S., Zhang, Z., & Lei, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121689 · Cited by 10
Towards Better Multilingual Code Search through Cross-Lingual Contrastive Learning
Huang, X., Ma, Y., Zhou, H., Jiang, Z., Zhang, Y., Wang, T., & Li, S. (2023) · Asia-Pacific Symposium on Internetware · doi:10.1145/3609437.3609439 · Cited by 7
“Recent advances in machine learning have benefited many software engineering-related tasks, such as code search [13, 17, 27, 48], code summarization [8, 18, 26, 34], and code synthesis [23, 35].”
BackgroundDeepCNN: A Dual Approach to Fault Localization and Repair in Convolutional Neural Networks
Wardat, M., & Al-Alaj, A. (2024) · IEEE Access · doi:10.1109/access.2024.3384981 · Cited by 8
“Inspired by prior works [48], [49], [50], we have developed a Transformer-based model designed specifically for the analysis of sequential data features.”
BackgroundMethodExploring potential implications of intelligent tools for human aspects of software engineering
Melegati, J., Nascimento, N., Chanin, R. M., Sales, A., & Wiese, I. (2024) · International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1145/3641822.3641877 · Cited by 7
A Systematic Evaluation of Large Code Models in API Suggestion: When, Which, and How
Wang, C., Gao, S., Gao, C., Wang, W., Chong, C. Y., Gao, S., & Lyu, M. R.-T. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695004 · Cited by 7
Qin, Z., Bai, Y., Sohrabizadeh, A., Ding, Z., Hu, Z., Sun, Y., & Cong, J. (2024) · ACM/IEEE Workshop on Machine Learning for CAD (MLCAD) · doi:10.1109/mlcad62225.2024.10740204 · Cited by 8
A Catalog of Data Smells for Coding Tasks
Vitale, A., Oliveto, R., & Scalabrino, S. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3707457 · Cited by 9
“The first common approach is to filter repositories based on the number of stars on GitHub [64, 67, 69, 75, 95, 97, 105, 106, 108, 110, 118, 127, 128, 138, 139, 149, 151].”
InfluentialTowards Practical Few-shot Federated NLP
Cai, D., Wu, Y., Yuan, H., Wang, S., Lin, F. X., & Xu, M. (2023) · Workshop on Machine Learning and Systems (EuroMLSys) · doi:10.1145/3578356.3592575 · Cited by 12
On the Applicability of Language Models to Block-Based Programs
Griebl, E., Fein, B., Obermüller, F., Fraser, G., & Just, R. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00199 · Cited by 9
“without or with only little fine-tuning to assist with tasks in the domain of source code processing [25]–[28].”
BackgroundHorváth, D., Csuvik, V., Gyimóthy, T., & Vidács, L. (2023) · IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1109/apr59189.2023.00013 · Cited by 7
“Another study by Svyatkovskiy et al. used transformer models to address the problem of synthesizing missing code in programs and found that their approach was able to generate high-quality repair patches for a variety of code faults [12].”
BackgroundShellGPT: Generative Pre-trained Transformer Model for Shell Language Understanding
Shi, J., Jiang, S., Xu, B., Liang, J., Xiao, Y., & Wang, W. (2023) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre59848.2023.00082 · Cited by 10
“For commonly-used naming conventions, including snake case, camelCase, and PascalCase [31], we propose a rule-based word segmentation to split the tokens (e.g., AppGalleryMessageService → App/Gallery/Message/Service ).”
MethodDo Programmers Prefer Predictable Expressions in Code?
Casalnuovo, C., Lee, K., Wang, H., Dévanbu, P., & Morgan, E. (2020) · Cognitive Science · doi:10.1111/cogs.12921 · Cited by 9
“, 2020) and have been used recently in industrial code completion (Svyatkovskiy et al., 2020).”
MethodDesolda, G., Esposito, A., Greco, F., Tucci, C., Buono, P., & Piccinno, A. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103648 · Cited by 7
Yang, Y., Xu, H., & Yao, S. (2024) · IEEE International Conference on Frontiers Technology of Information and Computer (ICFTIC) · doi:10.1109/icftic64248.2024.10913069 · Cited by 6
“Pre-trained models for programming languages have advanced significantly, inspired by successes in natural language processing [27], [28].”
Multimodal Retrieval Augmented Generation Evaluation Benchmark
Sun, T., Somalwar, A., & Chan, H. (2024) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2024-spring62846.2024.10683437 · Cited by 7
“Recent progress in generative Artificial Intelligence (AI) has led to notable achievements in fields such as text creation and summarization [1], [2], language translation [3], [4], code generation [5], [6], and many others [7], [8].”
VEGA: Automatically Generating Compiler Backends using a Pre-trained Transformer Model
Zhong, M., Lv, F., Wang, L., Qiu, L., Wang, Y., Liu, Y., Cui, H., Feng, X., & Xue, J. (2025) · Proceedings of the CGO/Proceedings of the ... CGO · doi:10.1145/3696443.3708931 · Cited by 6
Cao, Y., Ju, X., Chen, X., & Gong, L. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00543-3 · Cited by 9
“Additionally, many datasets suffer from the absence of code comments (Svyatkovskiy et al. 2020; Wen et al. 2019), thereby reducing the potential value of this modality.”
Jointly Learning to Repair Code and Generate Commit Message
Bai, J., Zhou, L., Blanco, A., Liu, S., Wei, F., Zhou, M., & Li, Z. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.771 · Cited by 6
“Recently, researchers have exploited deep learning to programming-language related tasks, such as code completion (Svyatkovskiy et al., 2020), automated code repair (Tufano et al., 2018), commit messages generation (Xu et al., 2019), code search (Gu et al., 2018), and so on.”
Code Comment Analysis–A Review Paper
Hassan, S. Z., Irshad, A., Mir, J., Aslam, A., Ayaz, K., & Bawazir, M. A. (2022) · Journal of Management Practices Humanities and Social Sciences · doi:10.33152/jmphss-6.1.9 · Cited by 6
Are NLP Metrics Suitable for Evaluating Generated Code?
Takaichi, R., Higo, Y., Matsumoto, S., Kusumoto, S., Kurabayashi, T., Kirinuki, H., & Tanno, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21388-5_38 · Cited by 6
Specializing Neural Networks for Cryptographic Code Completion Applications
Xiao, Y., Song, W., Qi, J., Viswanath, B., McDaniel, P. D., & Yao, D. D. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3265362 · Cited by 6
“Towards this direction, many studies [7], [15], [16], [17], [18] proposed solutions that incorporate program syntax or semantic properties.”
BackgroundInfluentialT5QL: Taming language models for SQL generation
Arcadinho, S., Aparício, D., Veiga, H., & Alegria, A. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.23 · Cited by 7
On the Generalizability of Deep Learning-based Code Completion Across Programming Language Versions
Ciniselli, M., Martin-Lopez, A., & Bavota, G. (2024) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3643916.3644411 · Cited by 6
Explainable Natural Language to Bash Translation using Abstract Syntax Tree
Bharadwaj, S., & Shevade, S. (2021) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2021.conll-1.20 · Cited by 6
“Advances have been made in code generation (Svyatkovskiy et al., 2020), code summarization (Ahmad et al., 2020), bug detection (Gupta et al., 2019), code translation (Lachaux et al., 2020) etc.”
BackgroundUniversal Representation for Code
Liu, L., Nguyen, H. A., Karypis, G., & Sengamedu, S. H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-75768-7_2 · Cited by 6
CGEMs: A Metric Model for Automatic Code Generation using GPT-3Preprint
Narasimhan, A., Rao, K. P. A. V., & Veena, M. B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10168 · Cited by 6
AI-Powered Code Reviews: Leveraging Large Language Models
Chowdhury, M. T., Chowdhury, M. T., Neha, F. F., & Haque, A. (2024) · International Conference on Signal Processing and Advance Research in Computing (SPARC) · doi:10.1109/sparc61891.2024.10829223 · Cited by 10
AdaCoder: An Adaptive Planning and Multi-Agent Framework for Function-Level Code Generation
Zhu, Y.-H., Liu, C., He, X., Ren, X.-X., Liu, Z.-X., Pan, R.-W., & Zhang, H.-Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3642621 · Cited by 14
“[28], [29], [30], ranging from template-based approaches [30], [31], [32] to deep learning models capable of comprehending complex requirements [33], [34], [35].”
A Survey on Automated Code Evaluation Systems and Their Resources for Code Analysis
Rahman, M. M., Watanobe, Y., & Hamada, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36822-6_33 · Cited by 5
Deep Multiple Assertions Generation
Wang, H., Xu, T., & Wang, B. (2024) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1145/3650105.3652293 · Cited by 5
C2B: A Semantic Source Code Retrieval Model Using CodeT5 and Bi-LSTM
Bibi, N., Maqbool, A., Rana, T. A., Afzal, F., & Khan, A. A. (2024) · Applied Sciences · doi:10.3390/app14135795 · Cited by 4
Effective Hard Negative Mining for Contrastive Learning-Based Code Search
Ye, F., Li, C., Ge, J., Huang, L., & Luo, B. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695994 · Cited by 5
“A significant number of PTM specifically designed for source code (CodePTMs) [9, 38, 39, 40, 48, 75] have been successfully developed in recent years and they can be divided into two groups.”
Ansible Lightspeed: A Code Generation Service for IT Automation
Sahoo, P., Pujar, S., Nalawade, G., Genhardt, R., Mandel, L., & Buratti, L. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695277 · Cited by 9
“● Ansible Lightspeed has relatively higher acceptance for Ansible, compared to more general tools for various programming languages.”
A Lightweight Transformer with Convolutional Attention
Zeng, K., & Paik, I. (2020) · International Conference on Awareness Science and Technology (iCAST) · doi:10.1109/icast51195.2020.9319489 · Cited by 5
Source code auto-completion using various deep learning models under limited computing resources
Sharma, M., Mishra, T. K., & Kumar, A. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-022-00708-7 · Cited by 5
“One important difference between our approach and MultiGPT-C[21] is that we have trained our model with a small GPT2 of only four layers and four heads and trained for just five epochs on just 1 Google Colab GPU.”
BackgroundMethodInfluentialNMT-Based Code Generation for Coding Assistance with Natural Language
Akinobu, Y., Kajiura, T., Obara, M., & Kuramitsu, K. (2022) · Journal of Information Processing · doi:10.2197/ipsjjip.30.443 · Cited by 5
Optimized Tokenization Process for Open-Vocabulary Code Completion: An Empirical Study
Hussain, Y., Huang, Z., Zhou, Y., Khan, I. A., Khan, N., & Abbas, M. Z. (2023) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3593434.3594236 · Cited by 6
“[27] developed IntelliCode Compose, a multilingual code completion tool that can generate complete statements with a perplexity of 1.”
MethodAstBERT: Enabling Language Model for Financial Code Understanding with Abstract Syntax Trees
Liang, R., Zhang, T., Lu, Y., Liu, Y., Huang, Z., & Chen, X. (2022) · Workshop on Financial Technology and Natural Language Processing (FinNLP) · doi:10.18653/v1/2022.finnlp-1.2 · Cited by 5
Measurement of Embedding Choices on Cryptographic API Completion Tasks
Xiao, Y., Song, W., Ahmed, S., Ge, X., Viswanath, B., Meng, N., & Yao, D. D. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3625291 · Cited by 5
“First, a line of research develops pure data-driven solutions on general source code tokens without program analysis [3, 12, 14, 20, 26, 27, 46, 49].”
BackgroundEfficient Text-to-Code Retrieval with Cascaded Fast and Slow Transformer Models
Gotmare, A. D., Li, J., Joty, S., & Hoi, S. C. H. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616369 · Cited by 7
Training AI Model that Suggests Python Code from Student Requests in Natural Language
Kuramitsu, K., Obara, M., Sato, M., & Akinobu, Y. (2024) · Journal of Information Processing · doi:10.2197/ipsjjip.32.69 · Cited by 7
On Calibration of Pre-trained Code Models
Zhou, Z., Sha, C., & Peng, X. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639126 · Cited by 5
“Examples of decoder-only code models include CodeGPT [27] and GPT-C [39], which are primarily designed for generative tasks such as code completion and code generation.”
MethodGuiding ChatGPT for Better Code Generation: An Empirical Study
Liu, C., Bao, X., Zhang, H., Zhang, N., Hu, H., Zhang, X., & Yan, M. L. (2024) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner60148.2024.00018 · Cited by 10
“GPT-C [34] is a variant of GPT-2 pre-trained on a large unsupervised multilingual source code dataset.”
BackgroundBoosting Large Language Models for System Software Retargeting: A Preliminary Study
Zhong, M., Lv, F., Wang, L., Qiu, L., Geng, H., Cui, H., & Feng, X. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00037 · Cited by 5
“To mitigate the impact of specific coding styles on template synthesis, common code across all targets is clustered using the affinity propagation algorithm [6] and “Edit Distance” [26] as the evaluation metric.”
Evaluating and improving transformers pre-trained on ASTs for Code Completion
Ochs, M., Narasimhan, K., & Mezini, M. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00096 · Cited by 7
“Automatic code completion or prediction is the most widely used and integral developer assistance feature [8, 16, 11].”
BackgroundCompleting Function Documentation Comments Using Structural Information
Ciurumelea, A., Alexandru, C. V., Gall, H., & Proksch, S. (2023) · Empirical Software Engineering · doi:10.1007/s10664-022-10284-6 · Cited by 6
“…(Intellij idea - code completion 2021) and many researchers have developed improved approaches over the years (Bruch et al. 2009; Alon et al. 2018; Svyatkovskiy et al. 2020) for completing source code, there are currently no studies investigating the perceived acceptance and usefulness of source…”
BackgroundChen, N., Sun, Q., Zhu, R., Li, X., Lu, X., & Gao, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.295 · Cited by 6
System Support for Environmentally Sustainable Computing in Data Centers
Chen, F. (2024) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi61997.2024.00094 · Cited by 5
“Our hardware estimator employs a transformer architecture [42] to directly extract more informative static features from the source code, capturing interactions among all kernels.”
MethodNeural Program Generation Modulo Static AnalysisPreprint
Mukherjee, R., Wen, Y., Chaudhari, D., Reps, T., Chaudhuri, S., & Jermaine, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.01633 · Cited by 6
“The recent trend is to use transformers [36, 16, 14, 24].”
BackgroundMethodExperimental Evaluation of Parameter-Efficient Fine-Tuning for Software Engineering Tasks
Zou, W., Shen, Z., Li, Q., Ge, J., Li, C., Chen, X., Shen, X., Huang, L., & Luo, B. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3722107 · Cited by 16
“GPT-C [79] is pre-trained in programming languages to complete multilingual [3] code.”
Augmenting the Interpretability of GraphCodeBERT for Code Similarity Tasks
Martínez-Gil, J. (2025) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194025500160 · Cited by 4
Xavier: Toward Better Coding Assistance in Authoring Tabular Data Wrangling Scripts
Zhou, Y., Cai, X., Shi, Q., Huang, Y., Li, H., Qu, H., Weng, D., & Wu, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714239 · Cited by 6
“More recently, as Generative Pre-trained Transformers (GPT) gained popularity in natural language processing, numerous GPT-based models and tools have emerged [6, 47, 70].”
Gambo, O. O., Onifade, O. J., Olanrewaju-Smart, W., Abuya, O. J., Salawu, R. E., Massenon, R., & Gambo, I. (2026) · Discover Education · doi:10.1007/s44217-026-01142-4 · Cited by 10
“Tools such as IntelliCode [95], Copilot [16], AlphaDev [55], Codex [24], and DeepMind’s AlphaCode [48] demonstrate AI’s growing capacity to facilitate learning across multiple disciplines.”
Which Syntactic Capabilities Are Statistically Learned by Masked Language Models for Code?
Velasco, A., Palacio, D. N., Rodríguez-Cárdenas, D., & Poshyvanyk, D. (2024) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1145/3639476.3639768 · Cited by 8
“Current research has focused on exploiting deep learning representations using LSTMs [33], GPT [32], RoBERTa [20], and T5 [6, 9].”
BackgroundGAP-Gen: Guided Automatic Python Code Generation
Zhao, J., Song, Y., Wang, J., & Harris, I. G. (2023) · Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop (EACL SRW) · doi:10.18653/v1/2023.eacl-srw.4 · Cited by 6
Artificial Intelligence Empowered Language Models: A Review
Negi, A., Verma, C. V., & Tayyebi, Y. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-99-9524-0_40 · Cited by 4
Optimizing Datasets for Code Summarization: Is Code-Comment Coherence Enough?
Vitale, A., Mastropaolo, A., Oliveto, R., Penta, M. D., & Scalabrino, S. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00033 · Cited by 8
“This approach has been proven effective for several SE tasks, including code generation [16], [17], [18], [19], program repair [20], [21], [22], [23], and, indeed, code summarization [24], [25], [10], [12], [26], [11].”
A methodology for refined evaluation of neural code completion approaches
Le, K. T., Rashidi, G., & Andrzejak, A. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00866-9 · Cited by 4
“Recent works such as IntelliCode Compose (Svyatkovskiy et al. 2020), Feeding Trees to Transformers (Kim et al. 2021; Liu et al. 2020) use Transformer models (Vaswanietal.2017),whichoutperformRNNs.”
BackgroundMethodExploring and Improving Code Completion for Test Code
Zhu, T., Liu, Z., Xu, T., Tang, Z. J., Zhang, T., Pan, M., & Xia, X. (2024) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3643916.3644421 · Cited by 4
Retracted: An Enhanced Role of AI in SDLC in Implementing Maintenance System
Kumar, Y., Babu, C. R. K., Deorari, M. M., Alzubaidi, L. H. J., Sharma, K., & Vinothkumar, K. (2024) · International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE) · doi:10.1109/icacite60783.2024.10616905 · Cited by 4
RetractedMachine Learning-Based Code Auto-Completion Implementation for Firmware Developers
Kim, J., Lee, K., & Choi, S.-H. (2020) · Applied Sciences · doi:10.3390/app10238520 · Cited by 8
A platform to help in generating code for machine learning and data science projects
Rajiv, C. S., Parkavi, K., & Leninisha, S. (2023) · AIP conference proceedings · doi:10.1063/5.0172751 · Cited by 4
Zhou, X., Weyssow, M., Widyasari, R., Zhang, T., He, J., Lyu, Y., Chang, J., Zhang, B., Huang, D., & Lo, D. (2025) · arXiv · doi:10.48550/arxiv.2502.06215 · Cited by 3
Metadata
- Authors
- Svyatkovskiy, A., Deng, S. K., Fu, S., & Sundaresan, N.
- Affiliations
- Microsoft (United States) (US)
- Microsoft, USA
- ORCID
- Sundaresan, Neel
- Venue
- Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2020-11-08
- Pages
- 1433-1443
- OpenAlex
- W3025993830
- Cited by
- 474
- References
- 38
- Citations by year
- 2026: 42 · 2025: 97 · 2024: 117 · 2023: 103 · 2022: 52 · 2021: 56 · 2020: 7
- FWCI
- 41.33
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- code generation, code completion, transformer models, integrated development environment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
