The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
IntelliCode compose: code generation using transformer
Open accessAbstract
In software development through integrated development environments (IDEs), code completion is one of the most widely used features. Nevertheless, majority of integrated development environments only support completion of methods and APIs, or arguments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Several methods have been proposed to incorporate repository context into the training of language models, including n-grams (Tu et al., 2014), LSTMs (Hellendoorn and Devanbu, 2017), and Transformers (Svyatkovskiy et al., 2020;Liu et al., 2022;Ding et al., 2022).”
From RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation (2023) · cites this paper as Svyatkovskiy et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The IntelliCode Compose system uses the term CTR (Click Through Rate) for this and reports a value of 10% in online trials [11].”
From Productivity Assessment of Neural Code Completion (2022) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“act syntax tree (AST) or concrete syntax tree (CST), and dataflow graph. A transformer block will typically consist of a multi-head self-attention, followed by a two-layer multi-layer perceptron (MLP) [17], optionally containing residual connections and layer normalization [23]. Recent neural architecture searches of t…”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 2×
“Several transformer models such as GPT-2, BERT, XLNet, and RoBERTa [19–22] have demonstrated the ability to learn effectively from unlabeled data to perform a wide variety of downstream tasks given supervised discriminative fine-tuning on each specific task.”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“Several transformer models such as GPT-2, BERT, XLNet, and RoBERTa [19–22] have demonstrated the ability to learn effectively from unlabeled data to perform a wide variety of downstream tasks given supervised discriminative fine-tuning on each specific task.”
BackgroundRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 2×
“We refer the interested reader to the numerous publications about transformer models [16, 20–22], and focus on code completion for the remainder of this section.”
BackgroundDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“lue and are most likely lower than the actual user sentiment. Over 150K requests, we have seen a surfacing rate of 9.2% and an acceptance rate of 10%. 11 Knowledge Distillation Knowledge distillation [38] is the model compression technique in which a smaller model – the student network – is trained to reproduce the re…”
InfluentialExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“(3) Language specific control codes, is an approach introduced in [29, 30] that uses target prefixes to facilitate constrained language modeling or multitask learning.”
Background(2004) · Cited by 9,718
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · cited 3×
“Transformers [18–21] are a family of neural networks designed to process ordered sequential data.”
BackgroundDistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653 · cited 1×
“n [38] is the model compression technique in which a smaller model – the student network – is trained to reproduce the results of a larger model – the teacher network. It has been shown in literature [39, 40], that it is possible to reach comparable performance on various tasks using distilled neural networks, resulti…”
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
Research works printed but not linked: 7
- Alexey Svyatkovskiy , Sebastian Lee , Anna Hadjitofi , Maik Riechert , Juliana Franco , and Miltiadis Allamanis . Fast and memory-eficient neural code completion , 2020 . Alexey Svyatkovskiy, Sebastian Lee, Anna Hadjitofi, Maik Riechert, Juliana Franco, and Miltiadis Allamanis. Fast and memory-eficient neural code completion, 2020.
- Alec Radford , Jef Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . Language models are unsupervised multitask learners . 2019 . Alec Radford, Jef Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
- Microsoft Corporation . Ai-assisted development. https://marketplace. visualstudio.com/items?itemName=VisualStudioExptTeam.VSIntelliCode. Visited Jan 2020 . Microsoft Corporation. Ai-assisted development. https://marketplace. visualstudio.com/items?itemName=VisualStudioExptTeam.VSIntelliCode. Visited Jan 2020.
- Alec Radford . Improving language understanding by generative pre-training . 2018 . Alec Radford. Improving language understanding by generative pre-training. 2018.
- Jimmy Lei Ba , Jamie Ryan Kiros, and Geofrey E. Hinton. Layer normalization , 2016 . Jimmy Lei Ba, Jamie Ryan Kiros, and Geofrey E. Hinton. Layer normalization, 2016.
- Eclipse Foundation . Code Recommenders . www.eclipse.org/recommenders. Visited June 2017 . Eclipse Foundation. Code Recommenders. www.eclipse.org/recommenders. Visited June 2017.
- Eclipse Recommenders . Eclipse SnipMatch . http://www.eclipse.org/ recommenders/manual/#snipmatch , 2014 . Visited Jun 2017. Eclipse Recommenders. Eclipse SnipMatch. http://www.eclipse.org/ recommenders/manual/#snipmatch, 2014. Visited Jun 2017.
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · arXiv · doi:10.48550/arxiv.2109.00859 · Cited by 1,495
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495
“CuBERT employs BERT’s powerful masked language modeling objective to derive generic code-specific representation, and CodeBERT further adds a replaced token detection (Clark et al., 2020) task to learn NL-PL cross-modal representation.”
BackgroundMethodInfluentialCompetition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
“Lanza, 2008) and large amounts of existing code data (Aye et al., 2021; Svyatkovskiy et al., 2020).”
BackgroundUniXcoder: Unified Cross-Modal Pre-training for Code Representation
Guo, D., Lu, S., Duan, N., Wang, Y., Zhou, M., & Yin, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.499 · Cited by 673
Metadata
- Authors
- Svyatkovskiy, A., Deng, S. K., Fu, S., & Sundaresan, N.
- Affiliations
- Microsoft (United States) (US)
- Microsoft, USA
- ORCID
- Sundaresan, Neel
- Venue
- Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2020-11-08
- Pages
- 1433-1443
- OpenAlex
- W3025993830
- Cited by
- 474
- References
- 38
- Citations by year
- 2026: 42 · 2025: 97 · 2024: 117 · 2023: 103 · 2022: 52 · 2021: 56 · 2020: 7
- FWCI
- 41.33
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- code generation, code completion, transformer models, integrated development environment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
