Big code != big vocabulary: open-vocabulary models for source code
Open accessAbstract
Statistical language modeling techniques have successfully been applied to large source code corpora, yielding a variety of new software development tools, such as tools for code suggestion, improving readability, and API migration. A major issue with these techniques is that code introduces new vocabulary at a far higher rate than natural language, as new identifier names proliferate. Both large vocabularies and out-of-vocabulary issues severely affect Neural Language Models (NLMs) of source code, degrading their performance and rendering them unable to scale.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
(2014) · Cited by 33,419 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · see citing works
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · see citing works
Pointer Sentinel Mixture ModelsPreprint
Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1609.07843 · Cited by 5,051 · see citing works
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · see citing works
An empirical study of smoothing techniques for language modeling
Chen, S. F., & Goodman, J. (1999) · Computer Speech & Language · doi:10.1006/csla.1999.0128 · Cited by 2,100 · see citing works
Vinyals, O., Fortunato, M., & Jaitly, N. (2015) · neural information processing systems · Cited by 2,024 · see citing works
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D. A., & Rush, A. M. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10362 · Cited by 1,546 · see citing works
Defects4J: a database of existing faults to enable controlled testing studies for Java programs
Just, R., Jalali, D., & Ernst, M. D. (2014) · Proceedings of the 2014 International Symposium on Software Testing and Analysis · doi:10.1145/2610384.2628055 · Cited by 1,447 · see citing works
Exploring the Limits of Language ModelingPreprint
Józefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., & Wu, Y.-H. (2016) · arXiv.org · doi:10.48550/arxiv.1602.02410 · Cited by 1,202 · see citing works
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974 · see citing works
On Using Very Large Target Vocabulary for Neural Machine Translation
Jean, S., Cho, K., Memisevic, R., & Bengio, Y. (2015) · Annual Meeting of the Association for Computational Linguistics · doi:10.3115/v1/p15-1001 · Cited by 839 · see citing works
Better Word Representations with Recursive Neural Networks for Morphology
Luong, M.-T., Socher, R., & Manning, C. D. (2013) · Conference on Computational Natural Language Learning · Cited by 789 · see citing works
Summarizing Source Code using a Neural Attention Model
Iyer, S. V., Konstas, I., Cheung, A., & Zettlemoyer, L. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1195 · Cited by 716 · see citing works
Hu, X., Li, G., Xia, X., Lo, D., & Jin, Z. G. (2018) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3196321.3196334 · Cited by 705 · see citing works
A new algorithm for data compression
Gage, P. (1994) · C/C++ users journal · Cited by 655 · see citing works
Deep learning code fragments for code clone detection
White, M., Tufano, M., Vendome, C., & Poshyvanyk, D. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970326 · Cited by 613 · see citing works
SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair
Chen, Z.-M., Kommrusch, S., Tufano, M., Pouchet, L., Poshyvanyk, D., & Martin, M. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2940179 · Cited by 585 · see citing works
On the naturalness of software
Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Dévanbu, P. (2012) · doi:10.5555/2337223.2337322 · Cited by 574 · see citing works
Code completion with statistical language models
Raychev, V., Vechev, M., & Yahav, E. (2014) · Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2594291.2594321 · Cited by 555 · see citing works
Gu, X., Zhang, H., Zhang, D., & Kim, S. (2016) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2950290.2950334 · Cited by 485 · see citing works
code2seq: Generating Sequences from Structured Representations of CodePreprint
Alon, U., Brody, S., Levy, O., & Yahav, E. (2018) · arXiv (Cornell University) · doi:10.4230/oasics.slate.2021.14 · Cited by 467 · see citing works
A Convolutional Attention Network for Extreme Summarization of Source CodePreprint
Allamanis, M., Peng, H. M., & Sutton, C. A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.03001 · Cited by 436 · see citing works
Self-citationDeepFix: Fixing Common C Language Errors by Deep Learning
Gupta, R., Pal, S., Kanade, A., & Shevade, S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10742 · Cited by 426 · see citing works
On the naturalness of software
Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Devanbu, P. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227135 · Cited by 425 · see citing works
The Porter stemming algorithm: then and now
Willett, P. (2006) · Data Technologies and Applications · doi:10.1108/00330330610681295 · Cited by 409 · see citing works
Mining source code repositories at massive scale using language modeling
Allamanis, M., & Sutton, C. (2013) · IEEE Working Conference on Mining Software Repositories · doi:10.1109/msr.2013.6624029 · Cited by 393 · see citing works
Self-citationLearning from examples to improve code completion systems
Bruch, M., Monperrus, M., & Mezini, M. (2009) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1595696.1595728 · Cited by 383 · see citing works
Learning natural coding conventions
Allamanis, M., Barr, E. T., Bird, C., & Sutton, C. A. (2014) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2635868.2635883 · Cited by 380 · see citing works
Self-citationSuggesting accurate method and class names
Allamanis, M., Barr, E. T., Bird, C., & Sutton, C. A. (2015) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2786805.2786849 · Cited by 379 · see citing works
Self-citationGulwani, S., Polozov, O., & Singh, R. (2017) · Foundations and Trends® in Programming Languages · doi:10.1561/2500000010 · Cited by 357 · see citing works
DeepBugs: a learning approach to name-based bug detection
Pradel, M., & Sen, K. (2018) · Proceedings of the ACM on Programming Languages · doi:10.1145/3276517 · Cited by 352 · see citing works
Improving Neural Language Models with a Continuous CachePreprint
Grave, E., Joulin, A., & Usunier, N. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1612.04426 · Cited by 316 · see citing works
Sharp nearby, fuzzy far away: How neural language models use context
Khandelwal, U., He, H., Qi, P., & Jurafsky, D. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1027 · Cited by 303 · see citing works
Are deep neural networks the best choice for modeling source code?
Hellendoorn, V. J., & Devanbu, P. (2017) · Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering · doi:10.1145/3106237.3106290 · Cited by 292 · see citing works
The adverse effects of code duplication in machine learning models of code
Allamanis, M. (2019) · Proceedings of the 2019 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software · doi:10.1145/3359591.3359735 · Cited by 284 · see citing works
On the "naturalness" of buggy code
Ray, B., Hellendoorn, V. J., Godhane, S., Tu, Z., Bacchelli, A., & Dévanbu, P. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884848 · Cited by 260 · see citing works
Code Completion with Neural Attention and Pointer Networks
Li, J., Wang, Y., Lyu, M. R., & King, I. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/578 · Cited by 252 · see citing works
Tu, Z., Su, Z., & Devanbu, P. (2014) · Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2635868.2635875 · Cited by 238 · see citing works
A statistical semantic language model for source code
Nguyen, T. T., Nguyen, A., Nguyen, H., & Nguyen, T. (2013) · ESEC/FSE 2013 · doi:10.1145/2491411.2491458 · Cited by 231 · see citing works
Toward Deep Learning Software Repositories
White, M., Vendome, C., Linares‐Vásquez, M., & Poshyvanyk, D. (2015) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2015.38 · Cited by 222 · see citing works
A study of the uniqueness of source code
Gabel, M., & Su, Z. (2010) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1882291.1882315 · Cited by 220 · see citing works
Using Natural Language Processing to Automatically Detect Self-Admitted Technical Debt
Maldonado, E. D. S., Shihab, E., & Tsantalis, N. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2654244 · Cited by 219 · see citing works
Learning to mine aligned code and natural language pairs from stack overflow
Yin, P., Deng, B., Chen, E., Vasilescu, B., & Neubig, G. (2018) · Proceedings of the 15th International Conference on Mining Software Repositories · doi:10.1145/3196398.3196408 · Cited by 218 · cited 1× · see citing works
“Yin et al. [90] mine pairs of natural language and code from Stack Overflow to support tasks such as code synthesis from natural language.”
BackgroundOn Learning Meaningful Code Changes Via Neural Machine Translation
Tufano, M., Pantiuchina, J., Watson, C., Bavota, G., & Poshyvanyk, D. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00021 · Cited by 212 · see citing works
Modelling out-of-vocabulary words for robust speech recognition
Bazzi, I., & Glass, J. R. (2002) · Cited by 189 · see citing works
Toward deep learning software repositories
White, M., Vendome, C., & Poshyvanyk, D. (2015) · Cited by 177 · see citing works
Mining source code to automatically split identifiers for software analysis
Enslen, E., Hill, E., Pollock, L., & Vijay‐Shanker, K. (2009) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2009.5069482 · Cited by 171 · see citing works
Polozov, O., Gulwani, S., & Singh, R. K. (2017) · Cited by 167 · see citing works
Raghothaman, M., Wei, Y., & Hamadi, Y. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884808 · Cited by 165 · see citing works
FRAGE: Frequency-Agnostic Word RepresentationPreprint
Gong, C., He, D., Tan, X., Qin, T., Wang, L.-W., & Liu, T.-Y. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1809.06858 · Cited by 155 · see citing works
Bimodal Modelling of Source Code and Natural Language
Allamanis, M., Tarlow, D., Gordon, A. D., & Wei, Y. (2015) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 154 · see citing works
A Survey of Machine Learning for Big Code and Naturalness
Allamanis, M., Barr, E. T., Devanbu, P., & Sutton, C. (2019) · ACM Computing Surveys · doi:10.1145/3212695 · Cited by 150 · see citing works
Self-citationLexical statistical machine translation for language migration
Nguyen, A. T., Nguyen, T. T., & Nguyen, T. N. (2013) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2491411.2494584 · Cited by 136 · see citing works
Morph-based speech recognition and modeling of out-of-vocabulary words across languages
Creutz, M., Hirsimäki, T., Kurimo, M., Puurula, A., Pylkkönen, J., Siivola, V., Varjokallio, M., Arisoy, E., Saraçlar, M., & Stolcke, A. (2007) · ACM Transactions on Speech and Language Processing · doi:10.1145/1322391.1322394 · Cited by 127 · see citing works
NL2Type: Inferring JavaScript Function Types from Natural Language Information
Malik, R. S., Patra, J., & Pradel, M. (2019) · IEEE/ACM International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00045 · Cited by 123 · see citing works
Program synthesis using natural language
Desai, A., Gulwani, S., Hingorani, V., Jain, N., Karkare, A., Marron, M., Sailesh, R., & Roy, S. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884786 · Cited by 116 · see citing works
Phrase-Based Statistical Translation of Programming Languages
Karaivanov, S., Raychev, V., & Vechev, M. (2014) · ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!) · doi:10.1145/2661136.2661148 · Cited by 115 · see citing works
PHOG: probabilistic model for code
Bielik, P., Raychev, V., & Vechev, M. (2016) · International Conference on Machine Learning · Cited by 115 · see citing works
Binkley, D., Davis, M., Lawrie, D., & Morrell, C. (2009) · 2009 IEEE 17th International Conference on Program Comprehension · doi:10.1109/icpc.2009.5090039 · Cited by 115 · see citing works
Syntax and sensibility: Using language models to detect and correct syntax errors
Santos, E. A., Campbell, J. C., Patel, D., Hindle, A., & Amaral, J. N. (2018) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2018.8330219 · Cited by 85 · see citing works
Syntax errors just aren't natural: improving error reporting with language models
Campbell, J. C., Hindle, A., & Amaral, J. N. (2014) · Working Conference on Mining Software Repositories (MSR) · doi:10.1145/2597073.2597102 · Cited by 82 · see citing works
When Code Completion Fails: A Case Study on Real-World Completions
Hellendoorn, V. J., Proksch, S., Gall, H. C., & Bacchelli, A. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00101 · Cited by 76 · see citing works
A deep language model for software codePreprint
Dam, H. K., Tran, T., & Pham, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.02715 · Cited by 72 · see citing works
Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural NetworksPreprint
Bhatia, S., & Singh, R. K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06129 · Cited by 69 · see citing works
Recovering clear, natural identifiers from obfuscated JS names
Vasilescu, B., Casalnuovo, C., & Dévanbu, P. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106289 · Cited by 64 · see citing works
DeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning
Gu, X., Zhang, H., Zhang, D., & Kim, S. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/514 · Cited by 62 · see citing works
LINSEN: An efficient approach to split identifiers and expand abbreviations
Corazza, A., Martino, S. D., & Maggio, V. (2012) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2012.6405277 · Cited by 62 · see citing works
T2API: synthesizing API code usage templates from English texts with statistical translation
Nguyen, T. V., Rigby, P. C., Nguyen, A. T., Karanfil, M., & Nguyen, T. N. (2016) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2950290.2983931 · Cited by 53 · see citing works
An empirical study of identifier splitting techniques
Hill, E., Binkley, D., Lawrie, D., Pollock, L., & Vijay‐Shanker, K. (2013) · Empirical Software Engineering · doi:10.1007/s10664-013-9261-0 · Cited by 48 · see citing works
Rahman, M., Palani, D., & Rigby, P. C. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00022 · Cited by 47 · see citing works
Spiral: splitters for identifiers in source code files
Hucka, M. (2018) · The Journal of Open Source Software · doi:10.21105/joss.00653 · Cited by 43 · see citing works
Meaningful variable names for decompiled code
Jaffe, A., Lacomis, J., Schwartz, E. J., Goues, C. L., & Vasilescu, B. (2018) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3196321.3196330 · Cited by 41 · see citing works
Maybe Deep Neural Networks are the Best Choice for Modeling Source CodePreprint
Karampatsis, R.-M., & Sutton, C. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.05734 · Cited by 41 · see citing works
Self-citationCACHECA: A Cache Language Model Based Code Suggestion Tool
Franks, C., Tu, Z., Dévanbu, P., & Hellendoorn, V. J. (2015) · Cited by 35 · see citing works
CACHECA: A Cache Language Model Based Code Suggestion Tool
Franks, C., Tu, Z., Devanbu, P., & Hellendoorn, V. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.228 · Cited by 30 · see citing works
Leveraging Small Software Engineering Data Sets with Pre-Trained Neural Networks
Robbes, R., & Janes, A. (2019) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier.2019.00016 · Cited by 27 · see citing works
Self-citationModeling Vocabulary for Big Code Machine LearningPreprint
Babii, H., Janes, A., & Robbes, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01873 · Cited by 22 · see citing works
Self-citationSplitting source code identifiers using Bidirectional LSTM Recurrent Neural NetworkPreprint
Markovtsev, V., Long, W., Bulychev, E., Keramitas, R., Slavnov, K., & Markowski, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11651 · Cited by 4 · see citing works
Algorithm for Data Compression
Samir, S., Ramdane, M., Lalam, M., & Ahmed-Ouamer, R. (2007) · E-ti · Cited by 3 · see citing works
Flagged first, then by citations
5 of the 91 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- Rohan Bavishi, Michael Pradel, and Koushik Sen. 2018. Context2Name: A Deep Learning-Based Approach to Infer Natural Variable Names from Usage Contexts. CoRR abs/1809.05193 (2018). arXiv:1809.05193 · arXiv
- Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A Neural Probabilistic Language Model. J. Mach. Learn. Res. 3 (March 2003), 1137-1155. http://dl.acm.org/citation.cfm?id=944919.944966
- James Bradbury, Stephen Merity, Caiming Xiong, and Richard Socher. 2017. Quasi- Recurrent Neural Networks. In Proceedings of ICLR 2017. https://openreview.net/ forum?id=H1zJ-v5xl
- Sergey Dudoladov. 2013. Statistical NLP for computer program source code: An information theoretic perspective on programming language verbosity. Master's thesis. School of Informatics, University of Edinburgh, United Kingdom.
- Stefan Fiott. 2015. An Investigation of Statistical Language Modelling of Different Programming Language Types Using Large Corpora. Master's thesis. School of Informatics, University of Edinburgh, United Kingdom.
- Clara Vania and Adam Lopez. 2017. From Characters to Words to in Between: Do We Capture Morphology?. In Proceedings of ACL 2017. 2016-2027. https: //doi.org/10.18653/v1/P17-1184 · DOI
Cited by
Sorted by citations
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., & Hellendoorn, V. J. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534862 · Cited by 540 · see citing works
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536 · see citing works
An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation
Schäfer, M., Nadi, S., Eghbali, A., & Tip, F. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3334955 · Cited by 431 · see citing works
“Neural techniques are rapidly being adopted for solving various Software Engineering problems, with promising results in several domains including code completion [34], [35], [36], [37], [38], program repair [39], [40], [41], and bug-finding [42], [43].”
MethodFu, M. C., & Tantithamthavorn, C. K. (2022) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3524842.3528452 · Cited by 424 · see citing works
Program Synthesis with Large Language Models
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07732 · Cited by 388 · see citing works
“Code completion has been a particular focus of interest [Raychev et al., 2016, Karampatsis et al., 2020, Svyatkovskiy et al., 2020, Kim et al., 2020].”
BackgroundSelf-citationLog-based Anomaly Detection Without Log Parsing
Le, V.-H., & Zhang, H. (2021) · 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678773 · Cited by 289 · see citing works
“For example, space/stemming/camel case based tokenization strategies can lead to many OOV words and a big vocabulary [41].”
BackgroundVulRepair: a T5-based automated software vulnerability repair
Fu, M. C., Tantithamthavorn, C. K., Le, T., Nguyen, V., & Phung, D. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549098 · Cited by 224 · see citing works
Code Prediction by Feeding Trees to Transformers
Kim, S., Zhao, J., Tian, Y., & Chandra, S. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00026 · Cited by 209 · see citing works
Multi-task Learning based Pre-trained Language Model for Code Completion
Liu, F., Li, G., Zhao, Y., & Jin, Z. (2020) · Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3324884.3416591 · Cited by 188 · see citing works
PalmTree: Learning an Assembly Language Model for Instruction Embedding
Li, X., Qu, Y., & Yin, H. (2021) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3460120.3484587 · Cited by 191 · see citing works
Natural Attack for Pre-trained Models of Code
Yang, Z., Shi, J., He, J., & Lo, D. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510146 · Cited by 166 · see citing works
Neural program repair with execution-based backpropagation
Ye, H., Martínez, M., & Monperrus, M. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510222 · Cited by 161 · see citing works
Code smell detection by deep direct-learning and transfer-learning
Sharma, T., Efstathiou, V., Λουρίδας, Π., & Spinellis, D. D. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.110936 · Cited by 149 · see citing works
A review of sentiment analysis: tasks, applications, and deep learning techniques
Sharma, N. A., Ali, A. B. M. S., & Kabir, M. A. (2024) · International Journal of Data Science and Analytics · doi:10.1007/s41060-024-00594-x · Cited by 145 · see citing works
A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research
Watson, C., Cooper, N., Palacio, D. N., Moran, K., & Poshyvanyk, D. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3485275 · Cited by 137 · see citing works
Karampatsis, R.-M., Sutton, C., Informatics, S. O., Institute for Machine Learning, & Intelligence, D. S. A. A. (2020) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.5281/zenodo.3653444 · Cited by 117 · see citing works
Self-citationCode Structure–Guided Transformer for Source Code Summarization
Gao, S., Gao, C., He, Y., Zeng, J., Nie, L., Xia, X., & Lyu, M. R.-T. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3522674 · Cited by 113 · see citing works
TypeWriter: neural type prediction with search-based validation
Pradel, M., Gousios, G., Liu, J., & Chandra, S. (2019) · ESEC/SIGSOFT FSE · doi:10.1145/3368089.3409715 · Cited by 141 · see citing works
Finding A Needle in a Haystack: Automated Mining of Silent Vulnerability Fixes
Zhou, J., Pacheco, M., Wan, Z., Xia, X., Lo, D., Wang, Y., & Hassan, A. E. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678720 · Cited by 97 · see citing works
Fast and Memory-Efficient Neural Code Completion
Svyatkovskiy, A., Lee, S., Hadjitofi, A., Riechert, M., Franco, J. V., & Allamanis, M. (2021) · 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00045 · Cited by 95 · see citing works
An Empirical Study on the Usage of Transformer Models for Code Completion
Ciniselli, M., Cooper, N., Pascarella, L., Mastropaolo, A., Aghajani, E., Poshyvanyk, D., Penta, M. D., & Bavota, G. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3128234 · Cited by 101 · see citing works
(2023) · doi:10.1145/3510003.3510060 · Cited by 88 · see citing works
An Empirical Study on the Usage of BERT Models for Code Completion
Ciniselli, M., Cooper, N., Pascarella, L., Poshyvanyk, D., Penta, M. D., & Bavota, G. (2021) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00024 · Cited by 93 · see citing works
CodeFill: multi-token code completion by jointly learning from structure and naming sequences
Izadi, M., Gismondi, R., & Gousios, G. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510172 · Cited by 86 · see citing works
Recommending metamodel concepts during modeling activities with pre-trained language models
Weyssow, M., Sahraoui, H. A., & Syriani, E. (2022) · Software & Systems Modeling · doi:10.1007/s10270-022-00975-5 · Cited by 83 · see citing works
Semantic bug seeding: a learning-based approach for creating realistic bugs
Patra, J., & Pradel, M. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468623 · Cited by 66 · see citing works
Thongtanunam, P., Pornprasit, C., & Tantithamthavorn, C. K. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510067 · Cited by 59 · see citing works
Empirical study of transformers for source code
Chirkova, N., & Troshin, S. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468611 · Cited by 60 · see citing works
Automating just-in-time comment updating
Liu, Z., Xia, X., Yan, M., & Li, S. (2020) · Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3324884.3416581 · Cited by 56 · see citing works
Exploring Software Naturalness through Neural Language ModelsPreprint
Buratti, L., Pujar, S., Bornea, M. A., McCarley, S., Zheng, Y., Rossiello, G., Morari, A., Laredo, J., Thost, V., Zhuang, Y., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12641 · Cited by 95 · see citing works
A Survey on Machine Learning Techniques for Source Code AnalysisPreprint
Sharma, T., Kechagia, M., Georgiou, S., Tiwari, R. K., Vats, I., Moazen, H., & Sarro, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09610 · Cited by 56 · see citing works
Automated detection of password leakage from public GitHub repositories
Feng, R., Yan, Z., Peng, S., & Zhang, Y. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510150 · Cited by 51 · see citing works
Ding, Y., Ray, B., Dévanbu, P., & Hellendoorn, V. J. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416587 · Cited by 50 · see citing works
On Multi-Modal Learning of Editing Source Code
Chakraborty, S., & Ray, B. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678559 · Cited by 50 · see citing works
Learning to Construct Better Mutation Faults
Tian, Z., Chen, J., Zhu, Q., Yang, J., & Zhang, L. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3556949 · Cited by 43 · see citing works
A survey on machine learning techniques applied to source code
Sharma, T., Kechagia, M., Georgiou, S., Tiwari, R., Vats, I., Moazen, H., & Sarro, F. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111934 · Cited by 130 · see citing works
Cruz-Benito, J., Vishwakarma, S., Martín-Fernández, F., & Faro, I. (2021) · AI · doi:10.3390/ai2010001 · Cited by 45 · see citing works
SeCNN: A semantic CNN parser for code comment generation
Li, Z., Wu, Y., Peng, B., Chen, X., Sun, Z., Liu, Y., & Yu, D. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111036 · Cited by 45 · see citing works
CrystalBLEU: Precisely and Efficiently Measuring the Similarity of Code
Eghbali, A., & Pradel, M. (2022) · Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3551349.3556903 · Cited by 42 · see citing works
Gros, D., Sezhiyan, H., Dévanbu, P., & Zhou, Y. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416546 · Cited by 42 · see citing works
Making the most of small Software Engineering datasets with modern machine learning
Prenner, J. A., & Robbes, R. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3135465 · Cited by 40 · see citing works
Self-citationLearning type annotation: is big data enough?
Jesse, K., Dévanbu, P., & Ahmed, T. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3473135 · Cited by 41 · see citing works
TRACED: Execution-aware Pre-training for Source Code
Ding, Y., Steenhoek, B. J., Pei, K., Kaiser, G. E., Le, W., & Ray, B. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3608140 · Cited by 41 · see citing works
DeepMerge: Learning to Merge Programs
Dinella, E., Mytkowicz, T., Svyatkovskiy, A., Bird, C., Naik, M., & Lahiri, S. K. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3183955 · Cited by 36 · see citing works
Greening Large Language Models of Code
Shi, J., Yang, Z., Kang, H. J., Xu, B., He, J., & Lo, D. (2024) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1145/3639475.3640097 · Cited by 35 · see citing works
Learning to recommend method names with global context
Liu, F., Li, G., Fu, Z., Lu, S., Hao, Y., & Jin, Z. G. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510154 · Cited by 35 · see citing works
Roy, D., Zhang, Z., Ma, M., Arnaoudova, V., Panichella, A., Panichella, S., Gonzalez, D., & Mirakhorli, M. T. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416622 · Cited by 34 · see citing works
Program merge conflict resolution via neural transformers
Svyatkovskiy, A., Fakhoury, S., Ghorbani, N., Mytkowicz, T., Dinella, E., Bird, C., Jang, J., Sundaresan, N., & Lahiri, S. K. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549163 · Cited by 38 · see citing works
Pre-training by Predicting Program Dependencies for Vulnerability Analysis Tasks
Liu, Z., Tang, Z., Zhang, J., Xia, X., & Yang, X. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639142 · Cited by 31 · see citing works
Finding the Dwarf: Recovering Precise Types from WebAssembly Binaries
Lehmann, D., & Pradel, M. (2022) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/3519939.3523449 · Cited by 31 · see citing works
Pradel, M., & Chandra, S. (2021) · Communications of the ACM · doi:10.1145/3460348 · Cited by 28 · see citing works
Poison Attack and Poison Detection on Deep Source Code Processing Models
Li, J., Li, Z., Zhang, H., Li, G., Jin, Z. G., Hu, X., & Xia, X. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3630008 · Cited by 43 · see citing works
A unified multi-task learning model for AST-level and token-level code completion
Liu, F., Li, G., Wei, B., Xia, X., Fu, Z., & Jin, Z. G. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10140-7 · Cited by 27 · see citing works
Unleashing the power of compiler intermediate representation to enhance neural program embeddings
Li, Z., Ma, P., Wang, H., Wang, S., Tang, Q., Nie, S., & Wu, S. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510217 · Cited by 27 · see citing works
Passport: Improving Automated Formal Verification Using Identifiers
Sanchez-Stern, A., First, E., Zhou, T., Kaufman, Z., Brun, Y., & Ringer, T. (2023) · ACM Transactions on Programming Languages and Systems · doi:10.1145/3593374 · Cited by 27 · see citing works
CCRep: Learning Code Change Representations via Pre-Trained Code Model and Query Back
Liu, Z., Tang, Z., Xia, X., & Yang, X. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00014 · Cited by 28 · see citing works
Predictive Mutation Analysis via the Natural Language Channel in Source Code
Kim, J.-H., Jeon, J., Hong, S., & Yoo, S. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3510417 · Cited by 26 · see citing works
Ciniselli, M., Pascarella, L., & Bavota, G. (2022) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3524842.3528440 · Cited by 26 · see citing works
(2020) · doi:10.1145/3379597.3387491 · Cited by 24 · see citing works
Self-citationTowards automating code review at scale
Hellendoorn, V. J., Tsay, J., Mukherjee, M., & Hirzel, M. J. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3473134 · Cited by 24 · see citing works
Simpler Hyperparameter Optimization for Software Analytics: Why, How, When?
Agrawal, A., Yang, X., Agrawal, R., Yedida, R., Shen, X., & Menzies, T. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3073242 · Cited by 23 · see citing works
An Investigation into Misuse of Java Security APIs by Large Language Models
Mousavi, Z., Islam, C., Moore, K., Abuadbba, A., & Babar, M. A. (2024) · ACM Asia Conference on Computer and Communications Security · doi:10.1145/3634737.3661134 · Cited by 34 · see citing works
Non-Autoregressive Line-Level Code Completion
Liu, F., Fu, Z., Li, G., Jin, Z., Liu, H., Hao, Y., & Zhang, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3649594 · Cited by 22 · see citing works
APIRecX: Cross-Library API Recommendation via Pre-Trained Language Model
Kang, Y., Wang, Z., Zhang, H., Chen, J., & You, H. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.275 · Cited by 22 · see citing works
PRHAN: Automated Pull Request Description Generation Based on Hybrid Attention Network
Fang, S., Zhang, T., Tan, Y., Xu, Z., Yuan, Z.-X., & Meng, L. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111160 · Cited by 22 · see citing works
Let's Chat to Find the APIs: Connecting Human, LLM and Knowledge Graph through AI Chain
Huang, Q. Y., Wan, Z., Xing, Z., Wang, C., Chen, J., Xu, X., & Lu, Q. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00075 · Cited by 24 · see citing works
How Often Do Single-Statement Bugs Occur? The ManySStuBs4J DatasetPreprint
Karampatsis, R.-M., & Sutton, C. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13334 · Cited by 21 · see citing works
Self-citationThinking Like a Developer? Comparing the Attention of Humans with Neural Models of Code
Paltenghi, M., & Pradel, M. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678712 · Cited by 20 · see citing works
Automated identification of libraries from vulnerability data
Haryono, S. A., Kang, H. J., Sharma, A., Sharma, A., Santosa, A. E., Ang, M. Y., & Lo, D. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527893 · Cited by 21 · see citing works
Learning lenient parsing & typing via indirect supervision
Ahmed, T., Devanbu, P., & Hellendoorn, V. J. (2021) · Empirical Software Engineering · doi:10.1007/s10664-021-09942-y · Cited by 18 · see citing works
Deep Just-In-Time Defect Localization
Qiu, F., Gao, Z., Xia, X., Lo, D., Grundy, J., & Wang, X. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3135875 · Cited by 17 · see citing works
Deep learning for code generation: a survey
Zhang, H., Zhang, K., Li, Z., Li, J., Li, J., Li, Y., Zhao, Y., Zhu, Y. S., Liu, F., Li, G., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-3956-3 · Cited by 17 · see citing works
DeepClone: Modeling Clones to Generate Code Predictions
Hammad, M., Babur, Ö., Basit, H. A., & van den Brand, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64694-3_9 · Cited by 17 · see citing works
Can Identifier Splitting Improve Open-Vocabulary Language Model of Code?
Shi, J., Yang, Z., He, J., Xu, B., & Lo, D. (2022) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner53432.2022.00130 · Cited by 16 · see citing works
Reasoning and Planning with Large Language Models in Code Development
Ding, H., Fan, Z., Guehring, I., Gupta, G. P., Ha, W., Huan, J., Liu, L. Q., Omidvar-Tehrani, B., Wang, S., & Zhou, H. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671452 · Cited by 17 · see citing works
Code to Comment "Translation": Data, Metrics, Baselining & EvaluationPreprint
Gros, D., Sezhiyan, H., Dévanbu, P., & Yu, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01410 · Cited by 16 · see citing works
Hliš, T., Četina, L., Beranič, T., & Pavlič, L. (2023) · Applied Sciences · doi:10.3390/app132413061 · Cited by 15 · see citing works
GrammarT5: Grammar-Integrated Pretrained Encoder-Decoder Neural Model for Code
Zhu, Q., Liang, Q., Sun, Z., Xiong, Y., Zhang, L., & Cheng, S. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639125 · Cited by 15 · see citing works
Razaque, A., Bektemyssova, G., Yoo, J., Hariri, S., Khan, M. J., Nalgozhina, N., Hwang, J., & Khan, M. A. (2025) · Cluster Computing · doi:10.1007/s10586-024-05017-x · Cited by 14 · see citing works
Phan, H., Sharma, A., & Jannesari, A. (2021) · IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) · doi:10.1109/asew52652.2021.00050 · Cited by 14 · see citing works
Can We Automatically Fix Bugs by Learning Edit Operations?
Connor, A., Harris, A., Cooper, N., & Poshyvanyk, D. (2022) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner53432.2022.00096 · Cited by 13 · see citing works
An Exploratory Literature Study on Sharing and Energy Use of Language Models for Source Code
Hort, M., Grishina, A., & Moonen, L. (2023) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem56168.2023.10304803 · Cited by 14 · see citing works
Can Neural Clone Detection Generalize to Unseen Functionalitiesƒ
Liu, C., Lin, Z., Lou, J., Wen, L., & Zhang, D. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678907 · Cited by 13 · see citing works
The Transformers’ Ability to Implement for Solving Intricacies of Language Processing
Singh, P. N., & Behera, S. (2022) · Asian Conference on Innovation in Technology (ASIANCON) · doi:10.1109/asiancon55314.2022.9909423 · Cited by 13 · see citing works
Cross-Dataset Design Discussion Mining
Mahadi, A., Tongay, K., & Ernst, N. (2020) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner48275.2020.9054792 · Cited by 13 · see citing works
Review4Repair: Code review aided automatic program repairing
Huq, F., Hasan, M., Haque, M. M. A., Mahbub, S., Iqbal, A., & Ahmed, T. (2021) · Information and Software Technology · doi:10.1016/j.infsof.2021.106765 · Cited by 13 · see citing works
NLP in SMEs for industry 4.0: opportunities and challenges
Bourdin, M., Paviot, T., Pellerin, R., & Samir Lamouri, A. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.06.186 · Cited by 14 · see citing works
PYInfer: Deep Learning Semantic Type Inference for Python VariablesPreprint
Cui, S., Zhao, G., Dai, Z., Wang, L., Huang, R., & Huang, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14316 · Cited by 20 · see citing works
Learning to Predict User-Defined Types
Jesse, K., Dévanbu, P., & Sawant, A. A. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3178945 · Cited by 11 · see citing works
CONCORD: Clone-Aware Contrastive Learning for Source Code
Ding, Y., Chakraborty, S., Buratti, L., Pujar, S., Morari, A., Kaiser, G. E., & Ray, B. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3598035 · Cited by 11 · see citing works
T5APR: Empowering automated program repair across languages through checkpoint ensemble
Gharibi, R., Sadreddini, M. H., & Fakhrahmad, S. M. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112083 · Cited by 13 · see citing works
NaturalCC: An Open-Source Toolkit for Code Intelligence
Wan, Y., He, Y., Bi, Z., Zhang, J., Sui, Y., Zhang, H.-Y., Hashimoto, K., Jin, H., Xu, G., Xiong, C.-M., et al. (2022) · 2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510454.3516863 · Cited by 18 · see citing works
Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion
Pei, H., Zhao, J., Lausen, L., Zha, S., & Karypis, G. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25653 · Cited by 10 · see citing works
ReACC: A Retrieval-Augmented Code Completion Framework
Lu, S., Duan, N., Han, H., Guo, D., Hwang, S.-W., & Svyatkovskiy, A. (2022) · arXiv · doi:10.48550/arxiv.2203.07722 · Cited by 96 · see citing works
“Primitive approaches take the given context only (Liu et al., 2016; Karampatsis et al., 2020), some methods use richer information, e.”
BackgroundMethodInfluentialToward Interpretable Graph Tensor Convolution Neural Network for Code Semantics Embedding
Yang, J., Fu, C., Deng, F., Wen, M., Guo, X., & Wan, C. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3582574 · Cited by 9 · see citing works
EGPT-SPE: story point effort estimation using improved GPT-2 by removing inefficient attention heads
Cheemaa, A. S., Azhar, M. H., Arif, F., Haq, Q. M. U., Sohail, M., & Iqbal, A. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06824-4 · Cited by 10 · see citing works
Tensor‐based gated graph neural network for automatic vulnerability detection in source code
Yang, J., Ruan, O., & Zhang, J. (2023) · Software Testing Verification and Reliability · doi:10.1002/stvr.1867 · Cited by 9 · see citing works
Identifying Incorrect Patches in Program Repair Based on Meaning of Source Code
Phung, Q.-N., Kim, M., & Lee, E. (2022) · IEEE Access · doi:10.1109/access.2022.3145983 · Cited by 9 · see citing works
Towards Full-line Code Completion with Neural Language ModelsPreprint
Wang, W., Shen, S., Li, G., & Jin, Z. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.08603 · Cited by 9 · see citing works
AsserT5: Test Assertion Generation Using a Fine-Tuned Code Language Model
Primbs, S., Fein, B., & Fraser, G. (2025) · doi:10.1109/ast66626.2025.00008 · Cited by 8 · see citing works
- Authors
- Karampatsis, R.-M., Babii, H., Robbes, R., Sutton, C., & Janes, A.
- Affiliations
- University of Edinburgh (GB)
- Free University of Bozen-Bolzano (IT)
- Google (United States) (US)
- Venue
- Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2020-06-27
- Pages
- 1073-1085
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W3011564318
- Cited by
- 182
- References
- 91
- Citations by year
- 2026: 3 · 2025: 14 · 2024: 19 · 2023: 26 · 2022: 51 · 2021: 48 · 2020: 20 · 2019: 1
- FWCI
- 27.87
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Open Research (University of Surrey) (submittedVersion, cc-by)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- ADVERB
- IDEALS
- Engineering and Physical Sciences Research Council (,EP/L016427/1)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- neural language models, vocabulary size, out-of-vocabulary words, identifier names, code completion, API migration, code readability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
