The DOI matches its Crossref record, with a small difference in title or year; 84 of the 91 references it cites were found in a registry or an index, and 2 are not checked yet.
Big code != big vocabulary: open-vocabulary models for source code
Open accessAbstract
Statistical language modeling techniques have successfully been applied to large source code corpora, yielding a variety of new software development tools, such as tools for code suggestion, improving readability, and API migration. A major issue with these techniques is that code introduces new vocabulary at a far higher rate than natural language, as new identifier names proliferate. Both large vocabularies and out-of-vocabulary issues severely affect Neural Language Models (NLMs) of source code, degrading their performance and rendering them unable to scale.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2014) · Cited by 33,419
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Pointer Sentinel Mixture ModelsPreprint
Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1609.07843 · Cited by 5,058
Research works printed but not linked: 8
- Rohan Bavishi, Michael Pradel, and Koushik Sen. 2018. Context2Name: A Deep Learning-Based Approach to Infer Natural Variable Names from Usage Contexts. CoRR abs/1809.05193 (2018). arXiv:1809.05193 · arXiv
- Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A Neural Probabilistic Language Model. J. Mach. Learn. Res. 3 (March 2003), 1137--1155. http://dl.acm.org/citation.cfm?id=944919.944966
- James Bradbury, Stephen Merity, Caiming Xiong, and Richard Socher. 2017. Quasi-Recurrent Neural Networks. In Proceedings of ICLR 2017. https://openreview.net/forum?id=H1zJ-v5xl
- Sergey Dudoladov. 2013. Statistical NLP for computer program source code: An information theoretic perspective on programming language verbosity. Master's thesis. School of Informatics, University of Edinburgh, United Kingdom.
- Stefan Fiott. 2015. An Investigation of Statistical Language Modelling of Different Programming Language Types Using Large Corpora. Master's thesis. School of Informatics, University of Edinburgh, United Kingdom.
- Tomas Mikolov, Ilya Sutskever, Anoop Deoras, Le Hai Son, Stefan Kombrink, and Jan Cernock. 2012. Subword Language Modeling With Neural Networks. (08 2012).
- 10.5555/2627435.2670313 · DOI
- 10.18653/v1/P17-1184 · DOI
Cited by
Sorted by citations
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., & Hellendoorn, V. J. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534862 · Cited by 540
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536
An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation
Schäfer, M., Nadi, S., Eghbali, A., & Tip, F. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3334955 · Cited by 431
“Neural techniques are rapidly being adopted for solving various Software Engineering problems, with promising results in several domains including code completion [34], [35], [36], [37], [38], program repair [39], [40], [41], and bug-finding [42], [43].”
MethodFu, M. C., & Tantithamthavorn, C. K. (2022) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3524842.3528452 · Cited by 424
Program Synthesis with Large Language Models
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07732 · Cited by 388
“Code completion has been a particular focus of interest [Raychev et al., 2016, Karampatsis et al., 2020, Svyatkovskiy et al., 2020, Kim et al., 2020].”
BackgroundSelf-citationLog-based Anomaly Detection Without Log Parsing
Le, V.-H., & Zhang, H. (2021) · 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678773 · Cited by 361
“For example, space/stemming/camel case based tokenization strategies can lead to many OOV words and a big vocabulary [41].”
BackgroundVulRepair: a T5-based automated software vulnerability repair
Fu, M. C., Tantithamthavorn, C. K., Le, T., Nguyen, V., & Phung, D. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549098 · Cited by 278
Code Prediction by Feeding Trees to Transformers
Kim, S., Zhao, J., Tian, Y., & Chandra, S. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00026 · Cited by 209
Multi-task Learning based Pre-trained Language Model for Code Completion
Liu, F., Li, G., Zhao, Y., & Jin, Z. (2020) · Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3324884.3416591 · Cited by 188
PalmTree: Learning an Assembly Language Model for Instruction Embedding
Li, X., Qu, Y., & Yin, H. (2021) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3460120.3484587 · Cited by 211
Natural Attack for Pre-trained Models of Code
Yang, Z., Shi, J., He, J., & Lo, D. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510146 · Cited by 166
Neural program repair with execution-based backpropagation
Ye, H., Martínez, M., & Monperrus, M. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510222 · Cited by 161
Code smell detection by deep direct-learning and transfer-learning
Sharma, T., Efstathiou, V., Λουρίδας, Π., & Spinellis, D. D. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.110936 · Cited by 167
A review of sentiment analysis: tasks, applications, and deep learning techniques
Sharma, N. A., Ali, A. B. M. S., & Kabir, M. A. (2024) · International Journal of Data Science and Analytics · doi:10.1007/s41060-024-00594-x · Cited by 191
A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research
Watson, C., Cooper, N., Palacio, D. N., Moran, K., & Poshyvanyk, D. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3485275 · Cited by 137
Karampatsis, R.-M., Sutton, C., Informatics, S. O., Institute for Machine Learning, & Intelligence, D. S. A. A. (2020) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.5281/zenodo.3653444 · Cited by 117
Self-citationCode Structure–Guided Transformer for Source Code Summarization
Gao, S., Gao, C., He, Y., Zeng, J., Nie, L., Xia, X., & Lyu, M. R.-T. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3522674 · Cited by 142
TypeWriter: neural type prediction with search-based validation
Pradel, M., Gousios, G., Liu, J., & Chandra, S. (2019) · ESEC/SIGSOFT FSE · doi:10.1145/3368089.3409715 · Cited by 141
Finding A Needle in a Haystack: Automated Mining of Silent Vulnerability Fixes
Zhou, J., Pacheco, M., Wan, Z., Xia, X., Lo, D., Wang, Y., & Hassan, A. E. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678720 · Cited by 97
Fast and Memory-Efficient Neural Code Completion
Svyatkovskiy, A., Lee, S., Hadjitofi, A., Riechert, M., Franco, J. V., & Allamanis, M. (2021) · 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00045 · Cited by 95
An Empirical Study on the Usage of Transformer Models for Code Completion
Ciniselli, M., Cooper, N., Pascarella, L., Mastropaolo, A., Aghajani, E., Poshyvanyk, D., Penta, M. D., & Bavota, G. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3128234 · Cited by 105
(2023) · doi:10.1145/3510003.3510060 · Cited by 88
An Empirical Study on the Usage of BERT Models for Code Completion
Ciniselli, M., Cooper, N., Pascarella, L., Poshyvanyk, D., Penta, M. D., & Bavota, G. (2021) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00024 · Cited by 101
CodeFill: multi-token code completion by jointly learning from structure and naming sequences
Izadi, M., Gismondi, R., & Gousios, G. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510172 · Cited by 86
Recommending metamodel concepts during modeling activities with pre-trained language models
Weyssow, M., Sahraoui, H. A., & Syriani, E. (2022) · Software & Systems Modeling · doi:10.1007/s10270-022-00975-5 · Cited by 86
Semantic bug seeding: a learning-based approach for creating realistic bugs
Patra, J., & Pradel, M. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468623 · Cited by 66
Thongtanunam, P., Pornprasit, C., & Tantithamthavorn, C. K. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510067 · Cited by 75
Empirical study of transformers for source code
Chirkova, N., & Troshin, S. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468611 · Cited by 64
Automating just-in-time comment updating
Liu, Z., Xia, X., Yan, M., & Li, S. (2020) · Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3324884.3416581 · Cited by 56
Exploring Software Naturalness through Neural Language ModelsPreprint
Buratti, L., Pujar, S., Bornea, M. A., McCarley, S., Zheng, Y., Rossiello, G., Morari, A., Laredo, J., Thost, V., Zhuang, Y., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12641 · Cited by 95
A Survey on Machine Learning Techniques for Source Code AnalysisPreprint
Sharma, T., Kechagia, M., Georgiou, S., Tiwari, R. K., Vats, I., Moazen, H., & Sarro, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09610 · Cited by 56
Automated detection of password leakage from public GitHub repositories
Feng, R., Yan, Z., Peng, S., & Zhang, Y. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510150 · Cited by 51
Ding, Y., Ray, B., Dévanbu, P., & Hellendoorn, V. J. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416587 · Cited by 52
On Multi-Modal Learning of Editing Source Code
Chakraborty, S., & Ray, B. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678559 · Cited by 55
Learning to Construct Better Mutation Faults
Tian, Z., Chen, J., Zhu, Q., Yang, J., & Zhang, L. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3556949 · Cited by 48
A survey on machine learning techniques applied to source code
Sharma, T., Kechagia, M., Georgiou, S., Tiwari, R., Vats, I., Moazen, H., & Sarro, F. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111934 · Cited by 159
Cruz-Benito, J., Vishwakarma, S., Martín-Fernández, F., & Faro, I. (2021) · AI · doi:10.3390/ai2010001 · Cited by 50
SeCNN: A semantic CNN parser for code comment generation
Li, Z., Wu, Y., Peng, B., Chen, X., Sun, Z., Liu, Y., & Yu, D. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111036 · Cited by 45
CrystalBLEU: Precisely and Efficiently Measuring the Similarity of Code
Eghbali, A., & Pradel, M. (2022) · Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3551349.3556903 · Cited by 42
Gros, D., Sezhiyan, H., Dévanbu, P., & Zhou, Y. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416546 · Cited by 58
Making the most of small Software Engineering datasets with modern machine learning
Prenner, J. A., & Robbes, R. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3135465 · Cited by 47
Self-citationLearning type annotation: is big data enough?
Jesse, K., Dévanbu, P., & Ahmed, T. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3473135 · Cited by 46
TRACED: Execution-aware Pre-training for Source Code
Ding, Y., Steenhoek, B. J., Pei, K., Kaiser, G. E., Le, W., & Ray, B. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3608140 · Cited by 41
DeepMerge: Learning to Merge Programs
Dinella, E., Mytkowicz, T., Svyatkovskiy, A., Bird, C., Naik, M., & Lahiri, S. K. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3183955 · Cited by 36
Greening Large Language Models of Code
Shi, J., Yang, Z., Kang, H. J., Xu, B., He, J., & Lo, D. (2024) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1145/3639475.3640097 · Cited by 54
Learning to Recommend Method Names with Global Context
Liu, F., Li, G., Fu, Z., Lu, S., Hao, Y.-Y., & Jin, Z. (2022) · International Conference on Software Engineering · doi:10.1145/3510003.3510154 · Cited by 40
Roy, D., Zhang, Z., Ma, M., Arnaoudova, V., Panichella, A., Panichella, S., Gonzalez, D., & Mirakhorli, M. T. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416622 · Cited by 34
Program merge conflict resolution via neural transformers
Svyatkovskiy, A., Fakhoury, S., Ghorbani, N., Mytkowicz, T., Dinella, E., Bird, C., Jang, J., Sundaresan, N., & Lahiri, S. K. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549163 · Cited by 38
Pre-training by Predicting Program Dependencies for Vulnerability Analysis Tasks
Liu, Z., Tang, Z., Zhang, J., Xia, X., & Yang, X. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639142 · Cited by 31
Finding the Dwarf: Recovering Precise Types from WebAssembly Binaries
Lehmann, D., & Pradel, M. (2022) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/3519939.3523449 · Cited by 31
Pradel, M., & Chandra, S. (2021) · Communications of the ACM · doi:10.1145/3460348 · Cited by 28
Poison Attack and Poison Detection on Deep Source Code Processing Models
Li, J., Li, Z., Zhang, H., Li, G., Jin, Z. G., Hu, X., & Xia, X. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3630008 · Cited by 63
A unified multi-task learning model for AST-level and token-level code completion
Liu, F., Li, G., Wei, B., Xia, X., Fu, Z., & Jin, Z. G. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10140-7 · Cited by 28
Unleashing the power of compiler intermediate representation to enhance neural program embeddings
Li, Z., Ma, P., Wang, H., Wang, S., Tang, Q., Nie, S., & Wu, S. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510217 · Cited by 27
Passport: Improving Automated Formal Verification Using Identifiers
Sanchez-Stern, A., First, E., Zhou, T., Kaufman, Z., Brun, Y., & Ringer, T. (2023) · ACM Transactions on Programming Languages and Systems · doi:10.1145/3593374 · Cited by 27
CCRep: Learning Code Change Representations via Pre-Trained Code Model and Query Back
Liu, Z., Tang, Z., Xia, X., & Yang, X. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00014 · Cited by 34
Predictive Mutation Analysis via the Natural Language Channel in Source Code
Kim, J.-H., Jeon, J., Hong, S., & Yoo, S. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3510417 · Cited by 29
Ciniselli, M., Pascarella, L., & Bavota, G. (2022) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3524842.3528440 · Cited by 34
(2020) · doi:10.1145/3379597.3387491 · Cited by 24
Self-citationTowards automating code review at scale
Hellendoorn, V. J., Tsay, J., Mukherjee, M., & Hirzel, M. J. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3473134 · Cited by 30
Simpler Hyperparameter Optimization for Software Analytics: Why, How, When?
Agrawal, A., Yang, X., Agrawal, R., Yedida, R., Shen, X., & Menzies, T. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3073242 · Cited by 23
An Investigation into Misuse of Java Security APIs by Large Language Models
Mousavi, Z., Islam, C., Moore, K., Abuadbba, A., & Babar, M. A. (2024) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3634737.3661134 · Cited by 28
Non-Autoregressive Line-Level Code Completion
Liu, F., Fu, Z., Li, G., Jin, Z., Liu, H., Hao, Y., & Zhang, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3649594 · Cited by 22
APIRecX: Cross-Library API Recommendation via Pre-Trained Language Model
Kang, Y., Wang, Z., Zhang, H., Chen, J., & You, H. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.275 · Cited by 23
PRHAN: Automated Pull Request Description Generation Based on Hybrid Attention Network
Fang, S., Zhang, T., Tan, Y., Xu, Z., Yuan, Z.-X., & Meng, L. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111160 · Cited by 24
Let's Chat to Find the APIs: Connecting Human, LLM and Knowledge Graph through AI Chain
Huang, Q. Y., Wan, Z., Xing, Z., Wang, C., Chen, J., Xu, X., & Lu, Q. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00075 · Cited by 26
How Often Do Single-Statement Bugs Occur? The ManySStuBs4J DatasetPreprint
Karampatsis, R.-M., & Sutton, C. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13334 · Cited by 21
Self-citationThinking Like a Developer? Comparing the Attention of Humans with Neural Models of Code
Paltenghi, M., & Pradel, M. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678712 · Cited by 20
Automated identification of libraries from vulnerability data
Haryono, S. A., Kang, H. J., Sharma, A., Sharma, A., Santosa, A. E., Ang, M. Y., & Lo, D. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527893 · Cited by 23
Learning lenient parsing & typing via indirect supervision
Ahmed, T., Devanbu, P., & Hellendoorn, V. J. (2021) · Empirical Software Engineering · doi:10.1007/s10664-021-09942-y · Cited by 20
Deep Just-In-Time Defect Localization
Qiu, F., Gao, Z., Xia, X., Lo, D., Grundy, J., & Wang, X. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3135875 · Cited by 19
Deep learning for code generation: a survey
Zhang, H., Zhang, K., Li, Z., Li, J., Li, J., Li, Y., Zhao, Y., Zhu, Y. S., Liu, F., Li, G., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-3956-3 · Cited by 20
DeepClone: Modeling Clones to Generate Code Predictions
Hammad, M., Babur, Ö., Basit, H. A., & van den Brand, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64694-3_9 · Cited by 17
Can Identifier Splitting Improve Open-Vocabulary Language Model of Code?
Shi, J., Yang, Z., He, J., Xu, B., & Lo, D. (2022) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner53432.2022.00130 · Cited by 16
Reasoning and Planning with Large Language Models in Code Development
Ding, H., Fan, Z., Guehring, I., Gupta, G. P., Ha, W., Huan, J., Liu, L. Q., Omidvar-Tehrani, B., Wang, S., & Zhou, H. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671452 · Cited by 23
Code to Comment "Translation": Data, Metrics, Baselining & EvaluationPreprint
Gros, D., Sezhiyan, H., Dévanbu, P., & Yu, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01410 · Cited by 16
Hliš, T., Četina, L., Beranič, T., & Pavlič, L. (2023) · Applied Sciences · doi:10.3390/app132413061 · Cited by 20
GrammarT5: Grammar-Integrated Pretrained Encoder-Decoder Neural Model for Code
Zhu, Q., Liang, Q., Sun, Z., Xiong, Y., Zhang, L., & Cheng, S. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639125 · Cited by 21
Razaque, A., Bektemyssova, G., Yoo, J., Hariri, S., Khan, M. J., Nalgozhina, N., Hwang, J., & Khan, M. A. (2025) · Cluster Computing · doi:10.1007/s10586-024-05017-x · Cited by 14
Phan, H., Sharma, A., & Jannesari, A. (2021) · IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) · doi:10.1109/asew52652.2021.00050 · Cited by 14
Can We Automatically Fix Bugs by Learning Edit Operations?
Connor, A., Harris, A., Cooper, N., & Poshyvanyk, D. (2022) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner53432.2022.00096 · Cited by 15
An Exploratory Literature Study on Sharing and Energy Use of Language Models for Source Code
Hort, M., Grishina, A., & Moonen, L. (2023) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem56168.2023.10304803 · Cited by 15
Can Neural Clone Detection Generalize to Unseen Functionalitiesƒ
Liu, C., Lin, Z., Lou, J., Wen, L., & Zhang, D. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678907 · Cited by 13
The Transformers’ Ability to Implement for Solving Intricacies of Language Processing
Singh, P. N., & Behera, S. (2022) · Asian Conference on Innovation in Technology (ASIANCON) · doi:10.1109/asiancon55314.2022.9909423 · Cited by 15
Cross-Dataset Design Discussion Mining
Mahadi, A., Tongay, K., & Ernst, N. (2020) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner48275.2020.9054792 · Cited by 13
Review4Repair: Code review aided automatic program repairing
Huq, F., Hasan, M., Haque, M. M. A., Mahbub, S., Iqbal, A., & Ahmed, T. (2021) · Information and Software Technology · doi:10.1016/j.infsof.2021.106765 · Cited by 22
NLP in SMEs for industry 4.0: opportunities and challenges
Bourdin, M., Paviot, T., Pellerin, R., & Lamouri, S. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.06.186 · Cited by 16
PYInfer: Deep Learning Semantic Type Inference for Python VariablesPreprint
Cui, S., Zhao, G., Dai, Z., Wang, L., Huang, R., & Huang, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14316 · Cited by 26
Learning to Predict User-Defined Types
Jesse, K., Dévanbu, P., & Sawant, A. A. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3178945 · Cited by 15
CONCORD: Clone-Aware Contrastive Learning for Source Code
Ding, Y., Chakraborty, S., Buratti, L., Pujar, S., Morari, A., Kaiser, G. E., & Ray, B. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3598035 · Cited by 13
T5APR: Empowering automated program repair across languages through checkpoint ensemble
Gharibi, R., Sadreddini, M. H., & Fakhrahmad, S. M. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112083 · Cited by 15
NaturalCC: An Open-Source Toolkit for Code Intelligence
Wan, Y., He, Y., Bi, Z., Zhang, J., Sui, Y., Zhang, H.-Y., Hashimoto, K., Jin, H., Xu, G., Xiong, C.-M., et al. (2022) · 2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510454.3516863 · Cited by 18
Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion
Pei, H., Zhao, J., Lausen, L., Zha, S., & Karypis, G. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25653 · Cited by 17
ReACC: A Retrieval-Augmented Code Completion Framework
Lu, S., Duan, N., Han, H., Guo, D., Hwang, S.-W., & Svyatkovskiy, A. (2022) · arXiv · doi:10.48550/arxiv.2203.07722 · Cited by 96
“Primitive approaches take the given context only (Liu et al., 2016; Karampatsis et al., 2020), some methods use richer information, e.”
BackgroundMethodInfluentialToward Interpretable Graph Tensor Convolution Neural Network for Code Semantics Embedding
Yang, J., Fu, C., Deng, F., Wen, M., Guo, X., & Wan, C. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3582574 · Cited by 9
EGPT-SPE: story point effort estimation using improved GPT-2 by removing inefficient attention heads
Cheemaa, A. S., Azhar, M. H., Arif, F., Haq, Q. M. U., Sohail, M., & Iqbal, A. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06824-4 · Cited by 10
Tensor‐based gated graph neural network for automatic vulnerability detection in source code
Yang, J., Ruan, O., & Zhang, J. (2023) · Software Testing Verification and Reliability · doi:10.1002/stvr.1867 · Cited by 9
Identifying Incorrect Patches in Program Repair Based on Meaning of Source Code
Phung, Q.-N., Kim, M., & Lee, E. (2022) · IEEE Access · doi:10.1109/access.2022.3145983 · Cited by 9
Towards Full-line Code Completion with Neural Language ModelsPreprint
Wang, W., Shen, S., Li, G., & Jin, Z. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.08603 · Cited by 18
AsserT5: Test Assertion Generation Using a Fine-Tuned Code Language Model
Primbs, S., Fein, B., & Fraser, G. (2025) · doi:10.1109/ast66626.2025.00008 · Cited by 15
Metadata
- Authors
- Karampatsis, R.-M., Babii, H., Robbes, R., Sutton, C., & Janes, A.
- Affiliations
- University of Edinburgh (GB)
- Free University of Bozen-Bolzano (IT)
- Google (United States) (US)
- Venue
- Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2020-06-27
- Pages
- 1073-1085
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W3011564318
- Cited by
- 182
- References
- 91
- Citations by year
- 2026: 3 · 2025: 14 · 2024: 19 · 2023: 26 · 2022: 51 · 2021: 48 · 2020: 20
- FWCI
- 27.87
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Open Research (University of Surrey) (submittedVersion, cc-by)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- ADVERB
- IDEALS
- Engineering and Physical Sciences Research Council (,EP/L016427/1)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- neural language models, vocabulary size, out-of-vocabulary words, identifier names, code completion, API migration, code readability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
