The DOI matches its Crossref record; 188 of the 188 references it cites were found in a registry or an index.
A Survey of Machine Learning for Big Code and Naturalness
Open accessAbstract
Research at the intersection of machine learning, programming languages, and software engineering has recently taken important steps in proposing learnable probabilistic models of source code that exploit the abundance of patterns of code. In this article, we survey this work. We contrast programming languages against natural languages and discuss how these similarities and differences drive the design of probabilistic models. We present a taxonomy based on the underlying design principles of each model and use it to navigate the literature. Then, we review how researchers have adapted these models to application areas and discuss cross-cutting and application-specific challenges and opportunities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Data Mining: Practical Machine Learning Tools and Techniques
Witten, I. H., Frank, E., & Hall, M. A. (2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,543
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687
(2016) · Cited by 11,679
Distributed Representations of Sentences and DocumentsPreprint
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 7,905
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014) · SSST@EMNLP · doi:10.3115/v1/w14-4012 · Cited by 7,251
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516
Research works printed but not linked: 68
- Alemi Alex A., 2016, Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS’16)
- Allamanis Miltiadis, 2018, Proceedings of the International Conference on Learning Representations (ICLR’18)
- Allamanis Miltiadis, 2017, Proceedings of the International Conference on Machine Learning (ICML’17)
- Allamanis Miltiadis, 2016, Proceedings of the International Conference on Machine Learning (ICML’16)
- 10.5555/2487085.2487127 · DOI
- Allamanis Miltiadis, 2015, Proceedings of the International Conference on Machine Learning (ICML’15)
- Bahdanau Dzmitry, 2015, Proceedings of the International Conference on Learning Representations (ICLR’15)
- Balog Matej, 2017, Proceedings of the International Conference on Learning Representations (ICLR’17)
- Miceli Barone Antonio Valerio, 2017, Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers) 2
- Bhatia Sahil, 2018, Proceedings of the International Conference on Software Engineering (ICSE’18)
- Bielik Pavol, 2015, Proceedings of the LIPIcs-Leibniz International Proceedings in Informatics.
- Bielik Pavol, 2016, Proceedings of the International Conference on Machine Learning (ICML’16)
- Raymond P., Proceedings of the International Conference on Software Engineering (ICSE’12)
- Deng Yuntian, Proceedings of the International Conference on Machine Learning (ICML’17)
- 10.5555/2819009.2819097 · DOI
- Devlin Jacob, 2017, Proceedings of the International Conference on Machine Learning (ICML’17)
- Dyer Robert, Proceedings of the International Conference on Software Engineering (ICSE’13)
- Ellis Kevin, 2017, Tenenbaum
- Ernst Michael D., 2017, Proceedings of the LIPIcs-Leibniz International Proceedings in Informatics.
- Feser John K., 2017, InProceedings of the International Conference on Learning Representations (ICLR’17)
- Foundation Eclipse, 2017, Retrieved
- 10.5555/2819009.2819143 · DOI
- Goodfellow Ian, Deep Learning
- Gordon Andrew D., Proceedings of the International Conference on Software Engineering (ICSE’14)
- Gotel Orlena, 2012, Andrea Zisman, Alexander Egyed, Paul Grünbacher, Alex Dekhtyar, Giuliano Antoniol, Jonathan Maletic, and Patrick Mäder.
- Gupta Rahul, 2017, Proceedings of the Conference of Artificial Intelligence (AAAI’17)
- 10.5555/2820518.2820539 · DOI
- Hinton G. E., Parallel Distributed Processing: Explorations in the Microstructure of Cognition, 1
- 10.5555/3155562.3155583 · DOI
- Johnson Daniel D., 2016, Proceedings of the International Conference on Learning Representations (ICLR’16)
- Dan Jurafsky. 2000. Speech 8 Language Processing (3rd. ed.). Pearson Education. Dan Jurafsky. 2000. Speech 8 Language Processing (3rd. ed.). Pearson Education.
- Koschke Rainer, 2007, Dagstuhl Seminar Proceedings. Schloss Dagstuhl-Leibniz-Zentrum für Informatik.
- Kremenek Ted, Proceedings of the International Joint Conference on Artifical intelligence (IJCAI’07)
- Kushman Nate, 2013, Proceedings of Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’13)
- Quoc, Proceedings of the International Conference on Machine Learning (ICML’14)
- Levy Dor, 2017, Proceedings of the International Conference on Machine Learning (ICML’17)
- Li Yujia, 2016, Proceedings of the International Conference on Learning Representations (ICLR’16)
- Liang Percy, 2010, Proceedings of the International Conference on Machine Learning (ICML’10)
- Lin Xi Victoria, Proceedings of the International Conference on Language Resources and Evaluation.
- Loos Sarah M., 2017, Proceedings of the International Conference on Logic for Programming Artificial Intelligence and Reasoning (LPAR’17)
- Maddison Chris, 2014, Proceedings of the International Conference on Machine Learning (ICML’14)
- Menon Aditya, 2013, Proceedings of the International Conference on Machine Learning (ICML’13)
- Mens Kim, Recommendation Systems in Software Engineering
- Mou Lili, 2016, Proceedings of the Conference of Artificial Intelligence (AAAI’16)
- Movshovitz-Attias Dana, Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’13)
- Movshovitz-Attias Dana, 2015, Cohen
- Murali Vijayaraghavan, 2018, Proceedings of the International Conference on Learning Representations (ICLR).
- Neelakantan Arvind, 2015, Proceedings of the International Conference on Learning Representations (ICLR’15)
- Nguyen Anh Tuan, Proceedings of the International Conference on Software Engineering (ICSE’15)
- Nguyen Anh Tuan, Proceedings of the International Symposium on Foundations of Software Engineering (FSE’13)
- Nguyen Anh T., Proceedings of the International Conference on Automated Software Engineering (ASE’15)
- Nguyen Trong Duc, Proceedings of the International Conference on Software Engineering (ICSE’16)
- Nguyen Trong Duc, Proceedings of the International Conference on Software Engineering (ICSE’17)
- Nguyen Tung Thanh, Proceedings of the Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering (ESEC/FSE’13)
- Omar Cyrus, 2017, Jonathan Aldrich, and Matthew A. Hammer.
- Parisotto Emilio, 2017, Proceedings of the International Conference on Learning Representations (ICLR’17)
- Pham Hung Viet, 2016, Proceedings of the International Conference on Software Engineering (ICSE’16)
- Piech Chris, Proceedings of the International Conference on Machine Learning (ICML’15)
- 10.5555/1667583.1667599 · DOI
- Reed Scott, 2016, Proceedings of the International Conference on Learning Representations (ICLR’16)
- Riedel Sebastian, 2017, Proceedings of the International Conference on Machine Learning (ICML’17)
- Rocktäschel Tim, 2017, Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS’17)
- 10.18653/v1/P16-1162 · DOI · Paper page
- Sutskever Ilya, Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS’14)
- Xin Wang Chang Liu Richard Shin Joseph E. Gonzalez and Dawn Song. 2016. Neural Code Completion. Retrieved from https://openreview.net/pdf?id=rJbPBt9lg. Xin Wang Chang Liu Richard Shin Joseph E. Gonzalez and Dawn Song. 2016. Neural Code Completion. Retrieved from https://openreview.net/pdf?id=rJbPBt9lg.
- 10.5555/2820518.2820559 · DOI
- Witten Ian H., 2016, Pal
- Zheng Alice X., 2003, Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS’03)
Cited by
Sorted by citations
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
“Prior work exists on evaluating readability (Allamanis et al., 2018) and security (Pearce et al.”
BackgroundSelf-citationHu, X., Li, G., Xia, X., Lo, D., & Jin, Z. G. (2018) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3196321.3196334 · Cited by 709
Automated Vulnerability Detection in Source Code Using Deep Representation Learning
Russell, R. L., Kim, L. Y., Hamilton, L., Lazovich, T., Harer, J., Özdemir, O., Ellingwood, P. M., & McConley, M. W. (2018) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2018.00120 · Cited by 758
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569
IntelliCode compose: code generation using transformer
Svyatkovskiy, A., Deng, S. K., Fu, S., & Sundaresan, N. (2020) · Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3368089.3417058 · Cited by 474
CoCoNuT: combining context-aware neural translation models using ensemble for program repair
Lutellier, T., Pham, H., Pang, L., Li, Y., Wei, M., & Tan, L. (2020) · International Symposium on Software Testing and Analysis · doi:10.1145/3395363.3397369 · Cited by 463
An empirical evaluation of GitHub copilot's code suggestions
Nguyen, N., & Nadi, S. (2022) · Proceedings of the 19th International Conference on Mining Software Repositories · doi:10.1145/3524842.3528470 · Cited by 313
A Neural Model for Generating Natural Language Summaries of Program Subroutines
LeClair, A., Jiang, S., & McMillan, C. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00087 · Cited by 319
Deep code comment generation with hybrid lexical and syntactical information
Hu, X., Li, G., Xia, X., Lo, D., & Jin, Z. G. (2019) · Empirical Software Engineering · doi:10.1007/s10664-019-09730-9 · Cited by 286
Investigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219
“The application of modern NLP techniques to programming language can be traced back to the naturalness hypothesis [4, 17, 37], that software is a form of human communication.”
Machine Learning in Compiler Optimization
Wang, Z., & O’boyle, M. F. P. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2817118 · Cited by 193
Big code != big vocabulary: open-vocabulary models for source code
Karampatsis, R.-M., Babii, H., Robbes, R., Sutton, C., & Janes, A. (2020) · Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering · doi:10.1145/3377811.3380342 · Cited by 182
Self-citationRetrieval-Based Prompt Selection for Code-Related Few-Shot Learning
Nashid, N., Sintaha, M., & Mesbah, A. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00205 · Cited by 178
Lessons Learned from Using a Deep Tree-Based Model for Software Defect Prediction in Practice
Dam, H. K., Pham, T., Ng, S. W., Tran, T., Grundy, J., Ghose, A., Kim, T., & Kim, C.-J. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00017 · Cited by 154
In-IDE Code Generation from Natural Language: Promise and Challenges
Xu, F. F., Vasilescu, B., & Neubig, G. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3487569 · Cited by 154
Code smell detection by deep direct-learning and transfer-learning
Sharma, T., Efstathiou, V., Λουρίδας, Π., & Spinellis, D. D. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.110936 · Cited by 167
A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Software
Ponta, S. E., Plate, H., Sabetta, A., Michele Bezzi, A., & Dangremont, C. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00064 · Cited by 140
VUDENC: Vulnerability Detection with Deep Learning on a Natural Codebase for Python
Wartschinski, L., Noller, Y., Vogel, T., Kehrer, T., & Grunske, L. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2021.106809 · Cited by 158
No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT
Liu, Z., Tang, Y., Luo, X., Zhou, Y., & Zhang, L. F. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3392499 · Cited by 128
Cyber Resilience in Healthcare Digital Twin on Lung Cancer
Zhang, J., Li, L., Lin, G., Da, F., Tai, Y., & Huang, J. (2020) · IEEE Access · doi:10.1109/access.2020.3034324 · Cited by 121
Comprehension of computer code relies primarily on domain-general executive brain regions
Ivanova, A. A., Srikant, S., Sueoka, Y., Kean, H., Dhamala, R., O’Reilly, U.-M., Bers, M. U., & Fedorenko, E. G. (2020) · eLife · doi:10.7554/elife.58906 · Cited by 120
Massarelli, L., Luna, G. A. D., Petroni, F., Querzoni, L., & Baldoni, R. (2019) · Workshop on Binary Analysis Research (BAR) · doi:10.14722/bar.2019.23020 · Cited by 107
A Bidirectional LSTM Language Model for Code Evaluation and Repair
Rahman, M. M., Watanobe, Y., & Nakamura, K. (2021) · Symmetry · doi:10.3390/sym13020247 · Cited by 110
Code Completion by Modeling Flattened Abstract Syntax Trees as Graphs
Wang, Y., & Li, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17650 · Cited by 86
Automatic detection of Long Method and God Class code smells through neural source code embeddings
Kovačević, A., Сливка, Ј., Vidaković, D., Grujić, K.-G., Luburić, N., Prokić, S., & Sladić, G. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117607 · Cited by 87
A Practical Approach to the Automatic Classification of Security-Relevant Commits
Sabetta, A., & Michele Bezzi, A. (2018) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2018.00058 · Cited by 77
Practices and Challenges of Using GitHub Copilot: An Empirical Study
Zhang, B., Liang, P., Zhou, X., Ahmad, A., & Waseem, M. (2023) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2023-077 · Cited by 124
The Language of Programming: A Cognitive Perspective
Fedorenko, E. G., Ivanova, A. A., Dhamala, R., & Bers, M. U. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.04.010 · Cited by 75
CoCoSoDa: Effective Contrastive Learning for Code Search
Shi, E., Wang, Y., Gu, W., Du, L., Zhang, H., Han, S., Zhang, D., & Sun, H. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00185 · Cited by 90
Kotenko, I. V., Евгеньевич, И. К., & Buinevich, M. (2022) · Sensors · doi:10.3390/s22041335 · Cited by 58
Generating correctness proofs with neural networks
Sanchez-Stern, A., Alhessi, Y., Saul, L. K., & Lerner, S. (2020) · ACM SIGPLAN International Workshop on Machine Learning and Programming Languages (MAPL) · doi:10.1145/3394450.3397466 · Cited by 49
An Extractive-and-Abstractive Framework for Source Code Summarization
Sun, W., Fang, C., Chen, Y., Zhang, Q., Tao, G., You, Y., Han, T., Ge, Y., Hu, Y., Luo, B., et al. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3632742 · Cited by 48
Bilateral Dependency Neural Networks for Cross-Language Algorithm Classification
Bui, N. D. Q., Yu, Y., & Jiang, L. (2019) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2019.8667995 · Cited by 40
Machine Learning for Software Engineering: A Tertiary Study
Kotti, Z., Galanopoulou, R., & Spinellis, D. D. (2022) · ACM Computing Surveys · doi:10.1145/3572905 · Cited by 54
Frequency Feature Learning from Vibration Information of GIS for Mechanical Fault Detection
Yuan, Y., Ma, S., Wu, J., Jia, B., Li, W., & Xiaowu, L. (2019) · Sensors · doi:10.3390/s19081949 · Cited by 43
The Essence of Software Engineering
Gruhn, V., & Striemer, R. (2018) · Springer eBooks · doi:10.1007/978-3-319-73897-0 · Cited by 37
Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit
Wan, Y., Bi, Z., He, Y., Zhang, J., Zhang, H., Sui, Y., Xu, G., Jin, H., & Yu, P. S. (2024) · ACM Computing Surveys · doi:10.1145/3664597 · Cited by 54
Wang, H., & Ma, S. (2021) · Socio-Economic Planning Sciences · doi:10.1016/j.seps.2021.101043 · Cited by 40
Predicting next Word using RNN and LSTM cells: Stastical Language Modeling
Ganai, A. F., & Khursheed, F. (2019) · International Conference on Image Information Processing (ICIIP) · doi:10.1109/iciip47207.2019.8985885 · Cited by 30
Memorization and Generalization in Neural Code Intelligence Models
Rabin, M. R. I., Hussain, A., Alipour, M. A., & Hellendoorn, V. J. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.107066 · Cited by 38
A Survey of Using Machine Learning in IoT Security and the Challenges Faced by Researchers
Harahsheh, K., & Chen, C. (2023) · Informatica · doi:10.31449/inf.v47i6.4635 · Cited by 38
Action Word Prediction for Neural Source Code Summarization
Haque, S., Bansal, A., Wu, L., & McMillan, C. (2021) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner50967.2021.00038 · Cited by 24
An Accurate Identifier Renaming Prediction and Suggestion Approach
Zhang, J., Luo, J., Liang, J., Gong, L., & Huang, Z. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3603109 · Cited by 24
Capturing source code semantics via tree-based convolution over API-enhanced AST
Chen, L., Ye, W., & Zhang, S. (2019) · ACM International Conference on Computing Frontiers (CF) · doi:10.1145/3310273.3321560 · Cited by 24
Long, T., Xie, Y., Chen, X., Zhang, W., Cao, Q., & Yu, Y. H. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20522 · Cited by 24
Thinking Like a Developer? Comparing the Attention of Humans with Neural Models of Code
Paltenghi, M., & Pradel, M. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678712 · Cited by 20
Identify and Update Test Cases When Production Code Changes: A Transformer-Based Approach
Hu, X., Liu, Z., Xia, X., Liu, Z., Xu, T., & Yang, X. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00165 · Cited by 20
Machine Learning in Compiler OptimisationPreprint
Wang, Z., & O’boyle, M. F. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.03441 · Cited by 19
REPFINDER: Finding Replacements for Missing APIs in Library Update
Huang, K., Chen, B., Pan, L., Wu, S., & Peng, X. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678905 · Cited by 18
CodeBERT-nt: Code Naturalness via CodeBERT
Khanfir, A., Jimenez, M., Papadakis, M., & Traon, Y. L. (2022) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs57517.2022.00098 · Cited by 20
Jimenez, M., Checkam, T. T., Cordy, M., Papadakis, M., Kintis, M., Traon, Y. L., & Harman, M. (2018) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3239235.3240500 · Cited by 18
Ortín, F., Rodriguez-Prieto, O., Pascual, N. M., & García, M. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.12.016 · Cited by 16
CodeS: Towards Code Model Generalization Under Distribution Shift
Hu, Q., Guo, Y., Xie, X., Cordy, M., Papadakis, M., Ma, L., & Traon, Y. L. (2023) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier58687.2023.00007 · Cited by 16
A Context-Aware Neural Embedding for Function-Level Vulnerability Detection
Wei, H., Lin, G., Li, L., & Jia, H. (2021) · Algorithms · doi:10.3390/a14110335 · Cited by 16
Machine Learning Method with Applications in Hardware Security of Post-Quantum Cryptography
Yi, H. (2023) · Journal of Grid Computing · doi:10.1007/s10723-023-09643-4 · Cited by 15
Leveraging Unsupervised Learning to Summarize APIs Discussed in Stack Overflow
Naghshzan, A., Guerrouj, L., & Baysal, O. (2021) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam52516.2021.00026 · Cited by 14
edgeTrans - Edge transport mode detection
Ferreira, P., Zavgorodnii, C., & Veiga, L. (2020) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2020.101268 · Cited by 13
Assessing and Analyzing the Correctness of GitHub Copilot's Code Suggestions
Mo, R., Wang, D., Zhan, W., Jiang, Y., Wang, Y., Zhao, Y., Li, Z., & Ma, Y. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715108 · Cited by 11
ProgramTransformer: A tool for generating semantically equivalent transformed programs
Rabin, M. R. I., & Alipour, M. A. (2022) · Software Impacts · doi:10.1016/j.simpa.2022.100429 · Cited by 11
Naturalness in Source Code Summarization. How Significant is it?
Ferretti, C., & Saletta, M. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00027 · Cited by 12
Neural Networks for Modeling Source Code EditsPreprint
Zhao, R. T., Bieber, D., Swersky, K., & Tarlow, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02818 · Cited by 11
Automatic Code Generation using Pre-Trained Language ModelsPreprint
Perez, L., Ottens, L., & Viswanathan, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10535 · Cited by 21
Self-Supervised Learning to Prove Equivalence Between Straight-Line Programs via Rewrite Rules
Kommrusch, S., Monperrus, M., & Pouchet, L.-N. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3271065 · Cited by 9
Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion
Pei, H., Zhao, J., Lausen, L., Zha, S., & Karypis, G. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25653 · Cited by 17
Towards Code Generation from BDD Test Case Specifications: A Vision
Chemnitz, L., Reichenbach, D., Aldebes, H., Naveed, M., Narasimhan, K., & Mezini, M. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00031 · Cited by 11
Unsupervised Classifying of Software Source Code Using Graph Neural Networks
Vytovtov, P., & Chuvilin, K. V. (2019) · Conference of Open Innovations Association (FRUCT) · doi:10.23919/fruct.2019.8711909 · Cited by 10
Learning the Relation Between Code Features and Code Transforms With Structured Prediction
Yu, Z., Martínez, M., Chen, Z., Bissyandé, T. F., & Monperrus, M. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3275380 · Cited by 10
A hybrid code representation learning approach for predicting method names
Zhang, F., Chen, B., Li, R., & Peng, X. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111011 · Cited by 10
Slice-Based Code Change Representation Learning
Zhang, F., Chen, B., Zhao, Y., & Peng, X. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00038 · Cited by 9
HoneyCode: Automating Deceptive Software Repositories with Deep Generative Models
Nguyen, D. D., Liebowitz, D., Nepal, ., & Kanhere, S. S. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.835 · Cited by 9
Distilled and Contextualized Neural Models Benchmarked for Vulnerable Function Detection
Lin, G., Jia, H., & Wu, D. (2022) · Mathematics · doi:10.3390/math10234482 · Cited by 8
Graph Confident Learning for Software Vulnerability Detection
Wang, Q., Li, Z., Liang, H., Pan, X., Li, H., Li, T., Li, X., Li, T., & Guo, S. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108296 · Cited by 8
Wang, B., Tong, Y., Ji, S., Dong, H., Luo, X., & Zhang, P. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3750042 · Cited by 8
Understanding security vulnerabilities in student code: A case study in a non-security course
Yilmaz, T., & Ulusoy, Ö. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111150 · Cited by 8
GRAPHSPY: Fused Program Semantic Embedding through Graph Neural Networks for Memory Efficiency
Guo, Y., Li, P., Luo, Y., Wang, X., & Wang, Z. (2021) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586120 · Cited by 8
Yao, K., Oliva, G. A., Hassan, A. E., Asaduzzaman, M., Malton, A. J., & Walenstein, A. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111651 · Cited by 8
Mavroudeas, G., Baudart, G., Cha, A., Hirzel, M. J., Laredo, J., Magdon‐Ismail, M., Mandel, L., & Wittern, E. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678513 · Cited by 7
SF6 High-Voltage Circuit Breaker Contact Status Detection at Different Currents
Guo, Z., Li, L., Han, W., & Guo, Z. (2022) · Sensors · doi:10.3390/s22218490 · Cited by 8
Graph Neural Networks Based Memory Inefficiency Detection Using Selective Sampling
Li, P., Guo, Y., Luo, Y., Wang, X., Wang, Z., & Liu, X. (2022) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41404.2022.00090 · Cited by 7
Implant Global and Local Hierarchy Information to Sequence based Code Representation Models
Zhang, K., Li, Z., Jin, Z. G., & Li, G. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00030 · Cited by 7
On the effectiveness of hybrid pooling in mixup-based graph learning for language processing
Dong, Z., Hu, Q., Zhang, Z., Guo, Y., Cordy, M., Papadakis, M., Traon, Y. L., & Zhao, J. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112139 · Cited by 6
Test-based and metric-based evaluation of code generation models for practical question answering
Kovalchuk, S. V., Fedrushkov, D., Lomshakov, V., & Aliev, A. (2023) · International Conference on Code Quality (ICCQ) · doi:10.1109/iccq57276.2023.10114665 · Cited by 6
A survey of coverage-guided greybox fuzzing with deep neural models
Qiu, J., Jiang, Y., Miao, Y., Luo, W., Pan, L., & Zheng, X. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107797 · Cited by 7
Senior DevOps Engineer — Cloud, D. A. A., & Kummarapurugu, C. S. (2024) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem17273 · Cited by 6
On Representation Learning-based Methods for Effective, Efficient, and Scalable Code Retrieval
França, C., Lima, R. C., de Andrade, C. M. V., da Cunha, W. L. M., de Melo, P. O. S. V., Ribeiro-Neto, B. A., da Rocha, L. C. D., Santos, R. L. T., Pagano, A. S., & Gonçalves, M. A. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128172 · Cited by 5
Automatic Acquisition of Annotated Training Corpora for Test-Code Generation
Kacmajor, M., & Kelleher, J. D. (2019) · Information · doi:10.3390/info10020066 · Cited by 4
Deep Code Search with Naming-Agnostic Contrastive Multi-View Learning
Feng, J., Li, W., Wu, S., Wei, Z., Xu, Y., Wang, J., & Li, H. (2025) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3737878 · Cited by 4
Can Code Representation Boost IR-Based Test Case Prioritization?
Yang, L., Chen, J., You, H., Han, J., Jiang, J., Sun, Z., Lin, X., Liang, F., & Kang, Y. (2023) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre59848.2023.00077 · Cited by 4
Towards semantically enhanced detection of emerging quality-related concerns in source code
Krasniqi, R., & Do, H. (2023) · Software Quality Journal · doi:10.1007/s11219-023-09614-8 · Cited by 5
Real-time pedestrian detection method based on improved YOLOv3
Luo, J., Wang, Y., & Wang, Y. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1453/1/012149 · Cited by 4
Predictive Analytics in IT Service Management (ITSM)
Sharon, C. I. L., & Suma, V. (2022) · doi:10.1002/9781119792529.ch7 · Cited by 4
Gated Graph Neural Networks (GG-NNs) for Abstractive Multi-Comment Summarization
Zhan, H., Zhang, K., Hu, C., & Sheng, V. S. (2021) · IEEE International Conference on Big Knowledge (ICBK) · doi:10.1109/ickg52313.2021.00050 · Cited by 3
DG-Trans: Automatic Code Summarization via Dynamic Graph Attention-based Transformer
Zeng, J., Zhang, T., & Zhou, X. (2021) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs54544.2021.00088 · Cited by 3
Combining Code Embedding with Static Analysis for Function-Call CompletionPreprint
Weyssow, M., Sahraoui, H. A., Frénay, B., & Vanderose, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03731 · Cited by 3
Enhancing Code LLM Training with Programmer Attention
Zhang, Y., Huang, C., Karas, Z., Nguyen, T. D., Leach, K., & Huang, Y. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728510 · Cited by 5
Feature Execution Graphs: A Human-AI Co-Programming Paradigm for Graph-Driven LLM Code Synthesis
Batatia, H., & Svetlichnyi, I. (2025) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v6i1.36052 · Cited by 2
The fools are certain; the wise are doubtful: Exploring LLM confidence in code completion
Kotti, Z., Dritsa, K., Spinellis, D. D., & Λουρίδας, Π. (2026) · Automated Software Engineering · doi:10.21203/rs.3.rs-7302683/v1 · Cited by 3
Evolutionary Approaches for Adversarial Attacks on Neural Source Code Classifiers
Mercuri, V., Saletta, M., & Ferretti, C. (2023) · Algorithms · doi:10.3390/a16100478 · Cited by 2
Statement-Grained Hierarchy Enhanced Code Summarization
Zhang, Q., Jin, D., Wang, Y., & Gong, Y. (2024) · Electronics · doi:10.3390/electronics13040765 · Cited by 2
Metadata
- Authors
- Allamanis, M., Barr, E. T., Devanbu, P., & Sutton, C.
- Affiliations
- Microsoft Research (United Kingdom) (GB)
- University College London (GB)
- University of California, Davis, California, USA
- The Alan Turing Institute (GB)
- Venue
- ACM Computing Surveys
- Published
- 2018-07-31
- Volume
- 51
- Issue
- 4
- Pages
- 1-37
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W2754629507
- Cited by
- 150
- References
- 186
- Citations by year
- 2026: 27 · 2025: 11 · 2024: 7 · 2023: 33 · 2022: 19 · 2021: 19 · 2020: 10 · 2019: 15 · 2018: 8
- FWCI
- 20.98
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Computing Surveys (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/K024043/1, EP/P005659/1, and EP/P005314/1)
- National Science Foundation (1414172)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Logic, programming, and type systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- probabilistic programming languages, naturalness, software engineering applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
