A Survey of Machine Learning for Big Code and Naturalness

Allamanis, Miltiadis; Barr, Earl T.; Devanbu, Premkumar; Sutton, Charles

doi:10.1145/3212695

Cited by 159Software Engineering ResearchLogic, programming, and type systemsNatural Language Processing Techniques

A Survey of Machine Learning for Big Code and Naturalness

Open access

Allamanis, Miltiadis, Barr, Earl T., Devanbu, Premkumar, Sutton, Charles·2019·ACM Computing Surveys

DOI 10.1145/3212695

Open this paper in CoChat — free account Open PDF

Abstract

Research at the intersection of machine learning, programming languages, and software engineering has recently taken important steps in proposing learnable probabilistic models of source code that exploit the abundance of patterns of code. In this article, we survey this work. We contrast programming languages against natural languages and discuss how these similarities and differences drive the design of probabilistic models. We present a taxonomy based on the underlying design principles of each model and use it to navigate the literature. Then, we review how researchers have adapted these models to application areas and discuss cross-cutting and application-specific challenges and opportunities.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1145/3212695

Cites

188 of 188 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
Data Mining: Practical Machine Learning Tools and Techniques
Witten, I. H., Frank, E., & Hall, M. A. (2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,493 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Anomaly detection: A survey
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687 · see citing works
Untitled work
(2016) · Cited by 11,679 · see citing works
Distributed Representations of Sentences and DocumentsPreprint
Le, Q. V., & Mikolov, T. (2014) · International Conference on Machine Learning · doi:10.48550/arxiv.1405.4053 · Cited by 10,307 · see citing works
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014) · SSST@EMNLP · doi:10.3115/v1/w14-4012 · Cited by 7,251 · see citing works
Probabilistic topic models
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516 · see citing works
Validity of the single processor approach to achieving large scale computing capabilities
Amdahl, G. M. (1967) · Proceedings of the April 18-20, 1967, spring joint computer conference on - AFIPS '67 (Spring) · doi:10.1145/1465482.1465560 · Cited by 4,339 · see citing works
An axiomatic basis for computer programming
Hoare, C. A. R. (1969) · Communications of the ACM · doi:10.1145/363235.363259 · Cited by 4,006 · see citing works
Gated Graph Sequence Neural NetworksPreprint
Li, Y., Tarlow, D., Brockschmidt, M., & Zemel, R. (2015) · International Conference on Learning Representations · doi:10.48550/arxiv.1511.05493 · Cited by 3,735 · see citing works
Drebin: Effective and Explainable Detection of Android Malware in Your Pocket
Arp, D., Spreitzenbarth, M., Hübner, M., Gascón, H., & Rieck, K. (2014) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2014.23247 · Cited by 2,417 · see citing works
An empirical study of smoothing techniques for language modeling
Chen, S. F., & Goodman, J. (1999) · Computer Speech & Language · doi:10.1006/csla.1999.0128 · Cited by 2,100 · see citing works
Literate Programming
Knuth, D. (1984) · Computer/law journal · doi:10.1093/comjnl/27.2.97 · Cited by 1,863 · see citing works
The Unreasonable Effectiveness of Data
Halevy, A. Y., Norvig, P., & Pereira, F. M. B. (2009) · IEEE Intelligent Systems · doi:10.1109/mis.2009.36 · Cited by 1,776 · see citing works
Improved backing-off for M-gram language modeling
Kneser, R., & Ney, H. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1995.479394 · Cited by 1,471 · see citing works
Defects4J: a database of existing faults to enable controlled testing studies for Java programs
Just, R., Jalali, D., & Ernst, M. D. (2014) · Proceedings of the 2014 International Symposium on Software Testing and Analysis · doi:10.1145/2610384.2628055 · Cited by 1,447 · see citing works
A Survey on Software Fault Localization
Wong, W. E., Gao, R., Li, Y., Abreu, R., & Wotawa, F. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2521368 · Cited by 1,110 · see citing works
Neural Turing Machines
Graves, A., Wayne, G., & Danihelka, I. (2014) · arXiv · doi:10.48550/arxiv.1410.5401 · Cited by 1,058 · see citing works
Distributed representations
Hinton, G. E. (2018) · KiltHub Repository · doi:10.1184/r1/6604925 · Cited by 1,029 · see citing works
A Survey on Multi-view LearningPreprint
Xu, C., Tao, D., & Xu, C. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.5634 · Cited by 983 · see citing works
Mining version histories to guide software changes
Zimmermann, T., Zeller, A., Weißgerber, P., & Diehl, S. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.72 · Cited by 869 · see citing works
Visualizing and Understanding Recurrent Networks
Karpathy, A., Johnson, J., & Fei-Fei, L. (2015) · arXiv · doi:10.48550/arxiv.1506.02078 · Cited by 845 · see citing works
Bugs as deviant behavior
Engler, D., Chen, D. Y., Hallem, S., Chou, A., & Chelf, B. (2001) · doi:10.1145/502034.502041 · Cited by 737 · see citing works
Scalable statistical bug isolation
Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065014 · Cited by 736 · see citing works
Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement LearningPreprint
Zhong, V., Xiong, C., & Socher, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.00103 · Cited by 733 · see citing works
Automatically learning semantic features for defect prediction
Wang, S., Liu, T., & Tan, L. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884804 · Cited by 720 · see citing works
Summarizing Source Code using a Neural Attention Model
Iyer, S. V., Konstas, I., Cheung, A., & Zettlemoyer, L. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1195 · Cited by 716 · see citing works
Convolutional Neural Networks over Tree Structures for Programming Language Processing
Mou, L., Li, G., Zhang, L., Wang, T., & Jin, Z. G. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10139 · Cited by 705 · see citing works
DeepCoder: Learning to Write ProgramsPreprint
Balog, M., Gaunt, A. L., Brockschmidt, M., Nowozin, S., & Tarlow, D. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1611.01989 · Cited by 641 · see citing works
A few billion lines of code later
Bessey, A., Block, K., Chelf, B., Chou, A., Fulton, B., Hallem, S., Henri-Gros, C., Kamsky, A., McPeak, S., & Engler, D. (2010) · Communications of the ACM · doi:10.1145/1646353.1646374 · Cited by 634 · see citing works
A Syntactic Neural Model for General-Purpose Code Generation
Yin, P., & Neubig, G. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1041 · Cited by 623 · see citing works
Deep learning code fragments for code clone detection
White, M., Tufano, M., Vendome, C., & Poshyvanyk, D. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970326 · Cited by 613 · see citing works
A Syntactic Neural Model for General-Purpose Code Generation
Yin, P., & Neubig, G. (2017) · arXiv · doi:10.48550/arxiv.1704.01696 · Cited by 585 · see citing works
On the naturalness of software
Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Dévanbu, P. (2012) · doi:10.5555/2337223.2337322 · Cited by 574 · see citing works
Self-citation
Code completion with statistical language models
Raychev, V., Vechev, M., & Yahav, E. (2014) · Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2594291.2594321 · Cited by 555 · see citing works
A cache-based natural language model for speech recognition
Kühn, R., & Mori, R. D. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.56193 · Cited by 503 · see citing works
Deep API learning
Gu, X., Zhang, H., Zhang, D., & Kim, S. (2016) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2950290.2950334 · Cited by 485 · see citing works
The Pragmatic Programmer: From Journeyman to Master
Hunt, A. P., & Thomas, D. A. (1999) · Cited by 468 · see citing works
Learning to Compose Neural Networks for Question Answering
Andreas, J., Rohrbach, M., Darrell, T., & Klein, D. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1181 · Cited by 450 · see citing works
A Convolutional Attention Network for Extreme Summarization of Source CodePreprint
Allamanis, M., Peng, H. M., & Sutton, C. A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.03001 · Cited by 436 · see citing works
Self-citation
The ASTREÉ Analyzer
Cousot, P., Cousot, R., Feret, J., Mauborgne, L., Miné, A., Monniaux, D., & Rival, X. (2005) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-31987-0_3 · Cited by 426 · see citing works
DeepFix: Fixing Common C Language Errors by Deep Learning
Gupta, R., Pal, S., Kanade, A., & Shevade, S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10742 · Cited by 426 · see citing works
Abstract Syntax Networks for Code Generation and Semantic Parsing
Rabinovich, M., Stern, M., & Klein, D. (2017) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p17-1105 · Cited by 395 · see citing works
Parseweb
Thummalapenta, S., & Xie, T. (2007) · doi:10.1145/1321631.1321663 · Cited by 394 · see citing works
Mining source code repositories at massive scale using language modeling
Allamanis, M., & Sutton, C. (2013) · IEEE Working Conference on Mining Software Repositories · doi:10.1109/msr.2013.6624029 · Cited by 393 · see citing works
Self-citation
Recommendation Systems for Software Engineering
Robillard, M. P., Walker, R. J., & Zimmermann, T. (2009) · IEEE Software · doi:10.1109/ms.2009.161 · Cited by 389 · see citing works
Learning from examples to improve code completion systems
Bruch, M., Monperrus, M., & Mezini, M. (2009) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1595696.1595728 · Cited by 383 · see citing works
Learning natural coding conventions
Allamanis, M., Barr, E. T., Bird, C., & Sutton, C. A. (2014) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2635868.2635883 · Cited by 380 · see citing works
Self-citation
Suggesting accurate method and class names
Allamanis, M., Barr, E. T., Bird, C., & Sutton, C. A. (2015) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2786805.2786849 · Cited by 379 · see citing works
Self-citation
Probabilistic programming
Gordon, A. D., Henzinger, T. A., Nori, A. V., & Rajamani, S. K. (2014) · Future of Software Engineering (FOSE) · doi:10.1145/2593882.2593900 · Cited by 372 · see citing works
Concise and consistent naming
Deissenboeck, F., & Pizka, M. (2006) · Software Quality Journal · doi:10.1007/s11219-006-9219-1 · Cited by 317 · see citing works
Learning to ExecutePreprint
Zaremba, W., & Sutskever, I. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.4615 · Cited by 314 · see citing works
Predicting Program Properties from "Big Code"
Raychev, V., Vechev, M., & Krause, A. (2014) · ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL) · doi:10.1145/2676726.2677009 · Cited by 311 · see citing works
Semantically Enhanced Software Traceability Using Deep Learning Techniques
Guo, J., Cheng, J., & Cleland-Huang, J. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2017.9 · Cited by 308 · see citing works
On the naturalness of software
Hindle, A., Barr, E. T., Gabel, M., Su, Z., & Devanbu, P. (2016) · Communications of the ACM · doi:10.1145/2902362 · Cited by 307 · see citing works
Self-citation
Latent Predictor Networks for Code Generation
Ling, W., Blunsom, P., Grefenstette, E., Hermann, K. M., Kočiský, T., Wang, F., & Senior, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1057 · Cited by 307 · see citing works
Are deep neural networks the best choice for modeling source code?
Hellendoorn, V. J., & Devanbu, P. (2017) · Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering · doi:10.1145/3106237.3106290 · Cited by 292 · see citing works
Convolutional Neural Networks over Tree Structures for Programming Language ProcessingPreprint
Mou, L., Li, G., Zhang, L., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5718 · Cited by 285 · see citing works
Behavioral consistency of C and verilog programs using bounded model checking
Clarke, E. M., Kroening, D., & Yorav, K. (2003) · Proceedings - ACM IEEE Design Automation Conference · doi:10.1145/775832.775928 · Cited by 283 · see citing works
Detecting object usage anomalies
Wasylkowski, A., Zeller, A., & Lindig, C. (2007) · doi:10.1145/1287624.1287632 · Cited by 274 · see citing works
Mining API patterns as partial orders from source code
Acharya, M. P., Xie, T., Pei, J., & Xu, J. (2007) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1287624.1287630 · Cited by 261 · see citing works
On the "naturalness" of buggy code
Ray, B., Hellendoorn, V. J., Godhane, S., Tu, Z., Bacchelli, A., & Dévanbu, P. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884848 · Cited by 260 · see citing works
Survey of Research on Software Clones
Koschke, R. (2007) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/dagsemproc.06301.13 · Cited by 259 · see citing works
Learning to Generate Pseudo-Code from Source Code Using Statistical Machine Translation
Oda, Y., Fudaba, H., Neubig, G., Hata, H., Sakti, S., Toda, T., & Nakamura, S. (2015) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.36 · Cited by 254 · see citing works
Boa: A Language and Infrastructure for Analyzing Ultra-Large-Scale Software Repositories
Dyer, R., Nguyen, H. A., Rajan, H., & Nguyen, T. N. (2013) · Cited by 246 · see citing works
Augmenting API documentation with insights from stack overflow
Treude, C., & Robillard, M. P. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884800 · Cited by 245 · see citing works
How developers search for code: a case study
Sadowski, C., Stolee, K. T., & Elbaum, S. G. (2015) · doi:10.1145/2786805.2786855 · Cited by 243 · see citing works
On the localness of software
Tu, Z., Su, Z., & Devanbu, P. (2014) · Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2635868.2635875 · Cited by 238 · see citing works
pix2code: Generating Code from a Graphical User Interface Screenshot
Beltramelli, T. (2018) · Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems · doi:10.1145/3220134.3220135 · Cited by 237 · see citing works
A statistical semantic language model for source code
Nguyen, T. T., Nguyen, A., Nguyen, H., & Nguyen, T. (2013) · ESEC/FSE 2013 · doi:10.1145/2491411.2491458 · Cited by 231 · see citing works
A study of the uniqueness of source code
Gabel, M., & Su, Z. (2010) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1882291.1882315 · Cited by 220 · see citing works
Recommendation Systems in Software Engineering
Robillard, M. P., Zimmermann, T. P., Walker, R. J., & Maalej, W. (2014) · Springer eBooks · doi:10.1007/978-3-642-45135-5 · Cited by 215 · see citing works
MAPO
Xie, T., & Pei, J. (2006) · doi:10.1145/1137983.1137997 · Cited by 204 · see citing works
Automatic mining of source code repositories to improve bug finding techniques
Williams, C. C., & Hollingsworth, J. K. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.63 · Cited by 198 · see citing works
Easy over hard: a case study on deep learning
Fu, W., & Menzies, T. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106256 · Cited by 195 · see citing works
Javert
Gabel, M., & Su, Z. (2008) · ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE) · doi:10.1145/1453101.1453150 · Cited by 192 · see citing works
End-to-End Deep Learning of Optimization Heuristics
Cummins, C., Petoumenos, P., Wang, Z., & Leather, H. (2017) · International Conference on Parallel Architectures and Compilation Techniques (PACT) · doi:10.1109/pact.2017.24 · Cited by 188 · see citing works
OverCode
Glassman, E. L., Scott, J., Singh, R., Guo, P. J., & Miller, R. C. (2015) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2699751 · Cited by 178 · see citing works
Toward deep learning software repositories
White, M., Vendome, C., & Poshyvanyk, D. (2015) · Cited by 177 · see citing works
Program Synthesis
Polozov, O., Gulwani, S., & Singh, R. K. (2017) · Cited by 167 · see citing works
Exploring API Embedding for API Usages and Applications
Nguyen, T. D., Nguyen, A. T., Phan, H., & Nguyen, T. N. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2017.47 · Cited by 164 · see citing works
Untitled work
(2014) · doi:10.1145/2635868.2635901 · Cited by 159 · see citing works
Self-citation
Learning to Understand Phrases by Embedding the Dictionary
Hill, F., Cho, K., Korhonen, A., & Bengio, Y. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00080 · Cited by 155 · see citing works
Evolutionary Linguistics
Croft, W. (2008) · Annual Review of Anthropology · doi:10.1146/annurev.anthro.37.081407.085156 · Cited by 154 · see citing works
Bugs as deviant behavior
Engler, D., Chen, D. Y., Hallem, S., Chou, A., & Chelf, B. (2001) · ACM SIGOPS Operating Systems Review · doi:10.1145/502059.502041 · Cited by 151 · see citing works
A Probabilistic Discriminative Model for Android Malware Detection with Decompiled Source Code
Cen, L., Gates, C. S., Si, L., & Li, N. (2014) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2014.2355839 · Cited by 150 · see citing works
Language to Code: Learning Semantic Parsers for If-This-Then-That Recipes
Quirk, C., Mooney, R. J., & Galley, M. (2015) · doi:10.3115/v1/p15-1085 · Cited by 150 · see citing works
Statistical debugging
Zheng, A. X., Jordan, M. I., Liblit, B., Naik, M., & Aiken, A. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143983 · Cited by 149 · see citing works
Merlin
Livshits, B., Nori, A. V., Rajamani, S. K., & Banerjee, A. (2009) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1542476.1542485 · Cited by 142 · see citing works
Graph-based statistical language model for code
Nguyen, A. T., & Nguyen, T. N. (2016) · Cited by 138 · see citing works
Lexical statistical machine translation for language migration
Nguyen, A. T., Nguyen, T. T., & Nguyen, T. N. (2013) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2491411.2494584 · Cited by 136 · see citing works
Bugram: bug detection with n-gram language models
Wang, S., Chollak, D., Movshovitz‐Attias, D., & Tan, L. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970341 · Cited by 130 · see citing works
Structured Generative Models of Natural Source CodePreprint
Maddison, C. J., & Tarlow, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.0514 · Cited by 129 · see citing works
Synthesizing API Usage Examples
Buse, R. P. L., & Weimer, W. R. (2012) · Cited by 129 · see citing works
Statistical Deobfuscation of Android Applications
Bichsel, B., Raychev, V., Tsankov, P., & Vechev, M. (2016) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/2976749.2978422 · Cited by 129 · see citing works
A Neural Architecture for Generating Natural Language Descriptions from Source Code Changes
Loyola, P., Marrese-Taylor, E., & Matsuo, Y. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p17-2045 · Cited by 128 · see citing works
Learning programs from noisy data
Raychev, V., Bielik, P., Vechev, M., & Krause, A. (2016) · ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL) · doi:10.1145/2837614.2837671 · Cited by 125 · see citing works
Deep Knowledge Tracing On Programming Exercises
Wang, L., Sy, A., Liu, L., & Piech, C. (2017) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3051457.3053985 · Cited by 122 · see citing works
NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System
Lin, X., Wang, C., Zettlemoyer, L., & Ernst, M. D. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/42er6x4bacdf · Cited by 116 · see citing works
Phrase-Based Statistical Translation of Programming Languages
Karaivanov, S., Raychev, V., & Vechev, M. (2014) · ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!) · doi:10.1145/2661136.2661148 · Cited by 115 · see citing works
Neural Programmer: Inducing Latent Programs with Gradient DescentPreprint
Neelakantan, A., Le, Q. V., & Sutskever, I. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04834 · Cited by 115 · see citing works
PHOG: probabilistic model for code
Bielik, P., Raychev, V., & Vechev, M. (2016) · International Conference on Machine Learning · Cited by 115 · see citing works
sk_p: a neural program corrector for MOOCs
Pu, Y., Narasimhan, K., Solar-Lezama, A., & Barzilay, R. (2016) · ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity (SPLASH) · doi:10.1145/2984043.2989222 · Cited by 112 · see citing works
Natural Language Models for Predicting Programming Comments
Movshovitz‐Attias, D., & Cohen, W. W. (2018) · Figshare · doi:10.1184/r1/6475913 · Cited by 108 · see citing works
A parallel corpus of Python functions and documentation strings for automated code documentation and code generationPreprint
Barone, A. V. M., & Sennrich, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.02275 · Cited by 103 · see citing works
Parameter-free probabilistic API mining across GitHub
Fowkes, J., & Sutton, C. A. (2016) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2950290.2950319 · Cited by 101 · see citing works
Self-citation
Learning to Compose Neural Networks for Question AnsweringPreprint
Andreas, J., Rohrbach, M., Darrell, T., & Klein, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.01705 · Cited by 100 · see citing works
NLyze
Gulwani, S., & Marron, M. (2014) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2588555.2612177 · Cited by 97 · see citing works
Deep Network Guided Proof Search
Loos, S. M., Irving, G., Szegedy, C., & Kaliszyk, C. (2018) · EPiC series in computing · doi:10.29007/8mwc · Cited by 95 · see citing works
Inducing Tree-Substitution Grammars
Cohn, T., Blunsom, P., & Goldwater, S. J. (2010) · doi:10.5555/1756006.1953031 · Cited by 91 · see citing works
Synthesizing benchmarks for predictive modeling
Cummins, C., Petoumenos, P., Wang, Z., & Leather, H. (2017) · Proceedings of the CGO/Proceedings of the ... CGO · doi:10.1109/cgo.2017.7863731 · Cited by 91 · see citing works
Portfolio
McMillan, C., Poshyvanyk, D., Grechanik, M., Xie, Q., & Fu, C. (2013) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2522920.2522930 · Cited by 86 · see citing works
Learning to rank code examples for code search engines
Niu, H., Keivanloo, I., & Zou, Y. (2016) · Empirical Software Engineering · doi:10.1007/s10664-015-9421-5 · Cited by 86 · see citing works
Syntax errors just aren't natural: improving error reporting with language models
Campbell, J. C., Hindle, A., & Amaral, J. N. (2014) · Working Conference on Mining Software Repositories (MSR) · doi:10.1145/2597073.2597102 · Cited by 82 · see citing works
Intelligent Code Completion with Bayesian Networks
Proksch, S., Lerch, J., & Mezini, M. (2015) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2744200 · Cited by 81 · see citing works
Learning Programs: A Hierarchical Bayesian Approach
Liang, P., Jordan, M. I., & Klein, D. (2010) · Cited by 80 · see citing works
Neural Sketch Learning for Conditional Program GenerationPreprint
Murali, V., Qi, L., Chaudhuri, S., & Jermaine, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.05698 · Cited by 79 · see citing works
A Study of Visual Studio Usage in Practice
Amann, S., Proksch, S., Nadi, S., & Mezini, M. (2016) · 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER) · doi:10.1109/saner.2016.39 · Cited by 77 · see citing works
TerpreT: A Probabilistic Programming Language for Program InductionPreprint
Gaunt, A. L., Brockschmidt, M., Singh, R., Kushman, N., Kohli, P., Taylor, J., & Tarlow, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04428 · Cited by 75 · see citing works
Synthesizing Java expressions from free-form queries
Gvero, T., & Kunčak, V. (2015) · Conference on Object-Oriented Programming Systems, Languages, and Applications · doi:10.1145/2814270.2814295 · Cited by 74 · see citing works
A user-guided approach to program analysis
Mangal, R., Zhang, X., Nori, A. V., & Naik, M. (2015) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2786805.2786851 · Cited by 73 · see citing works
A deep language model for software codePreprint
Dam, H. K., Tran, T., & Pham, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.02715 · Cited by 72 · see citing works
Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural NetworksPreprint
Bhatia, S., & Singh, R. K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06129 · Cited by 69 · see citing works
RCLinker: Automated Linking of Issue Reports and Commits Leveraging Rich Contextual Information
Le, T.-D. B., Linares‐Vásquez, M., Lo, D., & Poshyvanyk, D. (2015) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2015.13 · Cited by 68 · see citing works
Bayesian learning of a tree substitution grammar
Post, M., & Gildea, D. (2009) · doi:10.3115/1667583.1667599 · Cited by 66 · see citing works
pix2code: Generating Code from a Graphical User Interface ScreenshotPreprint
Beltramelli, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.07962 · Cited by 66 · see citing works
Predicting a Correct Program in Programming by Example
Singh, R. K., & Gulwani, S. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-21690-4_23 · Cited by 64 · see citing works
Recovering clear, natural identifiers from obfuscated JS names
Vasilescu, B., Casalnuovo, C., & Dévanbu, P. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106289 · Cited by 64 · see citing works
Learning a classifier for false positive error reports emitted by static code analysis tools
Koç, U., Saadatpanah, P., Foster, J. S., & Porter, A. (2017) · ACM SIGPLAN International Workshop on Machine Learning and Programming Languages (MAPL) · doi:10.1145/3088525.3088675 · Cited by 63 · see citing works
Learning Python Code Suggestion with a Sparse Pointer NetworkPreprint
Bhoopchand, A., Rocktäschel, T., Barr, E. T., & Riedel, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.08307 · Cited by 60 · see citing works
Self-citation
Statistical Debugging of Sampled Programs
Zheng, A. X., Jordan, M. I., Liblit, B., & Aiken, A. (2003) · Cited by 58 · see citing works
Learning a strategy for adapting a program analysis via bayesian optimisation
Oh, H., Yang, H., & Yi, K. (2015) · doi:10.1145/2814270.2814309 · Cited by 55 · see citing works
Internet-Scale Code Search
Gallardo-Valencia, R. E., & Sim, S. E. (2009) · doi:10.1109/suite.2009.5070022 · Cited by 52 · see citing works
Autofolding for Source Code Summarization
Fowkes, J., Chanthirasegaran, P., Ranca, R., Allamanis, M., Lapata, M., & Sutton, C. A. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2664836 · Cited by 49 · see citing works
Self-citation
Learning to Infer Graphics Programs from Hand-Drawn ImagesPreprint
Ellis, K., Ritchie, D., Solar-Lezama, A., & Tenenbaum, J. B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.09627 · Cited by 49 · see citing works
Mapping API elements for code migration with vector representations
Nguyen, T. D., Nguyen, A. T., & Nguyen, T. N. (2016) · doi:10.1145/2889160.2892661 · Cited by 46 · see citing works
Exploring the use of deep learning for feature location
Corley, C. S., Damevski, K., & Kraft, N. A. (2015) · IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsm.2015.7332513 · Cited by 45 · see citing works
Extracting code from programming tutorial videos
Yadid, S., & Yahav, E. (2016) · ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!) · doi:10.1145/2986012.2986021 · Cited by 45 · see citing works
Verifiable functional purity in java
Finifter, M., Mettler, A., Sastry, N., & Wagner, D. (2008) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/1455770.1455793 · Cited by 43 · see citing works
Bayesian specification learning for finding API usage errors
Murali, V., Chaudhuri, S., & Jermaine, C. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106284 · Cited by 42 · see citing works
Context2Name: A Deep Learning-Based Approach to Infer Natural Variable Names from Usage ContextsPreprint
Bavishi, R., Pradel, M., & Sen, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.05193 · Cited by 42 · see citing works
Natural Language is a Programming Language: Applying Natural Language Processing to Software Development
Ernst, M. D. (2017) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.snapl.2017.4 · Cited by 40 · see citing works
Programming by demonstration: a machine learning approach
Lau, T. A., Weld, D. S., & Domingos, P. (2001) · Cited by 38 · see citing works
RCLinker: automated linking of issue reports and commits leveraging rich contextual information
Le, T.-D. B., Linares‐Vásquez, M., Lo, D., et al. (2015) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · doi:10.5555/2820282.2820290 · Cited by 36 · see citing works
Using web corpus statistics for program analysis
Hsiao, C.-H., Cafarella, M., & Narayanasamy, S. (2014) · Conference on Object-Oriented Programming Systems, Languages, and Applications · doi:10.1145/2660193.2660226 · Cited by 36 · see citing works
Emergent, crowd-scale programming practice in the IDE
Fast, E., Steffee, D., Wang, L. L., Brandt, J., & Bernstein, M. S. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2556998 · Cited by 36 · see citing works
Program Synthesis
Gulwani, S., Polozov, O., & Singh, R. K. (2017) · doi:10.1561/9781680832938 · Cited by 36 · see citing works
CACHECA: A Cache Language Model Based Code Suggestion Tool
Franks, C., Tu, Z., Dévanbu, P., & Hellendoorn, V. J. (2015) · Cited by 35 · see citing works
Learning to Fuzz: Application-Independent Fuzz Testing with Probabilistic, Generative Models of Input Data
Patra, J., & Pradel, M. (2016) · TUbilio (Technical University of Darmstadt) · Cited by 34 · see citing works
Strathcona example recommendation tool
Holmes, R., Walker, R. J., & Murphy, G. C. (2005) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/1095430.1081744 · Cited by 34 · see citing works
NIRMAL: Automatic identification of software relevant tweets leveraging language model
Sharma, A. B., Tian, Y., & Lo, D. (2015) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2015.7081855 · Cited by 31 · see citing works
Neural Attribute Machines for Program GenerationPreprint
Amodio, M., Chaudhuri, S., & Reps, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.09231 · Cited by 29 · see citing works
Deep API LearningPreprint
Gu, X., Zhang, H., Zhang, D., & Kim, S. (2016) · arXiv (Cornell University) · Cited by 29 · see citing works
Evaluating the evaluations of code recommender systems: a reality check
Proksch, S., Amann, S., Nadi, S., & Mezini, M. (2016) · Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/2970276.2970330 · Cited by 26 · see citing works
Towards a universal code formatter through machine learning
Parr, T., & Vinju, J. J. (2016) · ACM SIGPLAN International Conference on Software Language Engineering (SLE) · doi:10.1145/2997364.2997383 · Cited by 26 · see citing works
Source Code-Based Recommendation Systems
Mens, K., & Lozano, A. (2013) · Springer eBooks · doi:10.1007/978-3-642-45135-5_5 · Cited by 25 · see citing works
Automatically generating features for learning program analysis heuristics for C-like languages
Chae, K., Oh, H., Heo, K., & Yang, H. (2017) · Proceedings of the ACM on Programming Languages · doi:10.1145/3133925 · Cited by 25 · see citing works
Deep Learning to Find Bugs
Pradel, M., & Sen, K. (2017) · TUbilio (Technical University of Darmstadt) · Cited by 24 · see citing works
A Neural Architecture for Generating Natural Language Descriptions from Source Code ChangesPreprint
Loyola, P., Marrese-Taylor, E., & Matsuo, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.04856 · Cited by 22 · see citing works
Using machine translation for converting Python 2 to Python 3 code
Aggarwal, K., Salameh, M., & Hindle, A. (2015) · doi:10.7287/peerj.preprints.1459v1 · Cited by 20 · see citing works
Deep Reinforcement Learning for Programming Language CorrectionPreprint
Gupta, R., Kanade, A., & Shevade, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.10467 · Cited by 19 · see citing works
Recent Advances in Neural Program SynthesisPreprint
Kant, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.02353 · Cited by 19 · see citing works
A factor graph model for software bug finding
Kremenek, T., Ng, A. Y., & Engler, D. (2007) · International Joint Conference on Artificial Intelligence · Cited by 18 · see citing works
KB-LDA: Jointly Learning a Knowledge Base of Hierarchy, Relations, and Facts
Movshovitz‐Attias, D., & Cohen, W. W. (2015) · doi:10.3115/v1/p15-1140 · Cited by 17 · see citing works
Programming with "Big Code": Lessons, Techniques and Applications
Bielik, P., Raychev, V., & Vechev, M. (2015) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.snapl.2015.41 · Cited by 14 · see citing works
CodeSum: Translate Program Language to Natural LanguagePreprint
Hu, X., Wei, Y., Li, G., & Jin, Z. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.01837 · Cited by 14 · see citing works
SmartPaste: Learning to Adapt Source Code
Allamanis, M., & Brockschmidt, M. (2017) · arXiv · doi:10.48550/arxiv.1705.07867 · Cited by 13 · see citing works
Self-citation
Learning Shape Analysis
Brockschmidt, M., Chen, Y., Kohli, P., Krishna, S., & Tarlow, D. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66706-5_4 · Cited by 13 · see citing works
Software Portability: An Advanced Course
Brown, P. J. (1977) · Medical Entomology and Zoology · Cited by 12 · see citing works
Parameter-Free Probabilistic API Mining at GitHub ScalePreprint
Fowkes, J., & Sutton, C. A. (2015) · arXiv (Cornell University) · Cited by 12 · see citing works
Self-citation
Learning to Align the Source Code to the Compiled Object Code
Levy, D., & Wolf, L. (2017) · International Conference on Machine Learning · Cited by 12 · see citing works
Toward Semantic Foundations for Program EditorsPreprint
Omar, C., Voysey, I., Hilton, M. L., Sunshine, J., Goues, C. L., Aldrich, J., & Hammer, M. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.08694 · Cited by 12 · see citing works
Data-Driven Program CompletionPreprint
Lu, Y., Chaudhuri, S., Jermaine, C., & Melski, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.09042 · Cited by 11 · see citing works
Products, developers, and milestones: how should I build my N-Gram language model
Saraiva, J. D. A. G., Bird, C., & Zimmermann, T. (2015) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2786805.2804431 · Cited by 10 · see citing works
Unsupervised Visual Sense Disambiguation for Verbs using Multimodal Embeddings
Gella, S., Lapata, M., & Keller, F. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1022 · Cited by 10 · see citing works
Towards better program obfuscation
Liu, H. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2889160.2891040 · Cited by 10 · see citing works
New Initiative: The Naturalness of Software
Dévanbu, P. (2015) · Cited by 9 · see citing works
Irish: A Hidden Markov Model to detect coded information islands in free text
Cerulo, L., Penta, M. D., Bacchelli, A., Ceccarelli, M., & Canfora, G. (2014) · Science of Computer Programming · doi:10.1016/j.scico.2014.11.017 · Cited by 6 · see citing works
Structured statistical syntax tree prediction
Omar, C. (2013) · doi:10.1145/2508075.2514876 · Cited by 4 · see citing works
Programming with “Big Code”
Yahav, E. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-26529-2_1 · Cited by 4 · see citing works
Finding Likely Errors with Bayesian SpecificationsPreprint
Murali, V., Chaudhuri, S., & Jermaine, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01370 · Cited by 4 · see citing works
Using web corpus statistics for program analysis
Hsiao, C.-H., Cafarella, M., & Narayanasamy, S. (2014) · ACM SIGPLAN Notices · doi:10.1145/2714064.2660226 · Cited by 4 · see citing works
Neural Functional ProgrammingPreprint
Feser, J., Brockschmidt, M., Gaunt, A. L., & Tarlow, D. (2016) · arXiv (Cornell University) · Cited by 3 · see citing works
Mining Semantic Loop Idioms from Big Code
Allamanis, M., Barr, E. T., Bird, C., Dévanbu, P., Marron, M., & Sutton, C. A. (2016) · Cited by 2 · see citing works
Self-citation
Neural Programmer-Interpreters
Reed, S., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06279

Flagged first, then by citations

Research works printed but not linked: 31

[3]3. Alex A. Alemi, Francois Chollet, Geoffrey Irving, Christian Szegedy, and Josef Urban. 2016. DeepMath-Deep se- quence models for premise selection. In Proceedings of the Annual Conference on Neural Information Processing Sys- tems (NIPS'16).
[8]8. Miltiadis Allamanis, Marc Brockschmidt, and Mahmoud Khademi. 2018. Learning to represent programs with graphs. In Proceedings of the International Conference on Learning Representations (ICLR'18).
[9]9. Miltiadis Allamanis, Pankajan Chanthirasegaran, Pushmeet Kohli, and Charles Sutton. 2017. Learning continuous semantic representations of symbolic expressions. In Proceedings of the International Conference on Machine Learning (ICML'17).
[12]12. Miltiadis Allamanis and Charles Sutton. 2014. Mining idioms from source code. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE'14).
[13]13. Miltiadis Allamanis, Daniel Tarlow, Andrew Gordon, and Yi Wei. 2015. Bimodal modelling of source code and natural language. In Proceedings of the International Conference on Machine Learning (ICML'15).
[19]19. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations (ICLR'15).
[51]51. Yuntian Deng, Anssi Kanervisto, Jeffrey Ling, and Alexander M. Rush. 2017. Image-to-markup generation with coarse-to-fine attention. In Proceedings of the International Conference on Machine Learning (ICML'17). 980-989.
[53]53. Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel rahman Mohamed, and Pushmeet Kohli. 2017. Robustfill: Neural program learning under noisy I/O. In Proceedings of the International Conference on Machine Learning (ICML'17).
[66]66. Mark Gabel and Zhendong Su. 2008. Javert: Fully automatic mining of general temporal properties from dynamic traces. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE'08).
[71]71. Elena L. Glassman, Jeremy Scott, Rishabh Singh, Philip J. Guo, and Robert C. Miller. 2015. OverCode: Visualizing variation in student solutions to programming problems at scale. ACM Transactions on Computer-Human Interaction (TOCHI) 22, 2 (2015), 7 pages.
[74]74. Orlena Gotel, Jane Cleland-Huang, Jane Huffman Hayes, Andrea Zisman, Alexander Egyed, Paul Grünbacher, Alex Dekhtyar, Giuliano Antoniol, Jonathan Maletic, and Patrick Mäder. 2012. Traceability fundamentals. In Software and Systems Traceability. Springer, 3-22.
[77]77. Sumit Gulwani and Mark Marron. 2014. NLyze: Interactive programming by natural language for spreadsheet data analysis and manipulation. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data.
[85]85. Vincent J. Hellendoorn, Premkumar T. Devanbu, and Alberto Bacchelli. 2015. Will they like this?: Evaluating code contributions with language models. In Proceedings of the Working Conference on Mining Software Repositories (MSR'15).
[97]97. Siyuan Jiang, Ameer Armaly, and Collin McMillan. 2017. Automatically generating commit messages from diffs using neural machine translation. In Proceedings of the International Conference on Automated Software Engineering (ASE'17).
[98]98. Daniel D. Johnson. 2016. Learning graphical state transitions. In Proceedings of the International Conference on Learn- ing Representations (ICLR'16).
[99]99. Dan Jurafsky. 2000. Speech & Language Processing (3rd. ed.). Pearson Education.
[110]110. Nate Kushman and Regina Barzilay. 2013. Using semantic unification to generate regular expressions from natural language. In Proceedings of Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT'13).
[122]122. Benjamin Livshits, Aditya V. Nori, Sriram K. Rajamani, and Anindya Banerjee. 2009. Merlin: Specification infer- ence for explicit information flow problems. In Proceedings of the Symposium on Programming Language Design and Implementation (PLDI'09).
[128]128. Collin Mcmillan, Denys Poshyvanyk, Mark Grechanik, Qing Xie, and Chen Fu. 2013. Portfolio: Searching for relevant functions and their usages in millions of lines of code. ACM Transactions on Software Engineering and Methodology (TOSEM) 22, 4 (2013), 37 pages.
[129]129. Aditya Menon, Omer Tamuz, Sumit Gulwani, Butler Lampson, and Adam Kalai. 2013. A machine learning framework for programming by example. In Proceedings of the International Conference on Machine Learning (ICML'13).
[141]141. Anh T. Nguyen, Tung Thanh Nguyen, and Tien N. Nguyen. 2015. Divide-and-conquer approach for multi-phase sta- tistical migration for source code. In Proceedings of the International Conference on Automated Software Engineering (ASE'15).
[151]151. Emilio Parisotto, Abdel-rahman Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, and Pushmeet Kohli. 2017. Neuro-symbolic program synthesis. In Proceedings of the International Conference on Learning Representations (ICLR'17).
[154]154. Hung Viet Pham, Phong Minh Vu, Tung Thanh Nguyen, and others. 2016. Learning API usages from bytecode: A statistical approach. In Proceedings of the International Conference on Software Engineering (ICSE'16).
[155]155. Chris Piech, Jonathan Huang, Andy Nguyen, Mike Phulsuksombati, Mehran Sahami, and Leonidas J. Guibas. 2015. Learning program embeddings to propagate feedback on student code. In Proceedings of the International Conference on Machine Learning (ICML'15).
[168]168. Sebastian Riedel, Matko Bosnjak, and Tim Rocktäschel. 2017. Programming with a differentiable forth interpreter. In Proceedings of the International Conference on Machine Learning (ICML'17).
[170]170. Martin P. Robillard, Walid Maalej, Robert J. Walker, and Thomas Zimmermann. 2014. Recommendation Systems in Software Engineering. Springer.
[171]171. Tim Rocktäschel and Sebastian Riedel. 2017. End-to-end differentiable proving. In Proceedings of the Annual Confer- ence on Neural Information Processing Systems (NIPS'17).
[174]174. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL'16).
[178]178. Suresh Thummalapenta and Tao Xie. 2007. Parseweb: A programmer assistant for reusing open source code on the web. In Proceedings of the International Conference on Automated Software Engineering (ASE'07).
[185]185. Xin Wang, Chang Liu, Richard Shin, Joseph E. Gonzalez, and Dawn Song. 2016. Neural Code Completion. Retrieved from https://openreview.net/pdf?id=rJbPBt9lg.
[192]192. Tao Xie and Jian Pei. 2006. MAPO: Mining API usages from open source repositories. In Proceedings of the Working Conference on Mining Software Repositories (MSR'06).

Sorted by citations

PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
“Prior work exists on evaluating readability (Allamanis et al., 2018) and security (Pearce et al.”
BackgroundSelf-citation
Deep code comment generation
Hu, X., Li, G., Xia, X., Lo, D., & Jin, Z. G. (2018) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3196321.3196334 · Cited by 705 · see citing works
Automated Vulnerability Detection in Source Code Using Deep Representation Learning
Russell, R. L., Kim, L. Y., Hamilton, L., Lazovich, T., Harer, J., Özdemir, O., Ellingwood, P. M., & McConley, M. W. (2018) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2018.00120 · Cited by 696 · see citing works
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569 · see citing works
IntelliCode compose: code generation using transformer
Svyatkovskiy, A., Deng, S. K., Fu, S., & Sundaresan, N. (2020) · Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3368089.3417058 · Cited by 474 · see citing works
CoCoNuT: combining context-aware neural translation models using ensemble for program repair
Lutellier, T., Pham, H., Pang, L., Li, Y., Wei, M., & Tan, L. (2020) · International Symposium on Software Testing and Analysis · doi:10.1145/3395363.3397369 · Cited by 463 · see citing works
A Neural Model for Generating Natural Language Summaries of Program Subroutines
LeClair, A., Jiang, S., & McMillan, C. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00087 · Cited by 305 · see citing works
Deep code comment generation with hybrid lexical and syntactical information
Hu, X., Li, G., Xia, X., Lo, D., & Jin, Z. G. (2019) · Empirical Software Engineering · doi:10.1007/s10664-019-09730-9 · Cited by 279 · see citing works
Investigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219 · see citing works
“The application of modern NLP techniques to programming language can be traced back to the naturalness hypothesis [4, 17, 37], that software is a form of human communication.”
Machine Learning in Compiler Optimization
Wang, Z., & O’boyle, M. F. P. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2817118 · Cited by 193 · see citing works
Big code != big vocabulary: open-vocabulary models for source code
Karampatsis, R.-M., Babii, H., Robbes, R., Sutton, C., & Janes, A. (2020) · Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering · doi:10.1145/3377811.3380342 · Cited by 182 · see citing works
Self-citation
Retrieval-Based Prompt Selection for Code-Related Few-Shot Learning
Nashid, N., Sintaha, M., & Mesbah, A. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00205 · Cited by 178 · see citing works
Lessons Learned from Using a Deep Tree-Based Model for Software Defect Prediction in Practice
Dam, H. K., Pham, T., Ng, S. W., Tran, T., Grundy, J., Ghose, A., Kim, T., & Kim, C.-J. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00017 · Cited by 155 · see citing works
In-IDE Code Generation from Natural Language: Promise and Challenges
Xu, F. F., Vasilescu, B., & Neubig, G. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3487569 · Cited by 154 · see citing works
Code smell detection by deep direct-learning and transfer-learning
Sharma, T., Efstathiou, V., Λουρίδας, Π., & Spinellis, D. D. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.110936 · Cited by 149 · see citing works
A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Software
Ponta, S. E., Plate, H., Sabetta, A., Michele Bezzi, A., & Dangremont, C. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00064 · Cited by 140 · see citing works
VUDENC: Vulnerability Detection with Deep Learning on a Natural Codebase for Python
Wartschinski, L., Noller, Y., Vogel, T., Kehrer, T., & Grunske, L. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2021.106809 · Cited by 130 · see citing works
Cyber Resilience in Healthcare Digital Twin on Lung Cancer
Zhang, J., Li, L., Lin, G., Da, F., Tai, Y., & Huang, J. (2020) · IEEE Access · doi:10.1109/access.2020.3034324 · Cited by 116 · see citing works
Comprehension of computer code relies primarily on domain-general executive brain regions
Ivanova, A. A., Srikant, S., Sueoka, Y., Kean, H., Dhamala, R., O’Reilly, U.-M., Bers, M. U., & Fedorenko, E. G. (2020) · eLife · doi:10.7554/elife.58906 · Cited by 111 · see citing works
Investigating Graph Embedding Neural Networks with Unsupervised Features Extraction for Binary Analysis
Massarelli, L., Luna, G. A. D., Petroni, F., Querzoni, L., & Baldoni, R. (2019) · Workshop on Binary Analysis Research (BAR) · doi:10.14722/bar.2019.23020 · Cited by 105 · see citing works
A Bidirectional LSTM Language Model for Code Evaluation and Repair
Rahman, M. M., Watanobe, Y., & Nakamura, K. (2021) · Symmetry · doi:10.3390/sym13020247 · Cited by 103 · see citing works
Code Completion by Modeling Flattened Abstract Syntax Trees as Graphs
Wang, Y., & Li, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17650 · Cited by 81 · see citing works
Automatic detection of Long Method and God Class code smells through neural source code embeddings
Kovačević, A., Сливка, Ј., Vidaković, D., Grujić, K.-G., Luburić, N., Prokić, S., & Sladić, G. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117607 · Cited by 81 · see citing works
A Practical Approach to the Automatic Classification of Security-Relevant Commits
Sabetta, A., & Michele Bezzi, A. (2018) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2018.00058 · Cited by 77 · see citing works
Practices and Challenges of Using GitHub Copilot: An Empirical Study
Zhang, B., Liang, P., Zhou, X., Ahmad, A., & Waseem, M. (2023) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2023-077 · Cited by 75 · see citing works
The Language of Programming: A Cognitive Perspective
Fedorenko, E. G., Ivanova, A. A., Dhamala, R., & Bers, M. U. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.04.010 · Cited by 75 · see citing works
CoCoSoDa: Effective Contrastive Learning for Code Search
Shi, E.-S., Wang, Y.-L., Gu, W., Du, L., Zhang, H.-Y., Han, S., Zhang, D.-M., & Sun, H. (2022) · International Conference on Software Engineering · doi:10.1109/icse48619.2023.00185 · Cited by 89 · see citing works
Static Analysis of Information Systems for IoT Cyber Security: A Survey of Machine Learning Approaches
Kotenko, I. V., Евгеньевич, И. К., & Buinevich, M. (2022) · Sensors · doi:10.3390/s22041335 · Cited by 58 · see citing works
Generating correctness proofs with neural networks
Sanchez-Stern, A., Alhessi, Y., Saul, L. K., & Lerner, S. (2020) · ACM SIGPLAN International Workshop on Machine Learning and Programming Languages (MAPL) · doi:10.1145/3394450.3397466 · Cited by 49 · see citing works
An Extractive-and-Abstractive Framework for Source Code Summarization
Sun, W., Fang, C., Chen, Y., Zhang, Q., Tao, G., You, Y., Han, T., Ge, Y., Hu, Y., Luo, B., et al. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3632742 · Cited by 44 · see citing works
Bilateral Dependency Neural Networks for Cross-Language Algorithm Classification
Bui, N. D. Q., Yu, Y., & Jiang, L. (2019) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2019.8667995 · Cited by 41 · see citing works
Machine Learning for Software Engineering: A Tertiary Study
Kotti, Z., Galanopoulou, R., & Spinellis, D. D. (2022) · ACM Computing Surveys · doi:10.1145/3572905 · Cited by 44 · see citing works
Frequency Feature Learning from Vibration Information of GIS for Mechanical Fault Detection
Yuan, Y., Ma, S., Wu, J., Jia, B., Li, W., & Xiaowu, L. (2019) · Sensors · doi:10.3390/s19081949 · Cited by 39 · see citing works
The Essence of Software Engineering
Gruhn, V., & Striemer, R. (2018) · Springer eBooks · doi:10.1007/978-3-319-73897-0 · Cited by 37 · see citing works
Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit
Wan, Y., Bi, Z., He, Y., Zhang, J., Zhang, H., Sui, Y., Xu, G., Jin, H., & Yu, P. S. (2024) · ACM Computing Surveys · doi:10.1145/3664597 · Cited by 34 · see citing works
Preventing crimes against public health with artificial intelligence and machine learning capabilities
Wang, H., & Ma, S. (2021) · Socio-Economic Planning Sciences · doi:10.1016/j.seps.2021.101043 · Cited by 34 · see citing works
Predicting next Word using RNN and LSTM cells: Stastical Language Modeling
Ganai, A. F., & Khursheed, F. (2019) · International Conference on Image Information Processing (ICIIP) · doi:10.1109/iciip47207.2019.8985885 · Cited by 30 · see citing works
Memorization and Generalization in Neural Code Intelligence Models
Rabin, M. R. I., Hussain, A., Alipour, M. A., & Hellendoorn, V. J. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.107066 · Cited by 30 · see citing works
A Survey of Using Machine Learning in IoT Security and the Challenges Faced by Researchers
Harahsheh, K., & Chen, C. (2023) · Informatica · doi:10.31449/inf.v47i6.4635 · Cited by 38 · see citing works
Action Word Prediction for Neural Source Code Summarization
Haque, S., Bansal, A., Wu, L., & McMillan, C. (2021) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner50967.2021.00038 · Cited by 24 · see citing works
An Accurate Identifier Renaming Prediction and Suggestion Approach
Zhang, J., Luo, J., Liang, J., Gong, L., & Huang, Z. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3603109 · Cited by 24 · see citing works
Capturing source code semantics via tree-based convolution over API-enhanced AST
Chen, L., Ye, W., & Zhang, S. (2019) · ACM International Conference on Computing Frontiers (CF) · doi:10.1145/3310273.3321560 · Cited by 24 · see citing works
Multi-View Graph Representation for Programming Language Processing: An Investigation into Algorithm Detection
Long, T., Xie, Y., Chen, X., Zhang, W., Cao, Q., & Yu, Y. H. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20522 · Cited by 21 · see citing works
Thinking Like a Developer? Comparing the Attention of Humans with Neural Models of Code
Paltenghi, M., & Pradel, M. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678712 · Cited by 20 · see citing works
Identify and Update Test Cases When Production Code Changes: A Transformer-Based Approach
Hu, X., Liu, Z., Xia, X., Liu, Z., Xu, T., & Yang, X. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00165 · Cited by 20 · see citing works
Machine Learning in Compiler OptimisationPreprint
Wang, Z., & O’boyle, M. F. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.03441 · Cited by 19 · see citing works
REPFINDER: Finding Replacements for Missing APIs in Library Update
Huang, K., Chen, B., Pan, L., Wu, S., & Peng, X. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678905 · Cited by 18 · see citing works
CodeBERT-nt: Code Naturalness via CodeBERT
Khanfir, A., Jimenez, M., Papadakis, M., & Traon, Y. L. (2022) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs57517.2022.00098 · Cited by 18 · see citing works
Are mutants really natural?
Jimenez, M., Checkam, T. T., Cordy, M., Papadakis, M., Kintis, M., Traon, Y. L., & Harman, M. (2018) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3239235.3240500 · Cited by 18 · see citing works
Heterogeneous tree structure classification to label Java programmers according to their expertise level
Ortín, F., Rodriguez-Prieto, O., Pascual, N. M., & García, M. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.12.016 · Cited by 16 · see citing works
CodeS: Towards Code Model Generalization Under Distribution Shift
Hu, Q., Guo, Y., Xie, X., Cordy, M., Papadakis, M., Ma, L., & Traon, Y. L. (2023) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier58687.2023.00007 · Cited by 16 · see citing works
A Context-Aware Neural Embedding for Function-Level Vulnerability Detection
Wei, H., Lin, G., Li, L., & Jia, H. (2021) · Algorithms · doi:10.3390/a14110335 · Cited by 15 · see citing works
Machine Learning Method with Applications in Hardware Security of Post-Quantum Cryptography
Yi, H. (2023) · Journal of Grid Computing · doi:10.1007/s10723-023-09643-4 · Cited by 14 · see citing works
Leveraging Unsupervised Learning to Summarize APIs Discussed in Stack Overflow
Naghshzan, A., Guerrouj, L., & Baysal, O. (2021) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam52516.2021.00026 · Cited by 14 · see citing works
edgeTrans - Edge transport mode detection
Ferreira, P., Zavgorodnii, C., & Veiga, L. (2020) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2020.101268 · Cited by 14 · see citing works
Assessing and Analyzing the Correctness of GitHub Copilot's Code Suggestions
Mo, R., Wang, D., Zhan, W., Jiang, Y., Wang, Y., Zhao, Y., Li, Z., & Ma, Y. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715108 · Cited by 11 · see citing works
ProgramTransformer: A tool for generating semantically equivalent transformed programs
Rabin, M. R. I., & Alipour, M. A. (2022) · Software Impacts · doi:10.1016/j.simpa.2022.100429 · Cited by 11 · see citing works
Naturalness in Source Code Summarization. How Significant is it?
Ferretti, C., & Saletta, M. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00027 · Cited by 11 · see citing works
Neural Networks for Modeling Source Code EditsPreprint
Zhao, R. T., Bieber, D., Swersky, K., & Tarlow, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02818 · Cited by 11 · see citing works
Automatic Code Generation using Pre-Trained Language ModelsPreprint
Perez, L., Ottens, L., & Viswanathan, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10535 · Cited by 21 · see citing works
Self-Supervised Learning to Prove Equivalence Between Straight-Line Programs via Rewrite Rules
Kommrusch, S., Monperrus, M., & Pouchet, L.-N. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3271065 · Cited by 9 · see citing works
Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion
Pei, H., Zhao, J., Lausen, L., Zha, S., & Karypis, G. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25653 · Cited by 10 · see citing works
Towards Code Generation from BDD Test Case Specifications: A Vision
Chemnitz, L., Reichenbach, D., Aldebes, H., Naveed, M., Narasimhan, K., & Mezini, M. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00031 · Cited by 11 · see citing works
Unsupervised Classifying of Software Source Code Using Graph Neural Networks
Vytovtov, P., & Chuvilin, K. V. (2019) · Conference of Open Innovations Association (FRUCT) · doi:10.23919/fruct.2019.8711909 · Cited by 10 · see citing works
Learning the Relation Between Code Features and Code Transforms With Structured Prediction
Yu, Z., Martínez, M., Chen, Z., Bissyandé, T. F., & Monperrus, M. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3275380 · Cited by 10 · see citing works
A hybrid code representation learning approach for predicting method names
Zhang, F., Chen, B., Li, R., & Peng, X. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111011 · Cited by 10 · see citing works
Slice-Based Code Change Representation Learning
Zhang, F., Chen, B., Zhao, Y., & Peng, X. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00038 · Cited by 9 · see citing works
HoneyCode: Automating Deceptive Software Repositories with Deep Generative Models
Nguyen, D. D., Liebowitz, D., Nepal‬, ‪., & Kanhere, S. S. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.835 · Cited by 9 · see citing works
Distilled and Contextualized Neural Models Benchmarked for Vulnerable Function Detection
Lin, G., Jia, H., & Wu, D. (2022) · Mathematics · doi:10.3390/math10234482 · Cited by 8 · see citing works
Graph Confident Learning for Software Vulnerability Detection
Wang, Q., Li, Z., Liang, H., Pan, X., Li, H., Li, T., Li, X., Li, T., & Guo, S. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108296 · Cited by 8 · see citing works
A Review of Learning-based Smart Contract Vulnerability Detection: A Perspective on Code Representation
Wang, B., Tong, Y., Ji, S., Dong, H., Luo, X., & Zhang, P. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3750042 · Cited by 8 · see citing works
Understanding security vulnerabilities in student code: A case study in a non-security course
Yilmaz, T., & Ulusoy, Ö. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111150 · Cited by 8 · see citing works
GRAPHSPY: Fused Program Semantic Embedding through Graph Neural Networks for Memory Efficiency
Guo, Y., Li, P., Luo, Y., Wang, X., & Wang, Z. (2021) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586120 · Cited by 8 · see citing works
Finding associations between natural and computer languages: A case-study of bilingual LDA applied to the bleeping computer forum posts
Yao, K., Oliva, G. A., Hassan, A. E., Asaduzzaman, M., Malton, A. J., & Walenstein, A. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111651 · Cited by 8 · see citing works
Learning GraphQL Query Cost
Mavroudeas, G., Baudart, G., Cha, A., Hirzel, M. J., Laredo, J., Magdon‐Ismail, M., Mandel, L., & Wittern, E. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678513 · Cited by 7 · see citing works
SF6 High-Voltage Circuit Breaker Contact Status Detection at Different Currents
Guo, Z., Li, L., Han, W., & Guo, Z. (2022) · Sensors · doi:10.3390/s22218490 · Cited by 7 · see citing works
Graph Neural Networks Based Memory Inefficiency Detection Using Selective Sampling
Li, P., Guo, Y., Luo, Y., Wang, X., Wang, Z., & Liu, X. (2022) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41404.2022.00090 · Cited by 7 · see citing works
Implant Global and Local Hierarchy Information to Sequence based Code Representation Models
Zhang, K., Li, Z., Jin, Z. G., & Li, G. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00030 · Cited by 7 · see citing works
On the effectiveness of hybrid pooling in mixup-based graph learning for language processing
Dong, Z., Hu, Q., Zhang, Z., Guo, Y., Cordy, M., Papadakis, M., Traon, Y. L., & Zhao, J. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112139 · Cited by 6 · see citing works
Test-based and metric-based evaluation of code generation models for practical question answering
Kovalchuk, S. V., Fedrushkov, D., Lomshakov, V., & Aliev, A. (2023) · International Conference on Code Quality (ICCQ) · doi:10.1109/iccq57276.2023.10114665 · Cited by 6 · see citing works
A survey of coverage-guided greybox fuzzing with deep neural models
Qiu, J., Jiang, Y., Miao, Y., Luo, W., Pan, L., & Zheng, X. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107797 · Cited by 6 · see citing works
A Framework for Real-Time AI-Driven Secure Code Analysis Integrated with DevSecOps in Cloud-Native CI/CD Pipelines
Senior DevOps Engineer — Cloud, D. A. A., & Kummarapurugu, C. S. (2024) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem17273 · Cited by 5 · see citing works
On Representation Learning-based Methods for Effective, Efficient, and Scalable Code Retrieval
França, C., Lima, R. C., de Andrade, C. M. V., da Cunha, W. L. M., de Melo, P. O. S. V., Ribeiro-Neto, B. A., da Rocha, L. C. D., Santos, R. L. T., Pagano, A. S., & Gonçalves, M. A. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128172 · Cited by 5 · see citing works
Automatic Acquisition of Annotated Training Corpora for Test-Code Generation
Kacmajor, M., & Kelleher, J. D. (2019) · Information · doi:10.3390/info10020066 · Cited by 4 · see citing works
Deep Code Search with Naming-Agnostic Contrastive Multi-View Learning
Feng, J., Li, W., Wu, S., Wei, Z., Xu, Y., Wang, J., & Li, H. (2025) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3737878 · Cited by 4 · see citing works
Can Code Representation Boost IR-Based Test Case Prioritization?
Yang, L., Chen, J., You, H., Han, J., Jiang, J., Sun, Z., Lin, X., Liang, F., & Kang, Y. (2023) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre59848.2023.00077 · Cited by 4 · see citing works
Towards semantically enhanced detection of emerging quality-related concerns in source code
Krasniqi, R., & Do, H. (2023) · Software Quality Journal · doi:10.1007/s11219-023-09614-8 · Cited by 5 · see citing works
Real-time pedestrian detection method based on improved YOLOv3
Luo, J., Wang, Y., & Wang, Y. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1453/1/012149 · Cited by 4 · see citing works
Predictive Analytics in IT Service Management (ITSM)
Sharon, C. I. L., & Suma, V. (2022) · doi:10.1002/9781119792529.ch7 · Cited by 4 · see citing works
Gated Graph Neural Networks (GG-NNs) for Abstractive Multi-Comment Summarization
Zhan, H., Zhang, K., Hu, C., & Sheng, V. S. (2021) · IEEE International Conference on Big Knowledge (ICBK) · doi:10.1109/ickg52313.2021.00050 · Cited by 3 · see citing works
DG-Trans: Automatic Code Summarization via Dynamic Graph Attention-based Transformer
Zeng, J., Zhang, T., & Zhou, X. (2021) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs54544.2021.00088 · Cited by 3 · see citing works
Combining Code Embedding with Static Analysis for Function-Call CompletionPreprint
Weyssow, M., Sahraoui, H. A., Frénay‬, B., & Vanderose, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03731 · Cited by 3 · see citing works
Enhancing Code LLM Training with Programmer Attention
Zhang, Y., Huang, C., Karas, Z., Nguyen, T. D., Leach, K., & Huang, Y. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728510 · Cited by 5 · see citing works
Feature Execution Graphs: A Human-AI Co-Programming Paradigm for Graph-Driven LLM Code Synthesis
Batatia, H., & Svetlichnyi, I. (2025) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v6i1.36052 · Cited by 2 · see citing works
Collaborative Causal Inference and Multi-Agent Dynamic Intervention for “Dual Carbon” Public Opinion Driven by Reinforced Large Language Models and Diffusion Models
Chen, X. (2025) · Systems · doi:10.3390/systems13080689 · Cited by 2 · see citing works
The fools are certain; the wise are doubtful: Exploring LLM confidence in code completion
Kotti, Z., Dritsa, K., Spinellis, D. D., & Λουρίδας, Π. (2026) · Automated Software Engineering · doi:10.21203/rs.3.rs-7302683/v1 · Cited by 2 · see citing works
Evolutionary Approaches for Adversarial Attacks on Neural Source Code Classifiers
Mercuri, V., Saletta, M., & Ferretti, C. (2023) · Algorithms · doi:10.3390/a16100478 · Cited by 2 · see citing works
Statement-Grained Hierarchy Enhanced Code Summarization
Zhang, Q., Jin, D., Wang, Y., & Gong, Y. (2024) · Electronics · doi:10.3390/electronics13040765 · Cited by 2 · see citing works
Study of Distractors in Neural Models of Code
Rabin, M. R. I., Hussain, A., Suneja, S., & Alipour, M. A. (2023) · doi:10.1109/intense59150.2023.00005 · Cited by 3 · see citing works
METHOD FOR SEMANTICALLY CORRECT CODE GENERATION FOR FUZZING TESTING JAVASCRIPT ENGINES
(2023) · Voprosy kiberbezopasnosti · doi:10.21681/2311-3456-2023-5-80-88 · Cited by 2 · see citing works

Authors: Allamanis, M., Barr, E. T., Devanbu, P., & Sutton, C.
Affiliations: Microsoft Research (United Kingdom) (GB)
University College London (GB)
University of California, Davis, California, USA
The Alan Turing Institute (GB)
ORCID: Allamanis, Miltiadis
Barr, Earl T.
Sutton, Charles
Venue: ACM Computing Surveys
Publisher: Association for Computing Machinery (ACM)
Published: 2018-07-31
Volume: 51
Issue: 4
Pages: 1-37
ISSN: 0360-0300 (ISSN-L)
1557-7341
DOI: 10.1145/3212695
OpenAlex: W2754629507
Cited by: 150
References: 186
Citations by year: 2026: 27 · 2025: 11 · 2024: 7 · 2023: 33 · 2022: 19 · 2021: 19 · 2020: 10 · 2019: 15 · 2018: 8 · 2017: 1
FWCI: 20.98
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: ACM Computing Surveys (publishedVersion)
arXiv (Cornell University) (submittedVersion)
UCL Discovery (University College London) (submittedVersion)
Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, other-oa)
arXiv (Cornell University)
Funders: Engineering and Physical Sciences Research Council (EP/K024043/1, EP/P005659/1, and EP/P005314/1)
National Science Foundation (1414172)
Topics: Physical Sciences › Computer Science › Information Systems › Software Engineering Research
Physical Sciences › Computer Science › Artificial Intelligence › Logic, programming, and type systems
Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords: probabilistic programming languages, naturalness, software engineering applications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Survey of Machine Learning for Big Code and Naturalness

Abstract

Integrity signals

Claims the paper makes

Self-check

Cites

Cited by

Related

Record

About these checks