On the naturalness of software
The DOI matches its Crossref record; 38 of the 39 references it cites were found in a registry or an index.
Abstract
Natural languages like English are rich, complex, and powerful. The highly creative and graceful use of languages like English and Tamil, by masters like Shakespeare and Avvaiyar, can certainly delight and inspire. But in practice, given cognitive constraints and the exigencies of daily life, most human utterances are far simpler and much more repetitive and predictable. In fact, these utterances can be very usefully modeled using modern statistical methods. This fact has led to the phenomenal success of statistical approaches to speech recognition, natural language translation, question-answering, and text mining and comprehension. We begin with the conjecture that most software is also natural, in the sense that it is created by humans at work, with all the attendant constraints and limitations - and thus, like natural language, it is also likely to be repetitive and predictable. We then proceed to ask whether a) code can be usefully modeled by statistical language models and b) such models can be leveraged to support software engineers. Using the widely adopted n-gram model, we provide empirical evidence supportive of a positive answer to both these questions. We show that code is also very repetitive, and in fact even more so than natural languages. As an example use of the model, we have developed a simple code completion engine for Java that, despite its simplicity, already improves Eclipse's built-in completion capability. We conclude the paper by laying out a vision for future research in this area.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The IntelliCode Compose system uses the term click through rate (CTR) for this and reports a value of 10% in online trials.20”
From Measuring GitHub Copilot's Impact on Productivity (2024) · cites this paper as [20]
In the citing paper
“It's been well-known right from the outset that language modeling for code has to deal with project-specific phenomena [10,11,22].”
From Few-shot training LLMs for project-specific code-summarization (2022) · cites this paper as [11]
Citations
Cites
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Chi, E. H., & Suh, B. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,923
Recovering traceability links between code and documentation
Antoniol, G., Canfora, G., Casazza, G., De Lucia, A., & Merlo, E. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1041053 · Cited by 967
Statistical Machine Translation
Koehn, P. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815829 · Cited by 871
Towards automatically generating summary comments for Java methods
Sridhara, G., Hill, E., Muppaneni, D., Pollock, L., & Vijay-Shanker, K. (2010) · Proceedings of the IEEE/ACM international conference on Automated software engineering · doi:10.1145/1858996.1859006 · Cited by 514
Jungloid mining: helping to navigate the API jungle
Mandelin, D., Xu, L., Bodík, R., & Kimelman, D. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065018 · Cited by 388
Learning from examples to improve code completion systems
Bruch, M., Monperrus, M., & Mezini, M. (2009) · Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering · doi:10.1145/1595696.1595728 · Cited by 378
Mining version histories to guide software changes
Zimmermann, T., Weibgerber, P., Diehl, S., & Zeller, A. (2004) · Proceedings. 26th International Conference on Software Engineering · doi:10.1109/icse.2004.1317478 · Cited by 322
Statistical language model adaptation: review and perspectives
Bellegarda, J. R. (2004) · Speech Communication · doi:10.1016/j.specom.2003.08.002 · Cited by 297
Cited by
Items 301–400 of 834
A Simple, Efficient, Context‐sensitive Approach for Code Completion
Asaduzzaman, M., Roy, C. K., Schneider, K. A., & Hou, D. (2016) · Journal of Software Evolution and Process · doi:10.1002/smr.1791 · Cited by 26
Are mutants really natural? a study on how "naturalness" helps mutant selection
Jimenez, M., Checkam, T. T., Cordy, M., Papadakis, M., Kintis, M., Traon, Y. L., & Harman, M. (2018) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3239235.3240500 · Cited by 25
Defect Prediction via LSTM Based on Sequence and Tree Structure
Zhou, X., & Lu, L. (2020) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs51102.2020.00055 · Cited by 25
Learning Scalable and Precise Representation of Program SemanticsPreprint
Wang, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05251 · Cited by 25
Towards demystifying dimensions of source code embeddings
Rabin, M. R. I., Mukherjee, A., Gnawali, O. D., & Alipour, M. A. (2020) · doi:10.1145/3416506.3423580 · Cited by 24
MetaTPTrans: A Meta Learning Approach for Multilingual Code Representation Learning
Pian, W., Peng, H., Tang, X., Sun, T., Tian, H., Habib, A., Klein, J., & Bissyandé, T. F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25654 · Cited by 24
PTM-APIRec: Leveraging Pre-trained Models of Source Code in API Recommendation
Li, Z., Li, C., Tang, Z. J., Huang, W., Ge, J., Luo, B., Ng, V., Wang, T., Hu, Y., & Zhang, X. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3632745 · Cited by 24
Programming with "Big Code": Lessons, Techniques and Applications
Bielik, P., Raychev, V., & Vechev, M. (2015) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.snapl.2015.41 · Cited by 24
LMDet: A “Naturalness” Statistical Method for Hardware Trojan Detection
Shen, H., Tan, H., Li, H., Zhang, F., & Li, X. (2017) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2017.2781423 · Cited by 24
Learning lenient parsing & typing via indirect supervision
Ahmed, T., Devanbu, P., & Hellendoorn, V. J. (2021) · Empirical Software Engineering · doi:10.1007/s10664-021-09942-y · Cited by 24
Self-citationOn the Quality of Identifiers in Test Code
Lin, B., Nagy, C., Bavota, G., Marcus, A., & Lanza, M. (2019) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam.2019.00031 · Cited by 24
Semantics-aware obfuscation scheme prediction for binary
Zhao, Y., Tang, Z., Ye, G., Peng, D., Fang, D., Chen, X., & Wang, Z. (2020) · Computers & Security · doi:10.1016/j.cose.2020.102072 · Cited by 24
SinkFinder: harvesting hundreds of unknown interesting function pairs with just one seed
Bian, P., Liang, B., Huang, J., Shi, W., Wang, X., & Zhang, J. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409678 · Cited by 24
Roundtable: What's Next in Software Analytics
Hassan, A. E., Hindle, A., Runeson, P., Shepperd, M., Dévanbu, P., & Kim, S. (2013) · IEEE Software · doi:10.1109/ms.2013.85 · Cited by 24
Self-citationModeling Vocabulary for Big Code Machine LearningPreprint
Babii, H., Janes, A., & Robbes, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01873 · Cited by 24
Learning API Usages from Bytecode: A Statistical ApproachPreprint
Nguyen, T. T., Pham, H. V., Vu, P. M., & Nguyen, T. T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.07306 · Cited by 24
Towards Full-line Code Completion with Neural Language ModelsPreprint
Wang, W., Shen, S., Li, G., & Jin, Z. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.08603 · Cited by 24
Code-line-level Bugginess Identification: How Far have We Come, and How Far have We Yet to Go?
Guo, Z., Liu, S., Liu, X., Lai, W., Ma, M., Zhang, X., Ni, C., Yang, Y., Li, Y., Chen, L., et al. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3582572 · Cited by 23
Generative API usage code recommendation with parameter concretization
Chen, C., Peng, X., Sun, J., Xing, Z., Wang, X., Zhao, Y., Zhang, H., & Zhao, W. (2019) · Science China Information Sciences · doi:10.1007/s11432-018-9821-9 · Cited by 23
A Source Code Similarity Based on Siamese Neural Network
Xie, C., Wang, X., Qian, C., & Mengqi, W. (2020) · Applied Sciences · doi:10.3390/app10217519 · Cited by 23
FRUGAL: Unlocking Semi-Supervised Learning for Software Analytics
Tu, H., & Menzies, T. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678617 · Cited by 23
Abstraction refinement guided by a learnt probabilistic model
Grigore, R., & Yang, H. (2016) · ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL) · doi:10.1145/2837614.2837663 · Cited by 23
Non-Autoregressive Line-Level Code Completion
Liu, F., Fu, Z., Li, G., Jin, Z., Liu, H., Hao, Y., & Zhang, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3649594 · Cited by 22
Leveraging structural properties of source code graphs for just-in-time bug prediction
Nadim, M., Mondal, D., & Roy, C. K. (2022) · Automated Software Engineering · doi:10.1007/s10515-022-00326-0 · Cited by 22
A Convolutional Neural Network for Language-Agnostic Source Code Summarization
Moore, J., Gelman, B. U., & Slater, D. (2019) · International Conference on Evaluation of Novel Approaches to Software Engineering · doi:10.5220/0007678100150026 · Cited by 22
Recovering Variable Names for Minified Code with Usage Contexts
Tran, H., Tran, N., Nguyen, S., Nguyen, H. A., & Nguyen, T. N. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00119 · Cited by 22
Suggesting Comment Completions for Python using Neural Language Models
Ciurumelea, A., Proksch, S., & Gall, H. C. (2020) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner48275.2020.9054866 · Cited by 22
Wen, F., Aghajani, E., Nagy, C., Lanza, M., & Bavota, G. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00025 · Cited by 22
Plagiarism Detection in Programming Assignments Using Deep Features
Yasaswi, J., Purini, S., & Jawahar, C. V. (2017) · Asian Conference on Pattern Recognition · doi:10.1109/acpr.2017.146 · Cited by 22
Slice-Based Cognitive Complexity Metrics for Defect Prediction
Alqadi, B. S., & Maletic, J. I. (2020) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner48275.2020.9054836 · Cited by 21
Semantic topic models for source code analysis
Mahmoud, A., & Bradshaw, G. L. (2016) · Empirical Software Engineering · doi:10.1007/s10664-016-9473-1 · Cited by 21
Toward Semantic Foundations for Program EditorsPreprint
Omar, C., Voysey, I., Hilton, M. L., Sunshine, J. S., Goues, C. L., Aldrich, J., & Hammer, M. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.08694 · Cited by 21
Ruiz, F. V., Grishina, A., Hort, M., & Moonen, L. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3771922 · Cited by 20
When less is more: on the value of “co-training” for semi-supervised software defect predictors
Majumder, S., Chakraborty, J., & Menzies, T. (2024) · Empirical Software Engineering · doi:10.1007/s10664-023-10418-4 · Cited by 20
An Adaptive Language-Agnostic Pruning Method for Greener Language Models for Code
Saad, M., López, J. A. H., Chen, B., Varró, D., & Sharma, T. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715773 · Cited by 20
SCStudio: a secure and efficient integrated development environment for smart contracts
Ren, M., Ma, F., Yin, Z., Li, H., Fu, Y., Chen, T., & Jiang, Y. (2021) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3460319.3469078 · Cited by 20
pycefr: Python competency level through code analysis
Robles, G., Kula, R. G., Ragkhitwetsagul, C., Sakulniwat, T., Matsumoto, K., & Gonzalez-Barahona, J. M. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527878 · Cited by 20
An Empirical Study of End-User Programmers in the Computer Music Community
Burlet, G., & Hindle, A. (2015) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2015.34 · Cited by 20
Self-citationDeep Learning to Detect Redundant Method CommentsPreprint
Louis, A. P., Dash, S. K., Barr, E. T., & Sutton, C. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.04616 · Cited by 20
Self-citationPyTorrent: A Python Library Corpus for Large-scale Language ModelsPreprint
Bahrami, M., Shrikanth, N. C., Ruangwan, S., Liu, L., Mizobuchi, Y., Fukuyori, M., Chen, W., Munakata, K., & Menzies, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.01710 · Cited by 20
Pre-training Code Representation with Semantic Flow Graph for Effective Bug Localization
Du, Y., & Yu, Z. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616338 · Cited by 19
CORAL: COde RepresentAtion learning with weakly-supervised transformers for analyzing data analysis
Zhang, G., Merrill, M. A., Liu, Y., Heer, J., & Althoff, T. (2022) · EPJ Data Science · doi:10.1140/epjds/s13688-022-00327-9 · Cited by 19
A language model for statements of software code
Yang, Y., Jiang, Y., Gu, M., Sun, J., Gao, J., & Liu, H. (2017) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115678 · Cited by 19
Statistical Translation of English Texts to API Code Templates
Nguyen, A. T., Rigby, P. C., Nguyen, T. V., Palani, D., Karanfil, M., & Nguyen, T. N. (2018) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2018.00029 · Cited by 19
A Context-Aware Neural Embedding for Function-Level Vulnerability Detection
Wei, H., Lin, G., Li, L., & Jia, H. (2021) · Algorithms · doi:10.3390/a14110335 · Cited by 19
How to Build a Recommendation System for Software Engineering
Proksch, S., Bauer, V., & Murphy, G. C. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-28406-4_1 · Cited by 19
Smart Contracts Vulnerability Auditing with Multi-semantics
Yang, Z., Keung, J., Zhang, M., Xiao, Y., Huang, Y., & Hui, T. (2020) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac48688.2020.0-153 · Cited by 19
Rahman, M., Khatoonabadi, S., Abdellatif, A., & Shihab, E. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.01382 · Cited by 19
Toward a Theory of Causation for Interpreting Neural Code Models
Nader Palacio, D., Velasco, A., Cooper, N., Rodriguez, A., Moran, K., & Poshyvanyk, D. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3379943 · Cited by 18
Wang, C., Zhang, J., Wu, R., & Zhang, C. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660816 · Cited by 18
Classifying Web Exploits with Topic Modeling
Ruohonen, J. (2017) · International Workshop on Database and Expert Systems Applications (DEXA) · doi:10.1109/dexa.2017.35 · Cited by 18
An Empirical Investigation into the Use of Image Captioning for Automated Software Documentation
Moran, K., Yachnes, A., Purnell, G., Mahmud, J., Tufano, M., Cardenas, C. B., Poshyvanyk, D., & H'Doubler, Z. (2022) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner53432.2022.00069 · Cited by 18
Martin, J., & Guo, J. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527872 · Cited by 18
Do Programmers Prefer Predictable Expressions in Code?
Casalnuovo, C., Lee, K., Wang, H., Dévanbu, P., & Morgan, E. (2020) · Cognitive Science · doi:10.1111/cogs.12921 · Cited by 18
Configuring topic models for software engineering tasks in TraceLab
Dit, B., Panichella, A., Moritz, E., Oliveto, R., Penta, M. D., Poshyvanyk, D., & Lucia, A. D. (2013) · International Workshop on Traceability in Emerging Forms of Software Engineering (TEFSE) · doi:10.1109/tefse.2013.6620164 · Cited by 18
LAMNER: code comment generation using character language model and named entity recognition
Sharma, R., Chen, F., & Fard, F. H. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527924 · Cited by 17
Chen, T.-H. P., Thomas, S. W., Hemmati, H., Nagappan, M., & Hassan, A. E. (2017) · IEEE Transactions on Reliability · doi:10.1109/tr.2017.2699938 · Cited by 17
Graph-based Vulnerability Detection via Extracting Features from Sliced Code
Wu, P., Yin, L., Du, X., Jia, L., & Dong, W. (2020) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c51114.2020.00018 · Cited by 17
Khalilian, A., Baraani-Dastjerdi, A., & Zamani, B. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114503 · Cited by 17
Predicting Future Developer Behavior in the IDE Using Topic Models
Damevski, K., Chen, H., Shepherd, D. C., Kraft, N. A., & Pollock, L. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2748134 · Cited by 17
Comparative Code Structure Analysis using Deep Learning for Performance Prediction
Ramadan, T., Islam, T. Z., Phelps, C., Pinnow, N., & Thiagarajan, J. J. (2021) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass51385.2021.00032 · Cited by 17
Large-Scale Code Clone Detection
Sajnani, H. (2016) · eScholarship (California Digital Library) · Cited by 17
Tailored Mutants Fit Bugs BetterPreprint
Allamanis, M., Barr, E. T., Just, R., & Sutton, C. A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02516 · Cited by 17
Self-citationSearch-Based Software Re-Modularization: A Case Study at Adyen
Schröder, C., van der Feltz, A., Panichella, A., & Aniche, M. F. (2021) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip52600.2021.00017 · Cited by 16
Do bugs lead to unnaturalness of source code?
Jiang, Y., Liu, H., Zhang, Y., Ji, W., Zhong, H., & Zhang, L. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549149 · Cited by 16
Towards Objective-Tailored Genetic Improvement Through Large Language Models
Kang, S., & Yoo, S. (2023) · IEEE/ACM International Workshop on Genetic Improvement (GI) · doi:10.1109/gi59320.2023.00013 · Cited by 16
Xiong, Y., Wang, B., Fu, G., & Zang, L. (2018) · doi:10.1145/3194810.3194816 · Cited by 16
Achievements, Failures, and the Future of Model-Based Software Engineering
Kautz, O., Roth, A., & Rumpe⋆, B. (2018) · Springer eBooks · doi:10.1007/978-3-319-73897-0_13 · Cited by 16
Syntax and Stack Overflow: A Methodology for Extracting a Corpus of Syntax Errors and Fixes
Wong, A. W., Salimi, A., Chowdhury, S., & Hindle, A. (2019) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2019.00048 · Cited by 16
Self-citationJSAC: A Novel Framework to Detect Malicious JavaScript via CNNs over AST and CFG
Liang, H., Yang, Y., Sun, L., & Jiang, L. C. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851760 · Cited by 16
A Hidden Markov Model to detect coded information islands in free text
Cerulo, L., Ceccarelli, M., Penta, M. D., & Canfora, G. (2013) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam.2013.6648197 · Cited by 16
Learning code summarization from a small and local datasetPreprint
Ahmed, T., & Dévanbu, P. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.00804 · Cited by 16
Mavroudeas, G., Baudart, G., Cha, A., Hirzel, M. J., Laredo, J., Magdon‐Ismail, M., Mandel, L., & Wittern, E. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678513 · Cited by 15
Fusing multi-abstraction vector space models for concern localization
Zhang, Y., Lo, D., Xia, X., Scanniello, G., Le, T.-D. B., & Sun, J. (2017) · Empirical Software Engineering · doi:10.1007/s10664-017-9585-2 · Cited by 15
Adaptive User Feedback for IR-based Traceability Recovery
Panichella, A., Lucia, A. D., & Zaidman, A. (n.d.) · Cited by 15
Data-Driven Program CompletionPreprint
Lu, Y., Chaudhuri, S., Jermaine, C., & Melski, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.09042 · Cited by 15
(2023) · doi:10.1109/ase56229.2023.00157 · Cited by 14
Do Contexts Help in Phrase-Based, Statistical Source Code Migration?
Nguyen, A. T., Tu, Z., & Nguyen, T. N. (2016) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2016.89 · Cited by 14
On the Naturalness and Localness of Software Logs
Gholamian, S., & Ward, P. A. S. (2021) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00028 · Cited by 14
Reducing Feedback Delay of Software Development Tools via Continuous Analysis
Muşlu, K., Brun, Y., Ernst, M. D., & Notkin, D. S. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2015.2417161 · Cited by 14
A Deep Learning Model for Source Code Generation
Tiwang, R., Oladunni, T., & Xu, W. (2019) · IEEE SoutheastCon · doi:10.1109/southeastcon42311.2019.9020360 · Cited by 14
Towards Learning Generalizable Code Embeddings Using Task-agnostic Graph Convolutional Networks
Ding, Z., Li, H., Shang, W., & Chen, T.-H. P. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3542944 · Cited by 14
Software Defect Prediction via GCN based on Structural and Context Information
Tang, L., Tao, C., Guo, H., & Zhang, J. (2022) · International Conference on Dependable Systems and Their Applications (DSA) · doi:10.1109/dsa56465.2022.00049 · Cited by 14
The Spartanizer: Massive automatic refactoring
Gil, Y., & Orrù, M. (2017) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2017.7884657 · Cited by 14
CodeSum: Translate Program Language to Natural LanguagePreprint
Hu, X., Wei, Y., Li, G., & Jin, Z. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.01837 · Cited by 14
Neural Networks for Modeling Source Code EditsPreprint
Zhao, R. T., Bieber, D., Swersky, K., & Tarlow, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02818 · Cited by 14
Program Classification Using Gated Graph Attention Neural Network for Online Programming ServicePreprint
Lu, M., Tan, D., Xiong, N. N., Chen, Z., & Li, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03804 · Cited by 14
Assessing and Improving Syntactic Adversarial Robustness of Pre-trained Models for Code TranslationPreprint
Yang, G., Zhou, Y., Zhang, X., Chen, X., Han, T., & Chen, T. (2023) · BIROn (Birkbeck, University of London) · doi:10.48550/arxiv.2310.18587 · Cited by 14
SmartPaste: Learning to Adapt Source Code
Allamanis, M., & Brockschmidt, M. (2017) · arXiv · doi:10.48550/arxiv.1705.07867 · Cited by 13
“Bhoopchand et al. [6], Hindle et al. [10] model the code as a sequence of tokens, while Maddison and Tarlow [11], Raychev et al. [17] model the syntax tree structure of code.”
Aging-Related Bugs Prediction Via Convolutional Neural Network
Liu, Q., Xiang, J., Xu, B., Zhao, D., Hu, W., & Wang, J. (2020) · International Conference on Dependable Systems and Their Applications (DSA) · doi:10.1109/dsa51864.2020.00020 · Cited by 13
“More Than Deep Learning”: post-processing for API sequence recommendation
Chen, C., Peng, X., Chen, B., Sun, J., Xing, Z., Wang, X., & Zhao, W. (2021) · Empirical Software Engineering · doi:10.1007/s10664-021-10040-2 · Cited by 13
Self-Supervised Learning to Prove Equivalence Between Straight-Line Programs via Rewrite Rules
Kommrusch, S., Monperrus, M., & Pouchet, L.-N. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3271065 · Cited by 13
On the Applicability of Language Models to Block-Based Programs
Griebl, E., Fein, B., Obermüller, F., Fraser, G., & Just, R. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00199 · Cited by 13
Diera, A., Dahou, A. H., Galke, L. P. A., Karl, F., Sihler, F., & Scherp, A. (2023) · GenBench Workshop on Generalisation (Benchmarking) in NLP (GenBench) · doi:10.18653/v1/2023.genbench-1.2 · Cited by 13
SLAMPA: Recommending Code Snippets with Statistical Language Model
Zhou, S., Zhong, H., & Shen, B. (2018) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec.2018.00022 · Cited by 13
Universal Representation for Code
Liu, L., Nguyen, H. A., Karypis, G., & Sengamedu, S. H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-75768-7_2 · Cited by 13
A hybrid code representation learning approach for predicting method names
Zhang, F., Chen, B., Li, R., & Peng, X. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111011 · Cited by 13
Cross-Dataset Design Discussion Mining
Mahadi, A., Tongay, K., & Ernst, N. (2020) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner48275.2020.9054792 · Cited by 13
Evaluating Vulnerability Prediction Models
Jimenez, M. (2018) · Open Repository and Bibliography (University of Luxembourg) · Cited by 13
Metadata
- Authors
- Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Devanbu, P.
- Affiliations
- University of California, Davis (US)
- The University of Texas at Dallas (US)
- Venue
- 2012 34th International Conference on Software Engineering (ICSE)
- Publisher
- IEEE
- Published
- 2012-06
- Pages
- 837-847
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W3146720657
- Cited by
- 425
- References
- 39
- Citations by year
- 2026: 19 · 2025: 24 · 2024: 36 · 2023: 44 · 2022: 49 · 2021: 50 · 2020: 48 · 2019: 46 · 2018: 22 · 2017: 35 · 2016: 13 · 2015: 23 · 2014: 2 · 2013: 14
- FWCI
- 9.81
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- statistical language models, n-gram language models, code completion, Java, eclipses, software engineering support, natural language modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.