On the naturalness of software
The DOI matches its Crossref record; 38 of the 39 references it cites were found in a registry or an index.
Abstract
Natural languages like English are rich, complex, and powerful. The highly creative and graceful use of languages like English and Tamil, by masters like Shakespeare and Avvaiyar, can certainly delight and inspire. But in practice, given cognitive constraints and the exigencies of daily life, most human utterances are far simpler and much more repetitive and predictable. In fact, these utterances can be very usefully modeled using modern statistical methods. This fact has led to the phenomenal success of statistical approaches to speech recognition, natural language translation, question-answering, and text mining and comprehension. We begin with the conjecture that most software is also natural, in the sense that it is created by humans at work, with all the attendant constraints and limitations - and thus, like natural language, it is also likely to be repetitive and predictable. We then proceed to ask whether a) code can be usefully modeled by statistical language models and b) such models can be leveraged to support software engineers. Using the widely adopted n-gram model, we provide empirical evidence supportive of a positive answer to both these questions. We show that code is also very repetitive, and in fact even more so than natural languages. As an example use of the model, we have developed a simple code completion engine for Java that, despite its simplicity, already improves Eclipse's built-in completion capability. We conclude the paper by laying out a vision for future research in this area.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The IntelliCode Compose system uses the term click through rate (CTR) for this and reports a value of 10% in online trials.20”
From Measuring GitHub Copilot's Impact on Productivity (2024) · cites this paper as [20]
In the citing paper
“It's been well-known right from the outset that language modeling for code has to deal with project-specific phenomena [10,11,22].”
From Few-shot training LLMs for project-specific code-summarization (2022) · cites this paper as [11]
Citations
Cites
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Chi, E. H., & Suh, B. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,923
Recovering traceability links between code and documentation
Antoniol, G., Canfora, G., Casazza, G., De Lucia, A., & Merlo, E. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1041053 · Cited by 967
Statistical Machine Translation
Koehn, P. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815829 · Cited by 871
Towards automatically generating summary comments for Java methods
Sridhara, G., Hill, E., Muppaneni, D., Pollock, L., & Vijay-Shanker, K. (2010) · Proceedings of the IEEE/ACM international conference on Automated software engineering · doi:10.1145/1858996.1859006 · Cited by 514
Jungloid mining: helping to navigate the API jungle
Mandelin, D., Xu, L., Bodík, R., & Kimelman, D. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065018 · Cited by 388
Learning from examples to improve code completion systems
Bruch, M., Monperrus, M., & Mezini, M. (2009) · Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering · doi:10.1145/1595696.1595728 · Cited by 378
Mining version histories to guide software changes
Zimmermann, T., Weibgerber, P., Diehl, S., & Zeller, A. (2004) · Proceedings. 26th International Conference on Software Engineering · doi:10.1109/icse.2004.1317478 · Cited by 322
Statistical language model adaptation: review and perspectives
Bellegarda, J. R. (2004) · Speech Communication · doi:10.1016/j.specom.2003.08.002 · Cited by 297
Cited by
Items 201–300 of 834
A Screening Test for Disclosed Vulnerabilities in FOSS Components
Dashevskyi, S., Brucker, A. D., & Massacci, F. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2816033 · Cited by 46
Deep Representation Learning for Code Smells Detection using Variational Auto-Encoder
Hadj-Kacem, M., & Bouassida, N. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851854 · Cited by 45
Ding, Z., Li, H., Shang, W., & Chen, T.-H. P. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10118-5 · Cited by 45
LoGenText: Automatically Generating Logging Texts Using Neural Machine Translation
Ding, Z., Li, H., & Shang, W. (2022) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner53432.2022.00051 · Cited by 45
Neural Attribute Machines for Program GenerationPreprint
Amodio, M., Chaudhuri, S., & Reps, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.09231 · Cited by 45
Analysis of issue and dependency management in open-source software projects
Kikas, R. (2018) · DSpace repository (University of Tartu) · Cited by 45
Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages
POPL 2016 Saint Petersburg, F., & Association for Computing Machinery (2016) · doi:10.1145/2837614 · Cited by 45
A study of the performance of general compressors on log files
Yao, K., Li, H., Shang, W., & Hassan, A. E. (2020) · Empirical Software Engineering · doi:10.1007/s10664-020-09822-x · Cited by 44
GypSum: learning hybrid representations for code summarization
Wang, Y., Dong, Y., Lu, X., & Zhou, A. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527903 · Cited by 44
EClone: detect semantic clones in Ethereum via symbolic transaction sketch
Liu, H., Yang, Z., Liu, C., Jiang, Y., Zhao, W., & Sun, J. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3264596 · Cited by 43
CodeGRU: Context-aware deep learning with gated recurrent unit for source code modeling
Hussain, Y., Huang, Z., Zhou, Y., & Wang, S. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106309 · Cited by 43
Identifying algorithm in program code based on structural features using CNN classification model
Watanobe, Y., Rahman, M. M., Amin, M. F. I., & Kabir, R. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04078-y · Cited by 43
Font, J., Arcega, L., Haugen, Ø., & Cetina, C. (2017) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2017.2751100 · Cited by 43
CrystalBLEU: Precisely and Efficiently Measuring the Similarity of Code
Eghbali, A., & Pradel, M. (2022) · Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3551349.3556903 · Cited by 42
Making smart contract development more secure and easier
Ren, M., Ma, F., Yin, Z., Fu, Y., Li, H., Chang, W., & Jiang, Y. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3473929 · Cited by 42
Stochastic Optimization of Program Obfuscation
Liu, H., Sun, C. P., Su, Z., Jiang, Y., Gu, M., & Sun, J. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2017.28 · Cited by 42
Self-citationDiet code is healthy: simplifying programs for pre-trained models of code
Zhang, Z., Zhang, H., Shen, B., & Gu, X. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549094 · Cited by 41
Automatically Tracing Dependability Requirements via Term-Based Relevance Feedback
Wang, W., Gupta, A., Niu, N., Xu, L. D., Cheng, J.-R. C., & Niu, Z. (2016) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2016.2637166 · Cited by 41
Pradel, M., & Chandra, S. (2021) · Communications of the ACM · doi:10.1145/3460348 · Cited by 41
Ciniselli, M., Pascarella, L., & Bavota, G. (2022) · Proceedings of the 19th International Conference on Mining Software Repositories · doi:10.1145/3524842.3528440 · Cited by 26
Learning to blame: localizing novice type errors with data-driven diagnosis
Seidel, E. L., Sibghat, H., Chaudhuri, K., Weimer, W. R., & Jhala, R. (2017) · Proceedings of the ACM on Programming Languages · doi:10.1145/3138818 · Cited by 41
Gholamian, S., & Ward, P. A. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.12489 · Cited by 41
SMT Solver Validation Empowered by Large Pre-Trained Language Models
Sun, M., Yang, Y., Wang, Y., Wen, M., Jia, H., & Zhou, Y. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00180 · Cited by 40
Sequence Model Design for Code Completion in the Modern IDE
Aye, G. A., & Kaiser, G. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.05249 · Cited by 17
Program merge conflict resolution via neural transformers
Svyatkovskiy, A., Fakhoury, S., Ghorbani, N., Mytkowicz, T., Dinella, E., Bird, C., Jang, J., Sundaresan, N., & Lahiri, S. K. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549163 · Cited by 38
“Neural language models (LMs) have shown great performance in natural language generation [36, 38], and have been successfully applied to the domain of source code [17, 24, 48].”
BackgroundAI-Assisted Programming Tasks Using Code Embeddings and Transformers
Kotsiantis, S. B., Verykios, V. S., & Tzagarakis, M. M. (2024) · Electronics · doi:10.3390/electronics13040767 · Cited by 38
Copilot for Xcode: Exploring AI-Assisted Programming by Prompting Cloud-based Large Language ModelsPreprint
Tan, C. W., Guo, S., Wong, M. F., & Hang, C. N. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.14349 · Cited by 39
StagedVulBERT: Multigranular Vulnerability Detection With a Novel Pretrained Code Model
Jiang, Y., Zhang, Y., Su, X., Treude, C., & Wang, T. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3493245 · Cited by 38
Automatically generating features for learning program analysis heuristics for C-like languages
Chae, K., Oh, H., Heo, K., & Yang, H. (2017) · Proceedings of the ACM on Programming Languages · doi:10.1145/3133925 · Cited by 37
Lancer: Your Code Tell Me What You Need
Zhou, S., Shen, B., & Zhong, H. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00137 · Cited by 37
Combining Program Analysis and Statistical Language Model for Code Statement Completion
Nguyen, S., Nguyen, T. N., Li, Y., & Wang, S. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00072 · Cited by 37
A unified multi-task learning model for AST-level and token-level code completion
Liu, F., Li, G., Wei, B., Xia, X., Fu, Z., & Jin, Z. G. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10140-7 · Cited by 37
Detecting Condition-Related Bugs with Control Flow Graph Neural Network
Zhang, J., Wang, X., Zhang, H., Sun, H., Liu, X., Hu, C., & Liu, Y. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3598142 · Cited by 37
Perceived language complexity in GitHub issue discussions and their effect on issue resolution
Kavaler, D., Sirovica, S., Hellendoorn, V. J., Aranovich, R., & Filkov, V. (2017) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115620 · Cited by 37
The Uniqueness of Changes: Characteristics and Applications
Ray, B., Bird, C., & Zimmermann, T. (2016) · Cited by 37
OptTyper: Probabilistic Type Inference by Optimising Logical and Natural ConstraintsPreprint
Pandi, I. V., Barr, E. T., Gordon, A. D., & Sutton, C. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00348 · Cited by 37
Self-citationSoftware Vulnerability Detection via Deep Learning over Disaggregated Code Graph RepresentationPreprint
Zhuang, Y., Suneja, S., Thost, V., Domeniconi, G., Morari, A., & Laredo, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03341 · Cited by 37
NIRMAL: Automatic identification of software relevant tweets leveraging language model
Sharma, A. B., Tian, Y., & Lo, D. (2015) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2015.7081855 · Cited by 36
Import2vec: Learning Embeddings for Software Libraries
Theeten, B., Vandeputte, F., & Cutsem, T. V. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00014 · Cited by 36
Always Provide Context: The Effects of Code Context on Programming Error Message Enhancement
Santos, E. A., Prasad, P., & Becker, B. A. (2023) · ACM Global Computing Education Conference (CompEd) · doi:10.1145/3576882.3617909 · Cited by 36
WebAPIRec: Recommending Web APIs to Software Projects via Personalized Ranking
Thung, F., Oentaryo, R. J., Lo, D., & Tian, Y. (2017) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2017.2699222 · Cited by 36
RefiNym: using names to refine types
Dash, S. K., Allamanis, M., & Barr, E. T. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3236042 · Cited by 36
Self-citationCode Execution with Pre-trained Language ModelsPreprint
Liu, C., Lu, S., Chen, W., Jiang, D., Svyatkovskiy, A., Fu, S., Sundaresan, N., & Duan, N. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.05383 · Cited by 36
Evolutionary Multi-objective Optimization for Contextual Adversarial Example Generation
Zhou, S., Huang, M., Sun, Y., & Li, K. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660808 · Cited by 35
Capturing source code semantics via tree-based convolution over API-enhanced AST
Chen, L., Ye, W., & Zhang, S. (2019) · ACM International Conference on Computing Frontiers (CF) · doi:10.1145/3310273.3321560 · Cited by 35
Leveraging Informal Documentation to Summarize Classes and Methods in Context
Guerrouj, L., Bourque, D., & Rigby, P. C. (2015) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2015.212 · Cited by 35
Syntactic Versus Semantic Similarity of Artificial and Real Faults in Mutation Testing Studies
Ojdanić, M., Garg, A., Khanfir, A., Degiovanni, R., Papadakis, M., & Traon, Y. L. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3277564 · Cited by 35
Beware of the Unexpected: Bimodal Taint Analysis
Chow, Y. W., Schäfer, M., & Pradel, M. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3598050 · Cited by 35
Investigating the Use of Code Analysis and NLP to Promote a Consistent Usage of Identifiers
Lin, B., Scalabrino, S., Mocci, A., Oliveto, R., Bavota, G., & Lanza, M. (2017) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam.2017.17 · Cited by 35
DeepTriage: Exploring the Effectiveness of Deep Learning for Bug TriagingPreprint
Mani, S., Sankaran, A., & Aralikatte, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.01275 · Cited by 35
Type error feedback via analytic program repair
Sakkas, G., Endres, M., Cosman, B., Weimer, W. R., & Jhala, R. (2020) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/3385412.3386005 · Cited by 34
CACHECA: A Cache Language Model Based Code Suggestion Tool
Franks, C., Tu, Z., Dévanbu, P., & Hellendoorn, V. J. (2015) · Cited by 34
COSET: A Benchmark for Evaluating Neural Program Embeddings
Wang, K., & Christodorescu, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11445 · Cited by 20
Studying the difference between natural and programming language corpora
Casalnuovo, C., Sagae, K., & Devanbu, P. (2019) · Empirical Software Engineering · doi:10.1007/s10664-018-9669-7 · Cited by 33
“We chose Java and Ruby due to their popularity on GitHub and Java in particular due to its past use as a research subject for ngram models (Hindle et al, 2012; Allamanis and Sutton, 2013).”
BackgroundMethodResultInfluentialEye of the mind: image processing for social coding
Nayebi, M. (2020) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1145/3377816.3381723 · Cited by 33
Learning Quick Fixes from Code Repositories
de Sousa, R. R., Soares, G., Gheyi, R., Barik, T., & D’Antoni, L. (2021) · Brazilian Symposium on Software Engineering · doi:10.1145/3474624.3474650 · Cited by 33
CodeMend: Assisting Interactive Programming with Bimodal Embedding
Rong, X., Yan, S., Oney, S., Dontcheva, M., & Adar, E. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984544 · Cited by 28
Vechev, M., & Yahav, E. (2016) · Foundations and Trends® in Programming Languages · doi:10.1561/9781680832310 · Cited by 33
Semantic Source Code Models Using Identifier Embeddings
Efstathiou, V., & Spinellis, D. D. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00015 · Cited by 33
Deng, J., Lu, L., Qiu, S., & Ou, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2985780 · Cited by 33
APIRecX: Cross-Library API Recommendation via Pre-Trained Language Model
Kang, Y., Wang, Z., Zhang, H., Chen, J., & You, H. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.275 · Cited by 32
Learning Realistic Mutations: Bug Creation for Neural Bug Detectors
Richter, C., & Wehrheim, H. (2022) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst53961.2022.00027 · Cited by 32
Boosting source code learning with text-oriented data augmentation: an empirical study
Dong, Z., Hu, Q., Guo, Y., Zhang, Z., Cordy, M., Papadakis, M., Traon, Y. L., & Zhao, J. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10624-2 · Cited by 32
Learning Code Context Information to Predict Comment Locations
Huang, Y., Hu, X., Jia, N., Chen, X., Xiong, Y., & Zheng, Z. (2019) · IEEE Transactions on Reliability · doi:10.1109/tr.2019.2931725 · Cited by 32
The Uniqueness of Changes: Characteristics and Applications
Ray, B., Nagappan, M., Bird, C., Nagappan, N., & Zimmermann, T. (2015) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2015.11 · Cited by 32
On the Utility of Domain Modeling Assistance with Large Language Models
Chaaben, M. B., Burgueño, L., Dávid, I., & Sahraoui, H. A. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3744920 · Cited by 31
Analyzing Text in Software Projects
Wagner, S., & Méndez, D. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-411519-4.00003-3 · Cited by 31
Action Word Prediction for Neural Source Code Summarization
Haque, S., Bansal, A., Wu, L., & McMillan, C. (2021) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner50967.2021.00038 · Cited by 31
Predictive Mutation Analysis via the Natural Language Channel in Source Code
Kim, J.-H., Jeon, J., Hong, S., & Yoo, S. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3510417 · Cited by 31
Cross-Entropy: A New Metric for Software Defect Prediction
Zhang, X., Ben, K., & Zeng, J. (2018) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs.2018.00025 · Cited by 31
Method name suggestion with hierarchical attention networks
Xu, S., Zhang, S., Wang, W., Cao, X., Guo, C., & Xu, J. (2018) · ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation (PEPM) · doi:10.1145/3294032.3294079 · Cited by 31
Learning from Examples to Find Fully Qualified Names of API Elements in Code Snippets
Saifullah, C. M. K., Asaduzzaman, M., & Roy, C. K. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00032 · Cited by 31
CACHECA: A Cache Language Model Based Code Suggestion Tool
Franks, C., Tu, Z., Devanbu, P., & Hellendoorn, V. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.228 · Cited by 30
From Triumph to Uncertainty: The Journey of Software Engineering in the AI Era
Mastropaolo, A., Escobar‐Velásquez, C., & Linares‐Vásquez, M. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3709360 · Cited by 30
Deep security analysis of program code: A systematic literature review
Sonnekalb, T., Heinze, T. S., & Mäder, P. (2021) · Empirical Software Engineering · doi:10.1007/s10664-021-10029-x · Cited by 30
Content Aware Source Code Change Description Generation
Loyola, P., Marrese-Taylor, E., Balazs, J., Matsuo, Y., & Satoh, F. (2018) · International Conference on Natural Language Generation · doi:10.18653/v1/w18-6513 · Cited by 30
Neural Bug Finding: A Study of Opportunities and ChallengesPreprint
Habib, A., & Pradel, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00307 · Cited by 30
Learning Autocompletion from Real-World Datasets
Aye, G. A., Kim, S., & Li, H. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip52600.2021.00022 · Cited by 29
Enriching in-IDE process information with fine-grained source code history
Proksch, S., Nadi, S., Amann, S., & Mezini, M. (2017) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2017.7884626 · Cited by 29
Chatting with AI: Deciphering Developer Conversations with ChatGPT
Mohamed, S., Parvin, A., & Parra, E. (2024) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3643991.3645078 · Cited by 29
ANNE: Improving Source Code Search using Entity Retrieval Approach
Vinayakarao, V., Sarma, A., Purandare, R., Jain, S., & Jain, S. (2017) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3018661.3018691 · Cited by 29
Deep Transfer Learning for Source Code Modeling
Hussain, Y., Huang, Z., Zhou, Y., & Wang, S. (2020) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194020500230 · Cited by 29
Adaptive User Feedback for IR-Based Traceability Recovery
Panichella, A., Lucia, A. D., & Zaidman, A. (2015) · IEEE/ACM International Symposium on Software and Systems Traceability (SST) · doi:10.1109/sst.2015.10 · Cited by 28
Augmenting Decompiler Output with Learned Variable Names and TypesPreprint
Chen, Q., Lacomis, J., Schwartz, E. J., Goues, C. L., Neubig, G., & Vasilescu, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06363 · Cited by 28
CodeBERT-nt: Code Naturalness via CodeBERT
Khanfir, A., Jimenez, M., Papadakis, M., & Traon, Y. L. (2022) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs57517.2022.00098 · Cited by 27
Syntax-aware on-the-fly code completion
Takerngsaksiri, W., Tantithamthavorn, C. K., & Li, Y.-F. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107336 · Cited by 27
On using machine learning to identify knowledge in API reference documentation
Fucci, D., Mollaalizadehbahnemiri, A., & Maalej, W. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3338943 · Cited by 27
Modeling programs hierarchically with stack-augmented LSTM
Liu, F., Zhang, L., & Jin, Z. G. (2020) · Journal of Systems and Software · doi:10.1016/j.jss.2020.110547 · Cited by 27
Use of Deep Learning Model with Attention Mechanism for Software Fault Prediction
Yu, T.-Y., Huang, C., & Fang, N. C. (2021) · International Conference on Dependable Systems and Their Applications (DSA) · doi:10.1109/dsa52907.2021.00025 · Cited by 27
Uncovering the Limits of Machine Learning for Automatic Vulnerability DetectionPreprint
Risse, N., & Böhme, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.17193 · Cited by 27
Show Me Your Code! Kill Code Poisoning: A Lightweight Method Based on Code Naturalness
Sun, W., Chen, Y., Yuan, M., Fang, C., Chen, Z., Wang, C., Liu, Y., Xu, B., & Chen, Z. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00247 · Cited by 26
Deep Just-In-Time Defect Localization
Qiu, F., Gao, Z., Xia, X., Lo, D., Grundy, J., & Wang, X. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3135875 · Cited by 26
A graph-based code representation method to improve code readability classification
Mi, Q., Zhan, Y., Weng, H., Bao, Q., Cui, L., & Ma, W. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10319-6 · Cited by 26
CONCORD: Clone-Aware Contrastive Learning for Source Code
Ding, Y., Chakraborty, S., Buratti, L., Pujar, S., Morari, A., Kaiser, G. E., & Ray, B. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3598035 · Cited by 26
On the Impact of Refactoring Operations on Code Naturalness
Lin, B., Nagy, C., Bavota, G., & Lanza, M. (2019) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2019.8667992 · Cited by 26
On the Impact of Tokenizer and Parameters on N-Gram Based Code Analysis
Jimenez, M., Cordy, M., Traon, Y. L., & Papadakis, M. (2018) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2018.00053 · Cited by 26
Variable Name Recovery in Decompiled Binary Code using Constrained Masked Language ModelingPreprint
Banerjee, P., Pal, K. K., Wang, F., & Baral, C. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.12801 · Cited by 26
Disentangled Code Representation Learning for Multiple Programming Languages
Zhang, J., Hong, H., Zhang⋆, Y., Wan, Y., Liu, Y., & Sui, Y. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.391 · Cited by 25
From Novice to Expert: Analysis of Token Level Effects in a Longitudinal Eye Tracking Study
Al Madi, N., Peterson, C. S., Sharif, B., & Maletic, J. I. (2021) · 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc52881.2021.00025 · Cited by 20
XCode : Towards Cross-Language Code Representation with Large-Scale Pre-Training
Lin, Z., Li, G., Zhang, J., Deng, Y., Zeng, X., Zhang, Y., & Wan, Y. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3506696 · Cited by 25
Metadata
- Authors
- Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Devanbu, P.
- Affiliations
- University of California, Davis (US)
- The University of Texas at Dallas (US)
- Venue
- 2012 34th International Conference on Software Engineering (ICSE)
- Publisher
- IEEE
- Published
- 2012-06
- Pages
- 837-847
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W3146720657
- Cited by
- 425
- References
- 39
- Citations by year
- 2026: 19 · 2025: 24 · 2024: 36 · 2023: 44 · 2022: 49 · 2021: 50 · 2020: 48 · 2019: 46 · 2018: 22 · 2017: 35 · 2016: 13 · 2015: 23 · 2014: 2 · 2013: 14
- FWCI
- 9.81
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- statistical language models, n-gram language models, code completion, Java, eclipses, software engineering support, natural language modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.