On the naturalness of software
The DOI matches its Crossref record; 55 of the 56 references it cites were found in a registry or an index.
Abstract
Natural languages like English are rich, complex, and powerful. The highly creative and graceful use of languages like English and Tamil, by masters like Shakespeare and Avvaiyar, can certainly delight and inspire. But in practice, given cognitive constraints and the exigencies of daily life, most human utterances are far simpler and much more repetitive and predictable. In fact, these utterances can be very usefully modeled using modern statistical methods. This fact has led to the phenomenal success of statistical approaches to speech recognition, natural language translation, question-answering, and text mining and comprehension. We begin with the conjecture that most software is also natural , in the sense that it is created by humans at work, with all the attendant constraints and limitations---and thus, like natural language, it is also likely to be repetitive and predictable. We then proceed to ask whether (a) code can be usefully modeled by statistical language models and (b) such models can be leveraged to support software engineers. Using the widely adopted n -gram model, we provide empirical evidence supportive of a positive answer to both these questions. We show that code is also very regular, and, in fact, even more so than natural languages. As an example use of the model, we have developed a simple code completion engine for Java that, despite its simplicity, already improves Eclipse's completion capability. We conclude the paper by laying out a vision for future research in this area.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
On The Language Instinct: (412952005-009)
Pinker, S., & American Psychological Association (1994) · PsycEXTRA Dataset · doi:10.1037/e412952005-009 · Cited by 5,646
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Chi, E. H., & Suh, B. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,923
Knuth, D. E. (1984) · The Computer Journal · doi:10.1093/comjnl/27.2.97 · Cited by 1,467
Statistical machine translation
(2010) · Choice Reviews Online · doi:10.5860/choice.47-6293 · Cited by 1,309
Recovering traceability links between code and documentation
Antoniol, G., Canfora, G., Casazza, G., De Lucia, A., & Merlo, E. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1041053 · Cited by 967
Towards automatically generating summary comments for Java methods
Sridhara, G., Hill, E., Muppaneni, D., Pollock, L., & Vijay-Shanker, K. (2010) · Proceedings of the IEEE/ACM international conference on Automated software engineering · doi:10.1145/1858996.1859006 · Cited by 514 · cited 1×
“Summarization and Concern Location A line of work on code summarization 9, 47, 48 uses semantic properties inferred by static analysis, rather than the “natural” regularities of software corpora capturable in statistical models; it is complementary to ours: properties derived by static analysis (as…”
MethodMining Version Histories to Guide Software Changes
Zimmermann, T., Member, S., Weißgerber, P., Diehl, S., Zeller, A., & Ieee Computer Society (2004) · Cited by 494
Building a Large Annotated Corpus of English: The
MarcusMitchell, P., Ann, M., & SantoriniBeatrice (1993) · Computational Linguistics · doi:10.5555/972470.972475 · Cited by 489
Jungloid mining: helping to navigate the API jungle
Mandelin, D., Xu, L., Bodík, R., & Kimelman, D. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065018 · Cited by 388
Research works printed but not linked: 6
- 10.5555/2819009.2819143 · doi:10.5555/2819009.2819143
- 10.5555/1734086 · doi:10.5555/1734086
- 10.1145/1062455.1062526 · doi:10.1145/1062455.1062526 · Paper page
- 10.5555/311445 · doi:10.5555/311445
- 10.5555/2820518.2820559 · doi:10.5555/2820518.2820559
- 10.5555/998675.999460 · doi:10.5555/998675.999460
Cited by
Items 101–200 of 330
RepoVis: Visual Overviews and Full-Text Search in Software Repositories
Feiner, J., & Andrews, K. (2018) · IEEE Working Conference on Software Visualization (VISSOFT) · doi:10.1109/vissoft.2018.00009 · Cited by 19
Toward Less Hidden Cost of Code Completion with Acceptance and Ranking Models
lijingxuan, L., Huang, R., Li, W., Yao, K., & Tan, W. (2021) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.26226/morressier.613b5419842293c031b5b638 · Cited by 24
Mining and relating design contexts and design patterns from Stack Overflow
Wijerathna, L., Aleti, A., Bi, T., & Tang, A. (2021) · Empirical Software Engineering · doi:10.1007/s10664-021-10034-0 · Cited by 18
Untangling Composite Commits by Attributed Graph Clustering
Chen, S., Xu, S., Yao, Y., & Feng, X. (2022) · Asia-Pacific Symposium on Internetware · doi:10.1145/3545258.3545267 · Cited by 15
Zhang, Y., Hu, Y. S., & Chen, X. (2024) · Sensors · doi:10.3390/s24051351 · Cited by 15
Statically Contextualizing Large Language Models with Typed Holes
Blinn, A., Li, X., Kim, J., & Omar, C. (2024) · Proc. ACM Program. Lang. · doi:10.1145/3689728 · Cited by 25
Khapre, S. P., Singh, P., Shankar, A., Nayak, S. R., & Diwakar, M. (2021) · International Journal of Intelligent Unmanned Systems · doi:10.1108/ijius-07-2021-0055 · Cited by 15
RetractedSimple or complex? together for a more accurate just-in-time defect predictor
Zhou, X., Han, D., & Lo, D. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527910 · Cited by 18
PragFormer: Data-Driven Parallel Source Code Classification with Transformers
Harel, R., Kadosh, T., Hasabnis, N., Mattson, T. G., Pinter, Y., & Oren, G. (2024) · International Journal of Parallel Programming · doi:10.1007/s10766-024-00778-9 · Cited by 21
Context-aware prompting for LLM-based program repair
Li, Y., Cai, M., Chen, J., Xu, Y., Huang, L., & Li, J. K. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00512-w · Cited by 24
Using Natural Language Processing Techniques to Improve Manual Test Case Descriptions
Viggiato, M., Paas, D., Buzon, C., & Bezemer, C. (2022) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip55303.2022.9794054 · Cited by 14
All you need is logs: improving code completion by learning from anonymous IDE usage logs
Bibaev, V., Kalina, A., Lomshakov, V., Golubev, Y., Bezzubov, A., Povarov, N. I., & Bryksin, T. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3558968 · Cited by 17
Analyzing the dependability of Large Language Models for code clone generation
Eagal, A., Stolee, K. T., & Ore, J. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112548 · Cited by 14
Predictive synthesis of API-centric code
Nam, D., Ray, B., Kim, S., Qu, X., & Chandra, S. (2022) · doi:10.1145/3520312.3534866 · Cited by 17
Efficacy of Various Large Language Models in Generating Smart Contracts
Chatterjee, S., & Ramamurthy, B. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-85363-0_31 · Cited by 12
Leveraging Reviewer Experience in Code Review Comment Generation
Lin, H. Y., Thongtanunam, P., Treude, C., Godfrey, M. W., Liu, C., & Charoenwet, W. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3762183 · Cited by 13
Kourtzanidis, S., Chatzigeorgiou, A., & Ampatzoglou, A. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3415305 · Cited by 22
A Novel Tree-based Neural Network for Android Code Smells Detection
Yu, J. S., Mao, C., & Ye, X. (2021) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs54544.2021.00083 · Cited by 19
Naturalness in Source Code Summarization. How Significant is it?
Ferretti, C., & Saletta, M. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00027 · Cited by 13
An Exploratory Study of Functional Redundancy in Code Repositories
Suzuki, M., de Paula, A. C., Guerra, E., Lopes, C. V., & Lemos, O. A. L. (2017) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam.2017.21 · Cited by 15
Meske, C., Hermanns, T., Weiden, E. V. D., Loser, K., & Berger, T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5378201 · Cited by 16
Moving Faster and Reducing Risk: Using LLMs in Release Deployment
Abreu, R., Murali, V., Rigby, P. C., Maddila, C., Sun, W., Ge, J., Chinniah, K., Mockus, A., Mehta, M., & Nagappan, N. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip66354.2025.00045 · Cited by 18
Talamadupula, K. (2021) · AI Matters · doi:10.1145/3465074.3465080 · Cited by 10
Do bugs lead to unnaturalness of source code?
Jiang, Y., Liu, H., Zhang, Y., Ji, W., Zhong, H., & Zhang, L. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549149 · Cited by 16
Unleashing the power of pseudo-code for binary code similarity analysis
Zhang, W., Xu, Z., Xiao, Y., & Xue, Y. (2022) · Cybersecurity · doi:10.1186/s42400-022-00121-0 · Cited by 12
Generating Variable Explanations via Zero-shot Prompt Learning
Wang, C., Lou, Y., Liu, J., & Peng, X. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00130 · Cited by 16
Git Merge Conflict Resolution Leveraging Strategy Classification and LLM
Shen, C., Yang, W., Pan, M., & Zhou, Y. (2023) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs60937.2023.00031 · Cited by 19
DSFM: Enhancing Functional Code Clone Detection with Deep Subtree Interactions
Xu, Z., Qiang, S., Song, D., Zhou, M. Q., Wan, H., Zhao, X., Luo, P., & Zhang, H. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639215 · Cited by 17
Heuristic and Neural Network Based Prediction of Project-Specific API Member Access
Jiang, L. C., Liu, H., Jiang, H., Zhang, L., & Hong, M. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.3017794 · Cited by 11
Cross-lingual Transfer in Programming Languages: An Extensive Empirical StudyPreprint
Baltaji, R., Pujar, S., Mandel, L., Hirzel, M. J., Buratti, L., & Varshney, L. R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.16937 · Cited by 11
Security of Language Models for Code: A Systematic Literature Review
Chen, Y., Sun, W., Fang, C., Chen, Z., Ge, Y., Han, T., Zhang, Q., Liu, Y., Chen, Z., & Xu, B. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3735554 · Cited by 10
FLAG: F inding L ine A nomalies (in RTL code) with G enerative AI
Ahmad, B., Ah-kiow, J., Tan, B., Karri, R., & Pearce, H. A. (2025) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3736411 · Cited by 10
Automatic Program Repair as Semantic Suggestions: An Empirical Study
Campos, D., Restivo, A., Ferreira, H. S., & Ramos, A. J. M. M. (2021) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst49551.2021.00032 · Cited by 16
The Effect of Perceptual Load on Performance Within IDE in People with ADHD Symptoms
Kasatskii, V., Sergeyuk, A., Serova, A., Titov, S., & Bryksin, T. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35017-7_9 · Cited by 12
CLeBPI: Contrastive Learning for Bug Priority Inference
Wang, W., Wu, C., & He, J. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107302 · Cited by 12
FCNN: Simple neural networks for complex code tasks
Sun, X., Liu, T., Liu, C., & Dong, W. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.101970 · Cited by 11
Enhancing Function Name Prediction using Votes-Based Name Tokenization and Multi-task Learning
Xiao-ling, Z., Xu, Z., Yang, S., Li, Z., Shi, Z., & Sun, L. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660782 · Cited by 13
Krasniqi, R., & Do, H. (2023) · Empirical Software Engineering · doi:10.1007/s10664-022-10280-w · Cited by 10
Vo, H. D., & Nguyen, S. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107304 · Cited by 10
Turn tree into graph: Automatic code review via simplified AST driven graph convolutional network
Wu, B., Liang, B., & Zhang, X. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109450 · Cited by 15
Deep Learning Based Identification of Suspicious Return Statements
Li, G., Liu, H., Jin, J., & Umer, Q. (2020) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner48275.2020.9054826 · Cited by 10
Bilgin, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.03131 · Cited by 10
Examination of Code generated by Large Language ModelsPreprint
Beer, R., Feix, A., Guttzeit, T., Muras, T., Müller, V. C., Rauscher, M., Schäffler, F., & Löwe, W. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.16601 · Cited by 10
ReACC: A Retrieval-Augmented Code Completion Framework
Lu, S., Duan, N., Han, H., Guo, D., Hwang, S.-W., & Svyatkovskiy, A. (2022) · arXiv · doi:10.48550/arxiv.2203.07722 · Cited by 96
Application of Deep Learning for Code Smell Detection: Challenges and Opportunities
Hadj-Kacem, M., & Bouassida, N. (2024) · SN Computer Science · doi:10.1007/s42979-024-02956-5 · Cited by 10
TACSan: Enhancing Vulnerability Detection with Graph Neural Network
Zeng, Q., Xiong, D., Wu, Z., Qian, K., Wang, Y., & Su, Y. (2024) · Electronics · doi:10.3390/electronics13193813 · Cited by 9
NeuDep: neural binary memory dependence analysis
Pei, K., She, D., Wang, M., Geng, S., Xuan, Z., David, Y., Yang, J., Jana, S., & Ray, B. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549147 · Cited by 14
ClassSum: a deep learning model for class-level code summarization
Li, M., Yu, H., Fan, G., Zhou, Z., & Huang, J. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07877-z · Cited by 9
A Catalog of Data Smells for Coding Tasks
Vitale, A., Oliveto, R., & Scalabrino, S. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3707457 · Cited by 11
Context-Encoded Code Change Representation for Automated Commit Message Generation
Vu, T. T., Do, T.-D., & Vo, H. D. (2023) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194023500493 · Cited by 9
Capturing Structural Locality in Non-parametric Language ModelsPreprint
Xu, F. F., He, J., Neubig, G., & Hellendoorn, V. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02870 · Cited by 9
Full Line Code Completion: Bringing AI to Desktop
Semenkin, A., Bibaev, V., Sokolov, Y., Krylov, K., Kalina, A., Khannanova, A., Savenkov, D., Rovdo, D., Davidenko, I., Karnaukhov, K., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.08704 · Cited by 8
“Since source code can be interpreted as sequential data, natural language processing (NLP) methods have found their application for the task [13].”
Recovering Traceability Links Between Code and Documentation: A Retrospective
Antoniol, G., Canfora, G., Casazza, G., Lucia, A. D., & Merlo, E. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3534027 · Cited by 12
Metamorphic-Based Many-Objective Distillation of LLMs for Code-Related Tasks
Panichella, A. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00230 · Cited by 14
Towards a Human-in-the-Loop Framework for Reliable Patch Evaluation Using an LLM-as-a-Judge
Shi, S., Wei, R., Tufano, M., Cambronero, J., Cheng, R., Ivančić, F., & Rondon, P. (2026) · doi:10.1145/3786175.3788348 · Cited by 13
Code for Machines, Not Just Humans: Quantifying AI-Friendliness with Code Health Metrics
Borg, M., Hagatulah, N., Tornhill, A., & Söderberg, E. (2026) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1145/3793655.3793722 · Cited by 10
Barchi, F., Parisi, E., Bartolini, A., & Acquaviva, A. (2022) · Journal of Low Power Electronics and Applications · doi:10.3390/jlpea12030037 · Cited by 8
Learning from User-driven Events to Generate Automation Sequences
Song, Y., Bian, Y., Wang, X., & Cai, Z. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3631427 · Cited by 10
Yadav, S., Qureshi, A. M., Kaushik, A., Sharma, S., Loughran, R., Kazhuparambil, S., Shaw, A., Sabry, M., Lynch, N. S. J., Singh, N., et al. (2026) · Applied AI Letters · doi:10.1002/ail2.70037 · Cited by 8
A Neural Network Architecture for Program Understanding Inspired by Human Behaviors
Zhu, R., Yuan, L., Li, X., Gao, M., & Cai, W. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.353 · Cited by 12
EnCoSum: enhanced semantic features for multi-scale multi-modal source code summarization
Gao, Y., Zhang, H., & Lyu, C. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10384-x · Cited by 9
A study of common bug fix patterns in Rust
Shirzad, M. R., & Lam, P. (2024) · Empirical Software Engineering · doi:10.1007/s10664-023-10437-1 · Cited by 8
Chen, N., Sun, Q., Zhu, R., Li, X., Lu, X., & Gao, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.295 · Cited by 14
A novel software defect prediction method based on hierarchical neural network
Yu, H., Sun, X., Zhou, Z., & Fan, G. (2021) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac51774.2021.00059 · Cited by 8
An LLM Agentic Approach for Legal-Critical Software: A Case Study for Tax Prep Software
Gogani-Khiabani, S., Trivedi, A., Saha, D., & Tizpaz-Niari, S. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3764575 · Cited by 8
Quantifying Memorization Advantage in Code LLMs
Djiré, A. E., Kaboré, A. K., Samhi, J., Barr, E. T., Klein, J., & Bissyandé, T. F. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3764554 · Cited by 8
Self-citationAI-boosted software automation: learning from human pair programmers
Peng, X., Xing, Z., & Sun, J. (2019) · Science China Information Sciences · doi:10.1007/s11432-018-9854-3 · Cited by 7
Better Modeling the Programming World with Code Concept Graphs-augmented Multi-modal Learning
Weyssow, M., Sahraoui, H. A., & Liu, B. (2022) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1145/3510455.3512771 · Cited by 10
Graph Neural Networks Based Memory Inefficiency Detection Using Selective Sampling
Li, P., Guo, Y., Luo, Y., Wang, X., Wang, Z., & Liu, X. (2022) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41404.2022.00090 · Cited by 7
Suggesting Descriptive Method Names: An Exploratory Study of Two Machine Learning Approaches
Zaitsev, O., Ducasse, S., Bergel, A., & Eveillard, M. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-58793-2_8 · Cited by 7
Combining Contexts from Multiple Sources for Documentation-Specific Code Example Generation
Khan, J. Y., & Uddin, G. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00071 · Cited by 11
Hammad, M., Babur, Ö., Basit, H. A., & van den Brand, M. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.737 · Cited by 8
Path context augmented statement and network for learning programs
Xiao, D., Hang, D., Ai, L., Li, S., & Liang, H. (2022) · Empirical Software Engineering · doi:10.1007/s10664-021-10098-y · Cited by 8
Wang, B., Wang, X., Li, G., Zhi, C., Han, J., Zhao, X., Wang, N., Deng, S., & Yin, J. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832136 · Cited by 6
Can test cases generated by large language models facilitate automated program repair?
Zhang, C., Wang, H., Xu, C., Liu, J., Liu, K., & Liu, Z. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10802-w · Cited by 8
Statistical patterns in the equations of physics and the emergence of a meta-law of nature
Constantin, A., Ferreira, P. G., Desmond, H., & Bartlett, D. J. (2026) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2025.0091 · Cited by 7
MGF-ESE: An Enhanced Semantic Extractor with Multi-Granularity Feature Fusion for Code Summarization
Xu, X., Cao, Y., Hu, H., Xiang, H., Qi, L., Xiong, J., & Dou, W. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714544 · Cited by 6
Understanding Similar Code through Comparative Comprehension
Middleton, J., & Stolee, K. T. (2022) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc53370.2022.9833117 · Cited by 8
The Language of Programming: On the Vocabulary of Names
Amit, N., & Feitelson, D. G. (2022) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec57359.2022.00014 · Cited by 10
Specializing Neural Networks for Cryptographic Code Completion Applications
Xiao, Y., Song, W., Qi, J., Viswanath, B., McDaniel, P. D., & Yao, D. D. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3265362 · Cited by 6
Multi-grained contextual code representation learning for commit message generation
Wang, C., Zhang, L., & Zhang, X. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107393 · Cited by 6
A methodology for refined evaluation of neural code completion approaches
Le, K. T., Rashidi, G., & Andrzejak, A. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00866-9 · Cited by 6
Splitting source code identifiers using Bidirectional LSTM Recurrent Neural NetworkPreprint
Markovtsev, V., Long, W., Bulychev, E., Keramitas, R., Slavnov, K., & Markowski, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11651 · Cited by 6
Automatic Static Bug Detection for Machine Learning Libraries: Are We There Yet?Preprint
Harzevili, N. S., Shin, J., Wang, J., Wang, S., & Nagappan, N. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.04080 · Cited by 6
AI Agents and Higher-Order WorkPreprint
Sarkar, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5713646 · Cited by 16
Najmi, A., El-dosuky, M. A., & El-dosuky, M. A. (2025) · Journal of Computer Science · doi:10.3844/jcssp.2025.1140.1155 · Cited by 5
SBugLocater: Bug Localization Based on Deep Matching and Information Retrieval
Huang, X., Xiang, C., Hua, L., & He, P. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/3987981 · Cited by 5
Jackson, I., & Rolf, B. (2023) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2023.10.1572 · Cited by 6
Zou, Y., Wang, H., Lv, H., Zhao, S., & Tian, H. (2024) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194024500165 · Cited by 5
Vision Paper: Proof-Carrying Code Completions
Kamran, P. S., Dévanbu, P., & Stanford, C. (2024) · IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) · doi:10.1145/3691621.3694932 · Cited by 8
Rong, Y., Xu, M., Li, R., Xin, L., & Bao, W. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00075-6 · Cited by 6
An Exploratory Study of Interface Redundancy in Code Repositories
de Paula, A. C., Guerra, E., Lopes, C. V., Sajnani, H., & Lemos, O. A. L. (2016) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam.2016.31 · Cited by 7
Building Hierarchically Disentangled Language Models for Text Generation with Named Entities
Agarwal, Y., Batra, D., & Bagler, G. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.3 · Cited by 8
Federated learning based multi‐task feature fusion framework for code expressive semantic extraction
Deng, F., Fu, C., Qian, Y., Yang, J., He, S., & Xu, H. (2022) · Software Practice and Experience · doi:10.1002/spe.3094 · Cited by 5
Liu, K., Wu, J., Yang, H., Sun, Q., & Wan, R. (2022) · IEEE Transactions on Reliability · doi:10.1109/tr.2022.3164645 · Cited by 5
A benchmark generator framework for evolving variant-rich software
Derks, C., Strüber, D., & Berger, T. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111736 · Cited by 5
Binary code similarity analysis based on naming function and common vector space
Xia, B., Pang, J., Zhou, X., Shan, Z., Wang, J., & Yue, F. (2023) · Scientific Reports · doi:10.1038/s41598-023-42769-9 · Cited by 5
Custom static analysis to enhance insight into the usage of in-house libraries
van de Laar, P., Corvino, R., Mooij, A. J., van Wezep, H., & Rosmalen, R. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112028 · Cited by 5
JEMMA: An extensible Java dataset for ML4Code applications
Karmakar, A., Allamanis, M., & Robbes, R. (2023) · Empirical Software Engineering · doi:10.1007/s10664-022-10275-7 · Cited by 4
SEART Data Hub: Streamlining Large-Scale Source Code Mining and Pre-Processing
Dabić, O., Tufano, R., & Bavota, G. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00097 · Cited by 7
Metadata
- Authors
- Hindle, A., Barr, E. T., Gabel, M., Su, Z., & Devanbu, P.
- Affiliations
- University of Alberta (CA)
- University College London (GB)
- University of California, Davis (US)
- UC Davis, CA
- Venue
- Communications of the ACM
- Published
- 2016-04-26
- Volume
- 59
- Issue
- 5
- Pages
- 122-131
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2344444819
- Cited by
- 307
- References
- 56
- Citations by year
- 2026: 34 · 2025: 58 · 2024: 48 · 2023: 45 · 2022: 66 · 2021: 26 · 2020: 16 · 2019: 5 · 2018: 6 · 2017: 2 · 2016: 1
- FWCI
- 10.42
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Communications of the ACM (publishedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/J017515/1)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- statistical language models, n-gram language models, code completion, Java, eclipses, software engineering support
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.