On the naturalness of software
The DOI matches its Crossref record; 38 of the 39 references it cites were found in a registry or an index.
Abstract
Natural languages like English are rich, complex, and powerful. The highly creative and graceful use of languages like English and Tamil, by masters like Shakespeare and Avvaiyar, can certainly delight and inspire. But in practice, given cognitive constraints and the exigencies of daily life, most human utterances are far simpler and much more repetitive and predictable. In fact, these utterances can be very usefully modeled using modern statistical methods. This fact has led to the phenomenal success of statistical approaches to speech recognition, natural language translation, question-answering, and text mining and comprehension. We begin with the conjecture that most software is also natural, in the sense that it is created by humans at work, with all the attendant constraints and limitations - and thus, like natural language, it is also likely to be repetitive and predictable. We then proceed to ask whether a) code can be usefully modeled by statistical language models and b) such models can be leveraged to support software engineers. Using the widely adopted n-gram model, we provide empirical evidence supportive of a positive answer to both these questions. We show that code is also very repetitive, and in fact even more so than natural languages. As an example use of the model, we have developed a simple code completion engine for Java that, despite its simplicity, already improves Eclipse's built-in completion capability. We conclude the paper by laying out a vision for future research in this area.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The IntelliCode Compose system uses the term click through rate (CTR) for this and reports a value of 10% in online trials.20”
From Measuring GitHub Copilot's Impact on Productivity (2024) · cites this paper as [20]
In the citing paper
“It's been well-known right from the outset that language modeling for code has to deal with project-specific phenomena [10,11,22].”
From Few-shot training LLMs for project-specific code-summarization (2022) · cites this paper as [11]
Citations
Cites
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Chi, E. H., & Suh, B. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,923
Recovering traceability links between code and documentation
Antoniol, G., Canfora, G., Casazza, G., De Lucia, A., & Merlo, E. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1041053 · Cited by 967
Statistical Machine Translation
Koehn, P. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815829 · Cited by 871
Towards automatically generating summary comments for Java methods
Sridhara, G., Hill, E., Muppaneni, D., Pollock, L., & Vijay-Shanker, K. (2010) · Proceedings of the IEEE/ACM international conference on Automated software engineering · doi:10.1145/1858996.1859006 · Cited by 514
Jungloid mining: helping to navigate the API jungle
Mandelin, D., Xu, L., Bodík, R., & Kimelman, D. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065018 · Cited by 388
Learning from examples to improve code completion systems
Bruch, M., Monperrus, M., & Mezini, M. (2009) · Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering · doi:10.1145/1595696.1595728 · Cited by 378
Mining version histories to guide software changes
Zimmermann, T., Weibgerber, P., Diehl, S., & Zeller, A. (2004) · Proceedings. 26th International Conference on Software Engineering · doi:10.1109/icse.2004.1317478 · Cited by 322
Statistical language model adaptation: review and perspectives
Bellegarda, J. R. (2004) · Speech Communication · doi:10.1016/j.specom.2003.08.002 · Cited by 297
Cited by
Items 601–700 of 834
A review on statistical language and neural network based code completion
Gao, Z. (2023) · Applied and Computational Engineering · doi:10.54254/2755-2721/22/20231222 · Cited by 2
Bringing Structure to Naturalness: On the Naturalness of ASTs
Pârţachi, P.-P., & Sugiyama, M. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3639478.3643517 · Cited by 2
Semantic-aware Source Code Modeling
Ding, Y. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695605 · Cited by 2
Nishikawa, K., Koreki, G., & Kanuka, H. (2024) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec65559.2024.00061 · Cited by 2
AI-Augmented Software Architecture: Autonomous Refactoring with Design Pattern Awareness
Konakanchi, M. S. K. (2025) · International Journal of Emerging Trends in Computer Science and Information Technology · doi:10.63282/3050-9246.ijetcsit-v6i3p113 · Cited by 2
Intrinsic Redundancy for Reliability and Beyond
Goffi, A., Gorla, A., Mattavelli, A., & Pezzè, M. (2017) · Springer eBooks · doi:10.1007/978-3-319-67425-4_10 · Cited by 2
Statistical Approach to Increase Source Code Completion Accuracy
Savchenko, V., & Volkov, A. S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-74313-4_25 · Cited by 2
TBCNN for Programs’ Abstract Syntax Trees
Mou, L., & Jin, Z. G. (2018) · SpringerBriefs in computer science · doi:10.1007/978-981-13-1870-2_4 · Cited by 2
API Misuse Detection Based on Stacked LSTM
OuYang, S., Ge, F., Kuang, L., & Yin, Y. (2021) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-67537-0_26 · Cited by 2
Learning natural ordering of tags in domain-specific Q&A sites
Jun-fang, J., & Li, G. (2021) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1900645 · Cited by 2
Multi-location cryptographic code repair with neural-network-based methodologies
Xiao, Y. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3473102 · Cited by 2
Formal Definition and Automatic Generation of Semantic Metrics: An Empirical Study on Bug Prediction
Hu, T., Mo, R., Xiong, P., Li, Z., & Feng, Q. (2021) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam52516.2021.00020 · Cited by 2
Yang, L., Ren, Y., Guan, J., Li, B., Ma, J., Han, P., & Tan, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-96772-7_20 · Cited by 2
Statistical Models and Machine Learning to Advance Code Completion: Are We There Yet?
Nguyen, T. N. (2023) · Natural computing series · doi:10.1007/978-981-19-9948-2_5 · Cited by 2
Software Bug Detection Using the N-gram Language Model
Chollak, D. (2015) · UWSpace (University of Waterloo) · Cited by 2
Helion: Enabling a Natural Perspective of Home AutomationPreprint
Manandhar, S., Moran, K., Kafle, K., Tang, R., Poshyvanyk, D., & Nadkarni, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.00124 · Cited by 2
Understanding How Developers Reuse Stack Overflow Code in Their GitHub Projects
Tekieh, R. (2021) · Carleton University's Institutional Repository (MacOdrum Library, Carleton University) · doi:10.22215/etd/2021-14656 · Cited by 2
Mining Android App Usages for Generating Actionable GUI-based Execution ScenariosPreprint
Linares‐Vásquez, M., White, M., Bernal-Cárdenas, C., Moran, K., & Poshyvanyk, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.06271 · Cited by 2
Tu, H., & Menzies, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.01997 · Cited by 2
Program Structure Aware Precondition GenerationPreprint
Dinella, E., Lahiri, S. K., & Naik, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.02154 · Cited by 2
Neural Models for Source Code Synthesis and CompletionPreprint
Niyogi, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.06690 · Cited by 2
Kobaladze, Z., Arnania, A., & Sanikidze, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.00013 · Cited by 2
Weyssow, M. (2024) · Papyrus : Institutional Repository (Université de Montréal) · doi:10.71781/10809 · Cited by 2
Code Understanding is a Bottleneck for Coding AgentsPreprint
Balepur, N., Tomlinson, K., & Schnabel, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.10610 · Cited by 2
Split Attention Pointer Network for Source Code Language Modeling
Zhou, Z., & Chen, Z. (2020) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194020500321 · Cited by 1
Learning Code Transformations Via Neural Machine Translation
Tufano, M. (2019) · W&M Publish (College of William & Mary) · doi:10.21220/s2-1z1j-d577 · Cited by 1
Enhancing Identifier Naming Through Multi-Mask Fine-Tuning of Language Models of Code
Vijayvargiya, S., Saad, M., & Sharma, T. (2024) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam63643.2024.00017 · Cited by 1
Transformer-based code model with compressed hierarchy representation
Zhang, K., Li, J., Li, Z., Jin, Z. G., & Li, G. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10612-6 · Cited by 1
Differential Testing of Concurrent Classes
Terragni, V., & Cheung, S.-C. (2025) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst62969.2025.10989027 · Cited by 1
Intelligent Dynamic Code Analysis using Large Language Models
Murugalakshmi, V., Venkadesh, P., Divya, S. V., Ageilan, K., Kishore, K., & Alden, C. S. (2025) · International Conference on Circuit, Power and Computing Technologies · doi:10.1109/iccpct65132.2025.11176641 · Cited by 1
Piccolo, S., & Stevens, H. P. (2025) · GigaScience · doi:10.1093/gigascience/giaf149 · Cited by 1
Ye, J., Li, Z., Tang, X., Xu, S., Zou, D., Yuan, Z., & Jin, H. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3840391 · Cited by 1
Promises and Perils of LLM- and Agent-Generated Code
Dévanbu, P. (2025) · Computer · doi:10.1109/mc.2025.3627694 · Cited by 1
PrevaRank: Ranking plausible patches by historic feature frequencies
Bhuiyan, S. S., Tiwari, A., Pei, Y., & Furia, C. A. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.112921 · Cited by 1
Pinto-Santos, F., Zato, C., Quintián, H., Li, T. C., & Chamoso, P. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1794125 · Cited by 1
A Case for Software Specific Natural Language Techniques
Binkley, D., & Lawrie, D. (2016) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam.2016.27 · Cited by 1
From software data to software theory
Whitehead, J. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-804206-9.00005-2 · Cited by 1
Infusing Topic Modeling into Interactive Program Comprehension: An Empirical Study
Wang, T., & Liu, Y. (2017) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac.2017.151 · Cited by 1
Facilitating Scenario-Based Program Comprehension with Topic Models
Wang, T., & Liu, Y. (2017) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec.2017.78 · Cited by 1
On Data Analysis of Software Repositories
Namiot, D., & Romanov, V. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-37436-5_24 · Cited by 1
Statistical Learning of Markov Chains of Programs
Incerto, E., Napolitano, A., & Tribastone, M. (2020) · Proceedings - International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascots50786.2020.9285947 · Cited by 1
Nourry, O., Kashiwa, Y., Kamei, Y., & Ubayashi, N. (2021) · Information and Software Technology · doi:10.1016/j.infsof.2021.106623 · Cited by 1
FeaRS: Recommending Complete Android Method Implementations
Wen, F., Ferrari, V., Aghajani, E., Nagy, C., Lanza, M., & Bavota, G. (2021) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme52107.2021.00062 · Cited by 1
Code Understanding Linter to Detect Variable Misuse
Ryou, Y., Joh, S., Yang, J., Kim, S., & Kim, Y. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3559497 · Cited by 1
Improved Methods of Pointer Mixture Network for Code Completion
Wei, C., Huang, Z., & Yu, Y. (2022) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs57517.2022.00095 · Cited by 1
Using the Deep Learning-Based Approaches for Program Debugging and Repair
Lin, T., Huang, C., & Fang, C. (2023) · International Conference on Dependable Systems and Their Applications (DSA) · doi:10.1109/dsa59317.2023.00059 · Cited by 1
GRRLN: Gated Recurrent Residual Learning Networks for code clone detection
Zhang, X., Xiao, Q., Hu, H., & Liu, J. (2024) · Journal of Software Evolution and Process · doi:10.1002/smr.2649 · Cited by 1
SG-Coder: Code completion algorithm applied on large software development platform
Li, S., Liu, Z., Liu, X., Chen, G., Gao, Y., & Liu, Y. (2023) · International Academic Exchange Conference on Science and Technology Innovation (IAECST) · doi:10.1109/iaecst60924.2023.10503165 · Cited by 1
Kumar, N. S. P. (2018) · International Journal of Science and Research (IJSR) · doi:10.21275/sr24608150741 · Cited by 1
Modeling Eye Movement for the Assessment of Programming Proficiency
Madi, N. S. A. (2020) · OhioLink ETD Center (Ohio Library and Information Network) · Cited by 1
Thota, M. R. (2022) · International Journal of Scientific Research in Science and Technology · doi:10.32628/ijsrst2295163 · Cited by 1
Intelligent Code Optimization Using Generative AI for Java and Node-Based Banking Applications
Bennett, D. A., Carter, E. J., Thompson, M., Richardson, D. O., Foster, M. D. K., & Srinivas, C. (2020) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit20631116 · Cited by 1
Search-based generation of human readable test data and its impact on human oracle costs
Afshan, S. (2013) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · Cited by 1
Jiang, Y. (2016) · PolyPublie (École Polytechnique de Montréal) · Cited by 1
Statistical Language Models applied to News Generation
Soares, J. R. P. (2017) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · doi:10.34626/q6mx-cg63 · Cited by 1
Exploring the Naturalness of Buggy Code with Recurrent Neural NetworksPreprint
Lanchantin, J., & Gao, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.08793 · Cited by 1
Context-Sensitive Code Completion
Asaduzzaman, M. (2018) · Cited by 1
Rahman, M. (2018) · Spectrum Research Repository (Concordia University) · Cited by 1
Leveraging Machine Learning to Improve Software Reliability
Wang, S. (2019) · UWSpace (University of Waterloo) · Cited by 1
Yang, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11503 · Cited by 1
Towards Automating Code Reviews
Fadhel, M. (2020) · MacSphere (McMaster University) · doi:10.71548/9264 · Cited by 1
Learning to Find Bugs in Programs and their Documentation
Habib, A. (2021) · Technischen Universität Darmstadt · doi:10.26083/tuprints-00017377 · Cited by 1
Extending maintainability analysis beyond code smells
Σαρμα, Τ. (2019) · doi:10.12681/eadd/46410 · Cited by 1
Towards Fully Declarative Program Analysis via Source Code TransformationPreprint
van Tonder, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.12398 · Cited by 1
Import2vec - Learning Embeddings for Software LibrariesPreprint
Theeten, B., Vandeputte, F., & Cutsem, T. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03990 · Cited by 1
A Code Completion Model based RWKV with Bimodal PretrainingPreprint
Zhou, L., & Li, S. (2023) · Research Square · doi:10.21203/rs.3.rs-3387525/v1 · Cited by 1
Dynamic Syntax Tree Model for Enhanced Source Code RepresentationPreprint
Roy, J., Patel, R., Nguyen, P., & Simon, S. (2023) · Preprints.org · doi:10.20944/preprints202312.1108.v1 · Cited by 1
Pauzi, Z., Capiluppi, A., & Sas, C. (2024) · doi:10.22541/au.172527329.98954722/v1 · Cited by 1
logram: efficient log paring using n-gram model
Dai, H. (2020) · Spectrum Research Repository (Concordia University) · Cited by 1
Retrieval-Augmented Validation of AUTOSAR ARXML : A Machine-Learning-Assisted Code-Review Framework
Mohamed, N. (2025) · DiVA (Digitala Vetenskapliga Arkivet) · Cited by 1
Improving the Precision of Automatic Program Repair with Machine Learning
Ye, H. (2023) · DiVA (Digitala Vetenskapliga Arkivet) · Cited by 1
Classification of computer programs in the Scratch online community
Abdalla, L. A. ). (2020) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 1
Menzies, T. (2019) · NCSU Libraries Repository (North Carolina State University Libraries) · Cited by 1
Plate, H. (2024) · theses.fr (ABES) · doi:10.70675/19d4cfb4za969z40a3z8c79z4060c26a7e44 · Cited by 1
Towards Personalizing Secure Programming Education with LLM-Injected VulnerabilitiesPreprint
Frazier, M., & Damevski, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.13955 · Cited by 1
On the role of context in AI-driven program repair and test generation
Nashid, N. (2026) · cIRcle (University of British Columbia) · Cited by 1
A Single Patch Is Not Enough: Deterministic Fusion of Repair CandidatesPreprint
Yang, B., Hu, X., Ren, L., Chen, Y., Le, B., Bissyandé, T. F., & Tian, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.01597 · Cited by 1
Rashidi, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.12524 · Cited by 1
CURE: Local Uncertainty Repair for Block-Parallel Speculative DecodingPreprint
Liu, A., Meng, J., Liu, F., & Chen, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.00531 · Cited by 1
Improving Code Autocompletion with Transfer Learning
Zhou, W., Kim, S., Murali, V., & Aye, G. A. (2021) · arXiv · doi:10.48550/arxiv.2105.05991
“Earlier attempts at modeling software languages and completion ranking were based on n-gram language models [4], [19], [20] or probabilistic models using the information from ASTs [21].”
Empirical Software Linguistics: An Investigation of Code Reviews, Recommendations and Faults
Hellendoorn, V. J. (2015) · Research Repository (Delft University of Technology)
Stepwise API usage assistance based on N-gram language models
Prendi, G. (2015) · Repositório do ISCTE-IUL
Inferring Semantic Information from Natural-Language Software Artifacts
Pandita, R. (2015) · NCSU Libraries Repository (North Carolina State University Libraries)
Study on Analysis of Program Collection for Classifying and Understanding Relations
神田, 哲. (2016) · Institutional Repositories DataBase (IRDB) · doi:10.18910/55841
Enhancing Software Development Techniques via Copy Codebases
Muşlu, K. (2015) · ResearchWorks at the University of Washington (University of Washington)
Guided Attention for Neural Language Models
Mularczyk, B. (2017) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-a-010890062
A Case for Deep Learning in Mining Software Repositories
Nijessen, R. (2017) · Research Repository (Delft University of Technology)
Continuous Semantic Inspection
Yan, Y. (2017) · eScholarship (California Digital Library)
Neural Language Models for Data-Driven Programming Support
Rong, X. (2017) · Deep Blue (University of Michigan)
Generated code in studies on clone rates
Koschke, R., & Weinig, M. (2018) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc.2018.8327314
Automatic Recall of Lessons Learned for Software Project Managers
Mohamed, T. M. A. (2018) · Scholarship@Western (Western University)
Assisting developers in the creation and maintenance of unit tests
Zhang, B. (2018) · Library, Museums and Press - UDSpace (University of Delaware) · doi:10.58088/7zs3-s762
Анализ данных для программных репозиториев
Намиот, Д. Е., & Романов, В. Ю. (2018) · International journal of open information technologies
PROGRAM INSPECTION AND TESTING TECHNIQUES FOR CODE CLONES AND REFACTORINGS IN EVOLVING SOFTWARE
Chen, Z. (2017) · DigitalCommons@UNO
Identifying Software Defects Using Neural Graph Classifiers
Hamel, O. (2019) · Carleton University's Institutional Repository (MacOdrum Library, Carleton University) · doi:10.22215/etd/2019-13526
An Improved Recurrent Neural Network Language Model for Programming Language
Wu, L., Wu, Y., Li, F., & Zheng, T. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852433
Deep Pattern Mining for Program Repair
Liu, K. (2019) · Open Repository and Bibliography (University of Luxembourg)
Improving the Usability of Static Analysis Tools Using Machine Learning
Koç, U. (2019) · University Libraries (University of Maryland) · doi:10.13016/bqo5-xlnp
JMentor: An Ontology-Based Framework for Software Understanding and Reuse
Rawashdeh, M., Alnusair, A., Almiani, M., & Sawalha, L. (2019) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa47632.2019.9035293
Machine Learning as a Mean to Uncover Latent Knowledge from Source Code
Çergani, E. (2020) · Technischen Universität Darmstadt · doi:10.25534/tuprints-00011658
Metadata
- Authors
- Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Devanbu, P.
- Affiliations
- University of California, Davis (US)
- The University of Texas at Dallas (US)
- Venue
- 2012 34th International Conference on Software Engineering (ICSE)
- Publisher
- IEEE
- Published
- 2012-06
- Pages
- 837-847
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W3146720657
- Cited by
- 425
- References
- 39
- Citations by year
- 2026: 19 · 2025: 24 · 2024: 36 · 2023: 44 · 2022: 49 · 2021: 50 · 2020: 48 · 2019: 46 · 2018: 22 · 2017: 35 · 2016: 13 · 2015: 23 · 2014: 2 · 2013: 14
- FWCI
- 9.81
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- statistical language models, n-gram language models, code completion, Java, eclipses, software engineering support, natural language modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.