Towards automatically generating summary comments for Java methods
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
Studies have shown that good comments can help programmers quickly understand what a method does, aiding program comprehension and software maintenance. Unfortunately, few software projects adequately comment the code. One way to overcome the lack of human-written summary comments, and guard against obsolete comments, is to automatically generate them. In this paper, we present a novel technique to automatically generate descriptive summary comments for Java methods. Given the signature and body of a method, our automatic comment generator identifies the content for the summary and generates natural language text that summarizes the method's overall actions. According to programmers who judged our generated comments, the summaries are accurate, do not miss important content, and are reasonably concise.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Initially, template-based approaches were popular [17,26,27,56,61]; however, creating a list of templates with good coverage is very challenging.”
From Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization) (2024) · cites this paper as [61]
Citations
Cites
Refactoring: Improving the Design of Existing Code
Fowler, M. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45672-4_31 · Cited by 5,591
Knuth, D. E. (1984) · The Computer Journal · doi:10.1093/comjnl/27.2.97 · Cited by 1,467 · cited 1×
“Specifically, they use symbolic execution to identify under what conditions the exception is thrown, and then use a set of templates to generate the textual comment for these conditions.”
BackgroundBuilding Natural-Language Generation Systems
Reiter, E. (1996) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9605002 · Cited by 1,228 · cited 1×
“Similar to how an abstract provides a summary for a natural language document [18], a descriptive summary comment for a method can help programmers quickly understand what a method does, aiding program comprehension and software maintenance.”
BackgroundMani, I. (2001) · Natural Language Processing · doi:10.1075/nlp.3 · Cited by 834
A Survey on Shari’Ah Governance Practices in Malaysia, GCC Countries and the UKPreprint
Hasan, Z. (2011) · SSRN Electronic Journal · Cited by 452
A study of the documentation essential to software maintenance
de Souza, S. C. B., Anquetil, N., & de Oliveira, K. M. (2005) · Proceedings of the 23rd annual international conference on Design of communication: documenting & designing for pervasive information · doi:10.1145/1085313.1085331 · Cited by 305
Ball, T., & Larus, J. R. (1993) · Proceedings of the ACM SIGPLAN 1993 conference on Programming language design and implementation · doi:10.1145/155090.155119 · Cited by 289 · cited 1×
“In the future, we will improve content selection by excluding if blocks handling special cases and error return paths, by possibly leveraging branch execution frequency estimation [1].”
BackgroundPickerell, A. G., & Dornin, M. (1968) · University of California Press eBooks · doi:10.1525/9780520339668 · Cited by 266
A survey on Shari'ah governance practices in Malaysia, GCC countries and the UK: Critical appraisal
Hasan, Z. (2011) · International Journal of Islamic and Middle Eastern Finance and Management · doi:10.1108/17538391111122195 · Cited by 240
Using profile information to assist classic code optimizations
Chang, P. P., Mahlke, S. A., & Hwu, W. W. (1991) · Software: Practice and Experience · doi:10.1002/spe.4380211204 · Cited by 237 · cited 1×
“While useful in targeting optimization [4], these statements do not represent content for a summary.”
BackgroundModernizing Legacy Systems: Software Technologies, Engineering Process and Business Practices
Seacord, R. C., Plakosh, D., & Lewis, G. A. (2003) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 221
Automatically capturing source code context of NL-queries for software maintenance and reuse
Hill, E., Pollock, L., & Vijay-Shanker, K. (2009) · 2009 IEEE 31st International Conference on Software Engineering · doi:10.1109/icse.2009.5070524 · Cited by 211 · cited 2×
“We use a method’s AST to identify void-return s units.”
MethodSelf-citationTheories, methods and tools in program comprehension: past, present and future
Storey, M.-A. (2005) · 13th International Workshop on Program Comprehension (IWPC'05) · doi:10.1109/wpc.2005.38 · Cited by 209 · cited 1×
“Program comprehension studies have shown that each line does not make an equal contribution to comprehension for expert programmers [ 27 ].”
BackgroundMining source code to automatically split identifiers for software analysis
Enslen, E., Hill, E., Pollock, L., & Vijay-Shanker, K. (2009) · 2009 6th IEEE International Working Conference on Mining Software Repositories · doi:10.1109/msr.2009.5069482 · Cited by 170 · cited 1×
“We will adopt more advanced splitting techniques after they are refined [7].”
MethodSelf-citationProgram readability: procedures versus comments
Tenny, T. (1988) · IEEE Transactions on Software Engineering · doi:10.1109/32.6171 · Cited by 150 · cited 1×
“In spite of numerous studies demonstrating the utility of comments for understanding software [30–32], few software projects adequately document the code to reduce future maintenance costs [6,14].”
BackgroundTakang, A. A., Grubb, P., & Macredie, R. D. (1996) · Journal of programming languages · Cited by 136
The effect of modularization and comments on program comprehension
Woodfield, S. N., Dunsmore, H. E., & Shen, V. Y. (1981) · doi:10.5555/800078.802534 · Cited by 131
A Survey of Documentation Practice within Corrective Maintenance
Kajko‐Mattsson, M. (2004) · Empirical Software Engineering · doi:10.1023/b:lida.0000048322.42751.ca · Cited by 129
The Roles Beacons Play in Comprehension for Novice and Expert Programmers.
Crosby, M. E., Scholtz, J. C., & Wiedenbeck, S. (2002) · PPIG · Cited by 129
Automatic documentation inference for exceptions
Buse, R. P. L., & Weimer, W. R. (2008) · Proceedings of the 2008 international symposium on Software testing and analysis · doi:10.1145/1390630.1390664 · Cited by 122 · cited 1×
“The system takes a method signature and body as input and outputs a natural language summary comment for the method.”
MethodCognitive Perspectives on the Role of Naming in Computer Programs
Liblit, B., Begel, A., & Sweetser, E. E. (2006) · Cited by 104
AMAP: automatically mining abbreviation expansions in programs to enhance software maintenance tools
Hill, E., Fry, Z. P., Boyd, H., Sridhara, G., Novikova, Y. S., Pollock, L., & Vijay‐Shanker, K. (2008) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/1370750.1370771 · Cited by 97 · cited 1×
“We use techniques from prior work [12] to automatically identify and expand abbreviations in code before generating text.”
MethodSelf-citationIdentifying Word Relations in Software: A Comparative Study of Semantic Similarity Tools
Sridhara, G., Hill, E., Pollock, L., & Vijay‐Shanker, K. (2008) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2008.18 · Cited by 96 · cited 1×
“Automatically detecting such semantic similarity in programs is not a trivial task [26].”
BackgroundSelf-citationA survey on the Software Maintenance Process
Sousa, M. J. C., & Moreira, H. (2002) · International Conference on Software Maintenance · doi:10.1109/icsm.1998.738518 · Cited by 65 · cited 1×
“Software maintenance demands as much as 90% of software engineering resources [24], and much of this time is spent understanding the maintenance task and any related software or documentation [25].”
BackgroundWhat users say they want in documentation
Novick, D. G., & Ward, K. (2006) · Proceedings of the 24th annual ACM international conference on Design of communication · doi:10.1145/1166324.1166346 · Cited by 56 · cited 1×
“From this high-level goal, we derive our problem statement: Given a method signature and body statements for a method M , generate natural language text that summarizes the overall actions of M accurately, adequately, and concisely.”
MethodCode extraction algorithms which unify slicing and concept assignment
Harman, M., Gold, N., Hierons, R. M., & Binkley, D. (2003) · Working Conference on Reverse Engineering (WCRE) · doi:10.1109/wcre.2002.1173060 · Cited by 55 · cited 1×
“In [10], Harman et al. defined Key Statements as the core of a program’s computation, statements through which the largest part of the program’s dependence appears to flow.”
BackgroundPollock, L., Shanker, V. K., & Hill, E. (2010) · Cited by 42
Self-citationApi hyperlinking via structural overlap
Long, F., Wang, X., & Cai, Y. (2009) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1595696.1595727 · Cited by 39 · cited 2×
“In contrast, we are aware of some techniques that could be used towards generating comments for legacy code [3,17].”
BackgroundMethodSchematic pseudocode for program constructs and its computer automation by SCHEMACODE
Robillard, P. N. (1986) · Communications of the ACM · doi:10.1145/7538.7541 · Cited by 31 · cited 1×
“Specifically, they use symbolic execution to identify under what conditions the exception is thrown, and then use a set of templates to generate the textual comment for these conditions.”
BackgroundImpact of Limited Memory Resources
Binkley, D., Lawrie, D., Maex, S., & Morrell, C. H. (2008) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2008.31 · Cited by 11 · cited 2×
“long identifier names [2, 16].”
BackgroundAn interactive source commenter for Prolog programs
Roach, D., Berghel, H., & Talburt, J. R. (1990) · doi:10.1145/97426.98002 · Cited by 9 · cited 1×
“In recent work [3], Buse and Weimer automatically generate comments about the exceptions thrown by a given Java method.”
BackgroundAn automated FORTRAN documenter
Erickson, T. E. (1982) · ACM International Conference on Systems Documentation (SIGDOC) · doi:10.1145/800065.801308 · Cited by 8 · cited 16×
“This paper describes a novel technique to automatically generate descriptive summary comments for Java methods.”
BackgroundInfluentialAn interactive source commenter for Prolog programs
Roach, D., Berghel, H., & Talburt, J. R. (1990) · ACM SIGDOC Asterisk Journal of Computer Documentation · doi:10.1145/97435.98002 · Cited by 3
Research works printed but not linked: 7
- Fowler M., 1999, Addison-Wesley
- 10.5555/882506.885157 · doi:10.5555/882506.885157
- Hill E., 2010, University of Delaware
- Marin D. P., 2005, University of California
- 10.5555/331955 · doi:10.5555/331955
- Seacord R. C., 2003, Inc.
- Takang A. A., 1996, J. Prog. Lang., 4
Cited by
Summarizing Source Code using a Neural Attention Model
Iyer, S., Konstas, I., Cheung, A., & Zettlemoyer, L. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1195 · Cited by 701
“Sridhara et al. (2010) present a pipeline that generates summaries of Java methods by selecting relevant content and generating phrases using templates to describe them.”
MethodHu, X., Li, G., Xia, X., Lo, D., & Jin, Z. (2018) · Proceedings of the 26th Conference on Program Comprehension · doi:10.1145/3196321.3196334 · Cited by 621
“Previ- ousstudieshaveshownthatgoodcommentsareimportanttopro-gramcomprehension,sincedeveloperscanunderstandthemeaning of a piece of code by using the natural language description of the comments [35].”
InfluentialOn the naturalness of software
Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Dévanbu, P. (2012) · doi:10.5555/2337223.2337322 · Cited by 541
A Transformer-based Approach for Source Code Summarization
Ahmad, W., Chakraborty, S., Ray, B., & Chang, K.-W. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.449 · Cited by 378
On the naturalness of software
Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Devanbu, P. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227135 · Cited by 425
“This line of work aims to generate natural language descriptions (summaries) of code [15–17].”
Background
Metadata
- Authors
- Sridhara, G., Hill, E., Muppaneni, D., Pollock, L., & Vijay-Shanker, K.
- Affiliations
- University of Delaware (US)
- University of Delaware, Newark, DE, USA
- Venue
- Proceedings of the IEEE/ACM international conference on Automated software engineering
- Publisher
- ACM
- Published
- 2010-09-20
- Pages
- 43-52
- OpenAlex
- W2082160726
- Cited by
- 514
- References
- 32
- Citations by year
- 2026: 9 · 2025: 29 · 2024: 31 · 2023: 52 · 2022: 46 · 2021: 41 · 2020: 56 · 2019: 50 · 2018: 41 · 2017: 34 · 2016: 23 · 2015: 30 · 2014: 26 · 2013: 16 · 2012: 16
- FWCI
- 10.33
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/ACM International Conference on Automated Software Engineering (ASE) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- Java methods, program comprehension, software maintenance, natural language generation, source code documentation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.