Retrieve and Refine: Exemplar-Based Neural Comment Generation
The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Code comment generation is a crucial task in the field of automatic software development. Most previous neural comment generation systems used an encoder-decoder neural network and encoded only information from source code as input. Software reuse is common in software development. However, this feature has not been introduced to existing systems. Inspired by the traditional IR-based approaches, we propose to use the existing comments of similar source code as exemplars to guide the comment generation process. Based on an open source search engine, we first retrieve a similar code and treat its comment as an exemplar. Then we applied a seq2seq neural network to conduct an exemplar-based comment generation. We evaluate our approach on a large-scale Java corpus, and experimental results demonstrate that our model significantly outperforms the state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Vinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400
DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones
Jiang, L., Misherghi, G., Su, Z., & Glondu, S. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.30 · Cited by 1,039
Ko, A. J., Myers, B. A., Coblenz, M. J., & Aung, H. H. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.116 · Cited by 685 · cited 1×
“Previous studies showed that during software maintenance, program comprehension takes more than half of the time [6, 9, 22, 44].”
BackgroundTowards automatically generating summary comments for Java methods
Sridhara, G., Hill, E., Muppaneni, D., Pollock, L., & Vijay-Shanker, K. (2010) · Proceedings of the IEEE/ACM international conference on Automated software engineering · doi:10.1145/1858996.1859006 · Cited by 514 · cited 2×
“Templatebased approaches [1], [12]–[14] generate comments based on manually defined rules and information from the source code.”
BackgroundMeasuring Program Comprehension: A Large-Scale Field Study with Professionals
Xia, X., Bao, L., Lo, D., Xing, Z., Hassan, A. E., & Li, S. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2734091 · Cited by 326
On the naturalness of software
Hindle, A., Barr, E. T., Gabel, M., Su, Z., & Devanbu, P. (2016) · Communications of the ACM · doi:10.1145/2902362 · Cited by 307
Supporting program comprehension with source code summarization
Haiduc, S., Aponte, J., & Marcus, A. (2010) · Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2 · doi:10.1145/1810295.1810335 · Cited by 291 · cited 1×
“The traditional IR-based comment generation are based on this idea [7]–[10] and treat the retrieved comments directly as comments for a given code.”
BackgroundThe adverse effects of code duplication in machine learning models of code
Allamanis, M. (2019) · Proceedings of the 2019 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software · doi:10.1145/3359591.3359735 · Cited by 284
Evaluating source code summarization techniques: Replication and expansion
Eddy, B. P., Robinson, J. A., Kraft, N. A., & Carver, J. C. (2013) · 2013 21st International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2013.6613829 · Cited by 144
Cited by
Retrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2024) · arXiv · doi:10.48550/arxiv.2402.19473 · Cited by 240
“It filters out lower-quality data to maintain a high standard of information mapping, offering a variety of retrieval system options like TF-IDF, DPR, RAG, and BLINK + flair to support evidence-based predictions or citations according to task requirements.”
(2023) · doi:10.1145/3510003.3510060 · Cited by 88
“Some previous studies [4, 15, 25, 26, 29, 34, 37, 64, 65, 69] accurately describe the BLEU metric used and compare models under the same BLEU metric [4, 15, 25, 26, 29, 34, 37, 64, 65, 69].”
BackgroundMethodInfluentialRACE: Retrieval-augmented Commit Message Generation
Shi, E., Wang, Y., Tao, W., Du, L., Zhang, H., Han, S., Zhang, D., & Sun, H. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.372 · Cited by 42
“Recently, introducing retrieved relevant results into the training process has been found useful in most generation tasks (Lewis et al., 2020; Yu et al., 2021; Wei et al., 2020).”
BackgroundCorrelating Automated and Human Evaluation of Code Documentation Generation Quality
Hu, X., Chen, Q., Wang, H., Xia, X., Lo, D., & Zimmermann, T. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3502853 · Cited by 33
ReACC: A Retrieval-Augmented Code Completion Framework
Lu, S., Duan, N., Han, H., Guo, D., Hwang, S.-W., & Svyatkovskiy, A. (2022) · arXiv · doi:10.48550/arxiv.2203.07722 · Cited by 96
“Both Wei et al. (2020) and Li et al. (2021) leverage the retrieve-and-edit/refine framework to improve model’s performance in code summarization.”
Method
Metadata
- Authors
- Wei, B.
- Venue
- 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE)
- Publisher
- IEEE
- Published
- 2019-11
- Pages
- 1250-1252
- OpenAlex
- W2999553660
- Cited by
- 25
- References
- 10
- FWCI
- 2.49
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Retrieve and Refine: Exemplar-Based Neural Comment Generation. https://app.cochat.ai/papers/10.1109/ase.2019.00152
- The paper
- Wei, B. (2019). Retrieve and Refine: Exemplar-Based Neural Comment Generation. 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). https://doi.org/10.1109/ase.2019.00152
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.