Retrieval-based neural source code summarization
Abstract
Source code summarization aims to automatically generate concise summaries of source code in natural language texts, in order to help developers better understand and maintain source code. Traditional work generates a source code summary by utilizing information retrieval techniques, which select terms from original source code or adapt summaries of similar code snippets. Recent studies adopt Neural Machine Translation techniques and generate summaries from code snippets using encoder-decoder neural networks. The neural-based approaches prefer the high-frequency words in the corpus and have trouble with the low-frequency ones. In this paper, we propose a retrieval-based neural source code summarization approach where we enhance the neural model with the most similar code snippets retrieved from the training set. Our approach can take advantages of both neural and retrieval-based techniques. Specifically, we first train an attentional encoder-decoder model based on the code snippets and the summaries in the training set; Second, given one input code snippet for testing, we retrieve its two most similar code snippets in the training set from the aspects of syntax and semantics, respectively; Third, we encode the input and two retrieved code snippets, and predict the summary by fusing them during decoding. We conduct extensive experiments to evaluate our approach and the experimental results show that our proposed approach can improve the state-of-the-art methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Selecting demonstration examples with higher similarity can improve ICL's performance.
“For example, Liu et al. [20] show that selecting demonstration examples with higher similarity or increasing the number of demonstration examples can improve ICL's performance.”
From What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs? (2023) · cites this paper as [20]
Increasing the number of demonstration examples can improve ICL's performance.
“For example, Liu et al. [20] show that selecting demonstration examples with higher similarity or increasing the number of demonstration examples can improve ICL's performance.”
From What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs? (2023) · cites this paper as [20]
Liu et al. first explored selecting the closest neighbors as the in-context examples.
“Liu et al. [20] first explore selecting the closest neighbors as the in-context examples.”
From What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs? (2023) · cites this paper as [20]
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588 · see citing works
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · see citing works
Supporting program comprehension with source code summarization
Haiduc, S., Aponte, J., & Marcus, A. (2010) · Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2 · doi:10.1145/1810295.1810335 · Cited by 291 · see citing works
Improving automated source code summarization via an eye-tracking study of programmers
Rodeghero, P., McMillan, C., McBurney, P. W., Bosch, N., & D'Mello, S. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568247 · Cited by 198 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029 · see citing works
An extensive study on pre-trained models for program understanding and generation
Zeng, Z., Tan, H., Zhang, H., Li, J., Zhang, Y., & Zhang, L. (2022) · Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3533767.3534390 · Cited by 153 · see citing works
Self-citationReassessing automatic evaluation metrics for code summarization tasks
Roy, D., Fakhoury, S., & Arnaoudova, V. (2021) · Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3468264.3468588 · Cited by 106 · see citing works
What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs?
Gao, S., Wen, X.-C., Gao, C., Wang, W., Zhang, H., & Lyu, M. R. (2023) · 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00109 · Cited by 106 · see citing works
Self-citationProblems and opportunities in training deep learning software systems: an analysis of variance
Pham, H. V., Qian, S., Wang, J., Lutellier, T., Rosenthal, J., Tan, L., Yu, Y., & Nagappan, N. (2020) · Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3324884.3416545 · Cited by 100 · see citing works
Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)
Ahmed, T., Pai, K. S., Devanbu, P., & Barr, E. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639183 · Cited by 112 · see citing works
“Some have combined previous approaches, such as template-based and retrieval-based approaches, using neural models [69], and have reported promising results.”
Multilingual training for software engineering
Ahmed, T., & Devanbu, P. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510049 · Cited by 57 · see citing works
RACE: Retrieval-augmented Commit Message Generation
Shi, E., Wang, Y., Tao, W., Du, L., Zhang, H., Han, S., Zhang, D., & Sun, H. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.372 · Cited by 42 · see citing works
Self-citationCorrelating Automated and Human Evaluation of Code Documentation Generation Quality
Hu, X., Chen, Q., Wang, H., Xia, X., Lo, D., & Zimmermann, T. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3502853 · Cited by 33 · see citing works
- Authors
- Zhang, J., Wang, X., Zhang, H., Sun, H., & Liu, X.
- Affiliations
- Beihang University (CN)
- University of Newcastle Australia (AU)
- Venue
- Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2020-06-27
- Pages
- 1385-1397
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W3091730360
- Cited by
- 275
- References
- 5
- Citations by year
- 2026: 15 · 2025: 42 · 2024: 55 · 2023: 53 · 2022: 65 · 2021: 36 · 2020: 9
- FWCI
- 30.91
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion)
- NOVA (University of Newcastle Australia) (submittedVersion)
- Funders
- National Key Research and Development Program of China (2018YFB1004805)
- National Natural Science Foundation of China (61702024, 61932007, 61972013, 61421003)
- ARC DP (200102940)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- source code summarization, neural machine translation, low-frequency words, program comprehension, code maintenance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
