Reinforcement-Learning-Guided Source Code Summarization Using Hierarchical Attention
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Code summarization (aka comment generation) provides a high-level natural language description of the function performed by code, which can benefit the software maintenance, code categorization and retrieval. To the best of our knowledge, the state-of-the-art approaches follow an encoder-decoder framework which encodes source code into a hidden space and later decodes it into a natural language space. Such approaches suffer from the following drawbacks: (a) they are mainly input by representing code as a sequence of tokens while ignoring code hierarchy; (b) most of the encoders only input simple features (e.g., tokens) while ignoring the features that can help capture the correlations between comments and code; (c) the decoders are typically trained to predict subsequent words by maximizing the likelihood of subsequent ground truth words, while in real world, they are excepted to generate the entire word sequence from scratch. As a result, such drawbacks lead to inferior and inconsistent comment generation accuracy. To address the above limitations, this paper presents a new code summarization approach using hierarchical attention network by incorporating multiple code features, including type-augmented abstract syntax trees and program control flows. Such features, along with plain code sequences, are injected into a deep reinforcement learning (DRL) framework (e.g., actor-critic network) for comment generation. Our approach assigns weights (pays “attention”) to tokens and statements when constructing the code representation to reflect the hierarchical code structure under different contexts regarding code features (e.g., control flows and abstract syntax trees). Our reinforcement learning mechanism further strengthens the prediction results through the actor network and the critic network, where the actor network provides the confidence of predicting subsequent words based on the current state, and the critic network computes the reward values of all the possible extensions of the current state to provide global guidance for explorations. Eventually, we employ an advantage reward to train both networks and conduct a set of experiments on a real-world dataset. The experimental results demonstrate that our approach outperforms the baselines by around 22 to 45 percent in BLEU-1 and outperforms the state-of-the-art approaches by around 5 to 60 percent in terms of S-BLEU and C-BLEU.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · cited 1×
“For the decoding, since we design a hierarchical and multi-dimensional input, we decide to adopt the Inputfeeding attention mechanism [64] to predict the tth word by using a softmax function.”
MethodTarget-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533 · cited 1×
“A visual navigation system [96] has been proposed recently based on actor-critic reinforcement learning model.”
MethodSequence Level Training with Recurrent Neural Networks
Ranzato, M., Chopra, S., Auli, M., & Zaremba, W. (2015) · arXiv · doi:10.48550/arxiv.1511.06732 · Cited by 1,426 · cited 2×
“We record the perplexity [37]/reward every 50 iterations.”
BackgroundClone detection using abstract syntax trees
Baxter, I. D., Yahin, A., Moura, L., Sant'Anna, M., & Bier, L. (2002) · Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272) · doi:10.1109/icsm.1998.738528 · Cited by 1,277
The oracle problem in software testing: A survey
Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2372785 · Cited by 1,104
Towards automatically generating summary comments for Java methods
Sridhara, G., Hill, E., Muppaneni, D., Pollock, L., & Vijay-Shanker, K. (2010) · Proceedings of the IEEE/ACM international conference on Automated software engineering · doi:10.1145/1858996.1859006 · Cited by 514
Research works printed but not linked: 69
- Lientz, Software maintenance management, 127
- 10.1109/ASE.2011.6100145 · DOI
- 10.1109/ICSE.2017.45 · DOI · Paper page
- 10.1145/3180155.3180222 · DOI
- 10.1145/2786805.2786880 · DOI
- 10.1007/s10009-019-00512-8 · DOI
- 10.1007/s11280-018-0562-5 · doi:10.1007/s11280-018-0562-5
- 10.1145/3180155.3180198 · DOI
- 10.1145/3238147.3238187 · DOI · Paper page
- 10.1109/ICST.2019.00033 · DOI
- 10.1145/3377811.3380422 · DOI
- 10.1109/SOSE.2017.26 · DOI
- 10.1145/3377811.3380358 · DOI
- 10.1109/TSE.2019.2911283 · DOI · Paper page
- 10.1109/ICSE.2015.47 · DOI
- 10.1145/3293882.3330559 · DOI · Paper page
- 10.1145/2950290.2950295 · DOI
- 10.1109/ASE.2019.00075 · DOI
- Abrahamsson, 2002, Agile software development methods review and analysis
- 10.1109/ICSE-C.2017.76 · DOI
- 10.1145/1085313.1085331 · DOI · Paper page
- 10.1023/b:lida.0000048322.42751.ca · DOI
- Movshovitz-Attias, Natural language models for predicting programming comments, 35
- 10.18653/v1/P16-1195 · DOI · Paper page
- Yang, From query to usable code: An analysis of stack overflow code snippets, 391
- 10.1109/TSC.2016.2560165 · DOI
- 10.1109/ICSE-C.2017.118 · DOI
- 10.1109/ASE.2015.36 · DOI · Paper page
- Allamanis, A convolutional attention network for extreme summarization of source code, 2091
- 10.1016/j.neunet.2014.09.003 · DOI · Paper page
- 10.24963/ijcai.2018/314 · DOI · Paper page
- 10.18653/v1/D16-1137 · DOI · Paper page
- 10.1145/3238147.3238206 · DOI · Paper page
- Wang, Bugram: Bug detection with N-gram language models, 708
- 10.1109/5.880083 · DOI · Paper page
- 10.1109/TNN.2004.842673 · DOI · Paper page
- 10.1609/aaai.v31i1.10804 · DOI · Paper page
- Sutton, 1998, Introduction to Reinforcement Learning, 135
- 10.1007/BF00992696 · DOI · Paper page
- Watkins, 1992, Q-learning, Mach. Learn., 8, 279, 10.1023/A:1022676722315 · DOI · Paper page
- 10.1137/s0363012901385691 · DOI · Paper page
- Konda, Actor-critic algorithms, 1008
- 10.1145/3308558.3313632 · DOI · Paper page
- Aho, 1986, Compilers, principles, techniques, Addison Wesley, 7
- 10.18653/v1/n16-1174 · DOI · Paper page
- 10.1007/978-3-030-29513-4_31 · DOI · Paper page
- Hu, Deep code comment generation, 200
- Duchi, 2011, Adaptive subgradient methods for online learning and stochastic optimization, J. Mach. Learn. Res., 12, 2121
- 10.1007/s10664-019-09730-9 · DOI · Paper page
- Banerjee, METEOR: An automatic metric for MT evaluation with improved correlation with human judgments, 29, 65
- Lin, 2004, ROUGE: A package for automatic evaluation of summaries, Proc. Workshop Text Summarization Branches Out, 1
- 10.18653/v1/P18-1142 · DOI
- 10.18653/v1/w19-0309 · doi:10.18653/v1/w19-0309
- 10.18653/v1/D18-1492 · DOI
- 10.3115/v1/W14-4009 · DOI
- 10.18653/v1/P17-1064 · DOI · Paper page
- Sutskever, Sequence to sequence learning with neural networks, 3104
- 10.18653/v1/E17-1019 · DOI
- 10.1145/2950290.2950334 · DOI · Paper page
- 10.1609/aaai.v30i1.10139 · DOI · Paper page
- Maddison, Structured generative models of natural source code, 649
- Allamanis, Bimodal modelling of source code and natural language, 2123
- 10.1145/3238147.3240471 · DOI · Paper page
- 10.1145/3238147.3238190 · DOI · Paper page
- Sutton, Policy gradient methods for reinforcement learning with function approximation, 1057
- 10.1038/nature14236 · DOI · Paper page
- 10.1109/CVPR.2017.128 · DOI · Paper page
- 10.18653/v1/D16-1127 · DOI · Paper page
- 10.18653/v1/D17-1062 · DOI · Paper page
Cited by
CURE: Code-Aware Neural Machine Translation for Automatic Program Repair
Jiang, N., Lutellier, T., & Tan, L. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00107 · Cited by 290
An extensive study on pre-trained models for program understanding and generation
Zeng, Z., Tan, H., Zhang, H., Li, J., Zhang, Y., & Zhang, L. (2022) · Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3533767.3534390 · Cited by 153
“In recent years, researchers have explored various techniques in the intersection of software engineering and machine learning [5, 8, 23, 36, 41, 70, 72, 78, 81, 82].”
BackgroundSelf-citationReassessing automatic evaluation metrics for code summarization tasks
Roy, D., Fakhoury, S., & Arnaoudova, V. (2021) · Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3468264.3468588 · Cited by 106
(2023) · doi:10.1145/3510003.3510060 · Cited by 88
“CodeNN [29] is an early attempt that uses only code token sequences, followed by various approaches that utilize AST [4, 25, 26, 33, 34, 37, 54], API knowledge [27], type information [9], global context [6, 22, 63], reinforcement learning [61, 62], multi-task learning [67], dual learning [64, 68], and pre-trained lang…”
MethodMultilingual training for software engineering
Ahmed, T., & Devanbu, P. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510049 · Cited by 57
Metadata
- Authors
- Wang, W., Zhang, Y., Sui, Y., Wan, Y., Zhao, Z., Wu, J., Yu, P. S., & Xu, G.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2022-01-01
- Volume
- 48
- Issue
- 1
- Pages
- 102-119
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W3011632945
- Cited by
- 103
- References
- 15
- FWCI
- 12.78
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- National Natural Science Foundation of China (61902169)
- Shenzhen Science and Technology Program (KQTD2016112514355531)
- Science and Technology Innovation Committee Foundation of Shenzhen (JCYJ20170817110848086)
- Australian Research Council (DP200101374, LP170100891, DE170101081, DP200101328)
- National Key Research and Development Program of China (2018AAA0102100, 2019YFC0118802)
- Zhejiang public welfare technology (LGF20F020013)
- National Institutes for Food and Drug Control (2019YFB1404802)
- National Science Foundation (III-1526499, III-1763325, III-1909323, CNS-1930941)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reinforcement-Learning-Guided Source Code Summarization Using Hierarchical Attention. https://app.cochat.ai/papers/10.1109/tse.2020.2979701
- The paper
- Wang, W., Zhang, Y., Sui, Y., Wan, Y., Zhao, Z., Wu, J., Yu, P. S., & Xu, G. (2022). Reinforcement-Learning-Guided Source Code Summarization Using Hierarchical Attention. IEEE Transactions on Software Engineering. https://doi.org/10.1109/tse.2020.2979701
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.