CoChat

Reinforcement-Learning-Guided Source Code Summarization Using Hierarchical Attention

Open access
Wang, Wenhua, Zhang, Yuqun, Sui, Yulei, Wan, Yao, Zhao, Zhou, Wu, Jian, Yu, Philip S., Xu, Guandong · 2022 · IEEE Transactions on Software Engineering
Cited by 103

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Abstract

Code summarization (aka comment generation) provides a high-level natural language description of the function performed by code, which can benefit the software maintenance, code categorization and retrieval. To the best of our knowledge, the state-of-the-art approaches follow an encoder-decoder framework which encodes source code into a hidden space and later decodes it into a natural language space. Such approaches suffer from the following drawbacks: (a) they are mainly input by representing code as a sequence of tokens while ignoring code hierarchy; (b) most of the encoders only input simple features (e.g., tokens) while ignoring the features that can help capture the correlations between comments and code; (c) the decoders are typically trained to predict subsequent words by maximizing the likelihood of subsequent ground truth words, while in real world, they are excepted to generate the entire word sequence from scratch. As a result, such drawbacks lead to inferior and inconsistent comment generation accuracy. To address the above limitations, this paper presents a new code summarization approach using hierarchical attention network by incorporating multiple code features, including type-augmented abstract syntax trees and program control flows. Such features, along with plain code sequences, are injected into a deep reinforcement learning (DRL) framework (e.g., actor-critic network) for comment generation. Our approach assigns weights (pays “attention”) to tokens and statements when constructing the code representation to reflect the hierarchical code structure under different contexts regarding code features (e.g., control flows and abstract syntax trees). Our reinforcement learning mechanism further strengthens the prediction results through the actor network and the critic network, where the actor network provides the confidence of predicting subsequent words based on the current state, and the critic network computes the reward values of all the possible extensions of the current state to provide global guidance for explorations. Eventually, we employ an advantage reward to train both networks and conduct a set of experiments on a real-world dataset. The experimental results demonstrate that our approach outperforms the baselines by around 22 to 45 percent in BLEU-1 and outperforms the state-of-the-art approaches by around 5 to 60 percent in terms of S-BLEU and C-BLEU.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tse.2020.2979701

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 15 references

Research works printed but not linked: 69

  1. Lientz, Software maintenance management, 127
  2. 10.1109/ASE.2011.6100145 · DOI
  3. 10.1109/ICSE.2017.45 · DOI · Paper page
  4. 10.1145/3180155.3180222 · DOI
  5. 10.1145/2786805.2786880 · DOI
  6. 10.1007/s10009-019-00512-8 · DOI
  7. 10.1007/s11280-018-0562-5 · doi:10.1007/s11280-018-0562-5
  8. 10.1145/3180155.3180198 · DOI
  9. 10.1145/3238147.3238187 · DOI · Paper page
  10. 10.1109/ICST.2019.00033 · DOI
  11. 10.1145/3377811.3380422 · DOI
  12. 10.1109/SOSE.2017.26 · DOI
  13. 10.1145/3377811.3380358 · DOI
  14. 10.1109/TSE.2019.2911283 · DOI · Paper page
  15. 10.1109/ICSE.2015.47 · DOI
  16. 10.1145/3293882.3330559 · DOI · Paper page
  17. 10.1145/2950290.2950295 · DOI
  18. 10.1109/ASE.2019.00075 · DOI
  19. Abrahamsson, 2002, Agile software development methods review and analysis
  20. 10.1109/ICSE-C.2017.76 · DOI
  21. 10.1145/1085313.1085331 · DOI · Paper page
  22. 10.1023/b:lida.0000048322.42751.ca · DOI
  23. Movshovitz-Attias, Natural language models for predicting programming comments, 35
  24. 10.18653/v1/P16-1195 · DOI · Paper page
  25. Yang, From query to usable code: An analysis of stack overflow code snippets, 391
  26. 10.1109/TSC.2016.2560165 · DOI
  27. 10.1109/ICSE-C.2017.118 · DOI
  28. 10.1109/ASE.2015.36 · DOI · Paper page
  29. Allamanis, A convolutional attention network for extreme summarization of source code, 2091
  30. 10.1016/j.neunet.2014.09.003 · DOI · Paper page
  31. 10.24963/ijcai.2018/314 · DOI · Paper page
  32. 10.18653/v1/D16-1137 · DOI · Paper page
  33. 10.1145/3238147.3238206 · DOI · Paper page
  34. Wang, Bugram: Bug detection with N-gram language models, 708
  35. 10.1109/5.880083 · DOI · Paper page
  36. 10.1109/TNN.2004.842673 · DOI · Paper page
  37. 10.1609/aaai.v31i1.10804 · DOI · Paper page
  38. Sutton, 1998, Introduction to Reinforcement Learning, 135
  39. 10.1007/BF00992696 · DOI · Paper page
  40. Watkins, 1992, Q-learning, Mach. Learn., 8, 279, 10.1023/A:1022676722315 · DOI · Paper page
  41. 10.1137/s0363012901385691 · DOI · Paper page
  42. Konda, Actor-critic algorithms, 1008
  43. 10.1145/3308558.3313632 · DOI · Paper page
  44. Aho, 1986, Compilers, principles, techniques, Addison Wesley, 7
  45. 10.18653/v1/n16-1174 · DOI · Paper page
  46. 10.1007/978-3-030-29513-4_31 · DOI · Paper page
  47. Hu, Deep code comment generation, 200
  48. Duchi, 2011, Adaptive subgradient methods for online learning and stochastic optimization, J. Mach. Learn. Res., 12, 2121
  49. 10.1007/s10664-019-09730-9 · DOI · Paper page
  50. Banerjee, METEOR: An automatic metric for MT evaluation with improved correlation with human judgments, 29, 65
  51. Lin, 2004, ROUGE: A package for automatic evaluation of summaries, Proc. Workshop Text Summarization Branches Out, 1
  52. 10.18653/v1/P18-1142 · DOI
  53. 10.18653/v1/w19-0309 · doi:10.18653/v1/w19-0309
  54. 10.18653/v1/D18-1492 · DOI
  55. 10.3115/v1/W14-4009 · DOI
  56. 10.18653/v1/P17-1064 · DOI · Paper page
  57. Sutskever, Sequence to sequence learning with neural networks, 3104
  58. 10.18653/v1/E17-1019 · DOI
  59. 10.1145/2950290.2950334 · DOI · Paper page
  60. 10.1609/aaai.v30i1.10139 · DOI · Paper page
  61. Maddison, Structured generative models of natural source code, 649
  62. Allamanis, Bimodal modelling of source code and natural language, 2123
  63. 10.1145/3238147.3240471 · DOI · Paper page
  64. 10.1145/3238147.3238190 · DOI · Paper page
  65. Sutton, Policy gradient methods for reinforcement learning with function approximation, 1057
  66. 10.1038/nature14236 · DOI · Paper page
  67. 10.1109/CVPR.2017.128 · DOI · Paper page
  68. 10.18653/v1/D16-1127 · DOI · Paper page
  69. 10.18653/v1/D17-1062 · DOI · Paper page

Cited by

See all 103

Metadata

Authors
Wang, W., Zhang, Y., Sui, Y., Wan, Y., Zhao, Z., Wu, J., Yu, P. S., & Xu, G.
Venue
IEEE Transactions on Software Engineering
Published
2022-01-01
Volume
48
Issue
1
Pages
102-119
ISSN
  • 0098-5589 (ISSN-L)
  • 1939-3520
  • 2326-3881
OpenAlex
W3011632945
Cited by
103
References
15
FWCI
12.78
Type
journal article
Language
en
Open access
green
Funders
  • National Natural Science Foundation of China (61902169)
  • Shenzhen Science and Technology Program (KQTD2016112514355531)
  • Science and Technology Innovation Committee Foundation of Shenzhen (JCYJ20170817110848086)
  • Australian Research Council (DP200101374, LP170100891, DE170101081, DP200101328)
  • National Key Research and Development Program of China (2018AAA0102100, 2019YFC0118802)
  • Zhejiang public welfare technology (LGF20F020013)
  • National Institutes for Food and Drug Control (2019YFB1404802)
  • National Science Foundation (III-1526499, III-1763325, III-1909323, CNS-1930941)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Reinforcement-Learning-Guided Source Code Summarization Using Hierarchical Attention. https://app.cochat.ai/papers/10.1109/tse.2020.2979701
The paper
Wang, W., Zhang, Y., Sui, Y., Wan, Y., Zhao, Z., Wu, J., Yu, P. S., & Xu, G. (2022). Reinforcement-Learning-Guided Source Code Summarization Using Hierarchical Attention. IEEE Transactions on Software Engineering. https://doi.org/10.1109/tse.2020.2979701

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.