Predictive Comment Updating With Heuristics and AST-Path-Based Neural Learning: A Two-Phase Approach
Open accessThe DOI matches its Crossref record; 79 of the 79 references it cites were found in a registry or an index.
Abstract
Just-in-time comment update is a promising way to reduce the burden of developers during software maintenance and evolution. Existing approaches can be divided into two categories: the heuristic-based approach and the deep-learning-based approach. The heuristic-based approach is restricted to a specific type of comment updates (i.e., code-indicative updates), but performs well on such type. The effectiveness of deep-learning-based approach is limited but it can handle diverse comment updates. Considering the complementary advantages of existing approaches, an intuitive idea is to combine them for better performance. To investigate this idea, we first conduct a pre-study experiment which shows that to construct an effective comment updater by combining heuristic-based and deep-learning-based approaches, we need to tackle two main challenges: 1) the heuristic-based approach may bring side effects to cases which cannot be updated by it; and 2) the current deep-learning-based approach is with limited effectiveness. Then, we propose a novel two-phase approach named Toper to cope with these two challenges and effectively perform comment updates. In the first phase, Toper integrates nine distinctive features identified through our large-scale empirical analysis into a predictive model, which can predict whether the contents of the comment updates can be found in the corresponding code changes, namely, the comment updates are code-indicative updates. If so, the updates are then generated by an off-the-shelf heuristic-based approach; otherwise, Toper leverages a deep learning model, which we specially designed for non-code-indicative updates, to infer the new comment based on the old comment and code change. Motivated by our manual observation on the limitation of existing approaches on non-code-indicative updates, our deep learning model adopts the Abstract Syntax Tree path technique, which can capture the program structure information for effectively embedding code changes. Our evaluation shows that our approach outperforms the state-of-the-art by around 20% with respect to the number of correct comments it generates. Via in-depth analysis, we illustrate the rationale of each design decision as well as point out potential directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures












Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354
On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other
Mann, H. B., & Whitney, D. R. (1947) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177730491 · Cited by 14,320
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775 · cited 2×
“We briefly introduce the pointer generator here and readers can refer to [25], [59] for details.”
BackgroundArcuri, A., & Briand, L. (2011) · Proceedings of the 33rd International Conference on Software Engineering · doi:10.1145/1985793.1985795 · Cited by 968
code2vec: Learning distributed representations of code
Alon, U., Zilberstein, M., Levy, O., & Yahav, E. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3290353 · Cited by 962 · cited 4×
“The main job of the attention mechanism is to compute a scalar weight for each path [28].”
BackgroundMethodInfluentialClassifying Software Changes: Clean or Buggy?
Kim, S., Whitehead, & Zhang, Y. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.70773 · Cited by 672 · cited 1×
“These features measure the code change complexity from different granularities and are thus widely used in the previous studies[34], [35], [36], [37], [38], [39], [40].”
MethodHu, X., Li, G., Xia, X., Lo, D., & Jin, Z. (2018) · Proceedings of the 26th Conference on Program Comprehension · doi:10.1145/3196321.3196334 · Cited by 621 · cited 1×
“instance, traditional code summarization techniques [31], [32] could be applied to directly generate comments for the updated code although doing so would ignore the code change information, which is important for guiding comment updates.”
MethodSelf-citationOrdinal Methods for Behavioral Data Analysis
Cliff, N. (2014) · Psychology Press eBooks · doi:10.4324/9781315806730 · Cited by 560 · cited 1×
“We also computed Cliff’s delta [43], a non-parametric effect size measure that can evaluate the amount of difference between two variables.”
MethodFine-grained and accurate source code differencing
Falleri, J.-R., Morandat, F., Blanc, X., Martinez, M., & Monperrus, M. (2014) · Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering · doi:10.1145/2642937.2642982 · Cited by 549 · cited 2×
“To extract the structure information, we first generate the ASTs of the old (AO) and new methods (AN ) using a widely-adopted tool, Gumtree [56].”
MethodTowards automatically generating summary comments for Java methods
Sridhara, G., Hill, E., Muppaneni, D., Pollock, L., & Vijay-Shanker, K. (2010) · Proceedings of the IEEE/ACM international conference on Automated software engineering · doi:10.1145/1858996.1859006 · Cited by 514 · cited 1×
“[70] proposed to generate natural language summarization for methods via manually defined templates.”
Method
Research works printed but not linked: 6
- 10.1109/ICSME52107.2021.00021 · doi:10.1109/icsme52107.2021.00021
- Zhang, Leveraging class hierarchy for code comprehension
- Alon, code2seq: Generating sequences from structured representations of code
- Mockus, Best practices and pitfalls for statistical analysis of se data
- Patil, 2013, Performance analysis of naive bayes and j48 classification algorithm for data classification, Int. J. Comput. Sci. Appl., 6, 256
- Wang, Automated patch correctness assessment: How far are we?, 968
Cited by
Geng, M., Wang, S., Dong, D., Wang, H., Li, G., Jin, Z., Mao, X., & Liao, X. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3608134 · Cited by 155
Self-citationCCT5: A Code-Change-Oriented Pre-trained Model
Lin, B., Wang, S., Liu, Z., Liu, Y., Xia, X., & Mao, X. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616339 · Cited by 69
“cent studies also utilize the AST path technique [3] for representing the code change [35, 36], where the AST is split into changed and unchanged parts and the paths connecting different leaf nodes are embedded to represent each part.”
BackgroundMethodInfluentialSelf-citationExploring the Capabilities of LLMs for Code-Change-Related Tasks
Fan, L., Liu, J., Liu, Z., Lo, D., Xia, X., & Li, S. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3709358 · Cited by 41
Self-citationMultilingual Code Co-evolution using Large Language Models
Zhang, J., Nie, P., Li, J. J., & Gligoric, M. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616350 · Cited by 60
Interpretation-based Code Summarization
Geng, M., Wang, S., Dong, D., Wang, H., Cao, S., Zhang, K., & Jin, Z. G. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00026 · Cited by 29
“A branch of studies focus on updating outdated comments according to code changes, which can be considered as another form of comment generation problem [30], [31].”
BackgroundSelf-citation
Metadata
- Authors
- Lin, B., Wang, S., Liu, Z., Xia, X., & Mao, X.
- Affiliations
- National University of Defense Technology (CN)
- Zhejiang University (CN)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2023-04-01
- Volume
- 49
- Issue
- 4
- Pages
- 1640-1660
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W4293211675
- Cited by
- 27
- References
- 77
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 11 · 2023: 8
- FWCI
- 5.94
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Transactions on Software Engineering (publishedVersion, cc-by)
- Funders
- National Natural Science Foundation of China (61872445, 61672529)
- Zhejiang University Education Foundation Qizhen Scholar Foundation
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- two-stage approach, software maintenance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Predictive Comment Updating With Heuristics and AST-Path-Based Neural Learning: A Two-Phase Approach. https://app.cochat.ai/papers/10.1109/tse.2022.3185458
- The paper
- Lin, B., Wang, S., Liu, Z., Xia, X., & Mao, X. (2023). Predictive Comment Updating With Heuristics and AST-Path-Based Neural Learning: A Two-Phase Approach. IEEE Transactions on Software Engineering. https://doi.org/10.1109/tse.2022.3185458
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.