CoChat

Predictive Comment Updating With Heuristics and AST-Path-Based Neural Learning: A Two-Phase Approach

Open access
Lin, Bo, Wang, Shangwen, Liu, Zhongxin, Xia, Xin, Mao, Xiaoguang · 2023 · IEEE Transactions on Software Engineering

The DOI matches its Crossref record; 79 of the 79 references it cites were found in a registry or an index.

Abstract

Just-in-time comment update is a promising way to reduce the burden of developers during software maintenance and evolution. Existing approaches can be divided into two categories: the heuristic-based approach and the deep-learning-based approach. The heuristic-based approach is restricted to a specific type of comment updates (i.e., code-indicative updates), but performs well on such type. The effectiveness of deep-learning-based approach is limited but it can handle diverse comment updates. Considering the complementary advantages of existing approaches, an intuitive idea is to combine them for better performance. To investigate this idea, we first conduct a pre-study experiment which shows that to construct an effective comment updater by combining heuristic-based and deep-learning-based approaches, we need to tackle two main challenges: 1) the heuristic-based approach may bring side effects to cases which cannot be updated by it; and 2) the current deep-learning-based approach is with limited effectiveness. Then, we propose a novel two-phase approach named Toper to cope with these two challenges and effectively perform comment updates. In the first phase, Toper integrates nine distinctive features identified through our large-scale empirical analysis into a predictive model, which can predict whether the contents of the comment updates can be found in the corresponding code changes, namely, the comment updates are code-indicative updates. If so, the updates are then generated by an off-the-shelf heuristic-based approach; otherwise, Toper leverages a deep learning model, which we specially designed for non-code-indicative updates, to infer the new comment based on the old comment and code change. Motivated by our manual observation on the limitation of existing approaches on non-code-indicative updates, our deep learning model adopts the Abstract Syntax Tree path technique, which can capture the program structure information for effectively embedding code changes. Our evaluation shows that our approach outperforms the state-of-the-art by around 20% with respect to the number of correct comments it generates. Via in-depth analysis, we illustrate the rationale of each design decision as well as point out potential directions.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tse.2022.3185458

Cites

79 of 79 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 An instance of code-indicative update.
Fig. 1 An instance of code-indicative update.CC-BY · From the PDF
Fig. 2 Value distributions of the eight features.
Fig. 2 Value distributions of the eight features.CC-BY · From the PDF
TABLE 6: An incorrect comment update from CUP.
TABLE 6: An incorrect comment update from CUP.CC-BY · From the PDF
Image 4
CC-BY · From the PDF
(a) An instance from our test set.
(a) An instance from our test set.CC-BY · From the PDF
Fig. 4 Overview of our approach.
Fig. 4 Overview of our approach.CC-BY · From the PDF
Image 7
CC-BY · From the PDF
by Toper .
by Toper .CC-BY · From the PDF
TABLE 13: A non-code-indicative instance misclassified by the classifier.
TABLE 13: A non-code-indicative instance misclassified by the classifier.CC-BY · From the PDF
TABLE 14: A code-indicative instance updated incorrectly by HebCup.
TABLE 14: A code-indicative instance updated incorrectly by HebCup.CC-BY · From the PDF
Fig. 6 The performance of code-indicative updater for different proportions of code-indicative instances from the training set.
Fig. 6 The performance of code-indicative updater for different proportions of code-indicative instances from the training set.CC-BY · From the PDF
Fig. 7 The distribution of the Levenshtein distance required to change each incorrect update into its corresponding groundtruth.
Fig. 7 The distribution of the Levenshtein distance required to change each incorrect update into its corresponding groundtruth.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 79 references

Research works printed but not linked: 6

  1. 10.1109/ICSME52107.2021.00021 · doi:10.1109/icsme52107.2021.00021
  2. Zhang, Leveraging class hierarchy for code comprehension
  3. Alon, code2seq: Generating sequences from structured representations of code
  4. Mockus, Best practices and pitfalls for statistical analysis of se data
  5. Patil, 2013, Performance analysis of naive bayes and j48 classification algorithm for data classification, Int. J. Comput. Sci. Appl., 6, 256
  6. Wang, Automated patch correctness assessment: How far are we?, 968

Cited by

See all 27

Metadata

Authors
Lin, B., Wang, S., Liu, Z., Xia, X., & Mao, X.
Affiliations
  • National University of Defense Technology (CN)
  • Zhejiang University (CN)
Venue
IEEE Transactions on Software Engineering
Published
2023-04-01
Volume
49
Issue
4
Pages
1640-1660
ISSN
  • 0098-5589 (ISSN-L)
  • 1939-3520
  • 2326-3881
OpenAlex
W4293211675
Cited by
27
References
77
Citations by year
2026: 3 · 2025: 5 · 2024: 11 · 2023: 8
FWCI
5.94
Type
journal article
Language
en
Open access
Locations
IEEE Transactions on Software Engineering (publishedVersion, cc-by)
Funders
  • National Natural Science Foundation of China (61872445, 61672529)
  • Zhejiang University Education Foundation Qizhen Scholar Foundation
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
Keywords
two-stage approach, software maintenance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Predictive Comment Updating With Heuristics and AST-Path-Based Neural Learning: A Two-Phase Approach. https://app.cochat.ai/papers/10.1109/tse.2022.3185458
The paper
Lin, B., Wang, S., Liu, Z., Xia, X., & Mao, X. (2023). Predictive Comment Updating With Heuristics and AST-Path-Based Neural Learning: A Two-Phase Approach. IEEE Transactions on Software Engineering. https://doi.org/10.1109/tse.2022.3185458

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.