Predictive Comment Updating With Heuristics and AST-Path-Based Neural Learning: A Two-Phase Approach
Open accessThe DOI matches its Crossref record; 79 of the 79 references it cites were found in a registry or an index.
Abstract
Just-in-time comment update is a promising way to reduce the burden of developers during software maintenance and evolution. Existing approaches can be divided into two categories: the heuristic-based approach and the deep-learning-based approach. The heuristic-based approach is restricted to a specific type of comment updates (i.e., code-indicative updates), but performs well on such type. The effectiveness of deep-learning-based approach is limited but it can handle diverse comment updates. Considering the complementary advantages of existing approaches, an intuitive idea is to combine them for better performance. To investigate this idea, we first conduct a pre-study experiment which shows that to construct an effective comment updater by combining heuristic-based and deep-learning-based approaches, we need to tackle two main challenges: 1) the heuristic-based approach may bring side effects to cases which cannot be updated by it; and 2) the current deep-learning-based approach is with limited effectiveness. Then, we propose a novel two-phase approach named Toper to cope with these two challenges and effectively perform comment updates. In the first phase, Toper integrates nine distinctive features identified through our large-scale empirical analysis into a predictive model, which can predict whether the contents of the comment updates can be found in the corresponding code changes, namely, the comment updates are code-indicative updates. If so, the updates are then generated by an off-the-shelf heuristic-based approach; otherwise, Toper leverages a deep learning model, which we specially designed for non-code-indicative updates, to infer the new comment based on the old comment and code change. Motivated by our manual observation on the limitation of existing approaches on non-code-indicative updates, our deep learning model adopts the Abstract Syntax Tree path technique, which can capture the program structure information for effectively embedding code changes. Our evaluation shows that our approach outperforms the state-of-the-art by around 20% with respect to the number of correct comments it generates. Via in-depth analysis, we illustrate the rationale of each design decision as well as point out potential directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures












Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354
On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other
Mann, H. B., & Whitney, D. R. (1947) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177730491 · Cited by 14,320
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775 · cited 2×
“We briefly introduce the pointer generator here and readers can refer to [25], [59] for details.”
BackgroundArcuri, A., & Briand, L. (2011) · Proceedings of the 33rd International Conference on Software Engineering · doi:10.1145/1985793.1985795 · Cited by 968
code2vec: Learning distributed representations of code
Alon, U., Zilberstein, M., Levy, O., & Yahav, E. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3290353 · Cited by 962 · cited 4×
“The main job of the attention mechanism is to compute a scalar weight for each path [28].”
BackgroundMethodInfluentialClassifying Software Changes: Clean or Buggy?
Kim, S., Whitehead, & Zhang, Y. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.70773 · Cited by 672 · cited 1×
“These features measure the code change complexity from different granularities and are thus widely used in the previous studies[34], [35], [36], [37], [38], [39], [40].”
MethodHu, X., Li, G., Xia, X., Lo, D., & Jin, Z. (2018) · Proceedings of the 26th Conference on Program Comprehension · doi:10.1145/3196321.3196334 · Cited by 621 · cited 1×
“instance, traditional code summarization techniques [31], [32] could be applied to directly generate comments for the updated code although doing so would ignore the code change information, which is important for guiding comment updates.”
MethodSelf-citationOrdinal Methods for Behavioral Data Analysis
Cliff, N. (2014) · Psychology Press eBooks · doi:10.4324/9781315806730 · Cited by 560 · cited 1×
“We also computed Cliff’s delta [43], a non-parametric effect size measure that can evaluate the amount of difference between two variables.”
MethodFine-grained and accurate source code differencing
Falleri, J.-R., Morandat, F., Blanc, X., Martinez, M., & Monperrus, M. (2014) · Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering · doi:10.1145/2642937.2642982 · Cited by 549 · cited 2×
“To extract the structure information, we first generate the ASTs of the old (AO) and new methods (AN ) using a widely-adopted tool, Gumtree [56].”
MethodTowards automatically generating summary comments for Java methods
Sridhara, G., Hill, E., Muppaneni, D., Pollock, L., & Vijay-Shanker, K. (2010) · Proceedings of the IEEE/ACM international conference on Automated software engineering · doi:10.1145/1858996.1859006 · Cited by 514 · cited 1×
“[70] proposed to generate natural language summarization for methods via manually defined templates.”
MethodRevisiting the Impact of Classification Techniques on the Performance of Defect Prediction Models
Ghotra, B., McIntosh, S., & Hassan, A. E. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.91 · Cited by 364 · cited 1×
“Indeed, it has been shown to perform well in various classification tasks in software engineering [51], [52], [53],”
MethodA Neural Model for Generating Natural Language Summaries of Program Subroutines
LeClair, A., Jiang, S., & McMillan, C. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00087 · Cited by 294 · cited 2×
“[32] designed deep learning models to generate comments by integrating source code with structure information.”
BackgroundMethodDeep code comment generation with hybrid lexical and syntactical information
Hu, X., Li, G., Xia, X., Lo, D., & Jin, Z. (2020) · Empirical Software Engineering · doi:10.1007/s10664-019-09730-9 · Cited by 278 · cited 1×
“Hu et al. [73] and LeClair et al. [32] designed deep learning models to generate comments by integrating source code with structure information.”
BackgroundSelf-citation/*icomment: bugs or bad comments?*/
Tan, L., Yuan, D., Krishna, G., & Zhou, Y. (2007) · Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles · doi:10.1145/1294261.1294276 · Cited by 268 · cited 2×
“source code during software evolution [15], [16], [17], [18].”
BackgroundDo Code and Comments Co-Evolve? On the Relation between Source Code and Comment Changes
Fluri, B., Wursch, M., & Gall, H. C. (2007) · 14th Working Conference on Reverse Engineering (WCRE 2007) · doi:10.1109/wcre.2007.21 · Cited by 228 · cited 1×
“As revealed by existing studies, such inconsistencies can lead to future project bugs [15] and thus hinder software maintenance activities [19], [20], [21], [22], [23].”
BackgroundS3: syntax- and semantic-guided repair synthesis via programming by examples
Le, X.-B. D., Chu, D.-H., Lo, D., Le Goues, C., & Visser, W. (2017) · Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering · doi:10.1145/3106237.3106309 · Cited by 220 · cited 1×
“These features measure the code change complexity from different granularities and are thus widely used in the previous studies[34], [35], [36], [37], [38], [39], [40].”
MethodQuality analysis of source code comments
Steidl, D., Hummel, B., & Juergens, E. (2013) · 2013 21st International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2013.6613836 · Cited by 199 · cited 1×
“THE natural-language comment is crucial for understanding the associated code [1], [2], [3], [4], [5], [6], [7].”
Background@tComment: Testing Javadoc Comments to Detect Comment-Code Inconsistencies
Tan, S. H., Marinov, D., Tan, L., & Leavens, G. T. (2012) · 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation · doi:10.1109/icst.2012.106 · Cited by 197 · cited 2×
“This threat is mitigated considering that (1) Java is one of the most popular programming languages and (2) Javadoc comment is a type of critical information for developers to comprehend the code [68].”
BackgroundWhat are the characteristics of high-rated apps? A case study on free Android Applications
Tian, Y., Nagappan, M., Lo, D., & Hassan, A. E. (2015) · 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsm.2015.7332476 · Cited by 164 · cited 1×
“Following the previous study [46], we consider the features are correlated if the correlations of features in the sub-hierarchy are above 0.”
MethodCloCom: Mining existing source code for automatic comment generation
Wong, E., Liu, T., & Tan, L. (2015) · 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER) · doi:10.1109/saner.2015.7081848 · Cited by 162 · cited 1×
“ColCom [71] generates a comment for a code snippet by reusing comments of open source code snippets that are similar to the studied one.”
BackgroundLeveraging syntax-related code for automated program repair
Xin, Q., & Reiss, S. P. (2017) · 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115676 · Cited by 156 · cited 1×
“These features measure the code change complexity from different granularities and are thus widely used in the previous studies[34], [35], [36], [37], [38], [39], [40].”
MethodDissection of a bug dataset: Anatomy of 395 patches from Defects4J
Sobreira, V., Durieux, T., Madeiral, F., Monperrus, M., & Maia, M. D. A. (2018) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2018.8330203 · Cited by 140
The effect of modularization and comments on program comprehension
Woodfield, S. N., Dunsmore, H. E., & Shen, V. Y. (1981) · doi:10.5555/800078.802534 · Cited by 131
Zhang, J., Zhang, L., Harman, M., Hao, D., Jia, Y., & Zhang, L. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2809496 · Cited by 128 · cited 1×
“Indeed, it has been shown to perform well in various classification tasks in software engineering [51], [52], [53], [54].”
MethodFlow2Vec: value-flow-based precise code embedding
Sui, Y., Cheng, X. Y., Zhang, G., & Wang, H. (2020) · Proceedings of the ACM on Programming Languages · doi:10.1145/3428301 · Cited by 121 · cited 2×
“, using data-flows and control-flows) [55], [65] would be beneficial to capture the code change semantics and thus deserve exploration in future studies.”
BackgroundResultStorey, M.-A., Ryall, J., Bull, R. I., Myers, D., & Singer, J. (2008) · Proceedings of the 13th international conference on Software engineering - ICSE '08 · doi:10.1145/1368088.1368123 · Cited by 120
Analyzing APIs Documentation and Code to Detect Directive Defects
Zhou, Y., Gu, R., Chen, T., Huang, Z., Panichella, S., & Gall, H. (2017) · 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2017.11 · Cited by 115 · cited 1×
“[74] focused on defects of Application Programming Interface (API) documentation and detected such issues based on the analysis results from a constraint solver.”
MethodChaff from the Wheat: Characterizing and Determining Valid Bug Reports
Fan, Y., Xia, X., Lo, D., & Hassan, A. E. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2864217 · Cited by 114 · cited 2×
“’s study [44], we investigate the possible associations between features and apply feature selection, aiming to remove correlated features that might lead to poor models [45].”
BackgroundMethodSelf-citationRodríguez-Pérez, G., Robles, G., & Gonzalez-Barahona, J. M. (2018) · Information and Software Technology · doi:10.1016/j.infsof.2018.03.009 · Cited by 109 · cited 1×
“Any different minor setting in this process may lead to major differences [69].”
BackgroundDissection of a Bug Dataset:Anatomy of 395 Patches from Defects4J
Sobreira, V. (., Durieux, T., Madeiral, F., Monperrus, M., & Maia, M. A. (. A. ). (2018) · LillOA (Université de Lille (University Of Lille)) · Cited by 107
The impact of test case summaries on bug fixing performance: an empirical investigation
Panichella, S., Panichella, A., Beller, M., Zaidman, A., & Gall, H. C. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884847 · Cited by 106 · cited 1×
“[72] generated the summarization of test cases by first obtaining the coverage information and then exploiting Java naming conventions to construct readable natural language sentences.”
MethodAutomatic Generation of Pull Request Descriptions
Liu, Z., Xia, X., Treude, C., Lo, D., & Li, S. (2019) · 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00026 · Cited by 106 · cited 1×
“[79] designed a novel seq2seq model where the pointer gen-”
MethodSelf-citationA Large-Scale Empirical Study on Code-Comment Inconsistencies
Wen, F., Nagy, C., Bavota, G., & Lanza, M. (2019) · 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2019.00019 · Cited by 105 · cited 2×
“[17] performed a large-scale investigation towards the introduction/fixing of code-comment inconsistencies where the empirical results can better guide the design of novel approaches towards the detection of inconsistent comments.”
BackgroundA Large Scale Study of Long-Time Contributor Prediction for GitHub Projects
Bao, L., Xia, X., Lo, D., & Murphy, G. C. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2918536 · Cited by 103 · cited 1×
“Indeed, it has been shown to perform well in various classification tasks in software engineering [51], [52], [53],”
MethodSelf-citation9.6 Million Links in Source Code Comments: Purpose, Evolution, and Decay
Hata, H., Treude, C., Kula, R. G., & Ishio, T. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00123 · Cited by 94 · cited 1×
“THE natural-language comment is crucial for understanding the associated code [1], [2], [3], [4], [5], [6], [7].”
BackgroundSuggesting natural method names to check name consistencies
Nguyen, S., Phan, H., Le, T., & Nguyen, T. N. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380926 · Cited by 89 · cited 1×
“Nguyen et al. [76] and Wang et al. [29] suggested high-quality names for methods following a seq2seq paradigm, while Alon et al. [27], [28] finished the same task by exploiting the code structure information.”
Method(2023) · doi:10.1145/3510003.3510060 · Cited by 88
Listening to programmers — Taxonomies and characteristics of comments in operating system code
Padioleau, Y., Tan, L., & Zhou, Y. (2009) · 2009 IEEE 31st International Conference on Software Engineering · doi:10.1109/icse.2009.5070533 · Cited by 86 · cited 1×
“THE natural-language comment is crucial for understanding the associated code [1], [2], [3], [4], [5], [6], [7].”
BackgroundA Self-Attentional Neural Architecture for Code Completion with Multi-Task Learning
Liu, F., Li, G., Wei, B., Xia, X., Fu, Z., & Jin, Z. (2020) · Proceedings of the 28th International Conference on Program Comprehension · doi:10.1145/3387904.3389261 · Cited by 84 · cited 1×
“[78] captured both the structure information and long-term”
BackgroundSelf-citationDuplicate Bug Report Detection Using Dual-Channel Convolutional Neural Networks
He, J., Xu, L., Yan, M. L., Xia, X., & Lei, Y. (2020) · International Conference on Program Comprehension · doi:10.1145/3387904.3389263 · Cited by 81
Self-citationA general path-based representation for predicting program properties
Alon, U., Zilberstein, M., Levy, O., & Yahav, E. (2018) · Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/3192366.3192412 · Cited by 79 · cited 1×
“[80] first pointed out that using paths in the program’s AST significantly lowers the learning effort (compared to learning over program text) and is still scalable and general.”
BackgroundFlakeFlagger: Predicting Flakiness Without Rerunning Tests
Alshammari, A., Morris, C., Hilton, M. L., & Bell, J. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00140 · Cited by 75 · cited 1×
“To answer RQ2, we measured the precision, recall, accuracy, F-score, and AUC, which are widely used in assessing predictive models [44], [51], [62], [63].”
MethodOn the relationship between comment update practices and Software Bugs
Ibrahim, W. M., Bettenburg, N., Adams, B., & Hassan, A. E. (2011) · Journal of Systems and Software · doi:10.1016/j.jss.2011.09.019 · Cited by 73
Ratol, I. K., & Robillard, M. P. (2017) · 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115624 · Cited by 70 · cited 3×
“Ratol and Robillard [16] proposed to detect comments that become inconsistent during identifier renaming via a set of human-written rules.”
BackgroundYing, A. T. T., Wright, J. L., & Abrams, S. (2005) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/1082983.1083152 · Cited by 67
Learning to Update Natural Language Comments Based on Code Changes
Panthaplackel, S., Nie, P., Gligoric, M., Li, J. J., & Mooney, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.168 · Cited by 66
Do automated program repair techniques repair hard and important bugs?
Motwani, M., Sankaranarayanan, S., Just, R., & Brun, Y. (2017) · Empirical Software Engineering · doi:10.1007/s10664-017-9550-0 · Cited by 63 · cited 1×
“These features measure the code change complexity from different granularities and are thus widely used in the previous studies[34], [35], [36], [37], [38], [39], [40].”
MethodBeyond accuracy: assessing software documentation quality
Treude, C., Middleton, J., & Atapattu, T. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3417045 · Cited by 60 · cited 1×
“As revealed by existing studies, such inconsistencies can lead to future project bugs [15] and thus hinder softwaremaintenance activities [19], [20], [21], [22], [23].”
BackgroundClassifying code comments in Java software systems
Pascarella, L., Bruntink, M., & Bacchelli, A. (2019) · Empirical Software Engineering · doi:10.1007/s10664-019-09694-w · Cited by 57 · cited 2×
“Usually, the comments record information, such as the intention of the functionality as well as the implementation details, which can facilitate the communication among developers [2], [8], [9], [10], [11], [12], [13], [14].”
BackgroundAutomating just-in-time comment updating
Liu, Z., Xia, X., Yan, M., & Li, S. (2020) · Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3324884.3416581 · Cited by 56 · cited 10×
“, it can work for both code-indicative and non-code-indicative updates), while the disadvantage is that its overall effectiveness is comparatively low [24], [25] (i.”
BackgroundMethodInfluentialSelf-citationDoes the modern code inspection have value?
Siy, H. P., & Votta, L. G. (2002) · doi:10.1109/icsm.2001.972741 · Cited by 55 · cited 1×
“As revealed by existing studies, such inconsistencies can lead to future project bugs [15] and thus hinder softwaremaintenance activities [19], [20], [21], [22], [23].”
BackgroundPathMiner: A Library for Mining of Path-Based Representations of Code
Kovalenko, V., Bogomolov, E., Bryksin, T., & Bacchelli, A. (2019) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2019.00013 · Cited by 55 · cited 1×
“The AST path extraction is based on an off-the-shelf AST path collector [57] (i.”
MethodAutomatic Detection and Repair Recommendation of Directive Defects in Java API Documentation
Zhou, Y., Wang, C., Yan, X., Chen, T., Panichella, S., & Gall, H. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2872971 · Cited by 51 · cited 2×
“DRONE [33], the detection and repair tool for API documentation defects, may also be used to recommend updates to specific types of old comments despite that its generality to code comments remains unknown.”
MethodAutomating the removal of obsolete TODO comments
Gao, Z., Xia, X., Lo, D., Grundy, J., & Zimmermann, T. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468553 · Cited by 44 · cited 1×
“As revealed by existing studies, such inconsistencies can lead to future project bugs [15] and thus hinder softwaremaintenance activities [19], [20], [21], [22], [23].”
BackgroundSelf-citationWang, S., Wen, M., Chen, L., Yi, X., & Mao, X. (2019) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2019.8870172 · Cited by 43 · cited 1×
“These features measure the code change complexity from different granularities and are thus widely used in the previous studies[34], [35], [36], [37], [38], [39], [40].”
MethodSelf-citationLocating Latent Design Information in Developer Discussions: A Study on Pull Requests
Viviani, G., Famelis, M., Xia, X., Janik-Jones, C., & Murphy, G. C. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2924006 · Cited by 40 · cited 1×
“To answer RQ2, we measured the precision, recall, accuracy, F-score, and AUC, which are widely used in assessing predictive models [44], [51], [62], [63].”
MethodSelf-citationLightweight global and local contexts guided method name recommendation with prior knowledge
Wang, S., Wen, M., Lin, B., & Mao, X. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468567 · Cited by 38 · cited 3×
“, in this study, all tokens are split into sub-tokens based on the camel case and underscore naming conventions and then transformed into lowercase, following previous studies [27], [29]) and (2) for code changes or comment updates.”
BackgroundMethodSelf-citationStructural Language Models of Code
Alon, U. S., Sadaka, R., Levy, O., & Yahav, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00577 · Cited by 37
An Empirical Study on Code Comment Completion
Mastropaolo, A., Aghajani, E., Pascarella, L., & Bavota, G. (2021) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.26226/morressier.613b5418842293c031b5b5da · Cited by 34
Automated Comment Update: How Far are We?
Lin, B., Wang, S., Liu, K., Mao, X., & Bissyandé, T. F. (2021) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc52881.2021.00013 · Cited by 33
Self-citationAn Empirical Study on Code Comment CompletionPreprint
Mastropaolo, A., Aghajani, E., Pascarella, L., & Bavota, G. (2021) · IEEE International Conference on Software Maintenance and Evolution · doi:10.48550/arxiv.2107.10544 · Cited by 31 · cited 1×
“Usually, the comments record information, such as the intention of the functionality as well as the implementation details, which can facilitate the communication among developers [2], [8], [9], [10], [11], [12], [13], [14].”
BackgroundFrom source code identifiers to natural language terms
Carvalho, N. R., Almeida, J. J., Henriques, P. R., & Pereira, M. J. V. (2014) · Journal of Systems and Software · doi:10.1016/j.jss.2014.10.013 · Cited by 30 · cited 1×
“3a, we foresee a way to facilitate the comment update, which is expanding the abbreviations in the code [66], [67].”
BackgroundAutomatic Detection of Outdated Comments During Code Changes
Liu, Z., Chen, H., Chen, X., Luo, X., & Zhou, F. (2018) · 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac.2018.00028 · Cited by 27 · cited 1×
“Liu et al. [75] identified the outdated comments via a machine learning classifier trained by mining the historical data of software repositories.”
MethodSelf-citationAutomating the removal of obsolete TODO comments
Gao, Z., Xia, X., Lo, D., Grundy, J., & Zimmermann, T. (2021) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 27
Self-citationDisentangled Code Representation Learning for Multiple Programming Languages
Zhang, J., Hong, H., Zhang⋆, Y., Wan, Y., Liu, Y., & Sui, Y. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.391 · Cited by 25 · cited 1×
“, using dataflows and control-flows) [55], [65] would be beneficial to”
MethodSuggesting Comment Completions for Python using Neural Language Models
Ciurumelea, A., Proksch, S., & Gall, H. C. (2020) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner48275.2020.9054866 · Cited by 22 · cited 1×
“THE natural-language comment is crucial for understanding the associated code [1], [2], [3], [4], [5], [6], [7].”
BackgroundWang, S., Wen, M., Mao, X., & Yang, D. (2019) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3319008.3319349 · Cited by 18 · cited 1×
“These features measure the code change complexity from different granularities and are thus widely used in the previous studies[34], [35], [36], [37], [38], [39], [40].”
MethodSelf-citationCode to Comment "Translation": Data, Metrics, Baselining & EvaluationPreprint
Gros, D., Sezhiyan, H., Dévanbu, P., & Yu, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01410 · Cited by 16
Associating Natural Language Comment and Source Code Entities
Panthaplackel, S., Gligoric, M., Mooney, R. J., & Li, J. J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6382 · Cited by 14
Which abbreviations should be expanded?
Jiang, Y., Liu, H., Zhang, Y., Niu, N., Zhao, Y., & Zhang, L. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468616 · Cited by 14 · cited 1×
“3a, we foresee a way to facilitate the comment update, which is expanding the abbreviations in the code [66], [67].”
BackgroundBeep: Fine-grained Fix Localization by Learning to Predict Buggy Code ElementsPreprint
Wang, S., Liu, K., Lin, B., Mao, X., Bissyandé, T. F., Li, L., & Klein, J. (2021) · arXiv (Cornell University) · doi:10.13140/rg.2.2.25272.26884 · Cited by 13
Self-citationDo Comments follow Commenting Conventions? A Case Study in Java and Python
Rani, P., Abukar, S., Stulova, N., Bergel, A., & Nierstrasz, O. (2021) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam52516.2021.00028 · Cited by 9 · cited 1×
“Usually, the comments record information, such as the intention of the functionality as well as the implementation details, which can facilitate the communication among developers [2], [8], [9], [10], [11], [12], [13], [14].”
BackgroundMachine Learning Through Data Classification and Reduction
Ziarko, W., & Shan, N. (1997) · Fundamenta Informaticae · doi:10.3233/fi-1997-303411 · Cited by 6 · cited 1×
“We experimented with four widely-used machine learning models: Decision Table [47], Naive Bayes [48], Logistic Regression [49], and Random Forest [50].”
MethodNeural Code Summarization: How Far Are We?Preprint
Shi, E., Wang, Y., Du, L., Chen, J., Han, S., Zhang, H., Zhang, D., & Sun, H. (2021) · arXiv (Cornell University) · Cited by 5
Beep: Fine-grained Fix Localization by Learning to Predict Buggy Code ElementsPreprint
Wang, S., Liu, K., Lin, B., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.07739 · Cited by 5 · cited 1×
“Generally, an operation path is an AST path associated with code change operator that works on a leaf node l [30], which is defined as a triple: op 1⁄4 ht; p; oi, where t is the code token of the leaf node l in the AST path p, and o 2 fDELETE; INSERT; KEEP; UPDATEg is an atomic code”
BackgroundSelf-citationLearning to Generate Code Comments from Class HierarchiesPreprint
Zhang, J., Panthaplackel, S., Nie, P., Mooney, R. J., Li, J. J., & Gligoric, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.13426 · Cited by 4 · cited 1×
“THE natural-language comment is crucial for understanding the associated code [1], [2], [3], [4], [5], [6], [7].”
BackgroundDeep Just-In-Time Inconsistency Detection Between Comments and Source Code
Panthaplackel, S., Li, J. J., Gligoric, M., & Mooney, R. J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16119 · Cited by 2
Afifi, A., May, S., & Clark, V. A. (2011) · Chapman & Hall/CRC eBooks · doi:10.1201/9781466503243-18 · Cited by 2
Learning About Components and Class Hierarchies
(2008) · Apress eBooks · doi:10.1007/978-1-4302-0542-5_7 · Cited by 1
Research works printed but not linked: 6
- J. Zhang, S. Panthaplackel, P. Nie, J. Li, R. Mooney, and M. Gligoric, "Leveraging class hierarchy for code comprehension," in NeurIPS 2020 Workshop on Computer-Assisted Programming, 2020. [Online]. Available: https://openreview.net/forum?id= WHjjpMNZFoD
- U. Alon, S. Brody, O. Levy, and E. Yahav, "code2seq: Generating sequences from structured representations of code," in Proceed- ings of the 7th International Conference on Learning Representations. OpenReview.net, 2019.
- A. Mockus, M. Nagappan, and A. Hassan, "Best practices and pitfalls for statistical analysis of se data," in Proc. ICSE, 2014.
- T. R. Patil and S. S. Sherekar, "Performance analysis of naive bayes and j48 classification algorithm for data classification," International journal of computer science and applications, vol. 6, no. 2, pp. 256-261, 2013.
- D. G. Kleinbaum, K. Dietz, M. Gail, M. Klein, and M. Klein, Logistic regression. Springer, 2002.
- S. Wang, M. Wen, B. Lin, H. Wu, Y. Qin, D. Zou, X. Mao, and H. Jin, "Automated patch correctness assessment: How far are we?" in Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering. ACM, 2020.
Cited by
Geng, M., Wang, S., Dong, D., Wang, H., Li, G., Jin, Z., Mao, X., & Liao, X. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3608134 · Cited by 155
Self-citationCCT5: A Code-Change-Oriented Pre-trained Model
Lin, B., Wang, S., Liu, Z., Liu, Y., Xia, X., & Mao, X. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616339 · Cited by 69
“cent studies also utilize the AST path technique [3] for representing the code change [35, 36], where the AST is split into changed and unchanged parts and the paths connecting different leaf nodes are embedded to represent each part.”
BackgroundMethodInfluentialSelf-citationExploring the Capabilities of LLMs for Code-Change-Related Tasks
Fan, L., Liu, J., Liu, Z., Lo, D., Xia, X., & Li, S. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3709358 · Cited by 41
Self-citationMultilingual Code Co-evolution using Large Language Models
Zhang, J., Nie, P., Li, J. J., & Gligoric, M. (2023) · ESEC/SIGSOFT FSE · doi:10.1145/3611643.3616350 · Cited by 47
Interpretation-based Code Summarization
Geng, M., Wang, S., Dong, D., Wang, H., Cao, S., Zhang, K., & Jin, Z. G. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00026 · Cited by 29
“A branch of studies focus on updating outdated comments according to code changes, which can be considered as another form of comment generation problem [30], [31].”
BackgroundSelf-citation
Metadata
- Authors
- Lin, B., Wang, S., Liu, Z., Xia, X., & Mao, X.
- Affiliations
- National University of Defense Technology (CN)
- Zhejiang University (CN)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2023-04-01
- Volume
- 49
- Issue
- 4
- Pages
- 1640-1660
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W4293211675
- Cited by
- 27
- References
- 77
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 11 · 2023: 8
- FWCI
- 5.94
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Transactions on Software Engineering (publishedVersion, cc-by)
- Funders
- National Natural Science Foundation of China (61872445, 61672529)
- Zhejiang University Education Foundation Qizhen Scholar Foundation
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- two-stage approach, software maintenance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.