The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index; none of the 9 sources its claims cite is checked for support yet.
Deep learning similarities from different representations of source code
Open accessAbstract
Assessing the similarity between code components plays a pivotal role in a number of Software Engineering (SE) tasks, such as clone detection, impact analysis, refactoring, etc. Code similarity is generally measured by relying on manually defined or hand-crafted features, e.g., by analyzing the overlap among identifiers or comparing the Abstract Syntax Trees of two code components. These features represent a best guess at what SE researchers can utilize to exploit and reliably assess code similarity for a given task. Recent work has shown, when using a stream of identifiers to represent the code, that Deep Learning (DL) can effectively replace manual feature engineering for the task of clone detection. However, source code can be represented at different levels of abstraction: identifiers, Abstract Syntax Trees, Control Flow Graphs, and Bytecode. We conjecture that each code representation can provide a different, yet orthogonal view of the same code fragment, thus, enabling a more reliable detection of similarities in code. In this paper, we demonstrate how SE tasks can benefit from a DL-based approach, which can automatically learn code similarities from different representations.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
55 Own result · 31 Own interpretation · 24 No basis given · 21 Own method · 5 No full text of the sources · 3 Sources not found
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also show that the combined model CloneDetector, trained on clone candidates belonging to a single project (hibernate), can be effectively used to detect clones in the remaining nine systems.”
§ Deep Learning Similarities from Different Representations of Source Code > 4.2.5 RQ 5 : Can trained DL-based models be reused on different, previously unseen projects?
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that the distances computed by the Identifier-based model for such clones were several orders of magnitude higher than a reasonable threshold”
§ Deep Learning Similarities from Different Representations of Source Code > Identifiers.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Identifier-based models detect the highest number of Type I, AST-based models identify the highest number of Type II, Bytecode models identify the highest number of Type III, and CFG models detect the highest number of Type IV.”
§ Deep Learning Similarities from Different Representations of Source Code > 5 RESULTS
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Better results are achieved at class-level, where false positives are mainly due to Java Beans”
§ Deep Learning Similarities from Different Representations of Source Code > Bytecode & CFG.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In fact, we show that the combined model achieves overall better results as compared to stand-alone code representations.”
§ Deep Learning Similarities from Different Representations of Source Code > 1 INTRODUCTION
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“From Table 2, we can notice that candidates detected only by Bytecode or only by CFG models (partitions with ID 1 and 2) tend to be false positives.”
§ Deep Learning Similarities from Different Representations of Source Code > Bytecode & CFG.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Identifier-based models detect the highest number of Type I, AST-based models identify the highest number of Type II, Bytecode models identify the highest number of Type III, and CFG models detect the highest number of Type IV.”
§ Deep Learning Similarities from Different Representations of Source Code > 5 RESULTS
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The Identifier-based models achieve the best precision, both when working at method-and class-level.”
§ Deep Learning Similarities from Different Representations of Source Code > 5 RESULTS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The notable exceptions are the partitions with ID 1 and 2, referring to clone candidates detected only by the Bytecode and by the CFG representations, respectively.”
§ Deep Learning Similarities from Different Representations of Source Code > 5 RESULTS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and a moderate/substantial agreement in terms of clone types classification (75% for methods and 57% for classes).”
§ Deep Learning Similarities from Different Representations of Source Code > 5 RESULTS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 30
- Probably not a substantial claim. 44
- Borderline: one of the tests for a substantial claim is unclear. 1
- The quote only partly states this claim. 1
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 2 shows that most of the partitions exhibit a good precision, with peaks of 100%.”
§ Deep Learning Similarities from Different Representations of Source Code > 5 RESULTS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However its estimated recall is respectively 52% and 40%, meaning that the model fails to detect a significant percentage of clones.”
§ Deep Learning Similarities from Different Representations of Source Code > Identifiers.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“a close inspection at the vocabulary of the candidates showed that they share a small percentage of identifiers, which in turn, makes them hard to detect with such a representation.”
§ Deep Learning Similarities from Different Representations of Source Code > Identifiers.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also demonstrate that the proposed models can be effectively used to compute similarities, not only in the context of a single project, but also to analyze code similarities among different (diverse) software projects (e.g., detecting clones or libraries across multiple projects).”
§ Deep Learning Similarities from Different Representations of Source Code > 1 INTRODUCTION
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The precision for such partitions is 5% and 9% at method-level and 49% and 58% at class-level.”
§ Deep Learning Similarities from Different Representations of Source Code > Bytecode & CFG.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The developers only took the needed functionalities out of the Gamma class, stripped it of unnecessary code, and built InternalGamma.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“At class level, the CloneClassifier has overall F-Measure of 68%, with average precision of 67% and recall of 68%.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Note that the classes mentioned in this study do not refer to imported libraries but to actual Java duplicated code, i.e., the source code files are in both libraries.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These classes share only a few identifiers (therefore not detected by the Identifier model) and differ in some if-structures, making them more difficult to detect by CFG and Bytecode representations.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When identifying clone instances, the model achieves 90% precision and 93% recall at class, and 98% precision 97% recall at method level.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As an example, both classes implement a method which sets a seed.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The classes share many similarities across the two libraries, however, they differ on a statement and structural level.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In one class, the seed is set by an integer passed through an argument, while the other class sets the seed by using a combination of the current time and system hash code of the instance.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“the”
§ Deep Learning Similarities from Different Representations of Source Code > 3.5 Combined Models
plaincorrelational- Asserted: unclear — It is unclear whether the authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
Borderline substantial claim. It is unclear whether the authors put it forward as true. It is unclear whether the paper’s conclusions depend on it. It is unclear why the paper says it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Gamma and InternalGamma, named respectively to the library they belong to, are clones since parts of the Gamma class were used to develop InternalGamma.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“An example of Type II clone within these two libraries is instead the pair EditScript and ReplacementsFinder classes.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These clones are classified as Type II, since only package information has been changed.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“At method level, the model obtains an overall F-Measure of 84%, with an average precision and recall of 84% across the different categories”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Therefore, many of the methods in Gamma either do not appear or are significantly smaller in InternalGamma.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“They both extend ResourceBase and offer the same features.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Single-representation models obtain significantly worse results in the classification task (with a bigger gap with respect to what observed for the CloneDetector).”
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while other clone types achieve a precision ≥ 74% and a recall ≥ 75%.”
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It achieved 98% precision and 92% recall at method-level and 99% precision and 95% recall at class-level.”
§ Deep Learning Similarities from Different Representations of Source Code > Table 8: Model Reusability
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For example, a language model and a recursive autoencoder trained on a given vocabulary would not be able to provide adequate results on a vocabulary containing terms not previously seen during the training phase.”
§ Deep Learning Similarities from Different Representations of Source Code > 4.2.5 RQ 5 : Can trained DL-based models be reused on different, previously unseen projects?
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Firstly, learning transformations of the data drastically reduces the cost of modeling code, since software (and various code representations) store a lot of data to improve the effectiveness of learning.”
§ Deep Learning Similarities from Different Representations of Source Code > 1 INTRODUCTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The data presented in this section is based on the agreement of three evaluators.”
§ Deep Learning Similarities from Different Representations of Source Code > 5 RESULTS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bytecode and CFG representations have a very high degree of abstraction.”
§ Deep Learning Similarities from Different Representations of Source Code > Bytecode & CFG.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While these classes are false positives in terms of code clones, they still represent a successful classification in terms of learning structural and low-level similarities from the code.”
§ Deep Learning Similarities from Different Representations of Source Code > Bytecode & CFG.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“we downloaded the j's source code and analyzed the building files (weaver-1.3relies on Maven, thus we investigated the .pomfiles) to check whether the library x is imported as a dependency in j.”
§ Deep Learning Similarities from Different Representations of Source Code > 4.2.4 RQ 4 : Are DL-based models applicable for detecting clones among different projects?
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Code similarity is generally measured by relying on manually defined or hand-crafted features, e.g., by analyzing the overlap among identifiers or comparing the Abstract Syntax Trees of two code components.”
§ Deep Learning Similarities from Different Representations of Source Code > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“differently from other techniques (e.g., SDNE), HOPE embeds directed graphs (as needed in the case of Control Flow Graphs).”
§ Deep Learning Similarities from Different Representations of Source Code > 3.3.2 Graph Embedding Strategy.
What the sources say
- Not checked — no full text of the sourcecites [55] 55. Annibale Panichella, Bogdan Dit, Rocco Oliveto, Massimiliano Di Penta, Denys Poshyvanyk, and Andrea De Lucia. 2016. Parameterizing and Assembling IR- Based Solutions for SE Tasks Using Genetic Algorithms. In IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering, SANER 2016, Suita, Osaka, Japan, March 14-18, 2016 -Volume 1. 314-325. https://doi.org/10.1109/ SANER.2016.97 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, clone pairs with very high AST similarity, but low identifiers similarity are likely to be Type-II clones (possibly parameterized clones, where all the identifiers have been systematically renamed).”
§ Deep Learning Similarities from Different Representations of Source Code > 3.5 Combined Models
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“In order to statistically assess the evaluators' agreement, we compute the Fleiss' kappa [24].”
§ Deep Learning Similarities from Different Representations of Source Code > 4.2.1 RQ 1 : How effective are different representations in detecting similar code fragments?
What the sources say
- Not matchedcites [24] 24. M. Gabel, L. Jiang, and Z. Su. [n. d.]. Scalable Detection of Semantic Clones (ICSE'08).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This (i) avoids any potential problem/inconsistency in compiling projects, and (ii) ensures reproducibility.”
§ Deep Learning Similarities from Different Representations of Source Code > 4.1.1 Projects.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarities from different code representations can be useful not only to detect clones, but also to classify them into types.”
§ Deep Learning Similarities from Different Representations of Source Code > 3.5 Combined Models
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Indeed, if bytecode is available (compiled code) CFGs can be extracted as well.”
§ Deep Learning Similarities from Different Representations of Source Code > Bytecode & CFG.
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In our study, AST and Bytecode representations both have a fixed vocabulary, limited respectively by the number of different AST node types and bytecode opcodes.”
§ Deep Learning Similarities from Different Representations of Source Code > 4.2.5 RQ 5 : Can trained DL-based models be reused on different, previously unseen projects?
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, we are not aware of any other learning-based approach that operates on code representations other than identifiers and comments.”
§ Deep Learning Similarities from Different Representations of Source Code > 2 DEEP LEARNING IN SE
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“we randomly draw a statistically significant sample with 95% confidence level and ±15% confidence interval.”
§ Deep Learning Similarities from Different Representations of Source Code > 4.2.1 RQ 1 : How effective are different representations in detecting similar code fragments?
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To train the two models we rely on Random Forest employing the commonly used 10-fold cross-validation technique to partition training and test sets.”
§ Deep Learning Similarities from Different Representations of Source Code > 4.2.3 RQ 3 : How effective are combined multi-representation models?
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“While, ideally, these thresholds should be tuned for each project and representation, we chose the same thresholds to facilitate the comparison among the four representations.”
§ Deep Learning Similarities from Different Representations of Source Code > 4.2.1 RQ 1 : How effective are different representations in detecting similar code fragments?
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Random Forest is one of the most effective ensemble learning method for classification, which relies on decision tree learning and bagging [17].”
§ Deep Learning Similarities from Different Representations of Source Code > 3.5 Combined Models
What the sources say
- Not checked — no full text of the sourcecites [17] 17. Bogdan Dit, Latifa Guerrouj, Denys Poshyvanyk, and Giuliano Antoniol. 2011. Can Better Identifier Splitting Techniques Help Feature Location?. In Proceedings of the 2011 IEEE 19th International Conference on Program Comprehension (ICPC '11). IEEE Computer Society, Washington, DC, USA, 11-20. https://doi.org/10. 1109/ICPC.2011.47 source
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We argue that our results should be useful for any SE task that relies on analyzing code similarities.”
§ Deep Learning Similarities from Different Representations of Source Code > 1 INTRODUCTION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
In the paper
“Recent work has shown, when using a stream of identifiers to represent the code, that Deep Learning (DL) can effectively replace manual feature engineering for the task of clone detection.”
- Also stated in: § 1 INTRODUCTION · plain
“one recent example by White et al. [74] shows that DL can effectively replace manual feature engineering for the task of clone detection.”
§ Deep Learning Similarities from Different Representations of Source Code > Abstract
What the sources say
- Not matchedcites White et al. [74]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § 1 INTRODUCTION · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Secondly, performance is increased because generally, learning algorithms are more efficient than humans at discovering correlations in high dimensional spaces.”
§ Deep Learning Similarities from Different Representations of Source Code > 1 INTRODUCTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Three authors independently evaluated the clone candidate samples.”
§ Deep Learning Similarities from Different Representations of Source Code > 4.2.1 RQ 1 : How effective are different representations in detecting similar code fragments?
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“we computed the two-judges agreement to obtain the final dataset (e.g., a candidate clone was marked as a true positive if at least two of the three evaluators classified it as such).”
§ Deep Learning Similarities from Different Representations of Source Code > 4.2.1 RQ 1 : How effective are different representations in detecting similar code fragments?
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The lower performance at class-level could be due to the smaller dataset available (i.e., 483 methods vs 362 classes).”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“suggests that these representations complement each other.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that the number of instances and percentages are computed only on the manually validated sample.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“their low value suggest that distances computed with different representations do not correlate and provide different perspectives about the similarity of code pairs.”
§ Deep Learning Similarities from Different Representations of Source Code > AST.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This allows for a more targeted choice by SE researchers of a code representation when applying representation-DL algorithms to a SE tasks.”
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We do not compare our approach against code clone detection techniques since the focus of this paper is to show a general technique on how to learn similarities from multiple code representations, rather than building a code clone detector tool.”
§ Deep Learning Similarities from Different Representations of Source Code > External validity.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To mitigate this threat, we applied our approach in different contexts and used two different datasets; Projects and Libraries.”
§ Deep Learning Similarities from Different Representations of Source Code > External validity.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All the Libraries, including all the Apache commons libraries, are publicly available, ensuring the replicability of the study.”
§ Deep Learning Similarities from Different Representations of Source Code > External validity.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All the results are available in our online appendix.”
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These findings speak to the vast amount of SE tasks which benefit from analyzing multiple representations of the code.”
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This work also emphasizes the attributes that different code representations can accentuate.”
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
In the paper
“Also, all our evaluations are publicly available [2].”
§ Deep Learning Similarities from Different Representations of Source Code > 6 THREATS TO VALIDITY
What the sources say
- Not matchedcites [2] 2. M. Allamanis, E. Barr, C. Bird, and C. Sutton. [n. d.]. Suggesting Accurate Method and Class Names (FSE'15).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Then, we computed two-judge agreement on the evaluated candidates.”
§ Deep Learning Similarities from Different Representations of Source Code > 6 THREATS TO VALIDITY
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results obtained in our study using the selected datasets might not generalize to other projects.”
§ Deep Learning Similarities from Different Representations of Source Code > External validity.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While the representation extraction steps are implemented for Java, all the subsequent steps are completely language-agnostic because they rely on a corpus of sentences with arbitrary tokens.”
§ Deep Learning Similarities from Different Representations of Source Code > External validity.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Recursive Autoencoders have previously been applied in multiple domains and with various input types, including natural language, source code, and images.
In the paper
“We did not utilize other clone-focused datasets (e.g., such as BigCloneBench [66]) since in order to extract some representations (i.e., CFG and Bytecode) we needed compiled code.”
§ Deep Learning Similarities from Different Representations of Source Code > External validity.
What the sources say
- Not checked — no full text of the sourcecites [66] 66. Ricardo Terra, Luis Fernando Miranda, Marco Tulio Valente, and Roberto S. Bigonha. 2013. Qualitas.class Corpus: A Compiled Version of the Qualitas Corpus. Software Engineering Notes 38, 5 (2013), 1-4. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We support this claim by performing different analyses: (i) complementarity metrics; and (ii) correlation test.”
§ Deep Learning Similarities from Different Representations of Source Code > 6 THREATS TO VALIDITY
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To mitigate such threat, we employed three evaluators who independently checked the candidates.”
§ Deep Learning Similarities from Different Representations of Source Code > 6 THREATS TO VALIDITY
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“For Projects, which is a subset of systems from the Qualitas.classcorpus, we selected diverse systems in terms of size and domain, focusing on popular ones.”
§ Deep Learning Similarities from Different Representations of Source Code > External validity.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This eliminates a large timesink, native to DL approaches, and broadens the applicability of our approach.”
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We instantiated our approach in different use case scenarios and datasets.”
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Another threat in this category is related to the fact that we apply our approach on Java code only.”
§ Deep Learning Similarities from Different Representations of Source Code > External validity.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
Deep learning models require a long time to train.
“One of the drawbacks of DL-based models is their long training time.”
The paper relies on this without support.
Enumerators tend to have bytecode structures that are highly similar even when their constants differ.
“Enumerators have a very similar bytecode structure even if containing different constants.”
The paper relies on this without support.
Recursive Autoencoders have previously been applied in multiple domains and with various input types, including natural language, source code, and images.
“As a matter of fact, Recursive Autoencoders have been used in several contexts with different inputs such as natural language, source code, images.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
“while other clone types achieve a precision ≥ 74% and a recall ≥ 75%.”
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
“| Type III | 74 | 75 | 75 | 61 | 59 | 60 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Deep Learning Similarities from Different Representations of Source Code > 1 INTRODUCTION
“one recent example by White et al. [74] shows that DL can effectively replace manual feature engineering for the task of clone detection.”
§ Deep Learning Similarities from Different Representations of Source Code > References
“74. Zhi-Hua Zhou. 2009. Ensemble Learning. Springer US, Boston, MA, 270-273. https://doi.org/10.1007/978-0-387-73003-5_293”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
“other clone types achieve a precision ≥ 74% and a recall ≥ 75%.”
§ Deep Learning Similarities from Different Representations of Source Code > 7 CONCLUSION
“| Type III | 74 | 75 | 75 | 61 | 59 | 60 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Deep Learning Similarities from Different Representations of Source Code > 1 INTRODUCTION
“We release all the data used in our studies [2].”
§ Deep Learning Similarities from Different Representations of Source Code > References
“2. M. Allamanis, E. Barr, C. Bird, and C. Sutton. [n. d.]. Suggesting Accurate Method and Class Names (FSE'15).”
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
Convolutional networks and applications in vision
LeCun, Y., Kavukcuoglu, K., & Farabet, C. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537907 · Cited by 2,213
Graph embedding techniques, applications, and performance: A survey
Goyal, P., & Ferrara, E. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.03.022 · Cited by 1,801
CCFinder: a multilinguistic token-based code clone detection system for large scale source code
Kamiya, T., Kusumoto, S., & Inoue, K. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1019480 · Cited by 1,715
Asymmetric Transitivity Preserving Graph Embedding
Ou, M.-D., Cui, P., Pei, J., Zhang, Z., & Zhu, W.-W. (2016) · Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939751 · Cited by 1,343
Clone detection using abstract syntax trees
Baxter, I. D., Yahin, A., Moura, L., Sant'Anna, M., & Bier, L. (2002) · Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272) · doi:10.1109/icsm.1998.738528 · Cited by 1,277
(2011) · Cited by 1,192
Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
Socher, R., Pennington, J., Huang, E., Ng, A. Y., & Manning, C. D. (2011) · Cited by 1,181
DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones
Jiang, L., Misherghi, G., Su, Z., & Glondu, S. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.30 · Cited by 1,039
Convolutional Neural Networks over Tree Structures for Programming Language Processing
Mou, L., Li, G., Zhang, L., Wang, T., & Jin, Z. G. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10139 · Cited by 762
On finding duplication and near-duplication in large software systems
Baker, B. S. (2002) · Working Conference on Reverse Engineering (WCRE) · doi:10.1109/wcre.1995.514697 · Cited by 722
Automatically learning semantic features for defect prediction
Wang, S., Liu, T., & Tan, L. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884804 · Cited by 720
Deep learning code fragments for code clone detection
White, M., Tufano, M., Vendome, C., & Poshyvanyk, D. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970326 · Cited by 636
Self-citationA language independent approach for detecting duplicated code
Ducasse, S., Rieger, M., & Demeyer, S. (1999) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.1999.792593 · Cited by 629
Identifying similar code with program dependence graphs
Krinke, J. (2002) · Working Conference on Reverse Engineering (WCRE) · doi:10.1109/wcre.2001.957835 · Cited by 623
CP-Miner: finding copy-paste and related bugs in large-scale software code
Li, Z., Lu, S., Myagmar, S., & Zhou, Y. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.28 · Cited by 612
Feature location in source code: a taxonomy and survey
Dit, B., Revelle, M., Gethers, M., & Poshyvanyk, D. (2011) · Journal of Software Evolution and Process · doi:10.1002/smr.567 · Cited by 588
Self-citationLearning task-dependent distributed representations by backpropagation through structure
Goller, C., & Kuchler, A. (2002) · Proceedings of International Conference on Neural Networks (ICNN'96) · doi:10.1109/icnn.1996.548916 · Cited by 581
Using Slicing to Identify Duplication in Source Code
Komondoor, R., & Horwitz, S. (2001) · Lecture notes in computer science · doi:10.1007/3-540-47764-0_3 · Cited by 537
Roy, C. K., & Cordy, J. R. (2008) · 2008 16th IEEE International Conference on Program Comprehension · doi:10.1109/icpc.2008.41 · Cited by 533
Gu, X., Zhang, H., Zhang, D., & Kim, S. (2016) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2950290.2950334 · Cited by 483
Proceedings of the 36th International Conference on Software Engineering
(2014) · doi:10.1145/2568225 · Cited by 477
A Convolutional Attention Network for Extreme Summarization of Source CodePreprint
Allamanis, M., Peng, H. M., & Sutton, C. A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.03001 · Cited by 456
Sajnani, H., Saini, V., Svajlenko, J., Roy, C. K., & Lopes, C. V. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884877 · Cited by 447
DeepFix: Fixing Common C Language Errors by Deep Learning
Gupta, R., Pal, S., Kanade, A., & Shevade, S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10742 · Cited by 426
GPLAG: detection of software plagiarism by program dependence graph analysis
Liu, C., Chen, C., Han, J., & Yu, P. S. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150522 · Cited by 422
Suggesting accurate method and class names
Allamanis, M., Barr, E. T., Bird, C., & Sutton, C. A. (2015) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2786805.2786849 · Cited by 379
Zhou, Z. (2009) · Springer eBooks · doi:10.1007/978-0-387-73003-5_293 · Cited by 369
Identifying syntactic differences between two programs
Yang, W. (1991) · Software Practice and Experience · doi:10.1002/spe.4380210706 · Cited by 346
Scalable detection of semantic clones
Gabel, M., Jiang, L., & Su, Z. (2008) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1368088.1368132 · Cited by 335
Clone Detection Using Abstract Syntax Suffix Trees
Koschke, R., Falke, R., & Frenzel, P. (2006) · Working Conference on Reverse Engineering (WCRE) · doi:10.1109/wcre.2006.18 · Cited by 323
SourcererCC: Scaling Code Clone Detection to Big CodePreprint
Sajnani, H., Saini, V., Svajlenko, J., Roy, C. K., & Lopes, C. V. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.06448 · Cited by 306
Wei, H.-H., & Li, M. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/423 · Cited by 296
Graph-based mining of multiple object usage patterns
Nguyen, T. T., Nguyen, H. A., Pham, N. H., Al-Kofahi, J. M., & Nguyen, T. N. (2009) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1595696.1595767 · Cited by 293
Are deep neural networks the best choice for modeling source code?
Hellendoorn, V. J., & Devanbu, P. (2017) · Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering · doi:10.1145/3106237.3106290 · Cited by 292
Achieving accuracy and scalability simultaneously in detecting application clones on Android markets
Chen, K., Liu, P., & Zhang, Y. (2014) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2568225.2568286 · Cited by 290
Convolutional Neural Networks over Tree Structures for Programming Language ProcessingPreprint
Mou, L., Li, G., Zhang, L., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5718 · Cited by 285
Identification of high-level concept clones in source code
(2005) · doi:10.1109/ase.2001.989796 · Cited by 259
Panichella, A., Dit, B., Oliveto, R., Penta, M. D., Poshynanyk, D., & Lucia, A. D. (2013) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2013.6606598 · Cited by 248
Identifying redundancy in source code using fingerprints
(1993) · Cited by 245
The plastic surgery hypothesis
Barr, E. T., Brun, Y., Dévanbu, P., Harman, M., & Sarro, F. (2014) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2635868.2635898 · Cited by 217
Testing advanced driver assistance systems using multi-objective search and neural networks
Abdessalem, R. B., Nejati, S., Briand, L., & Stifter, T. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970311 · Cited by 213
Panichella, A., Dit, B., Oliveto, R., Penta, M. D., Poshynanyk, D., & Lucia, A. D. (2013) · Cited by 207
Self-citationBug Localization with Combination of Deep Learning and Information Retrieval
Lam, A. N., Nguyen, A. T., Nguyen, H. A., & Nguyen, T. N. (2017) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2017.24 · Cited by 201
Easy over hard: a case study on deep learning
Fu, W., & Menzies, T. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106256 · Cited by 201
Substring matching for clone detection and change tracking
(1994) · doi:10.1109/icsm.1994.336783 · Cited by 197
Combining Deep Learning with Information Retrieval to Localize Buggy Files for Bug Reports (N)
Lam, A. N., Nguyen, A. T., Nguyen, H. A., & Nguyen, T. N. (2015) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.73 · Cited by 188
Parameterized Pattern Matching: Algorithms and Applications
Baker, B. S. (1996) · Journal of Computer and System Sciences · doi:10.1006/jcss.1996.0003 · Cited by 174
Evaluating clone detection tools with BigCloneBench
Svajlenko, J., & Roy, C. K. (2015) · 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsm.2015.7332459 · Cited by 172
(2014) · doi:10.1145/2635868.2635901 · Cited by 159
Methodbook: Recommending Move Method Refactorings via Relational Topic Models
Bavota, G., Oliveto, R., Gethers, M., Poshyvanyk, D., & Lucia, A. D. (2014) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.60 · Cited by 154
Self-citationSearching for better configurations: a rigorous approach to clone evaluation
Wang, T., Harman, M., Jia, Y., & Krinke, J. (2013) · Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2491411.2491420 · Cited by 147
Graph-based statistical language model for code
Nguyen, A. T., & Nguyen, T. N. (2016) · Cited by 142
Learning unified features from natural and programming languages for locating buggy source code
Huo, X., Li, M., & Zhou, Z.-H. (2016) · International Joint Conference on Artificial Intelligence · Cited by 142
Evaluating Modern Clone Detection Tools
Svajlenko, J., & Roy, C. K. (2014) · IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme.2014.54 · Cited by 121
Graph-Based Statistical Language Model for Code
Nguyen, A. T., & Nguyen, T. N. (2015) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2015.336 · Cited by 119
Improving software modularization via automated analysis of latent topics and dependencies
Bavota, G., Gethers, M., Oliveto, R., Poshyvanyk, D., & Lucia, A. D. (2014) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2559935 · Cited by 117
Self-citationDetecting similar software applications
McMillan, C., Grechanik, M., & Poshyvanyk, D. (2012) · Cited by 112
Self-citationUsing Data Fusion and Web Mining to Support Feature Location in Software
Revelle, M., Dit, B., & Poshyvanyk, D. (2010) · 2010 IEEE 18th International Conference on Program Comprehension · doi:10.1109/icpc.2010.10 · Cited by 105
Self-citationAn Empirical Study on the Developers ’ Perception of Software Coupling
(2013) · Cited by 95
Self-citation(2012) · doi:10.1007/s10664-011-9194-4 · Cited by 95
Self-citationCan Better Identifier Splitting Techniques Help Feature Location?
Dit, B., Guerrouj, L., Poshyvanyk, D., & Antoniol, G. (2011) · 2011 IEEE 19th International Conference on Program Comprehension · doi:10.1109/icpc.2011.47 · Cited by 89
Self-citationDetecting similar software applications
McMillan, C., Grechanik, M., & Poshyvanyk, D. (2012) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2012.6227178 · Cited by 87
Self-citationVisualizing Textual Redundancy in Legacy Source
(1994) · Cited by 84
(2013) · doi:10.1145/2507288.2507314 · Cited by 66
On automatically detecting similar Android apps
Linares-Vasquez, M., Holtzhauer, A., & Poshyvanyk, D. (2016) · 2016 IEEE 24th International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2016.7503721 · Cited by 43
Self-citationParameterizing and Assembling IR-Based Solutions for SE Tasks Using Genetic Algorithms
Panichella, A., Dit, B., Oliveto, R., Di Penta, M., Poshyvanyk, D., & De Lucia, A. (2016) · 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER) · doi:10.1109/saner.2016.97 · Cited by 36
Self-citation(2013) · doi:10.5555/2662708.2662710 · Cited by 34
AN EXPERIMENTAL DETERMINATION OF THE PROBABLE ERROR OF DR SPEARMAN'S CORRELATION COEFFICIENTS
STUDENT (1921) · Biometrika · doi:10.1093/biomet/13.2-3.263 · Cited by 27
Automatic Categorization with Deep Neural Network for Open-Source Java Projects
Nguyen, A. T., & Nguyen, T. N. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-c.2017.118 · Cited by 18
A mutation analysis based benchmarking framework for clone detectors
Svajlenko, J., Roy, C. K., & Cordy, J. R. (2013) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc.2013.6613033 · Cited by 18
Dit, B., Moritz, E., Linares‐Vásquez, M., Poshyvanyk, D., & Cleland‐Huang, J. (2014) · Empirical Software Engineering · doi:10.1007/s10664-014-9339-3 · Cited by 17
Self-citationVector abstraction and concretization for scalable detection of refactorings
Milea, N. A., Jiang, L., & Khoo, S. (2014) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2635868.2635926 · Cited by 15
Jun, W. (2018) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315169491-3 · Cited by 4
Graph Embedding Techniques, Applications, and Performance: A Survey
Goyal, P., & Ferrara, E. (2017) · arXiv · doi:10.48550/arxiv.1705.02801
An empirical study on the developers' perception of software coupling
Bavota, G., Dit, B., Oliveto, R., et al. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606615
Self-citationAn Experimental Determination of the Probable Error of Dr Spearman's Correlation Coefficients
Student (1921) · Biometrika · doi:10.2307/2331754
Li, Z., & Zhou, Y. (2005) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/1095430.1081755
A parallel and efficient approach to large scale clone detection
Sajnani, H., & Lopes, C. (2013) · 2013 7th International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc.2013.6613042
Research works printed but not linked: 9
- 2017. Online Appendix: https://sites.google.com/view/learningcodesimilarities , Source Code: https://github.com/micheletufano/AutoenCODE. ( 2017 ). 2017. Online Appendix: https://sites.google.com/view/learningcodesimilarities, Source Code: https://github.com/micheletufano/AutoenCODE. (2017).
- 10.5555/2486788.2486879 · DOI
- 10.5555/3060832.3060845 · DOI
- 10.5555/2337223.2337267 · DOI
- 10.5555/3015812.3016002 · DOI
- 10.5555/2486788.2486857 · DOI
- 10.5555/2662708.2662719 · DOI
- 10.1145/2491411.2491420 · DOI · Paper page
- 10.5555/3172077.3172312 · DOI
Cited by
Sorted by citations
A Novel Neural Source Code Representation Based on Abstract Syntax Tree
Zhang, J., Wang, X., Zhang, H., Sun, H., Wang, K., & Liu, X. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00086 · Cited by 738
An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation
Tufano, M., Watson, C., Bavota, G., Penta, M. D., White, M., & Poshyvanyk, D. (2019) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3340544 · Cited by 370
Self-citationOn Learning Meaningful Code Changes Via Neural Machine Translation
Tufano, M., Pantiuchina, J., Watson, C., Bavota, G., & Poshyvanyk, D. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00021 · Cited by 228
Self-citationSorting and Transforming Program Repair Ingredients via Deep Learning Code Similarities
White, M., Tufano, M., Martínez, M., Monperrus, M., & Poshyvanyk, D. (2019) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2019.8668043 · Cited by 173
Self-citationLi, Y., Wang, S., Nguyen, T. N., & Nguyen, S. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3360588 · Cited by 164
Metadata
- Authors
- Tufano, M., Watson, C., Bavota, G., Di Penta, M., White, M., & Poshyvanyk, D.
- Affiliations
- William & Mary (US)
- Università della Svizzera italiana (CH)
- University of Sannio, Benevento, Italy
- Venue
- Proceedings of the 15th International Conference on Mining Software Repositories
- Publisher
- ACM
- Published
- 2018-05-28
- Pages
- 542-553
- OpenAlex
- W2883359218
- Cited by
- 158
- References
- 75
- Citations by year
- 2026: 6 · 2025: 11 · 2024: 20 · 2023: 22 · 2022: 31 · 2021: 29 · 2020: 20 · 2019: 18 · 2018: 1
- FWCI
- 13.34
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Conference on Mining Software Repositories (MSR) (publishedVersion)
- Funders
- NSF (CCF-1525902)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- code similarity, clone detection, source code representation, abstract syntax tree, control flow graph, bytecode, feature engineering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
