The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Evaluating clone detection tools with BigCloneBench
Abstract
Many clone detection tools have been proposed in the literature. However, our knowledge of their performance in real software systems is limited, particularly their recall. In this paper, we use our big data clone benchmark, BigCloneBench, to evaluate the recall of ten clone detection tools. BigCloneBench is a collection of eight million validated clones within IJaDataset-2.0, a big data software repository containing 25,000 open-source Java systems. BigCloneBench contains both intra-project and inter-project clones of the four primary clone types. We use this benchmark to evaluate the recall of the tools per clone type and across the entire range of clone syntactical similarity. We evaluate the tools for both single-system and cross-project detection scenarios. Using multiple clone-matching metrics, we evaluate the quality of the tools' reporting of the benchmark clones with respect to refactoring and automatic clone analysis use-cases. We compare these real-world results against our Mutation and Injection Framework, a synthetic benchmark, to reveal deeper understanding of the tools. We found that the tools have strong recall for Type-1 and Type-2 clones, as well as Type-3 clones with high syntactical similarity. The tools have weaker detection of clones with lower syntactical similarity.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“leading a software repository usually containing 7-23% cloned codes (Svajlenko and Roy, 2015).”
From ReACC: A Retrieval-Augmented Code Completion Framework (2022) · cites this paper as Svajlenko and Roy, 2015
Citations
Cites
An Analysis and Survey of the Development of Mutation Testing
Jia, Y., & Harman, M. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.62 · Cited by 1,752
CCFinder: a multilinguistic token-based code clone detection system for large scale source code
Kamiya, T., Kusumoto, S., & Inoue, K. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1019480 · Cited by 1,715
DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones
Jiang, L., Misherghi, G., Su, Z., & Glondu, S. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.30 · Cited by 1,039
Comparison and evaluation of code clone detection techniques and tools: A qualitative approach
Roy, C. K., Cordy, J. R., & Koschke, R. (2009) · Science of Computer Programming · doi:10.1016/j.scico.2009.02.007 · Cited by 1,002
Self-citationComparison and Evaluation of Clone Detection Tools
Bellon, S., Koschke, R., Antoniol, G., Krinke, J., & Merlo, E. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.70725 · Cited by 782
A Survey on Software Clone Detection Research
Roy, C. K., & Cordy, J. R. (2007) · Cited by 732
Self-citationA language independent approach for detecting duplicated code
Ducasse, S., Rieger, M., & Demeyer, S. (1999) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.1999.792593 · Cited by 629
Roy, C. K., & Cordy, J. R. (2008) · 2008 16th IEEE International Conference on Program Comprehension · doi:10.1109/icpc.2008.41 · Cited by 533
Self-citationSoftware clone detection: A systematic review
Rattan, D., Bhatia, R., & Singh, M. (2013) · Information and Software Technology · doi:10.1016/j.infsof.2013.01.008 · Cited by 415
Towards a Big Data Curated Benchmark of Inter-project Code Clones
Svajlenko, J., Islam, J. F., Keivanloo, I., Roy, C. K., & Mia, M. M. (2014) · 2014 IEEE International Conference on Software Maintenance and Evolution · doi:10.1109/icsme.2014.77 · Cited by 339
Self-citationGöde, N., & Koschke, R. (2009) · Euromicro Conference on Software Maintenance and Reengineering (CSMR) · doi:10.1109/csmr.2009.20 · Cited by 222
A Mutation/Injection-Based Automatic Framework for Evaluating Code Clone Detection Tools
Roy, C. K., & Cordy, J. R. (2009) · International Conference on Software Testing, Verification and Validation Workshops · doi:10.1109/icstw.2009.18 · Cited by 194
Self-citationEvaluating clone detection tools for use during preventative maintenance
Burd, E., & Bailey, J. (2003) · IEEE International Workshop on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam.2002.1134103 · Cited by 150
Searching for better configurations: a rigorous approach to clone evaluation
Wang, T., Harman, M., Jia, Y., & Krinke, J. (2013) · Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2491411.2491420 · Cited by 147
An Empirical Study of Function Clones in Open Source Software
Roy, C. K., & Cordy, J. R. (2008) · 2008 15th Working Conference on Reverse Engineering · doi:10.1109/wcre.2008.54 · Cited by 137
Self-citationEvaluating Modern Clone Detection Tools
Svajlenko, J., & Roy, C. K. (2014) · IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme.2014.54 · Cited by 121
Self-citationCloneDetective - A workbench for clone detection research
Juergens, E., Deissenboeck, F., & Hummel, B. (2009) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2009.5070566 · Cited by 121
Finding Clones with Dup: Analysis of an Experiment
Baker, B. S. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.70720 · Cited by 85
Empirical evaluation of clone detection using syntax suffix trees
Falke, R., Frenzel, P., & Koschke, R. (2008) · Empirical Software Engineering · doi:10.1007/s10664-008-9073-9 · Cited by 77
SimCad: An extensible and faster clone detection tool for large scale software systems
Uddin, M. S., Roy, C. K., & Schneider, K. A. (2013) · International Conference on Program Comprehension · doi:10.1109/icpc.2013.6613857 · Cited by 39
Self-citationThreshold-free code clone detection for a large-scale heterogeneous Java repository
Keivanloo, I., Zhang, F., & Zou, Y. (2015) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2015.7081830 · Cited by 35
(2013) · doi:10.5555/2662708.2662710 · Cited by 34
Self-citationA dataset of clone references with gaps
Murakami, H., Higo, Y., & Kusumoto, S. (2014) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2597073.2597133 · Cited by 26
Towards a mutation-based automatic framework for evaluating code clone detection tools
Roy, C. K., & Cordy, J. R. (2008) · International C* Conference on Computer Science and Software Engineering (C3S2E) · doi:10.1145/1370256.1370279 · Cited by 23
Self-citationKrutz, D. E., & Le, W. (2014) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2597073.2597127 · Cited by 22
A mutation analysis based benchmarking framework for clone detectors
Svajlenko, J., Roy, C. K., & Cordy, J. R. (2013) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc.2013.6613033 · Cited by 18
Self-citationCtcompare: code clone detection using hashed token sequences
Toomey, W. (2012) · e-publications@bond (Bond University) · doi:10.5555/2664398.2664420 · Cited by 16
Ctcompare: Code clone detection using hashed token sequences
Toomey, W. (2012) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc.2012.6227881 · Cited by 14
Research works printed but not linked: 1
- 2013, IJaDataset 2.0, Ambient Software Evoluton Group
Cited by
Sorted by citations
Deep learning code fragments for code clone detection
White, M., Tufano, M., Vendome, C., & Poshyvanyk, D. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970326 · Cited by 636
Sajnani, H., Saini, V., Svajlenko, J., Roy, C. K., & Lopes, C. V. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884877 · Cited by 447
Self-citationSourcererCC: Scaling Code Clone Detection to Big CodePreprint
Sajnani, H., Saini, V., Svajlenko, J., Roy, C. K., & Lopes, C. V. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.06448 · Cited by 306
Self-citationCCLearner: A Deep Learning-Based Clone Detection Approach
Li, L., Feng, H., Zhuang, W., Meng, N., & Ryder, B. G. (2017) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2017.46 · Cited by 237
DéjàVu: a map of code duplicates on GitHub
Lopes, C. V., Maj, P., Martins, P., Saini, V., Yang, D., Zitny, J., Sajnani, H., & Vitek, J. (2017) · Proceedings of the ACM on Programming Languages · doi:10.1145/3133908 · Cited by 183
Metadata
- Authors
- Svajlenko, J., & Roy, C. K.
- Affiliations
- University of Saskatchewan (CA)
- Venue
- 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME)
- Publisher
- IEEE
- Published
- 2015-09
- Pages
- 131-140
- ISSN
- 1063-6773 (ISSN-L)
- OpenAlex
- W2158439356
- Cited by
- 172
- References
- 28
- Citations by year
- 2026: 7 · 2025: 16 · 2024: 10 · 2023: 16 · 2022: 22 · 2021: 28 · 2020: 21 · 2019: 15 · 2018: 17 · 2017: 9 · 2016: 10 · 2015: 1
- FWCI
- 10.72
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Software Maintenance (ICSM) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- clone detection, BigCloneBench, memory retrieval, refactoring, synthetic benchmark
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
