The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Evaluating clone detection tools with BigCloneBench
Abstract
Many clone detection tools have been proposed in the literature. However, our knowledge of their performance in real software systems is limited, particularly their recall. In this paper, we use our big data clone benchmark, BigCloneBench, to evaluate the recall of ten clone detection tools. BigCloneBench is a collection of eight million validated clones within IJaDataset-2.0, a big data software repository containing 25,000 open-source Java systems. BigCloneBench contains both intra-project and inter-project clones of the four primary clone types. We use this benchmark to evaluate the recall of the tools per clone type and across the entire range of clone syntactical similarity. We evaluate the tools for both single-system and cross-project detection scenarios. Using multiple clone-matching metrics, we evaluate the quality of the tools' reporting of the benchmark clones with respect to refactoring and automatic clone analysis use-cases. We compare these real-world results against our Mutation and Injection Framework, a synthetic benchmark, to reveal deeper understanding of the tools. We found that the tools have strong recall for Type-1 and Type-2 clones, as well as Type-3 clones with high syntactical similarity. The tools have weaker detection of clones with lower syntactical similarity.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“leading a software repository usually containing 7-23% cloned codes (Svajlenko and Roy, 2015).”
From ReACC: A Retrieval-Augmented Code Completion Framework (2022) · cites this paper as Svajlenko and Roy, 2015
Citations
Cites
An Analysis and Survey of the Development of Mutation Testing
Jia, Y., & Harman, M. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.62 · Cited by 1,752
CCFinder: a multilinguistic token-based code clone detection system for large scale source code
Kamiya, T., Kusumoto, S., & Inoue, K. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1019480 · Cited by 1,715
DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones
Jiang, L., Misherghi, G., Su, Z., & Glondu, S. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.30 · Cited by 1,039
Comparison and evaluation of code clone detection techniques and tools: A qualitative approach
Roy, C. K., Cordy, J. R., & Koschke, R. (2009) · Science of Computer Programming · doi:10.1016/j.scico.2009.02.007 · Cited by 1,002
Self-citationComparison and Evaluation of Clone Detection Tools
Bellon, S., Koschke, R., Antoniol, G., Krinke, J., & Merlo, E. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.70725 · Cited by 782
A Survey on Software Clone Detection Research
Roy, C. K., & Cordy, J. R. (2007) · Cited by 732
Self-citationA language independent approach for detecting duplicated code
Ducasse, S., Rieger, M., & Demeyer, S. (1999) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.1999.792593 · Cited by 629
Roy, C. K., & Cordy, J. R. (2008) · 2008 16th IEEE International Conference on Program Comprehension · doi:10.1109/icpc.2008.41 · Cited by 533
Self-citationSoftware clone detection: A systematic review
Rattan, D., Bhatia, R., & Singh, M. (2013) · Information and Software Technology · doi:10.1016/j.infsof.2013.01.008 · Cited by 415
Towards a Big Data Curated Benchmark of Inter-project Code Clones
Svajlenko, J., Islam, J. F., Keivanloo, I., Roy, C. K., & Mia, M. M. (2014) · 2014 IEEE International Conference on Software Maintenance and Evolution · doi:10.1109/icsme.2014.77 · Cited by 339
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- 2013, IJaDataset 2.0, Ambient Software Evoluton Group
Cited by
Sorted by citations
Items 101–173 of 173
On the Use of Deep Learning Models for Semantic Clone Detection
Pinku, S. N., Mondal, D., & Roy, C. K. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00053 · Cited by 6
Self-citationBigCloneBench: A Retrospective and Roadmap
Svajlenko, J., & Roy, C. K. (2022) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc55060.2022.00009 · Cited by 6
Self-citationTo Share, or Not to Share: Exploring Test-Case Reusability in Fork Ecosystems
Mukelabai, M., Derks, C., Krüger, J., & Berger, T. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00191 · Cited by 6
Hammad, M., Babur, Ö., Basit, H. A., & van den Brand, M. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.737 · Cited by 7
Quradaa, F. H., Shahzad, S., Almoqbily, R. S., & Sufyan, M. M. A. (2024) · PLoS ONE · doi:10.1371/journal.pone.0302333 · Cited by 5
Detecting Semantic Clones in Microservices Using Components
Abdelfattah, A. S., Rodriguez, A., Walker, A., & Cerny, T. (2023) · SN Computer Science · doi:10.1007/s42979-023-01910-1 · Cited by 5
Das, J. K., Mondal, S., & Roy, C. K. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00015 · Cited by 5
Self-citationTowards Semantic Clone Detection via Probabilistic Software Modeling
Thaller, H., Linsbauer, L., & Egyed, A. F. (2020) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc50091.2020.9047635 · Cited by 5
Precise Code Clone Detection with Architecture of Abstract Syntax Trees
Guo, X., Zhang, R., Zhou, L., & Lu, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19211-1_10 · Cited by 5
Intelligent token-based code clone detection system for large scale source code
Elkhail, A. A., Svacina, J., & Cerny, T. (2019) · ACM Conference on Research in Adaptive and Convergent Systems (RACS) · doi:10.1145/3338840.3355654 · Cited by 5
Development and benchmarking of multilingual code clone detector
Zhu, W., Yoshida, N., Kamiya, T., Choi, E., & Takada, H. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112215 · Cited by 4
JEMMA: An extensible Java dataset for ML4Code applications
Karmakar, A., Allamanis, M., & Robbes, R. (2023) · Empirical Software Engineering · doi:10.1007/s10664-022-10275-7 · Cited by 4
Finding Code-Clone Snippets in Large Source-Code Collection by ccgrep
Inoue, K., Miyamoto, Y., Germán, D. M., & Ishio, T. (2021) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-75251-4_3 · Cited by 4
Efficiently Measuring an Accurate and Generalized Clone Detection Precision using Clone Clustering
Svajlenko, J., & Roy, C. K. (2016) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2016-150 · Cited by 4
Self-citationAlomari, H. W., & Stephan, M. (2020) · International Conference on Program Comprehension · doi:10.1145/3387904.3389271 · Cited by 4
Code Clone Matching: A Practical and Effective Approach to Find Code SnippetsPreprint
Inoue, K., Miyamoto, Y., Germán, D. M., & Ishio, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.05615 · Cited by 4
Detecting Semantic Clones of Unseen Functionality
Kitsios, K., Sovrano, F., Barr, E. T., & Bacchelli, A. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00112 · Cited by 4
A Summary on the Stability of Code Clones and Current Research Trends
Mondal, M., Roy, C. K., & Schneider, K. A. (2021) · Springer eBooks · doi:10.1007/978-981-16-1927-4_12 · Cited by 3
Self-citationA Novel Method for Code Clone Detection Based on Minimally Random Kernel Convolutional Transform
Abdelkader, M. (2024) · IEEE Access · doi:10.1109/access.2024.3484995 · Cited by 3
Identification of Promoted Eclipse Unstable Interfaces Using Clone Detection Technique
Kawuma, S., & Nabaasa, E. (2018) · International Journal of Software Engineering & Applications · doi:10.5121/ijsea.2018.9502 · Cited by 3
Code Clone Detection Using Boosting Algorithms
Thanoshan, M. V., Kuhaneswaran, B., Kumara, B. T. G. S., Prasanth, S., Li, Z., & Paik, I. (2023) · International Conference on Advanced Research in Computing (ICARC) · doi:10.1109/icarc57651.2023.10145638 · Cited by 3
Cross-Language Binary-Source Code Matching Based on Rust and Intermediate Representation
Mao, J., Tang, Z., & Rao, W. (2023) · International Conference on Computer Communication and Artificial Intelligence (CCAI) · doi:10.1109/ccai57533.2023.10201266 · Cited by 3
Redundancy and Concept Analysis for Code-trained Language Models
Sharma, A., Hu, Z., Quinn, C., & Jannesari, A. (2024) · IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) · doi:10.1145/3691621.3694931 · Cited by 9
Pyclone: A Python Code Clone Test Bank Generator
Duncan, S., Walker, A., DeHaan, C., Alvord, S., Cerny, T., & Tisnovsky, P. (2021) · Lecture notes in electrical engineering · doi:10.1007/978-981-33-6385-4_22 · Cited by 3
An empirical study on how project context impacts on code cloning
Pérez‐Castillo, R., & Piattini, M. G. (2018) · Journal of Software Evolution and Process · doi:10.1002/smr.2115 · Cited by 3
Semantic Clone Detection via Probabilistic Software Modeling
Thaller, H., Linsbauer, L., & Egyed, A. F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99429-7_16 · Cited by 3
Fundamental Approaches to Software Engineering
Johnsen, E. B., & Wimmer, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99429-7 · Cited by 3
A Study of Software Clone Detection Techniques for Better Software Maintenance and Reliability
Ralhan, C., & Malik, N. (2021) · International Conference on Computing Sciences (ICCS) · doi:10.1109/iccs54944.2021.00056 · Cited by 3
CodeGuard: enhancing accuracy in detecting clones within java source code
Glani, Y., & Luo, P. (2024) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1455860 · Cited by 2
Yang, K., Wang, J., & Song, Z. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107741 · Cited by 3
Are Classical Clone Detectors Good Enough for the AI Era?
Alam, A. I., Roy, P. R., Al-omari, F., Roy, C. K., Roy, B., & Schneider, K. A. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00035 · Cited by 2
Self-citationImproving Large-Gap Clone Detection Recall Using Multiple Features
Dai, P., Zhang, Q., Wang, Y., Jin, D., & Gong, Y. (2022) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194022500413 · Cited by 2
Zhang, Z., & Saber, T. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00033 · Cited by 2
Cycle Discrepancy of Cubic Toeplitz Graphs
Aslam, L., Sarwar, S., Yousaf, M. M., & Jaffry, S. W. (2018) · Pakistan journal of engineering and applied sciences · doi:10.67635/pjeas.2018.22.1.601 · Cited by 2
Pattern Analysis of TXL Programs
Rahman, M. A., & Cordy, J. R. (2016) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2016.110 · Cited by 2
Comparison and Visualization of Code Clone Detection Results
Kazuki, M., & Inoue, K. (2020) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc50091.2020.9047633 · Cited by 2
TOWARDS AN APPROACH TO RECOMMEND AND RANK CLONES FOR REFACTORING
Kaur, M., & Rattan, D. (2020) · Advances in Mathematics Scientific Journal · doi:10.37418/amsj.9.6.56 · Cited by 2
An Intelligent Tutoring System for API Misuse Correction by Instant Quality Feedback
Zhao, R., Siy, H. P., Pack, C., Soh, L., & Song, M. (2022) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac54236.2022.00026 · Cited by 2
Experiments on Code Clone Detection and Machine Learning
Schäfer, A., Amme, W., & Heinze, T. S. (2022) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc55060.2022.00016 · Cited by 2
Extractive Multi Product-Line Engineering
Rosiak, K. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion52605.2021.00122 · Cited by 2
Vijayanandan, T., Kuhaneswaran, B., Induranga, A., Kumara, B. T. G. S., & Koswattage, K. R. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9070187 · Cited by 1
Semantic code clone detection using hybrid intermediate representations and BiLSTM networks
Ismail, M. S., Shahzad, S., & Quradaa, F. H. (2026) · PLoS ONE · doi:10.1371/journal.pone.0340971 · Cited by 1
Towards summarizing program statements in source code search
Marin, V. J., Bansal, I., & Rivero, C. R. (2020) · ACM Symposium on Applied Computing · doi:10.1145/3341105.3374055 · Cited by 1
Assessing and Improving Dataset and Evaluation Methodology in Deep Learning for Code Clone Detection
Li, H., Gao, Q. Z., & Zhang, S. (2023) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre59848.2023.00044 · Cited by 1
Hossain, M. M., Roy, B., Roy, C. K., & Schneider, K. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60441-6_13 · Cited by 1
Self-citationGPT-Based Wasm Instruction Analysis for Program Language Processing
Deng, L., Zhong, Q., Lei, H., Qiu, Y., & Chen, J. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-80419-9_9 · Cited by 1
Quradaa, F. H., Shahzad, S., Almoqbily, R. S., & Al-Hakimi, M. H. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-89813-6_7 · Cited by 1
Roy, C. K., & Cordy, J. R. (2018) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3196321.3196325 · Cited by 1
Self-citationMoCoP: Towards a Model Clone Portal
Babur, Ö., & Stephan, M. (2019) · International Workshop on Modeling in Software Engineering (MiSE) · doi:10.1109/mise.2019.00019 · Cited by 1
Intelligent Multilevel Node Authentication in Mobile Computing Using Clone Node
Malhotra, N., & Bala, M. (2021) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.020920 · Cited by 1
Analyzing Clone Evolution for Identifying the Important Clones for Management
Mondal, M. (2017) · University Library - University of Saskatchewan (University of Saskatchewan) · Cited by 1
Fast and Precise Token-Based Code Clone Detection
村上, 寛. (2016) · OUKA (Osaka University Knowledge Archive) (Osaka University) · doi:10.18910/55840
Clone Type Classification Based on Tree-Based Convolution Neural Network
Jo, Y.-B., Yoo, C.-J., & Lee, J. (2020) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2020.18.3.79
Bui, N. D. Q., Yu, Y., & Jiang, L. (2020) · arXiv (Cornell University)
Extractive Multi Product-Line EngineeringPreprint
Rosiak, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.05602
Sahar, H., Liu, Y., Hindle, A., & Barbosa, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.10789
Finding repeated strings in code repositories and its applications to code-clone detection
Yamagata, Y., Herve, F., Fujiwara, Y., & Inoue, K. (2021) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec53868.2021.00057
Sclabel: Finding Performance Bugs Through Clone Detection Based on Labeled Code TemplatesPreprint
Deng, Z., Wang, W., Xue, Y., & Xu, Y. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4150166
A Feasibility Study of Using Code Clone Detection for Secure Programming Education
Menard, M., Nelson, T., Shahi, M., Morton, H., DeTavernier, A., Siy, H. P., Zhao, R., & Song, M. (2022) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac54236.2022.00238
MTCD: An Efficient Cloning Detection Technique Based on Method Table
Liao, F., Long, S., Zhu, W., Chen, W., Cao, S., & Guan, X. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35257-7_19
Obfuscated Clone Search in JavaScript based on Reinforcement Subsequence Learning
Song, L., Ding, S. H. H., Tian, Y., Li, L. T., Ou, W., Charland, P., & Walenstein, A. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3711903
Integrating Large Language Models and Reinforcement Learning for Non-linear Reasoning
Alon, Y., & David, C. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715761
Unveiling code clone patterns in open source VR software: an empirical study
Chen, H., Huang, Z., Xu, Y., Huang, W., Wang, X., Chen, J., Li, H., Peng, K., Liu, F., & He, S. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00536-2
Quantifying cross-language code reuse via function-level clone detection
Rong, Y., & Zhou, Y. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00362-2
Enhancing Semantic Clone Detection with Siamese Bi-LSTM for Multi-Level Source Code Representations
Quradaa, F. H., Shahzad, S., Almoqbily, R. S., & Al-Hakimi, M. H. (2025) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-025-10675-z
An Empirical Study on the Characteristics of Reusable Code Clones
Yu, C., Noei, S., Zhang, H., & Zou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3793251
Adhikari, P. K., Kumar, A., & Deshwal, V. (2025) · International Conference on Emerging Technologies and Innovation for Sustainability (EmergIN) · doi:10.1109/emergin67762.2025.11450760
Scalable Hybrid LSH-Based Framework for Structural Code Clone Detection
Wickramasooriya, D., Awshid, B., Dilshan, H., Jayasekara, S., Kodagoda, N., & Manathunga, K. (2026) · International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA) · doi:10.1109/ichora69329.2026.11537226
A Comparative Study of Threshold Determination Algorithms for Code Similarity Detection
Bhattacharyya, A., & Singh, J. (2025) · International Conference on Computational Engineering, Sensing Technology and Management (ICCETM) · doi:10.1109/iccetm66557.2025.11558115
Understanding Code Similarity across Instruction Set Architectures: An Empirical Study
Yu, H., Zhu, J., Zheng, Y., Zhang, Y., Wang, W., Wei, J., & Tao, H. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808119
Find My Code Twin: Improving SNIPPET SEARCH Performance Using LLMs in Practice
Ko, S., Jang, E., Choi, D., Ryu, D., Park, J., & Park, C. (2026) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3786583.3786881
Shazam4Code: An explainable method for detecting derivative clones for ensuring code providence
Heller, J., & Guyer, S. Z. (2026) · IEEE International Conference on AI and Data Analytics (ICAD) · doi:10.1109/icad69378.2026.11608599
Depiction Model clones through Design Metrics
Shobha, G., Agarwal, R., Kansal, V., & Tanwar, S. (2026) · International Conference on Sustainable Engineering and Technology Innovations (ICSETI) · doi:10.1109/icseti67678.2026.11636851
Metadata
- Authors
- Svajlenko, J., & Roy, C. K.
- Affiliations
- University of Saskatchewan (CA)
- Venue
- 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME)
- Publisher
- IEEE
- Published
- 2015-09
- Pages
- 131-140
- ISSN
- 1063-6773 (ISSN-L)
- OpenAlex
- W2158439356
- Cited by
- 172
- References
- 28
- Citations by year
- 2026: 7 · 2025: 16 · 2024: 10 · 2023: 16 · 2022: 22 · 2021: 28 · 2020: 21 · 2019: 15 · 2018: 17 · 2017: 9 · 2016: 10 · 2015: 1
- FWCI
- 10.72
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Software Maintenance (ICSM) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- clone detection, BigCloneBench, memory retrieval, refactoring, synthetic benchmark
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
