Evaluating clone detection tools with BigCloneBench
Abstract
Many clone detection tools have been proposed in the literature. However, our knowledge of their performance in real software systems is limited, particularly their recall. In this paper, we use our big data clone benchmark, BigCloneBench, to evaluate the recall of ten clone detection tools. BigCloneBench is a collection of eight million validated clones within IJaDataset-2.0, a big data software repository containing 25,000 open-source Java systems. BigCloneBench contains both intra-project and inter-project clones of the four primary clone types. We use this benchmark to evaluate the recall of the tools per clone type and across the entire range of clone syntactical similarity. We evaluate the tools for both single-system and cross-project detection scenarios. Using multiple clone-matching metrics, we evaluate the quality of the tools' reporting of the benchmark clones with respect to refactoring and automatic clone analysis use-cases. We compare these real-world results against our Mutation and Injection Framework, a synthetic benchmark, to reveal deeper understanding of the tools. We found that the tools have strong recall for Type-1 and Type-2 clones, as well as Type-3 clones with high syntactical similarity. The tools have weaker detection of clones with lower syntactical similarity.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A software repository usually contains 7-23% cloned codes.
“leading a software repository usually containing 7-23% cloned codes (Svajlenko and Roy, 2015).”
From ReACC: A Retrieval-Augmented Code Completion Framework (2022) · cites this paper as Svajlenko and Roy, 2015
Cites
An Analysis and Survey of the Development of Mutation Testing
Jia, Y., & Harman, M. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.62 · Cited by 1,752 · see citing works
CCFinder: a multilinguistic token-based code clone detection system for large scale source code
Kamiya, T., Kusumoto, S., & Inoue, K. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1019480 · Cited by 1,715 · see citing works
DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones
Jiang, L., Misherghi, G., Su, Z., & Glondu, S. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.30 · Cited by 1,039 · see citing works
Comparison and evaluation of code clone detection techniques and tools: A qualitative approach
Roy, C. K., Cordy, J. R., & Koschke, R. (2009) · Science of Computer Programming · doi:10.1016/j.scico.2009.02.007 · Cited by 1,002 · see citing works
Self-citationComparison and Evaluation of Clone Detection Tools
Bellon, S., Koschke, R., Antoniol, G., Krinke, J., & Merlo, E. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.70725 · Cited by 782 · see citing works
A Survey on Software Clone Detection Research
Roy, C. K., & Cordy, J. R. (2007) · Cited by 732 · see citing works
Self-citationA language independent approach for detecting duplicated code
Ducasse, S., Rieger, M., & Demeyer, S. (1999) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.1999.792593 · Cited by 629 · see citing works
Roy, C. K., & Cordy, J. R. (2008) · 2008 16th IEEE International Conference on Program Comprehension · doi:10.1109/icpc.2008.41 · Cited by 533 · see citing works
Self-citationSoftware clone detection: A systematic review
Rattan, D., Bhatia, R., & Singh, M. (2013) · Information and Software Technology · doi:10.1016/j.infsof.2013.01.008 · Cited by 415 · see citing works
Towards a Big Data Curated Benchmark of Inter-project Code Clones
Svajlenko, J., Islam, J. F., Keivanloo, I., Roy, C. K., & Mia, M. M. (2014) · 2014 IEEE International Conference on Software Maintenance and Evolution · doi:10.1109/icsme.2014.77 · Cited by 339 · see citing works
Self-citationGöde, N., & Koschke, R. (2009) · Euromicro Conference on Software Maintenance and Reengineering (CSMR) · doi:10.1109/csmr.2009.20 · Cited by 222 · see citing works
A Mutation/Injection-Based Automatic Framework for Evaluating Code Clone Detection Tools
Roy, C. K., & Cordy, J. R. (2009) · International Conference on Software Testing, Verification and Validation Workshops · doi:10.1109/icstw.2009.18 · Cited by 194 · see citing works
Self-citationEvaluating clone detection tools for use during preventative maintenance
Burd, E., & Bailey, J. (2003) · IEEE International Workshop on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam.2002.1134103 · Cited by 150 · see citing works
Searching for better configurations: a rigorous approach to clone evaluation
Wang, T., Harman, M., Jia, Y., & Krinke, J. (2013) · Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2491411.2491420 · Cited by 147 · see citing works
An Empirical Study of Function Clones in Open Source Software
Roy, C. K., & Cordy, J. R. (2008) · 2008 15th Working Conference on Reverse Engineering · doi:10.1109/wcre.2008.54 · Cited by 137 · see citing works
Self-citationEvaluating Modern Clone Detection Tools
Svajlenko, J., & Roy, C. K. (2014) · IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme.2014.54 · Cited by 121 · see citing works
Self-citationCloneDetective - A workbench for clone detection research
Juergens, E., Deissenboeck, F., & Hummel, B. (2009) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2009.5070566 · Cited by 121 · see citing works
Finding Clones with Dup: Analysis of an Experiment
Baker, B. S. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.70720 · Cited by 85 · see citing works
Empirical evaluation of clone detection using syntax suffix trees
Falke, R., Frenzel, P., & Koschke, R. (2008) · Empirical Software Engineering · doi:10.1007/s10664-008-9073-9 · Cited by 77 · see citing works
SimCad: An extensible and faster clone detection tool for large scale software systems
Uddin, M. S., Roy, C. K., & Schneider, K. A. (2013) · International Conference on Program Comprehension · doi:10.1109/icpc.2013.6613857 · Cited by 39 · see citing works
Self-citationThreshold-free code clone detection for a large-scale heterogeneous Java repository
Keivanloo, I., Zhang, F., & Zou, Y. (2015) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2015.7081830 · Cited by 35 · see citing works
(2013) · doi:10.5555/2662708.2662710 · Cited by 34 · see citing works
Self-citationA dataset of clone references with gaps
Murakami, H., Higo, Y., & Kusumoto, S. (2014) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2597073.2597133 · Cited by 26 · see citing works
Towards a mutation-based automatic framework for evaluating code clone detection tools
Roy, C. K., & Cordy, J. R. (2008) · International C* Conference on Computer Science and Software Engineering (C3S2E) · doi:10.1145/1370256.1370279 · Cited by 23 · see citing works
Self-citationKrutz, D. E., & Le, W. (2014) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2597073.2597127 · Cited by 22 · see citing works
A mutation analysis based benchmarking framework for clone detectors
Svajlenko, J., Roy, C. K., & Cordy, J. R. (2013) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc.2013.6613033 · Cited by 18 · see citing works
Self-citationCtcompare: code clone detection using hashed token sequences
Toomey, W. (2012) · e-publications@bond (Bond University) · doi:10.5555/2664398.2664420 · Cited by 16 · see citing works
Ctcompare: Code clone detection using hashed token sequences
Toomey, W. (2012) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc.2012.6227881 · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Deep learning code fragments for code clone detection
White, M., Tufano, M., Vendome, C., & Poshyvanyk, D. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970326 · Cited by 613 · see citing works
Sajnani, H., Saini, V., Svajlenko, J., Roy, C. K., & Lopes, C. V. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884877 · Cited by 447 · see citing works
Self-citationSourcererCC: Scaling Code Clone Detection to Big CodePreprint
Sajnani, H., Saini, V., Svajlenko, J., Roy, C. K., & Lopes, C. V. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.06448 · Cited by 306 · see citing works
Self-citationCCLearner: A Deep Learning-Based Clone Detection Approach
Li, L., Feng, H., Zhuang, W., Meng, N., & Ryder, B. G. (2017) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2017.46 · Cited by 227 · see citing works
DéjàVu: a map of code duplicates on GitHub
Lopes, C. V., Maj, P., Martins, P., Saini, V., Yang, D., Zitny, J., Sajnani, H., & Vitek, J. (2017) · Proceedings of the ACM on Programming Languages · doi:10.1145/3133908 · Cited by 183 · see citing works
Oreo: detection of clones in the twilight zone
Saini, V., Farmahinifarahani, F., Lu, Y., Baldi, P., & Lopes, C. V. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3236026 · Cited by 181 · see citing works
Deep learning similarities from different representations of source code
Tufano, M., Watson, C., Bavota, G., Di Penta, M., White, M., & Poshyvanyk, D. (2018) · Proceedings of the 15th International Conference on Mining Software Repositories · doi:10.1145/3196398.3196431 · Cited by 158 · see citing works
Neural Detection of Semantic Code Clones Via Tree-Based Convolution
Yu, H., Lam, W., Chen, L., Li, G., Xie, T., & Wang, Q. (2019) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2019.00021 · Cited by 140 · see citing works
Data Quality Matters: A Case Study on Data Label Correctness for Security Bug Report Prediction
Wu, X., Zheng, W., Xia, X., & Lo, D. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3063727 · Cited by 134 · see citing works
Wang, P., Svajlenko, J., Wu, Y., Xu, Y., & Roy, C. K. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180179 · Cited by 126 · see citing works
Self-citationA Systematic Literature Review on Bad Smells–5 W's: Which, When, What, Who, Where
Sobrinho, E. V. D. P., Lucia, A. D., & Maia, M. D. A. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2880977 · Cited by 118 · see citing works
Bui, N. D. Q., Yu, Y., & Jiang, L. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462840 · Cited by 103 · see citing works
FCCA: Hybrid Code Representation for Functional Clone Detection Using Attention Networks
Hua, W., Sui, Y., Wan, Y., Liu, G., & Xu, G. (2020) · IEEE Transactions on Reliability · doi:10.1109/tr.2020.3001918 · Cited by 100 · see citing works
Learning-Based Recursive Aggregation of Abstract Syntax Trees for Code Clone Detection
Büch, L., & Andrzejak, A. (2019) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2019.8668039 · Cited by 94 · see citing works
BigCloneEval: A Clone Detection Tool Evaluation Framework with BigCloneBench
Svajlenko, J., & Roy, C. K. (2016) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2016.62 · Cited by 89 · see citing works
Self-citationKim, K., Kim, D., Bissyandé, T. F., Choi, E., Li, L., Klein, J., & Traon, Y. L. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180187 · Cited by 87 · see citing works
Siamese: scalable and incremental code clone search via multiple code representations
Ragkhitwetsagul, C., & Krinke, J. (2019) · Empirical Software Engineering · doi:10.1007/s10664-019-09697-7 · Cited by 73 · see citing works
Assessing the Generalizability of Code2vec Token Embeddings
Kang, H. J., Bissyandé, T. F., & Lo, D. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00011 · Cited by 73 · see citing works
Nasrabadi, M. Z., Parsa, S. A., Ramezani, M., Roy, C. K., & Ekhtiarzadeh, M. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111796 · Cited by 77 · see citing works
Self-citationFast and Flexible Large-Scale Clone Detection with CloneWorks
Svajlenko, J., & Roy, C. K. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-c.2017.3 · Cited by 55 · see citing works
Self-citationScalable code clone detection and search based on adaptive prefix filtering
Nishi, M. A., & Damevski, K. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2017.11.039 · Cited by 52 · see citing works
Semantic code search via equational reasoning
Premtoon, V., Koppel, J., & Solar-Lezama, A. (2020) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/3385412.3386001 · Cited by 52 · see citing works
Open-source tools and benchmarks for code-clone detection
Walker, A., Cerny, T., & Song, E. (2020) · ACM SIGAPP Applied Computing Review · doi:10.1145/3381307.3381310 · Cited by 50 · see citing works
Challenging Machine Learning-Based Clone Detectors via Semantic-Preserving Code Transformations
Zhang, W., Guo, S., Zhang, H., Sui, Y., Xue, Y., & Xu, Y. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3240118 · Cited by 54 · see citing works
CloneWorks: A Fast and Flexible Large-Scale Near-Miss Clone Detection Tool
Svajlenko, J., & Roy, C. K. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-c.2017.78 · Cited by 48 · see citing works
Self-citationYue, Z., Ban, B., Xue, Y., & Xu, Y. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416541 · Cited by 46 · see citing works
NIL: large-scale detection of large-variance clones
Nakagawa, T., Higo, Y., & Kusumoto, S. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468564 · Cited by 45 · see citing works
Using compilation/decompilation to enhance clone detection
Ragkhitwetsagul, C., & Krinke, J. (2017) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc.2017.7880502 · Cited by 43 · see citing works
Tao, C., Zhang, Q., Hu, X., & Xia, X. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527911 · Cited by 43 · see citing works
Cross-language code search using static and dynamic analyses
Mathew, G., & Stolee, K. T. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468538 · Cited by 41 · see citing works
TreeCen: Building Tree Graph for Scalable Semantic Code Clone Detection
Hu, Y., Zou, D., Peng, J., Wu, Y., Shan, J., & Jin, H. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3556927 · Cited by 41 · see citing works
Benchmarks for software clone detection: A ten-year retrospective
Roy, C. K., & Cordy, J. R. (2018) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2018.8330194 · Cited by 41 · see citing works
Self-citationSoftware systems at risk: An empirical study of cloned vulnerabilities in practice
Kim, S., & Lee, H. (2018) · Computers & Security · doi:10.1016/j.cose.2018.02.007 · Cited by 40 · see citing works
Fast Code Clone Detection Based on Weighted Recursive Autoencoders
Zeng, J., Ben, K., Li, X., & Zhang, X. (2019) · IEEE Access · doi:10.1109/access.2019.2938825 · Cited by 41 · see citing works
LVMapper: A Large-Variance Clone Detector Using Sequencing Alignment Approach
Wu, M., Wang, P., Yin, K., Cheng, H., Xu, Y., & Roy, C. K. (2020) · IEEE Access · doi:10.1109/access.2020.2971545 · Cited by 33 · see citing works
Self-citationLearning Graph-based Code Representations for Source-level Functional Similarity Detection
Liu, J., Zeng, J., Wang, X., & Liang, Z. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00040 · Cited by 31 · see citing works
Efficient transformer with code token learner for code clone detection
Zhang, A., Fang, L., Ge, C., Li, P., & Liu, Z. (2022) · Journal of Systems and Software · doi:10.1016/j.jss.2022.111557 · Cited by 30 · see citing works
Chen, J., Shang, W., Hassan, A. E., Wang, Y., & Lin, J. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00068 · Cited by 28 · see citing works
Micro-clones in evolving software
Mondai, M., Roy, C. K., & Schneider, K. A. (2018) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2018.8330196 · Cited by 27 · see citing works
Self-citationYuan, D., Fang, S., Zhang, T., Xu, Z., & Luo, X. (2022) · IEEE Transactions on Reliability · doi:10.1109/tr.2022.3176922 · Cited by 25 · see citing works
The Mutation and Injection Framework: Evaluating Clone Detection Tools with Mutation Analysis
Svajlenko, J., & Roy, C. K. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2912962 · Cited by 24 · see citing works
Self-citationCapturing source code semantics via tree-based convolution over API-enhanced AST
Chen, L., Ye, W., & Zhang, S. (2019) · ACM International Conference on Computing Frontiers (CF) · doi:10.1145/3310273.3321560 · Cited by 24 · see citing works
Custom-tailored clone detection for IEC 61131-3 programming languages
Rosiak, K., Schlie, A., Linsbauer, L., Vogel‐Heuser, B., & Schaefer, I. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111070 · Cited by 23 · see citing works
Analysis of Source Code Duplication in Ethreum Smart Contracts
Pierro, G. A., & Tonelli, R. (2021) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner50967.2021.00089 · Cited by 21 · see citing works
Comparison and Evaluation of Clone Detection Techniques with Different Code Representations
Wang, Y., Ye, Y., Wu, Y., Zhang, W., Xue, Y., & Liu, Y. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00039 · Cited by 21 · see citing works
BUI, D. Q. N., Yu, Y., & Jiang, L. (2021) · Open Research Online (The Open University) · Cited by 20 · see citing works
Jo, Y.-B., Lee, J., & Yoo, C.-J. (2021) · Applied Sciences · doi:10.3390/app11146613 · Cited by 20 · see citing works
On Precision of Code Clone Detection Tools
Farmahinifarahani, F., Saini, V., Yang, D., Sajnani, H., & Lopes, C. V. (2019) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2019.8668015 · Cited by 20 · see citing works
Clone detection through srcClone: A program slicing based approach
Alomari, H. W., & Stephan, M. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111115 · Cited by 19 · see citing works
A Large-Gap Clone Detection Approach Using Sequence Alignment via Dynamic Parameter Optimization
Liu, J., Wang, T., Feng, C., Wang, H., & Li, D. (2019) · IEEE Access · doi:10.1109/access.2019.2940710 · Cited by 19 · see citing works
Toward a Theory of Causation for Interpreting Neural Code Models
Nader Palacio, D., Velasco, A., Cooper, N., Rodriguez, A., Moran, K., & Poshyvanyk, D. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3379943 · Cited by 18 · see citing works
Generalizability of Code Clone Detection on CodeBERT
Sonnekalb, T., Gruner, B., Brust, C.-A., & Mäder, P. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3561165 · Cited by 18 · see citing works
AST-path Based Compare-Aggregate Network for Code Clone Detection
Liang, H., & Ai, L. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534099 · Cited by 17 · see citing works
Stubber: Compiling Source Code into Bytecode without Dependencies for Java Code Clone Detection
Schäfer, A., Amme, W., & Heinze, T. S. (2021) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc53727.2021.00011 · Cited by 17 · see citing works
Big Code Search: A Bibliography
Kim, K., Ghatpande, S., Kim, D., Zhou, X., Liu, K., Bissyandé, T. F., Klein, J., & Traon, Y. L. (2023) · ACM Computing Surveys · doi:10.1145/3604905 · Cited by 17 · see citing works
BigCloneBench Considered Harmful for Machine Learning
Krinke, J., & Ragkhitwetsagul, C. (2022) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc55060.2022.00008 · Cited by 17 · see citing works
Inferring Bug Signatures to Detect Real Bugs
Zhong, H., Wang, X., & Mei, H. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.2996975 · Cited by 17 · see citing works
DeepClone: Modeling Clones to Generate Code Predictions
Hammad, M., Babur, Ö., Basit, H. A., & van den Brand, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64694-3_9 · Cited by 17 · see citing works
Review Sharing via Deep Semi-Supervised Code Clone Detection
Guo, C., Yang, H., Huang, D., Zhang, J., Dong, N., Xu, J., & Zhu, J. (2020) · IEEE Access · doi:10.1109/access.2020.2966532 · Cited by 16 · see citing works
Svajlenko, J., & Roy, C. K. (2021) · Springer eBooks · doi:10.1007/978-981-16-1927-4_7 · Cited by 16 · see citing works
Self-citationDIP: Dead code Insertion based Black-box Attack for Programming Language Model
Na, C., Choi, Y., & Lee, J.-H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.430 · Cited by 16 · see citing works
Building a Coding Assistant via the Retrieval-Augmented Language Model
Li, X., Wang, H., Liu, Z., Yu, S., Wang, S., Yan, Y., Fu, Y.-K., Gu, Y., & Yu, G. (2024) · ACM Transactions on Information Systems · doi:10.1145/3695868 · Cited by 16 · see citing works
CPPCD: A Token-Based Approach to Detecting Potential Clones
Hung, Y.-L., & Takada, S. (2020) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc50091.2020.9047636 · Cited by 15 · see citing works
Cheers, H., Lin, Y., & Smith, S. P. (2021) · Empirical Software Engineering · doi:10.1007/s10664-021-09990-4 · Cited by 14 · see citing works
Can static analysis tools find more defects?
Mehrpour, S., & LaToza, T. D. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10232-4 · Cited by 14 · see citing works
MuMonDE: A framework for evaluating model clone detectors using model mutation analysis
Stephan, M., & Cordy, J. R. (2018) · Software Testing Verification and Reliability · doi:10.1002/stvr.1669 · Cited by 14 · see citing works
Code Clone Detection with Hierarchical Attentive Graph Embedding
Ji, X., Liu, L., & Zhu, J. (2021) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s021819402150025x · Cited by 14 · see citing works
You Look so Different: Finding Structural Clones and Subclones in Java Source Code
Amme, W., Heinze, T. S., & Schäfer, A. (2021) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme52107.2021.00013 · Cited by 14 · see citing works
Towards Automating Precision Studies of Clone Detectors
Saini, V., Farmahinifarahani, F., Lu, Y., Yang, D., Martins, P., Sajnani, H., Baldi, P., & Lopes, C. V. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00023 · Cited by 14 · see citing works
Zhu, W., Yoshida, N., Kamiya, T., Choi, E., & Takada, H. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3529161 · Cited by 13 · see citing works
Can Neural Clone Detection Generalize to Unseen Functionalitiesƒ
Liu, C., Lin, Z., Lou, J., Wen, L., & Zhang, D. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678907 · Cited by 13 · see citing works
How Stable Are Eclipse Application Framework Internal Interfaces?
Businge, J., Kawuma, S., Openja, M., Bainomugisha, E., & Serebrenik, A. (2019) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2019.8668018 · Cited by 12 · see citing works
Model-Driven Evaluation of Software Architecture Quality Using Model Clone Detection
Stephan, M., & Cordy, J. R. (2016) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs.2016.21 · Cited by 12 · see citing works
Transformer-based networks over tree structures for code classification
Hua, W., & Liu, G. (2021) · Applied Intelligence · doi:10.1007/s10489-021-02894-2 · Cited by 11 · see citing works
An approach to clone detection in sequence diagrams and its application to security analysis
Alalfi, M. H., Antony, E. P., & Cordy, J. R. (2016) · Software & Systems Modeling · doi:10.1007/s10270-016-0557-6 · Cited by 11 · see citing works
Investigating Vector-Based Detection of Code Clones Using BigCloneBench
Yokoi, K., Choi, E., Yoshida, N., & Inoue, K. (2018) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec.2018.00095 · Cited by 11 · see citing works
A technique to detect multi-grained code clones
Yuki, Y., Higo, Y., & Kusumoto, S. (2017) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc.2017.7880510 · Cited by 11 · see citing works
Akram, J., Mumtaz, M., & Luo, P. (2019) · Software Practice and Experience · doi:10.1002/spe.2759 · Cited by 11 · see citing works
SJBCD: A Java Code Clone Detection Method Based on Bytecode Using Siamese Neural Network
Wan, B., Dong, S., Zhou, J., & Qian, Y. (2023) · Applied Sciences · doi:10.3390/app13179580 · Cited by 10 · see citing works
Focus : Function clone identification on cross‐platform
Fu, L., Ji, S., Liu, C., Liu, P., Duan, F., Wang, Z., Chen, W., & Wang, T. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22752 · Cited by 10 · see citing works
Pathways to Leverage Transcompiler based Data Augmentation for Cross-Language Clone Detection
Pinku, S. N., Mondal, D., & Roy, C. K. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00031 · Cited by 10 · see citing works
Self-citationDetection of Similar Functions Through the Use of Dominator Information
Schäfer, A., Amme, W., & Heinze, T. S. (2020) · IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C) · doi:10.1109/acsos-c51401.2020.00057 · Cited by 10 · see citing works
Generating clone references with less human subjectivity
Yuki, Y., Higo, Y., Hotta, K., & Kusumoto, S. (2016) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2016.7503739 · Cited by 10 · see citing works
Large-Scale Code Clone Detection
Sajnani, H. (2016) · eScholarship (California Digital Library) · Cited by 10 · see citing works
A Survey on the Evaluation of Clone Detection Performance and BenchmarkingPreprint
Svajlenko, J., & Roy, C. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.15682 · Cited by 10 · see citing works
Self-citationReACC: A Retrieval-Augmented Code Completion Framework
Lu, S., Duan, N., Han, H., Guo, D., Hwang, S.-W., & Svyatkovskiy, A. (2022) · arXiv · doi:10.48550/arxiv.2203.07722 · Cited by 96 · see citing works
“This is unnatural from human perspective, as studies demonstrate that programmers tend to reuse an existing code snippet by copying part of code with or without minor modifications to accelerate software development (Roy and Cordy, 2008; Baker, 2007), leading a software repository usually containing 7-23% cloned codes…”
BackgroundCRAM: Code Recommendation With Programming Context Based on Self-Attention Mechanism
Tao, C.-Q., Lin, K., Huang, Z.-Q., & Sun, X.-B. (2023) · IEEE Transactions on Reliability · doi:10.1109/tr.2022.3171309 · Cited by 6 · see citing works
Code2Img: Tree-Based Image Transformation for Scalable Code Clone Detection
Hu, Y., Fang, Y., Sun, Y., Jia, Y., Wu, Y., Zou, D., & Jin, H. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3295801 · Cited by 9 · see citing works
Towards a framework for generating program dependence graphs from source code
Marin, V. J., & Rivero, C. R. (2018) · doi:10.1145/3278142.3278144 · Cited by 9 · see citing works
Clone Swarm: A Cloud Based Code-Clone Analysis Tool
Bandi, V., Roy, C. K., & Gutwin, C. A. (2020) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc50091.2020.9047642 · Cited by 9 · see citing works
Self-citationClone Detection on Large Scala Codebases
Rahman, W., Xu, Y., Pu, F., Xuan, J., Jia, X., Basios, M., Kanthan, L., Li, L., Wu, F., & Xu, B. (2020) · IEEE International Workshop on Software Clones (IWSC) · doi:10.1109/iwsc50091.2020.9047640 · Cited by 9 · see citing works
Large-Scale Clone Detection and Benchmarking
Svajlenko, J. (2018) · University Library - University of Saskatchewan (University of Saskatchewan) · Cited by 9 · see citing works
Self-citationARCC: assistant for repetitive code comprehension
Nunez, W. Z., Marin, V. J., & Rivero, C. R. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3122824 · Cited by 8 · see citing works
DCCD: An Efficient and Scalable Distributed Code Clone Detection Technique for Big Code
Akram, J., Shi, Z., Mumtaz, M., & Luo, P. (2018) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2018-117 · Cited by 8 · see citing works
A Neural Network Architecture for Program Understanding Inspired by Human Behaviors
Zhu, R., Yuan, L., Li, X., Gao, M., & Cai, W. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.353 · Cited by 8 · see citing works
An enhanced transformer-based framework for interpretable code clone detection
Nashaat, M., Amin, R., Eid, A. H., & Abdel-Kader, R. F. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112347 · Cited by 7 · see citing works
Unlearnable Examples: Protecting Open-Source Software from Unauthorized Neural Code Learning
Ji, Z., Ma, P., & Wang, S. (2022) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2022-066 · Cited by 7 · see citing works
Adding Context to Source Code Representations for Deep Learning
Tian, F., & Treude, C. (2022) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme55016.2022.00042 · Cited by 7 · see citing works
Enhancing Source Code Representations for Deep Learning with Static Analysis
Guan, X., & Treude, C. (2024) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3643916.3644396 · Cited by 7 · see citing works
Assessing and Improving an Evaluation Dataset for Detecting Semantic Code Clones via Deep Learning
Yu, H., Hu, X., Li, G., Li, Y., Wang, Q., & Xie, T. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3502852 · Cited by 7 · see citing works
- Authors
- Svajlenko, J., & Roy, C. K.
- Affiliations
- University of Saskatchewan (CA)
- Venue
- 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME)
- Publisher
- IEEE
- Published
- 2015-09
- Pages
- 131-140
- ISSN
- 1063-6773 (ISSN-L)
- OpenAlex
- W2158439356
- Cited by
- 172
- References
- 28
- Citations by year
- 2026: 7 · 2025: 16 · 2024: 10 · 2023: 16 · 2022: 22 · 2021: 28 · 2020: 21 · 2019: 15 · 2018: 17 · 2017: 9 · 2016: 10 · 2015: 1
- FWCI
- 10.72
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Software Maintenance (ICSM) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Software Engineering Methodologies
- Keywords
- clone detection, BigCloneBench, memory retrieval, refactoring, synthetic benchmark
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
