CoChat
Sign Up
Cited by 3Software Engineering ResearchData Quality and ManagementAcademic integrity and plagiarism

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

SimClone: Detecting Tabular Data Clones Using Value Similarity

Open access
Yang, Xu, Rajbahadur, Gopi Krishnan, Lin, Dayi, Wang, Shaowei, Jiang, Zhen Ming (Jack)·2025·ACM Transactions on Software Engineering and Methodology
DOI 10.1145/3676961
Open this paper in CoChat — free accountOpen PDF

Abstract

Data clones are defined as multiple copies of the same data among datasets. The presence of data clones between datasets can cause issues such as difficulties in managing data assets and data license violations when using datasets with clones to build AI software. However, detecting data clones is not trivial. The majority of the prior studies in this area rely on structural information to detect data clones (e.g., font size, column header). However, tabular datasets used to build AI software are typically stored without any structural information. In this article, we propose a novel method called SimClone for data clone detection in tabular datasets without relying on structural information. SimClone method utilizes value similarities for data clone detection. We also propose a visualization approach as a part of our SimClone method to help locate the exact position of the cloned data between a dataset pair. Our results show that our SimClone outperforms the current state-of-the-art method by at least 20% in terms of both F1-score and AUC. In addition, SimClone’s visualization component helps identify the exact location of the data clone in a dataset with a Precision@10 value of 0.80 in the top 20 true positive predictions.

Overview

Integrity signals

Registry

Crossref · 10.1145/3676961

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Yang et al. (2024b)'s method using entropy-delta for patch ranking and classification can rank correct patches more effectively than state-of-the-art machine learning tools.

    “By using entropy-delta for patch ranking and classification, this method can rank correct patches more effectively than state-of-the-art machine learning tools.”

    From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Yang et al. (2024b)

  • MORepair's fine-tuning strategy enables LLM to achieve superior performance in program repair.

    “This fine-tuning strategy enables LLM to achieve superior performance in program repair.”

    From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Yang et al. 2024c

  • Integrating code into LLMs significantly enhances its ability to perform more complex tasks as an intelligent agent.

    “Additionally, Yang etal. (2024d) demonstrates that integrating code into LLMs significantly enhances its ability to perform more complex tasks as an intelligent agent.”

    From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Yang et al. 2024d

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 78

  1. 10.1109/ACCESS.2019.2918202 · DOI
  2. Ibrahim Alabdulmohsin Jessica Schrouff and Oluwasanmi Koyejo. 2022. A reduction to binary approach for debiasing multiclass datasets. arXiv preprint arXiv:2205.15860. Retrieved from https://arxiv.org/abs/2205.15860 · arXiv
  3. 10.1016/j.jnca.2016.01.008 · DOI
  4. 10.1145/3133264.3133296 · DOI
  5. Mamdouh Farouk and. 2019. Measuring sentences similarity: A survey. Indian Journal of Science and Technology 12 25 (Jul 2019) 1–11. DOI: 10.17485/ijst/2019/v12i25/143977 · DOI
  6. 10.4103/2229-3485.100662 · DOI
  7. 10.1145/1541880.1541883 · DOI
  8. Misha Benjamin Paul Gagnon Negar Rostamzadeh Chris Pal Yoshua Bengio and Alex Shee. 2019. Towards standardization of data licenses: The montreal data license. arXiv:1903.12262. Retrieved from https://arxiv.org/abs/1903.12262 · arXiv
  9. 10.1007/s10462-020-09896-5 · DOI · Paper page
  10. 10.1007/s11749-016-0481-7 · DOI · Paper page
  11. 10.1145/956750.956759 · DOI
  12. 10.1109/TNNLS.2022.3229161 · DOI
  13. 10.1145/509907.509965 · DOI
  14. Surajit Chaudhuri, Bee-Chung Chen, Venkatesh Ganti, and Raghav Kaushik. 2007. Example-driven design of efficient record matching queries. In Proceedings of the 33rd International Conference on Very Large Data Bases (VLDB ’07), Vol. 7, 327–338.
  15. 10.1109/ICDE.2005.125 · DOI
  16. 10.1145/2939672.2939785 · DOI · Paper page
  17. 10.1109/ICCV.2019.00942 · DOI
  18. Gao Cong, Wenfei Fan, Floris Geerts, Xibei Jia, and Shuai Ma. 2007. Improving data quality: Consistency and accuracy. In Proceedings of the 33rd international Conference on Very Large Data Bases, 315–326.
  19. 10.1109/MIS.2003.1234770 · DOI
  20. 10.1145/2950290.2950359 · DOI
  21. 10.1117/1.JMI.10.6.061104 · DOI · Paper page
  22. 10.1007/s10515-020-00277-4 · DOI
  23. Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng Ming Gong Linjun Shou Bing Qin Ting Liu Daxin Jiang and Ming Zhou. 2020. CodeBERT: A pre-trained model for programming and natural languages. arXiv:2002.08155. Retrieved from https://arxiv.org/abs/2002.08155 · DOI · Paper page
  24. 10.1016/j.asoc.2021.107366 · DOI
  25. 10.1145/1082983.1083242 · DOI
  26. 10.1109/TNNLS.2013.2292894 · DOI · Paper page
  27. 10.1145/2588555.2588576 · DOI
  28. Venkat Gudivada, Amy Apon, and Junhua Ding. 2017. Data quality considerations for big data and machine learning: Going beyond data cleaning and transformations. International Journal on Advances in Software 10, 1 (2017), 1–20.
  29. Daya Guo Shuo Ren Shuai Lu Zhangyin Feng Duyu Tang Shujie Liu Long Zhou Nan Duan Alexey Svyatkovskiy Shengyu Fu Michele Tufano Shao Kun Deng Colin Clement Dawn Drain Neel Sundaresan Jian Yin Daxin Jiang and Ming Zhou. 2020. GraphCodeBERT: Pre-training code representations with data flow. arXiv:2009.08366. Retrieved from https://arxiv.org/abs/2009.08366 · arXiv
  30. Awni Hannun, Chuan Guo, and Laurens van der Maaten. 2021. Measuring data leakage in machine-learning models with fisher information. In Proceedings of the 37th Conference on Uncertainty in Artificial Intelligence. PMLR, 760–770.
  31. 10.1145/1148170.1148222 · DOI
  32. 10.1109/ICSE.2015.129 · DOI
  33. 10.1109/ICSE.2013.6606575 · DOI
  34. 10.1023/A:1009761603038 · DOI
  35. 10.1145/3310205 · DOI
  36. 10.1109/ICSME52107.2021.00054 · DOI
  37. 10.1145/3394486.3406477 · DOI
  38. Xiao Jin, Pin-Yu Chen, Chia-Yi Hsu, Chia-Mu Yu, and Tianyi Chen. 2021. Cafe: Catastrophic data leakage in vertical federated learning. Advances in Neural Information Processing Systems 34 (2021), 994–1006.
  39. Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. LightGBM: A highly efficient gradient boosting decision tree. In Proceedings of the 31st International Conference on Neural Information Processing Systems.
  40. 10.3233/IP-150357 · DOI
  41. 10.2134/agronj2000.922345x · DOI
  42. 10.5555/647170.718283 · DOI
  43. Alex Krizhevsky, Geoffrey Hinton. 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report TR-2009, University of Toronto, Toronto.
  44. 10.1162/dint_a_00042 · DOI
  45. 10.1109/ACCESS.2019.2904286 · DOI
  46. V. I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady 10 (Feb. 1966), 707.
  47. 10.1145/3468264.3468597 · DOI
  48. José Antonio Hernández López Boqi Chen Tushar Sharma and Dániel Varró. 2024. On inter-dataset code duplication and data leakage in large language models. arXiv:2401.07930. Retrieved from https://arxiv.org/abs/2401.07930 · arXiv
  49. Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems.
  50. Rada Mihalcea and Paul Tarau. 2004. Textrank: Bringing order into text. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, 404–411.
  51. Alvaro E. Monge and Charles Elkan. 1996. The field matching problem: Algorithms and applications. In Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining, Vol. 2, 267–270.
  52. Suphakit Niwattanakul, Jatsada Singthongchai, Ekkachai Naenudorn, and Supachanun Wanapu. 2013. Using of Jaccard coefficient for keywords similarity. In Proceedings of the International Multiconference of Engineers and Computer Scientists, Vol. 1, 380–384.
  53. 10.5555/3294771.3294994 · DOI
  54. 10.5555/3495724.3497154 · DOI
  55. Alin C. Popescu and Hany Farid. 2004. Exposing Digital Forgeries by Detecting Duplicated Image Regions. Computer Science Technical Report TR2004-515 (2004).
  56. Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2018. CatBoost: Unbiased boosting with categorical features. In Proceedings of the 32nd International Conference on Neural Information Processing Systems
  57. 10.1145/3236024.3236065 · DOI
  58. Gopi Krishnan Rajbahadur Erika Tuck Li Zi Dayi Lin Boyuan Chen Zhen Ming Jiang and Daniel Morales German. 2021. Can I use this publicly available dataset to build commercial AI software? – A case study on publicly available image datasets. arXiv:2111.02374. Retrieved from https://arxiv.org/abs/2111.02374 · arXiv
  59. 10.1109/TSE.2021.3056941 · DOI
  60. 10.1007/978-1-4614-2053-8 · DOI
  61. G. Shobha, Ajay Rana, Vineet Kansal, and Sarvesh Tanwar. 2021. Code clone detection—A systematic review. In Proceedings of the International Conference on Emerging Technologies in Data Mining and Information Security (IEMIS ’20), Vol. 2, 645–655.
  62. Svetlana Sicular, Jim Hare, and Kenneth Brant. 2019. Hype Cycle for Artificial Intelligence, 2019. Technical Report. Gartner, Inc.
  63. 10.1145/3540250.3558915 · DOI
  64. Robert Szczepanek. 2022. Daily streamflow forecasting in mountainous catchment using XGBoost, LightGBM and CatBoost. Hydrology 9, 12 (2022), 226.
  65. 10.5120/1939-2590 · DOI
  66. 10.1145/3524610.3527911 · DOI
  67. Anthony J. Viera and Joanne M. Garrett. 2005. Understanding interobserver agreement: The kappa statistic. Family Medicine 37, 5 (2005), 360–363.
  68. 10.1109/ICME.2006.262509 · DOI
  69. Jiannan Wang Tim Kraska Michael J Franklin and Jianhua Feng. 2012. Crowder: Crowdsourcing entity resolution. arXiv:1208.1927. Retrieved from https://arxiv.org/abs/1208.1927 · arXiv
  70. 10.14778/1454159.1454165 · DOI
  71. William E. Winkler. 1999. The State of Record Linkage and Current Research Problems. Statistical Research Division, US Bureau of the Census, Wachington, DC (1999).
  72. 10.1109/ICSE48619.2023.00219 · DOI
  73. 10.1109/SANER.2018.8330229 · DOI
  74. Yang Xu, Wang Shaowei, Li Yi, and Wang Shaohua. 2023. Does data sampling improve deep learning-based vulnerability detection? Yeas! and Nays!. In Proceedings of the 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 2287–2298. · DOI
  75. Jinsung Yoon, James Jordon, and Mihaela van der Schaar. 2019. PATE-GAN: Generating synthetic data with differential privacy guarantees. In Proceedings of the 7th International Conference on Learning Representations. Retrieved from https://openreview.net/forum?id=S1zk9iRqF7
  76. 10.1145/3395363.3397384 · DOI
  77. 10.1016/j.aiopen.2021.01.001 · DOI · Paper page
  78. 10.1016/j.neucom.2017.01.026 · DOI

Cited by

Sorted by citations

Metadata

Authors
Yang, X., Rajbahadur, G. K., Lin, D., Wang, S., & Jiang, Z. M. (.
Affiliations
  • University of Manitoba (CA)
  • Huawei Technologies (Canada) (CA)
  • York University (CA)
ORCID
Venue
ACM Transactions on Software Engineering and Methodology
Publisher
Association for Computing Machinery (ACM)
Published
2024-12-27
Volume
34
Issue
1
Pages
1-27
ISSN
  • 1049-331X (ISSN-L)
  • 1557-7392
DOI
10.1145/3676961
OpenAlex
W4400685466
Cited by
3
References
3
Citations by year
2026: 2 · 2025: 1
FWCI
1.12
Type
journal article
Language
en
Open access
Licence
Locations
  • ACM Transactions on Software Engineering and Methodology (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Data Quality and Management
  • Social Sciences › Education › Academic integrity and plagiarism
Keywords
tabular data, value similarity, data deduplication, data reuse, AI software, data asset management

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review