The DOI matches its Crossref record; 174 of the 178 references it cites were found in a registry or an index.
Source-code Similarity Detection and Detection Tools Used in Academia: A Systematic Review
Abstract
Teachers deal with plagiarism on a regular basis, so they try to prevent and detect plagiarism, a task that is complicated by the large size of some classes. Students who cheat often try to hide their plagiarism (obfuscate), and many different similarity detection engines (often called plagiarism detection tools) have been built to help teachers. This article focuses only on plagiarism detection and presents a detailed systematic review of the field of source-code plagiarism detection in academia. This review gives an overview of definitions of plagiarism, plagiarism detection tools, comparison metrics, obfuscation methods, datasets used for comparison, and algorithm types. Perspectives on the meaning of source-code plagiarism detection in academia are presented, together with categorisations of the available detection tools and analyses of their effectiveness. While writing the review, some interesting insights have been found about metrics and datasets for quantitative tool comparison and categorisation of detection algorithms. Also, existing obfuscation methods classifications have been expanded together with a new definition of “source-code plagiarism detection in academia.”
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Hence, this has resulted in different interpretations of what is considered an indication of plagiarism for use in SCPDT evaluations, and how to evaluate SCPDTs in general [9].”
From Academic Source Code Plagiarism Detection by Measuring Program Behavioral Similarity (2021) · cites this paper as [9]
In the citing paper
“Furthermore, all known available SCPDTs implement structural measures of similarity only.”
From Academic Source Code Plagiarism Detection by Measuring Program Behavioral Similarity (2021) · cites this paper as [9]
In the citing paper
“The lack of similar approaches to SCPD is supported by the recent literature reviews [8], [9] that identify various structural and/or semantic SCPDTs, however, fail to identify any behavioural SCPDTs.”
From Academic Source Code Plagiarism Detection by Measuring Program Behavioral Similarity (2021) · cites this paper as [9]
In the citing paper
“All known proposed SCPDTs implement structural or semantic measures of similarity.”
From Academic Source Code Plagiarism Detection by Measuring Program Behavioral Similarity (2021) · cites this paper as [9]
In the citing paper
“We chose JPlag as it is state-of-the-art [32], open-source, and was used for the evaluation of Mossad [13].”
From Detecting Automatic Software Plagiarism via Token Sequence Normalization (2024) · cites this paper as [32]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Halstead, M. H. (1977) · CERN Document Server (European Organization for Nuclear Research) · Cited by 3,639
Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M., & Khalil, M. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2006.07.009 · Cited by 2,394 · cited 1×
“(software engineering) [18], Spolaor and Benitti (learning theories) [159], and Breivold et al.”
MethodSchleimer, S., S, W. D., & Aiken, A. (2003) · Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data · doi:10.1145/872757.872770 · Cited by 1,099 · cited 2×
“The best example is the paper about the Winnowing algorithm [148], is one of the most referenced papers, but the paper was not picked up by the queries.”
BackgroundMethodPlagiarism in programming assignments
Joy, M., & Luck, M. (1999) · IEEE Transactions on Education · doi:10.1109/13.762946 · Cited by 299
Self-citationYAP3: improved detection of similarities in computer program and other texts
Wise, M. J. (1996) · Proceedings of the twenty-seventh SIGCSE technical symposium on Computer science education · doi:10.1145/236452.236525 · Cited by 294
Shared Information and Program Plagiarism Detection
Chen, X., Francia, B., Li, M., McKinnon, B., & Seker, A. (2004) · IEEE Transactions on Information Theory · doi:10.1109/tit.2004.830793 · Cited by 251
Computer algorithms for plagiarism detection
Parker, A., & Hamblen, J. O. (1989) · IEEE Transactions on Education · doi:10.1109/13.28038 · Cited by 216 · cited 8×
“Routine transformations, typically text substitutions, do not require a detailed understanding of the program” [131].”
BackgroundMethodInfluentialAn algorithmic approach to the detection and prevention of plagiarism
Ottenstein, K. J. (1976) · ACM SIGCSE Bulletin · doi:10.1145/382222.382462 · Cited by 208 · cited 2×
“Research into source-code plagiarism detection had already been performed in the 1970s, for example, a paper [130] from Ottenstein was published in 1976, but some mechanisms that he used to detect plagiarism had already been presented a few years earlier.”
BackgroundFaidhi, J. A. W., & Robinson, S. K. (1987) · Computers & Education · doi:10.1016/0360-1315(87)90042-x · Cited by 198 · cited 10×
“Another common classification are the six levels (plus level zero, which means no changes) from Faidhi and Robinson [42].”
BackgroundMethodInfluentialAn Approach to Source-Code Plagiarism Detection and Investigation Using Latent Semantic Analysis
Cosma, G., & Joy, M. (2012) · IEEE Transactions on Computers · doi:10.1109/tc.2011.223 · Cited by 176
Self-citation
Research works printed but not linked: 16
- F. W. Blackwell . 1980. Computer assistance in detecting plagiarsim in student programs . California Education Computer Consortium , 28--30. F. W. Blackwell. 1980. Computer assistance in detecting plagiarsim in student programs. California Education Computer Consortium, 28--30.
- K. W. Bowyer and L. O. Hall . 1999. Experience using “MOSS” to detect cheating on programming assignments . In Proceedings of the 29th Annual Frontiers in Education Conference: Designing the Future of Science and Engineering Education , Vol. 3 . Stripes Publishing L.L.C, San Juan, Puerto Rico, USA, 13B3/18--13B3/22. K. W. Bowyer and L. O. Hall. 1999. Experience using “MOSS” to detect cheating on programming assignments. In Proceedings of the 29th Annual Frontiers in Education Conference: Designing the Future of Science and Engineering Education, Vol. 3. Stripes Publishing L.L.C, San Juan, Puerto Rico, USA, 13B3/18--13B3/22.
- 10.5555/1228662.1228664 · doi:10.5555/1228662.1228664
- Fintan Culwin , Anna Macleod , and Thomas Lancaster . 2001 . Source code plagiarism in UK HE computing schools . In Proceedings of the 2nd Annual LTSN-ICS Conference. LTSN Centre for Information and Computer Sciences , London, United Kingdom. Fintan Culwin, Anna Macleod, and Thomas Lancaster. 2001. Source code plagiarism in UK HE computing schools. In Proceedings of the 2nd Annual LTSN-ICS Conference. LTSN Centre for Information and Computer Sciences, London, United Kingdom.
- M. C. Johnson , C. Watson , S. Davidson , and D. Eschbach . 2004. Gene sequence inspired design plagiarism screening . In Proceedings of the Annual Conference and Exposition: Engineering Researchs New Heights. 6087--6105 . M. C. Johnson, C. Watson, S. Davidson, and D. Eschbach. 2004. Gene sequence inspired design plagiarism screening. In Proceedings of the Annual Conference and Exposition: Engineering Researchs New Heights. 6087--6105.
- B. Karthikeyan , V. Vaithiyanathan , and C. V. Lavanya . 2011 . Similarity detection in source code using data mining techniques . Eur. J. Sci. Res. 62 , 4 (2011), 500 -- 505 . B. Karthikeyan, V. Vaithiyanathan, and C. V. Lavanya. 2011. Similarity detection in source code using data mining techniques. Eur. J. Sci. Res. 62, 4 (2011), 500--505.
- T. Lancaster and M. Tetlow . 2005. Does automated anti-plagiarism have to be complex? Evaluating more appropriate software metrics for finding collusion . In Proceedings of the 22nd Annual Conference of the Australian Society for Computers in Tertiary Education. 361--370 . T. Lancaster and M. Tetlow. 2005. Does automated anti-plagiarism have to be complex? Evaluating more appropriate software metrics for finding collusion. In Proceedings of the 22nd Annual Conference of the Australian Society for Computers in Tertiary Education. 361--370.
- V. Mateljan , V. Juričić , and K. Peter . 2011. Analysis of programming code similarity by using intermediate language . In Proceedings of the 34th International Convention on Information and Communication Technology, Electronics and Microelectronics. 1235--1240 . V. Mateljan, V. Juričić, and K. Peter. 2011. Analysis of programming code similarity by using intermediate language. In Proceedings of the 34th International Convention on Information and Communication Technology, Electronics and Microelectronics. 1235--1240.
- A. Ohno and H. Murao . 2011 . A two-step in-class source code plagiarism detection method utilizing improved CM algorithm and SIM . Int. J. Innov. Comput. Inf. Contr. 7 , 8 (2011), 4729 -- 4739 . A. Ohno and H. Murao. 2011. A two-step in-class source code plagiarism detection method utilizing improved CM algorithm and SIM. Int. J. Innov. Comput. Inf. Contr. 7, 8 (2011), 4729--4739.
- G. K. Rambally and M. Lesage . 1990. An inductive inference approach to plagiarsim detection in computer-programs . In Proceedings of the National Educational Computing Conference (INTSOC’90) , 23--29. G. K. Rambally and M. Lesage. 1990. An inductive inference approach to plagiarsim detection in computer-programs. In Proceedings of the National Educational Computing Conference (INTSOC’90), 23--29.
- J. C. Rodriguez , E. Rubio Royo , and Z. Hernandez . 2011. USES OF VPL . In Proceedings of the 5th International Technology, Education and Development Conference. IATED-INT, 743--748 . J. C. Rodriguez, E. Rubio Royo, and Z. Hernandez. 2011. USES OF VPL. In Proceedings of the 5th International Technology, Education and Development Conference. IATED-INT, 743--748.
- A. Semerdzhiev and T. Trifonov . 2013. Practical aspects of plagiarsim detection in coputer science e-learning . In Proceedings of the 6th International Conference of Education, Research and Inovation. IATED-INT, 3953--3961 . A. Semerdzhiev and T. Trifonov. 2013. Practical aspects of plagiarsim detection in coputer science e-learning. In Proceedings of the 6th International Conference of Education, Research and Inovation. IATED-INT, 3953--3961.
- I. Smeureanu and B. Iancu . 2013. Source code plagiarsim detection method using ontologies . In Proceedings of the International Conference on Informatics in Economy. 594--597 . I. Smeureanu and B. Iancu. 2013. Source code plagiarsim detection method using ontologies. In Proceedings of the International Conference on Informatics in Economy. 594--597.
- N. D. Tselikas , M. Samarakou , D. Karolidis , P. Prentakis , and S. Athineos . 2014. Automatic plagiarism detection in programming laboratory courses . In Proceedings of the 7th IADIS International Conference Information Systems. IADIS, 232--238 . N. D. Tselikas, M. Samarakou, D. Karolidis, P. Prentakis, and S. Athineos. 2014. Automatic plagiarism detection in programming laboratory courses. In Proceedings of the 7th IADIS International Conference Information Systems. IADIS, 232--238.
- A. Wojdyga . 2013 . Towards intellectual property theft prevention: Economic significance of automatic software plagiarism verification . Act. Probl. Econ. 142 , 4 (2013), 300 -- 306 . A. Wojdyga. 2013. Towards intellectual property theft prevention: Economic significance of automatic software plagiarism verification. Act. Probl. Econ. 142, 4 (2013), 300--306.
- L. Zhang , Y. T. Zhuang , and Z. M. Yuan . 2007. A program plagiarism detection model based on information distance and clustering . In Proceedings of the 2007 International Conference on Intelligent Pervasive Computing. IEEE, 431--436 . L. Zhang, Y. T. Zhuang, and Z. M. Yuan. 2007. A program plagiarism detection model based on information distance and clustering. In Proceedings of the 2007 International Conference on Intelligent Pervasive Computing. IEEE, 431--436.
Cited by
Exploring DeepSeek: A Survey on Advances, Applications, Challenges and Future Directions
Deng, Z., Ma, W., Han, Q., Zhou, W., Zhu, X., Wen, S., & Xiang, Y. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125498 · Cited by 224
SoK: Decentralized Finance (DeFi) Attacks
Zhou, L., Xiong, X., Ernstberger, J., Chaliasos, S., Wang, Z., Wang, Y., Qin, K., Wattenhofer, R., Song, D., & Gervais, A. (2023) · 2023 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46215.2023.10179435 · Cited by 200
Nasrabadi, M. Z., Parsa, S. A., Ramezani, M., Roy, C. K., & Ekhtiarzadeh, M. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111796 · Cited by 129
Combéfis, S. (2022) · Software · doi:10.3390/software1010002 · Cited by 96
A Systematic Mapping Study of Code Quality in Education
Keuning, H., Jeuring, J. T., & Heeren, B. J. (2023) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3587102.3588777 · Cited by 66
Metadata
- Authors
- Novak, M., Joy, M., & Kermek, D.
- Affiliations
- University of Zagreb (HR)
- University of Warwick (GB)
- Venue
- ACM Transactions on Computing Education
- Published
- 2019-05-21
- Volume
- 19
- Issue
- 3
- Pages
- 1-37
- ISSN
- 1946-6226 (ISSN-L)
- OpenAlex
- W2945111072
- Cited by
- 117
- References
- 178
- Citations by year
- 2026: 11 · 2025: 14 · 2024: 19 · 2023: 21 · 2022: 19 · 2021: 19 · 2020: 10 · 2019: 4
- FWCI
- 22.73
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Computing Education (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (submittedVersion)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Keywords
- code plagiarism detection, academic dishonesty, plagiarism detection software, code obfuscation, similarity measures, programming education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Source-code Similarity Detection and Detection Tools Used in Academia: A Systematic Review. https://app.cochat.ai/papers/10.1145/3313290
- The paper
- Novak, M., Joy, M., & Kermek, D. (2019). Source-code Similarity Detection and Detection Tools Used in Academia: A Systematic Review. ACM Transactions on Computing Education. https://doi.org/10.1145/3313290
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.