Hermetic and Web Plagiarism Detection Systems for Student Essays—An Evaluation of the State-of-the-Art
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Plagiarism has become a serious problem in education, and several plagiarism detection systems have been developed for dealing with this problem. This study provides an empirical evaluation of eight plagiarism detection systems for student essays. We present a categorical hierarchy of the most common types of plagiarism that are encountered in student texts. Our purpose-built test set contains texts in which instances of several commonly utilized plagiaristic techniques have been embedded. While Sherlock was clearly the overall best hermetic detection system, SafeAssignment performed best in detecting web plagiarism. TurnitIn was found to be the most advanced system for detecting semi-automatic forms of plagiarism such as the substitution of Cyrillic equivalents for certain characters or the insertion of fake whitespaces. The survey indicates that none of the systems are capable of reliably detecting plagiarism from both local and Internet sources while at the same time being able to identify the technical tricks that plagiarizers use to conceal plagiarism.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Schleimer, S., S, W. D., & Aiken, A. (2003) · Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data · doi:10.1145/872757.872770 · Cited by 1,099 · cited 1×
“…systems that are sold commercially, most hermetic detectors are available without cost (Bloomfield, 2009; Prechelt, Malpohl, & Philippsen, 2002; Schleimer, Wilkerson, & Aiken, 2003), are also, in some cases, open source (Ahtiainen, Surakka, & Rahikainen, 2006; Joy & Luck, 1999) and utilize…”
BackgroundPlagiarism in programming assignments
Joy, M., & Luck, M. (1999) · IEEE Transactions on Education · doi:10.1109/13.762946 · Cited by 299
An algorithmic approach to the detection and prevention of plagiarism
Ottenstein, K. J. (1976) · ACM SIGCSE Bulletin · doi:10.1145/382222.382462 · Cited by 208 · cited 1×
“While academic interest in plagiarism detection in software source codes dates back to the 1970s (Ottenstein, 1976), plagiarism detection systems for texts have only been developed over the past two decades.”
BackgroundFaidhi, J. A. W., & Robinson, S. K. (1987) · Computers & Education · doi:10.1016/0360-1315(87)90042-x · Cited by 198 · cited 1×
“Most of the current hermetic systems compare input files on the basis of the content (although other approaches, such as attribute counting, are also used; Faidhi & Robinson, 1987).”
BackgroundFast Plagiarism Detection System
Mozgovoy, M., Fredriksson, K., White, D., Joy, M., & Sutinen, E. (2005) · Lecture Notes in Computer Science · doi:10.1007/11575832_30 · Cited by 49 · cited 1×
“Research by Mozgovoy et al. (2005) has also shown how “a jury” that consists of several high-quality plagiarism detection systems can be used to test another system with good results and a high degree of reliability.”
BackgroundSelf-citation
Cited by
Ferreira‐Mello, R., André, M., Pinheiro, A., Costa, E., & Romero, C. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1332 · Cited by 193
Reducing plagiarism through academic misconduct education
Perkins, M., Gezgin, U. B., & Roe, J. (2020) · International Journal for Educational Integrity · doi:10.1007/s40979-020-00052-8 · Cited by 134
“Although it is very easy for software solutions to identify text that is already present in its database, the ability of current software is not yet advanced enough to detect the complexities of contract cheating (Kakkonen and Mozgovoy 2010; Mozgovoy et al. 2010).”
BackgroundState-of-the-art in detecting academic plagiarism
Meuschke, N., & Gipp, B. (2013) · International Journal for Educational Integrity · doi:10.21913/ijei.v9i1.847 · Cited by 112
“The results of the PAN competitions, the HTW PDS Test and other studies (Hill & Page, 2009; Kakkonen & Mozgovoy, 2010; Maurer & Zaka, 2007; Scaife, 2007) prove that state-of-the-art PDS, which implement external detection methods, find incidences of verbatim and slightly modified copying with high…”
BackgroundAn integrated approach for intrinsic plagiarism detection
AlSallal, M., Iqbal, R., Palade, V., Amin, S., & Chang, V. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2017.11.023 · Cited by 48
Plagiarism Detection in Texts Obfuscated with Homoglyphs
Alvi, F., Stevenson, M., & Clough, P. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-56608-5_64 · Cited by 15
Metadata
- Authors
- Kakkonen, T., & Mozgovoy, M.
- Venue
- Journal of Educational Computing Research
- Publisher
- SAGE Publications
- Published
- 2010-02-09
- Volume
- 42
- Issue
- 2
- Pages
- 135-159
- ISSN
- 0735-6331 (ISSN-L)
- 1541-4140
- OpenAlex
- W1980106054
- Cited by
- 43
- References
- 5
- FWCI
- 7.51
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.