Syntax-based Improvements to Plagiarism Detectors and their Evaluations
Open accessAbstract
Software plagiarism cheats students out of their own education and leads to unfair grading, making software plagiarism detection an important problem. However, many popular plagiarism detection tools are inaccurate, language-specific, or closed source, limiting their applicability. In this work, we seek to address these problems via a novel approach. We adapt the optimal Smith-Waterman sequence alignment algorithm to precisely measure the similarity between programs, greatly improving detection accuracy relative to competitors. Our approach is applicable to any language describable by an ANTLR grammar, which includes most programming languages. We also provide a new type of evaluation based on random program generation and obfuscation. Finally, we make our approach freely available, allowing for customizations and transparent reasoning about detection behavior.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
State-of-the-art plagiarism detectors operate token-based and compare the code's structure.
“State-ofthe-art plagiarism detectors operate token-based and compare the code's structure [31].”
From Detecting Automatic Software Plagiarism via Token Sequence Normalization (2024) · cites this paper as [31]
Section 2.2 describes token-based detectors, including JPLAG, and detectors that compare syntax trees. But it distinguishes these approaches: token sequences lose some program structure, which is why the authors’ detector uses parse trees instead. The source does not support the blanket claim that state-of-the-art detectors are both token-based and structural. (LLM, conf=0.85)
Experimental — accuracy not yet measuredMossad's approach is mainly designed against token-based plagiarism detectors but has the potential for attacks on other structure-based software plagiarism detectors.
“Their approach is mainly designed against token-based plagiarism detectors but has the potential for attacks on other structure-based [31] software plagiarism detectors.”
From Detecting Automatic Software Plagiarism via Token Sequence Normalization (2024) · cites this paper as [31]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Identification of common molecular subsequences
Smith, T. F., & Waterman, M. S. (1981) · Journal of Molecular Biology · doi:10.1016/0022-2836(81)90087-5 · Cited by 9,962 · see citing works
Schleimer, S., S, W. D., & Aiken, A. (2003) · Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data · doi:10.1145/872757.872770 · Cited by 1,099 · cited 5× · see citing works
“While there is a large existing body of work on plagiarism detection (e.g., [4–6, 8, 10, 12, 14], we observe that plagiarism detection remains an unsolved problem.”
BackgroundMethodInfluentialANTLR: A predicated‐ LL ( k ) parser generator
Parr, T. J., & Quong, R. W. (1995) · Software Practice and Experience · doi:10.1002/spe.4380250705 · Cited by 619 · see citing works
Gitchell, D., & Trân, N. (1999) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/299649.299783 · Cited by 163 · see citing works
JPlag: finding plagiarisms among a set of programs
Prechelt, L., Malpohl, G., & Philippsen, M. (2000) · Karlsruhe · doi:10.5445/ir/542000 · Cited by 144 · see citing works
Barone, L. M., Marinari, E., Organtini, G., & Ricci‐Tersenghi, F. (2013) · WORLD SCIENTIFIC eBooks · doi:10.1142/8842 · Cited by 99 · see citing works
Syntax tree fingerprinting for source code similarity detection
Chilowicz, M., Duris, É., & Roussel, G. (2009) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2009.5090050 · Cited by 89 · see citing works
PDetect: A Clustering Approach for Detecting Plagiarism in Source Code Datasets
Moussiades, L., & Vakali, A. I. (2005) · The Computer Journal · doi:10.1093/comjnl/bxh119 · Cited by 86 · see citing works
Sim: a utility for detecting similarity in computer programs
Gitchell, D., & Tran, N. (1999) · ACM SIGCSE Bulletin · doi:10.1145/384266.299783 · Cited by 62 · see citing works
Plagiarism Detection: A Tool Survey and Comparison
Martins, V. T., Fonte, D., Henriques, P. R., & da Cruz, D. (2014) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/oasics.slate.2014.143 · Cited by 53 · see citing works
WASTK: A Weighted Abstract Syntax Tree Kernel Method for Source Code Plagiarism Detection
Fu, D., Xu, Y., Yu, H., & Yang, B. (2017) · Scientific Programming · doi:10.1155/2017/7809047 · Cited by 44 · see citing works
Program Plagiarism Detection Using Parse Tree Kernels
Son, J.-W., Park, S.-B., & Park, S. (2006) · Lecture notes in computer science · doi:10.1007/978-3-540-36668-3_122 · Cited by 34 · see citing works
ES‐Plag: Efficient and sensitive source code plagiarism detection tool for academic environment
Sulistiani, L., & Karnalim, O. (2018) · Computer Applications in Engineering Education · doi:10.1002/cae.22066 · Cited by 34 · see citing works
Automated Plagiarism Detection for Computer Programming Exercises Based on Patterns of Resubmission
Tahaei, N., & Noelle, D. C. (2018) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3230977.3231006 · Cited by 30 · see citing works
Narayanan, S., & Surendran, S. (2012) · International Conference on Computer Science & Education (ICCSE) · doi:10.1109/iccse.2012.6295247 · Cited by 20 · see citing works
Prado, B., Bispo, K., & Andrade, R. (2018) · Proceedings of the 20th International Conference on Enterprise Information Systems · doi:10.5220/0006668705170524 · Cited by 9 · see citing works
AST-based multi-language plagiarism detection method
Zhang, L. P., & Liu, D. (2013) · IEEE International Conference on Software Engineering and Service Science (ICSESS) · doi:10.1109/icsess.2013.6615411 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Mossad: defeating software plagiarism detection
Devore-McDonald, B., & Berger, E. D. (2020) · Proceedings of the ACM on Programming Languages · doi:10.1145/3428206 · Cited by 35 · see citing works
IoT-based green city architecture using secured and sustainable android services
Ullah, F., Al-Turjman, F. M., & Nayyar, A. (2020) · Environmental Technology & Innovation · doi:10.1016/j.eti.2020.101091 · Cited by 31 · see citing works
Paraphrase type identification for plagiarism detection using contexts and word embeddings
Alvi, F., Stevenson, M., & Clough, P. D. (2021) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-021-00277-8 · Cited by 28 · see citing works
Ullah, F., Naeem, M. R., Naeem, H., Cheng, X., & Alazab, M. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.22813 · Cited by 23 · see citing works
Source Code Plagiarism Detection in an Educational Context: A Literature Mapping
Aniceto, R. C., de Holanda, M. T., Castanho, C. D., & Silva, D. D. (2021) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie49875.2021.9637155 · Cited by 22 · see citing works
Programmers' de-anonymization using a hybrid approach of abstract syntax tree and deep learning
Ullah, F., Jabbar, S., & Al-Turjman, F. M. (2020) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2020.120186 · Cited by 18 · see citing works
Automated Detection of AI-Obfuscated Plagiarism in Modeling Assignments
Sağlam, T., Hahner, S., Schmid, L., & Burger, E. (2024) · Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training · doi:10.1145/3639474.3640084 · Cited by 17 · see citing works
Maintaining Academic Integrity in Programming: Locality-Sensitive Hashing and Recommendations
Karnalim, O. (2023) · Education Sciences · doi:10.3390/educsci13010054 · Cited by 14 · see citing works
Disguising Code to Help Students Understand Code Similarity
Karnalim, O., & Simon (2020) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3428029.3428064 · Cited by 11 · see citing works
A Game-Based Framework to Compare Program Classifiers and Evaders
Damásio, T., Canesche, M., Pacheco, V., Botacin, M. F., da Silva, A. F., & Pereira, F. M. Q. (2023) · Proceedings of the CGO/Proceedings of the ... CGO · doi:10.1145/3579990.3580012 · Cited by 11 · see citing works
Detecting Automatic Software Plagiarism via Token Sequence Normalization
Sağlam, T., Brödel, M., Schmid, L., & Hahner, S. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639192 · Cited by 9 · see citing works
Work-in-Progress: Syntactic Code Similarity Detection in Strongly Directed Assessments
Karnalim, O., Simon, Ayub, M., Kurniawati, G., Nathasya, R. A., & Wijanto, M. C. (2021) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon46332.2021.9454152 · Cited by 9 · see citing works
Layered similarity detection for programming plagiarism and collusion on weekly assessments
Karnalim, O., Simon, N. P., & Chivers, W. (2022) · Computer Applications in Engineering Education · doi:10.1002/cae.22553 · Cited by 9 · see citing works
Identifying AI Generated Code with Parallel KNN Weight Outlier Detection
Karnalim, O. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-71530-3_29 · Cited by 1 · see citing works
Criminal Minds: How First-Year CS Students Plagiarize Code
Maisch, R., Schmid, L., Glassey, R. J., Fuchß, D., Niehues, N., Liu, H., & Koziolek, A. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3805789 · Cited by 1 · see citing works
Mapping Similarity Detectors of Code Clone to Academic Integrity in Programming
Pangestu, M. A., Karnalim, O., & Simon (2021) · IEEE World Engineering Education Conference (EDUNINE) · doi:10.1109/edunine51952.2021.9429164 · Cited by 1 · see citing works
Damerau-Levenshtein Distance Algorithm Based on Abstract Syntax Tree to Detect Code Plagiarism
Nuraminah, A., & Ammar, A. (2023) · Scientific Journal of Informatics · doi:10.15294/sji.v11i1.48064 · Cited by 1 · see citing works
Kuropiatnyk, O. (2022) · System technologies · doi:10.34185/1562-9945-1-138-2022-04
BIOPLAG: An Approach to Detect Programming Plagiarism
Gomes, K. P., Matos, S. N., & Bini, T. A. (2023) · Anais da Academia Brasileira de Ciências · doi:10.1590/0001-3765202320220684
Plagiarism on Mobile Programming Assessments: Automated Detection
Karnalim, O., Handoyo, E. D., Johan, M. C., & Surjawan, D. J. (2024) · International Conference on Technology Innovation and Its Applications (ICTIIA) · doi:10.1109/ictiia61827.2024.10761687
Mitigating Obfuscation Attacks on Software Plagiarism Detectors via Subsequence Merging
Sağlam, T., Niehues, N., Hahner, S., & Schmid, L. (2025) · IEEE International Conference on Software Engineering Education and Training (CSEE&T) · doi:10.1109/cseet66350.2025.00028
ROBiT - A Binary Optimization Anti-Plagiarism Method
Robert, R., da Silva, B. C., & Nobre, J. C. (2025) · Simpósio Brasileiro de Cibersegurança (SBSeg) · doi:10.5753/sbseg.2025.11514
RKRGST Mismatching to Detect Contract Cheating and Automated Obfuscation in Academia
Karnalim, O., & Sujadi, S. F. (2026) · International Conference on Information and Education Technology (ICIET) · doi:10.1109/iciet69664.2026.11561597
- Authors
- Nichols, L., Dewey, K., Emre, M., Chen, S., & Hardekopf, B.
- Affiliations
- University of California, Santa Barbara (US)
- California State University, Northridge (US)
- Venue
- Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education
- Publisher
- ACM
- Published
- 2019-07-02
- Pages
- 555-561
- OpenAlex
- W2963625196
- Semantic Scholar
- dc330291a0ae02ced176b670b5a1a4bc103e698c
- Cited by
- 22
- References
- 17
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 4 · 2023: 4 · 2022: 3 · 2021: 4 · 2020: 3
- FWCI
- 1.99
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) (publishedVersion, other-oa)
- Funders
- NSF (CCF-1319060)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Keywords
- software plagiarism detection, code similarity, programming language syntax, code obfuscation, academic dishonesty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
