Who wrote this? Evaluating the reliability of AI detection tools in higher education
Open accessAbstract
The widespread and increasing adoption of generative artificial intelligence (GenAI) in higher education has raised urgent questions about academic integrity and the reliability of AI detection tools used in high-stakes assessments. This study compares the accuracy of four popular detection tools: GPTZero, Pangram, Copyleaks, and Turnitin on four kinds of academic papers: fully human-written, fully AI-written, hybrid (human with GenAI-inserted passages), and humanised GenAI (AI-generated passages were humanised using a prompt designed to resemble possible student behaviour). Using a synthetic dataset of 160 documents with known ground truth values, we assessed each tool’s detection accuracy. Results show that Pangram consistently performed better than the other tools, achieving high accuracy in detecting fully AI-generated, hybrid, and humanised texts. In contrast, the other tools significantly underestimated GenAI content, particularly for texts generated with the most advanced model. All tools correctly identified fully human texts. To illustrate how the best-performing tool performs in an authentic academic context, Pangram was applied to 1,163 master’s theses submitted in academic year 2024–2025, without known ground truth. The analysis describes the distribution of Pangram’s flagging scores, with flagged cases (45.5%) typically indicating low to moderate levels of AI-associated text. False positives were rare across all tools, suggesting improvement compared to earlier studies. Findings show that while detection tools can provide useful initial flags, they should not be used as sole evidence in high-stakes decision-making but should be implemented in a broader evaluation strategy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI content detectors produce inconsistent results and are readily circumvented by minor edits to AI-generated text, limiting the weight detection-based enforcement alone can carry.
“AI content detectors have been shown to produce inconsistent results and are readily circumvented by minor edits to AI-generated text, which limits how much weight detection-based enforcement alone can carry (Hadra et al., 2026;Van Vlasselaer et al., 2026).”
From Convenience Amid Crisis: Explaining Generative AI Misuse and Compliance Intention Among Myanmar University Students (2026) · cites this paper as Van Vlasselaer et al., 2026
The Classification accuracy and error analysis section reports sharply different results across detectors, including reduced accuracy for Turnitin after AI-generated passages were humanised. The Discussion says detector flags should prompt further review rather than serve as a final conclusion. But the tested alteration was a GPT-4o humanising prompt, not necessarily a minor edit, and Pangram remained largely unaffected; the paper does not establish that detectors are generally readily circumvented by minor edits. Page numbers were unavailable in our copy. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · see citing works
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079 · see citing works
Students' voices on generative ai: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · arXiv · doi:10.48550/arxiv.2305.00290 · Cited by 2,504 · cited 1× · see citing works
“Baek et al. (2024) found that 33.1% used ChatGPT monthly for writing, and Chan and Hu (2023) documented generally positive attitudes toward its integration in learning.”
InfluentialStudents’ voices on generative AI: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00411-8 · Cited by 2,301 · cited 1× · see citing works
“Baek et al. (2024) found that 33.1% used ChatGPT monthly for writing, and Chan and Hu (2023) documented generally positive attitudes toward its integration in learning.”
InfluentialChatGPT in higher education: considerations for academic integrity and student learning
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.17 · Cited by 853 · see citing works
Perkins, M. (2023) · Journal of University Teaching and Learning Practice · doi:10.53761/1.20.02.07 · Cited by 839 · see citing works
Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13544 · Cited by 805 · see citing works
Testing of detection tools for AI-generated text
Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023) · International Journal for Educational Integrity · doi:10.1007/s40979-023-00146-z · Cited by 582 · cited 5× · see citing works
“Similarly, Weber-Wulff et al. (2023) also highlighted serious limitations of current AI-generated text detection tools, noting inconsistent results, high rates of false positives and negatives, and their unsuitability as sole evidence in academic misconduct cases.”
InfluentialJin, Y., Yan, L., Echeverria, V., Gašević, D., & Martinez-Maldonado, R. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100348 · Cited by 452 · see citing works
Academic integrity: A review of the literature
Macfarlane, B., Zhang, J., & Pun, A. (2014) · Studies in Higher Education · doi:10.1080/03075079.2012.709495 · Cited by 359 · see citing works
Perkins, M., Furze, L., Roe, J., & MacVaugh, J. (2024) · Journal of University Teaching and Learning Practice · doi:10.53761/q3azde36 · Cited by 291 · cited 2× · see citing works
“…approaches may also support institutions in setting clearer expectations for students and in designing assessments that protect academic integrity (Perkins et al. 2024a, b) As the study by An (2025) also shows, the majority of top-ranked U.S. universities did not completely ban these tools,…”
Wang, H., Dang, A., Wu, Z., & Mac, S. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100326 · Cited by 272 · see citing works
Dalalah, D., & Dalalah, O. M. A. (2023) · The International Journal of Management Education · doi:10.1016/j.ijme.2023.100822 · Cited by 218 · see citing works
Plagiarism in Higher Education: Tackling Tough Topics in Academic Integrity
Eaton, S. E. (2021) · doi:10.5040/9798400697142 · Cited by 199 · see citing works
Ibrahim, H., Liu, F., Asim, R., Battu, B., Benabderrahmane, S., Alhafni, B., Adnan, W., Alhanai, T., AlShebli, B., Baghdadi, R., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-38964-3 · Cited by 154 · see citing works
“ChatGPT seems too good to be true”: College students’ use and perceptions of generative AI
Baek, C., Tate, T., & Warschauer, M. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100294 · Cited by 150 · cited 1× · see citing works
“Baek et al. (2024) found that 33.1% used ChatGPT monthly for writing, and Chan and Hu (2023) documented generally positive attitudes toward its integration in learning.”
InfluentialSimple techniques to bypass GenAI text detectors: implications for inclusive education
Perkins, M., Roe, J., Vu, B. H., Postma, D., Hickerson, D., McGaughran, J., & Khuat, H. Q. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00487-w · Cited by 117 · cited 2× · see citing works
“…approaches may also support institutions in setting clearer expectations for students and in designing assessments that protect academic integrity (Perkins et al. 2024a, b) As the study by An (2025) also shows, the majority of top-ranked U.S. universities did not completely ban these tools,…”
Walters, W. H. (2023) · Open Information Science · doi:10.1515/opis-2022-0158 · Cited by 109 · cited 4× · see citing works
“We chose 40 texts per category to ensure sufficient statistical power for detecting performance differences between tools and were consistent with sample sizes used in prior GenAI detection evaluation studies (e.g., Walters 2023; Weber-Wulff et al. 2023).”
InfluentialHistory, development, and principles of large language models: an introductory survey
Wang, Z., Chu, Z., Doan, T. V., Ni, S., Yang, M., & Zhang, W. (2025) · AI and Ethics · doi:10.1007/s43681-024-00583-7 · Cited by 109 · see citing works
Gustilo, L., Ong, E., & Lapinid, M. R. (2024) · International Journal for Educational Integrity · doi:10.1007/s40979-024-00153-8 · Cited by 89 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- Van Vlasselaer et al. International Journal for Educational Integrity (2026) 22:16
- Plecerda LP (2024) Academic integrity surrounding the use of generative AI in higher education: Lenses from ICT college students. EnvironSocial Psychol 9(12). https://doi.org/10.59429/esp.v9i12.3177 · DOI
- Pudasaini S, Miralles-Pechuán L, Lillis D, Llorens Salvador M (2024) Survey on AI-generated plagiarism detection: The impact of large language models on academic integrity. J Acad Ethics 23:1137-1170. https://doi.org/10.1007/s10805-024-09576-x · DOI · Paper page
- Reiter L, Jörling M, Fuchs C, Böhm R (2025) Student (mis)use of generative AI tools for university-related tasks. Int J Human- computer Interact 41(19). https://doi .org/10.108 0/10447318. 2025.246 2083 · DOI
- Russell J, Karpinska M, Iyyer, M (2025) People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text. In W Che, J Nabende, E Shutova, MT Pilehvar (Eds.), Proceedings of the 63rd Annual Meeting of the Associa- tion for Computational Linguistics (Volume 1: Long Papers) (pp. 5342-5373). Association for Computational Linguistics. https://doi .org/10.186 53/v1/2025. acl-long .267 · DOI
- Sol K, Sok S, Heng K (2025) Rethinking assessment in higher education in the age of generative AI. In M A Peters, R Heraud (Eds.), Encyclopedia of educational innovation (pp. 1-5). Springer. https://doi .org/10.100 7/978-981-1 3-2262-4 _327-2 · DOI
- Tiwari S, Sharma R, Sikarwar RS, Dubey GP, Bajpai N, Singhatiya S (2024) Detecting AI generated content: A study of methods and applications. In S Kumar, R Garg S Hiranwal, SD Purohit (Eds.), Proceedings of the International Conference on Com- munication and Computational Technologies (ICCCT 2024) (Vol. 1, pp. 162-171). Springer. h t t p s : / / d o i . o r g / 1 0 . 1 0 0 7 / 9 7 8 -9 8 1 -9 7 -7 4 2 3 -4 · DOI
Cited by
Sorted by citations
Swanson, B. A. (2026) · Journal of ICT and Education · doi:10.69739/jicte.v1i2.2263
- Authors
- Van Vlasselaer, M., Van Droogenbroeck, F., & Spruyt, B.
- Affiliations
- Vrije Universiteit Brussel (BE)
- Venue
- International Journal for Educational Integrity
- Published
- 2026-06-29
- Volume
- 22
- Issue
- 1
- ISSN
- 1833-2595 (ISSN-L)
- OpenAlex
- W7166448951
- Cited by
- 14
- References
- 20
- Citations by year
- 2026: 14
- FWCI
- 53.37
- Type
- journal article
- Language
- en
- Update notices
- correction (2026-10-06)
- Open access
- gold
- doi.org
- Licence
- Locations
- International Journal for Educational Integrity (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- VUBIR (Vrije Universiteit Brussel) (submittedVersion)
- Topics
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- generative AI, AI detection tools, academic integrity, GPTZero, Turnitin, detection accuracy, master's theses, high-stakes testing, false positives, synthetic dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
