The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
AI Detection Tools: A Systematic Review of Empirical Evidence and Their Implications for Education
Open accessAbstract
The rapid advancement of generative artificial intelligence has significantly transformed academic writing practices, prompting institutions to implement tools designed to verify authorship and uphold academic integrity. Artificial intelligence detection systems have emerged as a prominent, albeit increasingly debated response to these challenges. This systematic review synthesizes empirical evidence to assess the reliability, fairness, and pedagogical implications of artificial intelligence text-detection tools in educational settings. Adhering to PRISMA 2020 standards, this review identified twenty-five peer-reviewed empirical studies, 18 of which were conducted directly within educational settings. This review synthesizes empirical studies published between 2022 and 2025, encompassing quantitative, qualitative, and mixed-methods designs across diverse disciplinary and linguistic contexts. The findings indicate that AI text detection tools are unsuitable for high-stakes academic integrity decisions in their current form. Furthermore, there is substantial variability and instability in detection accuracy across tools, genres, and linguistic backgrounds; a noticeable weakness in paraphrasing, translation, and other adversarial techniques; and systemic biases that disproportionately affect non-native English writers. Human judgment was also found to be inconsistent, reinforcing the difficulty in reliably distinguishing AI-generated text from human-authored text. Collectively, these results raise significant ethical, pedagogical and institutional concerns. This review underscores the need for integrity strategies that prioritize transparency, AI literacy, fairness-aware design, and process-based assessment rather than relying on detection-centered approaches. The findings suggest the necessity of hybrid approaches that combine watermarking and fairness-aware detection algorithms with process-oriented assessment, AI literacy initiatives, and cross-linguistic benchmarking, alongside interpretability-focused and longitudinal research on students’ perceptions of AI detection.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Studies further illustrate that even minor textual alterations, such as synonym replacement, sentence reconfiguration, or translation cycles, can markedly diminish detection accuracy.”
From Harms and Ethical Risks of AI Detection Tools in Education: False Positives, Bias, Surveillance, and Student Rights (2026) · cites this paper as Caner, 2025
The cited source says what the claim attributes to it.
In the citing paper
“Thus, false positives are structural features of current detection architectures (Caner, 2025; Elkhatat et al., 2023; Walters, 2023).”
From Harms and Ethical Risks of AI Detection Tools in Education: False Positives, Bias, Surveillance, and Student Rights (2026) · cites this paper as Caner, 2025
The cited source says what the claim attributes to it.
In the citing paper
“They are also highly vulnerable to simple evasion strategies, including paraphrasing and translation.”
From Harms and Ethical Risks of AI Detection Tools in Education: False Positives, Bias, Surveillance, and Student Rights (2026) · cites this paper as Caner, 2025
The cited source says what the claim attributes to it.
In the citing paper
“A systematic review of 25 peer-reviewed studies conducted in educational contexts demonstrates that AI detection tools exhibit significant variability in accuracy across genres, disciplines, and linguistic backgrounds.”
From Harms and Ethical Risks of AI Detection Tools in Education: False Positives, Bias, Surveillance, and Student Rights (2026) · cites this paper as Caner, 2025
The cited source says what the claim attributes to it.
In the citing paper
“In the absence of clear explanatory frameworks, these scores are at risk of being overinterpreted, thereby increasing their potential for harm (Caner, 2025; Lieberman, 2024).”
From Harms and Ethical Risks of AI Detection Tools in Education: False Positives, Bias, Surveillance, and Student Rights (2026) · cites this paper as Caner, 2025
The cited source says what the claim attributes to it.
In the citing paper
“This bias is not incidental but structurally embedded in the design and training of these systems (Caner, 2025; Ibrahim, 2023; Jiang et al., 2024; Liang et al., 2023).”
From Harms and Ethical Risks of AI Detection Tools in Education: False Positives, Bias, Surveillance, and Student Rights (2026) · cites this paper as Caner, 2025
The cited source says what the claim attributes to it.
In the citing paper
“Empirical research indicates that probabilistic AI-likelihood scores are often misinterpreted by both educators and students, who may regard them as definitive indicators rather than as uncertain estimates.”
From Harms and Ethical Risks of AI Detection Tools in Education: False Positives, Bias, Surveillance, and Student Rights (2026) · cites this paper as Caner, 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00819-6 · Cited by 1,099
GPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · Patterns · doi:10.1016/j.patter.2023.100779 · Cited by 673
Testing of detection tools for AI-generated text
Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023) · International Journal for Educational Integrity · doi:10.1007/s40979-023-00146-z · Cited by 582
Elkhatat, A. M., Elsaid, K., & Almeer, S. (2023) · International Journal for Educational Integrity · doi:10.1007/s40979-023-00140-5 · Cited by 392
Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda
Bittle, K., & El-Gayar, O. (2025) · Information · doi:10.3390/info16040296 · Cited by 305
Casal, J. E., & Kessler, M. (2023) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100068 · Cited by 261
Perkins, M., Roe, J., Postma, D., McGaughran, J., & Hickerson, D. (2024) · Journal of Academic Ethics · doi:10.1007/s10805-023-09492-6 · Cited by 234
Dalalah, D., & Dalalah, O. M. A. (2023) · The International Journal of Management Education · doi:10.1016/j.ijme.2023.100822 · Cited by 218
The death of the short-form physics essay in the coming AI revolution
Yeadon, W., Inyang, O.-O., Mizouri, A., Peach, A., & Testrow, C. P. (2023) · Physics Education · doi:10.1088/1361-6552/acc5cf · Cited by 199
Ethical and regulatory challenges of Generative AI in education: a systematic review
García-López, I. M., & Trujillo-Liñán, L. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1565938 · Cited by 144
Evaluating the authenticity of ChatGPT responses: a study on text-matching capabilities
Elkhatat, A. M. (2023) · International Journal for Educational Integrity · doi:10.1007/s40979-023-00137-0 · Cited by 127
Do teachers spot AI? Evaluating the detectability of AI-generated texts among student essays
Fleckenstein, J., Meyer, J., Jansen, T., Keller, S. D., Köller, O., & Möller, J. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100209 · Cited by 126
Simple techniques to bypass GenAI text detectors: implications for inclusive education
Perkins, M., Roe, J., Vu, B. H., Postma, D., Hickerson, D., McGaughran, J., & Khuat, H. Q. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00487-w · Cited by 117
Walters, W. H. (2023) · Open Information Science · doi:10.1515/opis-2022-0158 · Cited by 109
Jiang, Y., Hao, J., Fauss, M., & Li, C. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105070 · Cited by 68
Ibrahim, K. (2023) · Language Testing in Asia · doi:10.1186/s40468-023-00260-2 · Cited by 66
Assisting academics to identify computer generated writing
Abd-Elaal, E.-S., Gamage, S. H. P. W., & Mills, J. E. (2022) · European Journal of Engineering Education · doi:10.1080/03043797.2022.2046709 · Cited by 62
The Problem with False Positives: AI Detection Unfairly Accuses Scholars of AI Plagiarism
Giray, L. (2024) · The Serials Librarian · doi:10.1080/0361526x.2024.2433256 · Cited by 60
Baron, P. (2024) · Scholarship of Teaching and Learning in the South · doi:10.36615/sotls.v8i2.411 · Cited by 44
Popkov, A. A., & Barrett, T. S. (2024) · Accountability in Research · doi:10.1080/08989621.2024.2331757 · Cited by 40
Can we trust academic AI detective? Accuracy and limitations of AI-output detectors
Erol, G., Ergen, A., Gülşen Erol, B., Kaya Ergen, Ş., Bora, T. S., Çölgeçen, A. D., Araz, B., Şahin, C., Bostancı, G., Kılıç, İ., et al. (2025) · Acta Neurochirurgica · doi:10.1007/s00701-025-06622-4 · Cited by 28
Pratama, A. R. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2953 · Cited by 22
Robustness of generative AI detection: adversarial attacks on black-box neural text detectors
Fishchuk, V., & Braun, D. (2024) · International Journal of Speech Technology · doi:10.1007/s10772-024-10144-2 · Cited by 7
Beyond Reproach: Navigating Usage, Detection, and Future Pathways of AI in Education
Kumar, R. (2024) · Brock Education Journal · doi:10.26522/brocked.v33i3.1173 · Cited by 6
Alharthi, R. S., Ojo, S., Nathaniel, T. I., Samee, N. M. A., Umer, M., Jamjoom, M. M., Alsubai, S., & Khan, J. (2025) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-025-01025-w · Cited by 5
Research works printed but not linked: 3
- Chaka, C. (2023). Detecting AI content in responses generated by ChatGPT, YouChat, and ChatSonic: The case of five AI content detection tools. Journal of Applied Learning & Teaching, 6(2). https://doi.org/10.37074/jalt. 2023.6.2.12. Cingillioglu, I. (2023). Detecting AI-generated essays: The ChatGPT challenge. International Journal of Information and Learning Technology, 40(3). 259-268. https://doi.org/10.1108/IJILT-03-2023-0043. · DOI
- Liu, J. Q. J., Hui, K. T. K., Al Zoubi, F., Zhou, Z. Z. X., Samartzis, D., Yu, C. C. H., Chang, J. R., & Wong, A. Y. L. (2024). The great detectives: Humans versus AI detectors in catching large language model-generated medical writing. International Journal for Educational Integrity, 20, Article 8. https://doi.org/10.1007/s40979- 024-00141-8. · DOI
- Lieberman, G. (2024). The use and detection of AI-based tools in higher education. Journal of Instructional Research, 13, 70-80. https://doi.org/10.9743/jir.2024.13.4. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., . . . Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. International journal of surgery, 88, 105906. https://doi.org/10.1016/j.ijsu.2021.105906. · DOI · Paper page
Cited by
Sorted by citations
GİRAY, L., CANER, H. N., & AMOOZEGAR, A. (2026) · Journal of Narrative and Language Studies · doi:10.59045/nalans.2026.109
Self-citation
Metadata
- Authors
- CANER, H. N.
- Affiliations
- Akdeniz University (TR)
- Corresponding author
- CANER, Halime Nuran
- ORCID
- CANER, Halime Nuran
- Venue
- Khazar Journal of Humanities and Social Sciences
- Publisher
- Khazar University Press
- Published
- 2025-12-31
- Volume
- 28
- Issue
- 4
- Pages
- 234-246
- ISSN
- 2223-2613 (ISSN-L)
- 2223-2621
- OpenAlex
- W7125884845
- Cited by
- 2
- References
- 25
- Citations by year
- 2026: 2
- FWCI
- 0.31
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Khazar journal of humanities and social sciences (publishedVersion)
- Khazar University Institutional Repository (Khazar University) (acceptedVersion)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- AI detection tools, generative AI, academic integrity, AI-generated text detection, detection accuracy, non-native English speakers, algorithmic bias, human judgment, AI literacy, process-based assessment, watermarking, student perceptions of AI, educational implications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
