Accuracy pecking order – How 30 AI detectors stack up in detecting generative artificial intelligence content in university English L1 and English L2 student essays
Open accessAbstract
This study set out to evaluate the accuracy of 30 AI detectors in identifying generative artificial intelligence (GenAI)-generated and human-written content in university English L1 and English L2 student essays. 40 student essays were divided into four essay sets of English L1 and English L2 and two undergraduate modules: a second-year module and a third-year module. There are ten essays in each essay set. The 30 AI detectors comprised freely available detectors and non-premium versions of online AI detectors. Employing a critical studies approach to artificial intelligence, the study had three research questions. It focused on and calculated the accuracy, false positive rates (FPRs), and true negative rates (TNRs) of all 30 AI detectors for all essays in each of the four sets to determine the accuracy of each AI detector to identify the GenAI content of each essay. It also used confusion matrices to determine the specificity of best- and worst-performing AI detectors. Some of the results of this study are worth mentioning. Firstly, only two AI detectors, Copyleaks and Undetectable AI, managed to correctly detect all of the essay sets of the two English language categories (English L1 and English L2) as human written. As a result, these two AI detectors jointly shared the first spot in terms of the GenAI detection accuracy ranking. Secondly, nine of the 30 AI detectors completely misidentified all the essays in each of the four essay sets of the two language categories in both modules. Thus, they collectively shared the last spot. Thirdly, the remaining 19 AI detectors both correctly and incorrectly classified the four essay sets in varying degrees without any bias to any essay set of the two English language categories. Fourthly, none of the 30 AI detectors tended to have a bias toward a specific English language category in classifying the four essay sets. Lastly, the results of the current study suggest that the bulk of the currently available AI detectors, especially the currently available free-to-use AI detectors, are not fit for purpose.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Detection accuracy differs significantly by language background, with L2 writers consistently facing higher false-positive rates.
“Chaka (2024), evaluating 30 AI detectors across English L1 and English L2 student essays, finds that detection accuracy differs significantly by language background, with L2 writers consistently facing higher false-positive rates.”
From Harms and Ethical Risks of AI Detection Tools in Education: False Positives, Bias, Surveillance, and Student Rights (2026) · cites this paper as Chaka (2024)
The section on differential classification says no language category received more false positives and that differences in scores were neither substantial nor consistently skewed toward L1 or L2. The Conclusion likewise reports no language-category bias in classification or false positives, contrary to the claim that L2 writers consistently faced higher false-positive rates. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Yaşlı Kadınlarda Üreme Sağlığı
Kokanalı, D., & Engin Üstün, Y. (n.d.) · DergiPark (Istanbul University) · Cited by 170,090 · see citing works
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.9 · Cited by 1,806 · see citing works
Data colonialism: rethinking big data’s relation to the contemporary subject
Couldry, N., & Mejias, U. A. (2019) · Television & New Media · doi:10.1177/1527476418796632 · Cited by 1,617 · see citing works
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00819-6 · Cited by 1,099 · see citing works
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
Mohamed, S., Png, M.-T., & Isaac, W. (2020) · Philosophy & Technology · doi:10.1007/s13347-020-00405-8 · Cited by 912 · see citing works
ChatGPT in higher education: considerations for academic integrity and student learning
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.17 · Cited by 853 · see citing works
Perkins, M. (2023) · Journal of University Teaching and Learning Practice · doi:10.53761/1.20.02.07 · Cited by 839 · see citing works
Dergaa, I., Chamari, K., Zmijewski, P., & Ben Saad, H. (2023) · Biology of Sport · doi:10.5114/biolsport.2023.125623 · Cited by 796 · see citing works
An investigation of the false discovery rate and the misinterpretation of <i>p</i> -values
Colquhoun, D. (2014) · Royal Society Open Science · doi:10.1098/rsos.140216 · Cited by 792 · see citing works
GPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · Patterns · doi:10.1016/j.patter.2023.100779 · Cited by 673 · see citing works
Data Epistemologies, The Coloniality of Power, and Resistance
Quijano, P. R. (2019) · Television & New Media · doi:10.1177/1527476419831640 · Cited by 451 · see citing works
Swedberg, R. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108762519.002 · Cited by 408 · see citing works
Elkhatat, A. M., Elsaid, K., & Almeer, S. (2023) · International Journal for Educational Integrity · doi:10.1007/s40979-023-00140-5 · Cited by 392 · see citing works
Online Delivery and Assessment during COVID-19: Safeguarding Academic Integrity
Gamage, K. A. A., de Silva, E. K., & Gunawardhana, N. (2020) · Education Sciences · doi:10.3390/educsci10110301 · Cited by 307 · see citing works
Should ChatGPT be biased? Challenges and risks of bias in large language models
Ferrara, E. (2023) · First Monday · doi:10.5210/fm.v28i11.13346 · Cited by 282 · see citing works
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models
Ferrara, E. (2023) · arXiv · doi:10.48550/arxiv.2304.03738 · Cited by 282
Perkins, M., Roe, J., Postma, D., McGaughran, J., & Hickerson, D. (2024) · Journal of Academic Ethics · doi:10.1007/s10805-023-09492-6 · Cited by 234 · see citing works
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212 · see citing works
AI-generated research paper fabrication and plagiarism in the scientific community
Elali, F. R., & Rachid, L. N. (2023) · Patterns · doi:10.1016/j.patter.2023.100706 · Cited by 197 · see citing works
Grounded Theory: A Guide for Exploratory Studies in Management Research
Makri, C., & Neely, A. D. (2021) · International Journal of Qualitative Methods · doi:10.1177/16094069211013654 · Cited by 193 · see citing works
Can AI-Generated Text be Reliably Detected?
Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., & Feizi, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11156 · Cited by 161 · see citing works
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1195 · Cited by 150 · see citing works
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2023) · arXiv · doi:10.48550/arxiv.2303.13408 · Cited by 150 · see citing works
Detecting AI-generated essays: the ChatGPT challenge
Cingillioglu, I. (2023) · International Journal of Information and Learning Technology · doi:10.1108/ijilt-03-2023-0043 · Cited by 135 · see citing works
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.2.12 · Cited by 132 · see citing works
Self-citationDesaire, H., Chua, A. E., Isom, M., Jarošová, R., & Hua, D. (2023) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2023.101426 · Cited by 122 · see citing works
Walters, W. H. (2023) · Open Information Science · doi:10.1515/opis-2022-0158 · Cited by 109 · see citing works
Alexander, K., Savvidou, C., & Alexander, C. (2023) · Teaching English With Technology · doi:10.56297/buka4060/xhld5365 · Cited by 101 · see citing works
Chaka, C. (2024) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2024.7.1.14 · Cited by 84 · see citing works
Self-citationOdri, G. A., & Yoon, D. J. Y. (2023) · Orthopaedics & Traumatology Surgery & Research · doi:10.1016/j.otsr.2023.103706 · Cited by 76 · see citing works
Bias in Automatic Speech Recognition: The Case of African American Language
Martin, J., & Wright, K. E. (2022) · Applied Linguistics · doi:10.1093/applin/amac066 · Cited by 71 · see citing works
Rethinking online assessment strategies: Authenticity versus AI chatbot intervention
Ifelebuegu, A. O. (2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.2.2 · Cited by 70 · see citing works
Habibzadeh, F. (2023) · Journal of Korean Medical Science · doi:10.3346/jkms.2023.38.e319 · Cited by 68 · see citing works
Ibrahim, K. (2023) · Language Testing in Asia · doi:10.1186/s40468-023-00260-2 · Cited by 66 · see citing works
Bisi, T., Risser, A., Clavert, P., Migаud, H., & Dаrtus, J. (2023) · Orthopaedics & Traumatology Surgery & Research · doi:10.1016/j.otsr.2023.103694 · Cited by 38 · see citing works
Chaka, C. (2023) · International Journal of Learning Teaching and Educational Research · doi:10.26803/ijlter.22.6.1 · Cited by 36 · see citing works
Self-citationAn investigation of the false discovery rate and the misinterpretation of P values
Colquhoun, D. (2014) · UCL Discovery (University College London) · doi:10.48550/arxiv.1407.5296 · Cited by 34 · see citing works
Testing of Detection Tools for AI-Generated Text
Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023) · arXiv · doi:10.48550/arxiv.2306.15666 · Cited by 32 · see citing works
Blau, I., Goldberg, S. C., Friedman, A., & Eshet‐Alkalai, Y. (2020) · Journal of Computing in Higher Education · doi:10.1007/s12528-020-09260-0 · Cited by 32 · see citing works
Scholarly Communication and Machine-Generated Text: Is it Finally AI vs AI in Plagiarism Detection?
Santra, P. P., & Majhi, D. (2023) · Journal of Information and Knowledge · doi:10.17821/srels/2023/v60i3/171028 · Cited by 29 · see citing works
Homolak, J. (2023) · Croatian Medical Journal · doi:10.3325/cmj.2023.64.205 · Cited by 24 · see citing works
Sobaih, A. E. (2024) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2024.7.1.21 · Cited by 23 · see citing works
AI-Generated Content Detectors: Boon or Bane for Scientific Writing
Ladha, N., Yadav, K., & Rathore, P. (2023) · Indian Journal of Science and Technology · doi:10.17485/ijst/v16i39.1632 · Cited by 20 · see citing works
Anil, A., Saravanan, A., Singh, S., Shamim, M. A., Tiwari, K., Lal, H., Seshatri, S., Gomaz, S. B., Karat, T. P., Dwivedi, P., et al. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e19194 · Cited by 18 · see citing works
Chaka, C. (2022) · DOAJ (DOAJ: Directory of Open Access Journals) · doi:10.20368/1971-8829/1135678 · Cited by 18 · see citing works
Self-citationIntroducing critical studies of artificial intelligence
Lindgren, S. (2023) · Edward Elgar Publishing eBooks · doi:10.4337/9781803928562.00005 · Cited by 17 · see citing works
ChatGPT and scientific papers in veterinary neurology; is the genie out of the bottle?
Abani, S., Volk, H. A., Decker, S. D., Fenn, J., Rusbridge, C., Charalambous, M., Gonçalves, R., Gutierrez‐Quintana, R., Loderstedt, S., Flegel, T., et al. (2023) · Frontiers in Veterinary Science · doi:10.3389/fvets.2023.1272755 · Cited by 15 · see citing works
Non-English academics face inequality via AI-generated essays and countermeasure tools
Wee, H. B., & Reimer, J. D. (2023) · BioScience · doi:10.1093/biosci/biad034 · Cited by 8 · see citing works
Evade ChatGPT Detectors via A Single Space
Cai, S., & Cui, W. (2023) · arXiv · doi:10.48550/arxiv.2307.02599 · Cited by 7 · see citing works
Identifying Text Classification Failures in Multilingual AI-Generated Content
Subramaniam, R. (2023) · International Journal of Artificial Intelligence & Applications · doi:10.5121/ijaia.2023.14505 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Lee, D. (2023). How hard can it be? Testing the reliability of AI detection tools. https://www.researchgate.net/ profile/Daniel-Lee-95/publication/374170650_How_hard_ can_it_be_Testing_the_reliability_of_AI_detection_tools/ links/6512b65237d0df2448edc358/How-hard-can-it-be- Testing-the-reliability-of-AI-detection-tools.pdf
- Chaka Chaka. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
- Uzun, L. (2023). ChatGPT and academic integrity concerns: Detecting artificial intelligence generated content. Language Education & Technology, 3(1), 45-54. https:// www.researchgate.net/publication/370299956_ChatGPT_ and_Academic_Integrity_Concerns_Detecting_Artificial_ Intelligence_Generated_Content
Cited by
Sorted by citations
The AI Assessment Scale (AIAS) in action: A pilot implementation of GenAI supported assessment
Furze, L., Perkins, M., Roe, J., & MacVaugh, J. (2024) · Australasian Journal of Educational Technology · doi:10.14742/ajet.9434 · Cited by 92 · see citing works
Giray, L., Sevnarayan, K., & Ranjbaran, F. (2025) · Internet Reference Services Quarterly · doi:10.1080/10875301.2024.2437174 · Cited by 62 · see citing works
Hua, H., & Yao, C.-J. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1469197 · Cited by 27 · see citing works
Student self-reflection as a tool for managing GenAI use in large class assessment
Combrinck, C., & Loubser, N. (2025) · Discover Education · doi:10.1007/s44217-025-00461-2 · Cited by 26 · see citing works
The EAP‐AIAS: Adapting the AI Assessment Scale for English for Academic Purposes
Roe, J., Perkins, M., & Tregubova, Y. (2026) · TESOL Journal · doi:10.1002/tesj.70122 · Cited by 33 · see citing works
AI detecting AI in academic writing: Why most AI detector findings are false
Tsigaris, P., & Teixeira da Silva, J. A. (2026) · Next Research · doi:10.1016/j.nexres.2026.101396 · Cited by 11 · see citing works
Tyler, D. H. F., George, S. S., Gagnon, A., & Howard, J. (2025) · Journal of Criminal Justice Education · doi:10.1080/10511253.2025.2506435 · Cited by 5 · see citing works
(2025) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2025.8.1.20 · Cited by 7 · see citing works
The Misclassification of Autistic Writing as AI-Generated
Chambers, S., & Kelley, M. C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98420-4_7 · Cited by 3 · see citing works
Shamsi, A., Wang, T., Amraei, M., & Raju, N. V. (2025) · Acta Informatica Pragensia · doi:10.18267/j.aip.293 · Cited by 4 · see citing works
Chaka, C. (2024) · International Journal of Learning Teaching and Educational Research · doi:10.26803/ijlter.23.12.9 · Cited by 4 · see citing works
Self-citationHey, ChatGPT: How should we teach law to Generation AI?
Divino, S. B. S. (2024) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2024.7.2.6 · Cited by 1 · see citing works
The Reliability of AI Text Detectors on Filipino Student Essays
Chan, A. L., Chua, R. C., Rio, D. D., Lee, B. C., Ong, R. E., & Tiam-Lee, T. J. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-05070-0_5 · Cited by 1 · see citing works
Kaşarcı, İ. (2026) · International Journal of Current Educational Studies · doi:10.46328/ijces.315 · Cited by 1 · see citing works
GİRAY, L., CANER, H. N., & AMOOZEGAR, A. (2026) · Journal of Narrative and Language Studies · doi:10.59045/nalans.2026.109
Revisión bibliométrica sobre la adopción de inteligencia artificial en la educación superior
Jiménez, M. U., López-Mallama, O. M., & Espinosa, A. A. (2025) · Revista Guillermo de Ockham · doi:10.21500/22563202.7780
Dynamic Monitoring Model of English Classroom Attention Driven by Generative Artificial Intelligence
Chen, P., & Wang, T. (2026) · International Journal of Information Technologies and Systems Approach · doi:10.4018/ijitsa.400907
Atamhenwan, L. (2026) · Education and Information Technologies · doi:10.1007/s10639-026-14049-2 · Cited by 3 · see citing works
Angelier, V. (2026) · AI and Ethics · doi:10.1007/s43681-026-01376-w
- Authors
- Chaka, C.
- Affiliations
- University of South Africa (ZA)
- Corresponding author
- Chaka Chaka
- ORCID
- Chaka Chaka
- Venue
- Journal of Applied Learning & Teaching
- Publisher
- Kaplan Business School, Australia
- Published
- 2024-04-10
- Volume
- 7
- Issue
- 1
- ISSN
- 2591-801X (ISSN-L)
- OpenAlex
- W4394707607
- Cited by
- 16
- References
- 46
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 3
- FWCI
- 6.66
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Journal of Applied Learning & Teaching (publishedVersion, cc-by)
- Topics
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- AI detection tools, generative AI, academic integrity, student essays, English as a second language, English as a first language, detection accuracy, false positive rate, test specificity, university students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
