Using AI-based detectors to control AI-assisted plagiarism in ESL writing: “The Terminator Versus the Machines”
Open accessThe DOI matches its Crossref record; 48 of the 49 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
The release of ChatGPT marked the beginning of a new era of AI-assisted plagiarism that disrupts traditional assessment practices in ESL composition. In the face of this challenge, educators are left with little guidance in controlling AI-assisted plagiarism, especially when conventional methods fail to detect AI-generated texts. One approach to managing AI-assisted plagiarism is using fine-tuned AI classifiers, such as RoBERTa, to identify machine-generated texts; however, the reliability of this approach is yet to be established. To address the challenge of AI-assisted plagiarism in ESL contexts, the present cross-disciplinary descriptive study examined the potential of two RoBERTa-based classifiers to control AI-assisted plagiarism on a dataset of 240 human-written and ChatGPT-generated essays. Data analysis revealed that both platforms could identify AI-generated texts, but their detection accuracy was inconsistent across the dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures














Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This bias is not incidental but structurally embedded in the design and training of these systems (Caner, 2025; Ibrahim, 2023; Jiang et al., 2024; Liang et al., 2023).”
From Harms and Ethical Risks of AI Detection Tools in Education: False Positives, Bias, Surveillance, and Student Rights (2026) · cites this paper as Ibrahim, 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These flaws weaken detection tools' credibility and call into question their admissibility as evidence in academic conduct proceedings [11].”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [11]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While AI detectors offer scalable solutions, they frequently produce false positives and lack transparency, especially for multilingual or non-native English speakers.”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [11]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Non-native English students are most at risk for such mistakes because of the similarities in syntax and structure with AI-written language.”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [11]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Inconsistencies were partly a result of the fact that both detectors were trained on GPT-2 output and were hence less adept at identifying content created by more advanced models like GPT-3.5 or GPT-4 [11].”
From Evaluating the Effectiveness and Ethical Implications of AI Detection Tools in Higher Education (2025) · cites this paper as [11]
Experimental — accuracy not yet measured
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Chatting about ChatGPT: how may AI and GPT impact academia and libraries?
Lund, B. D., & Wang, T. (2023) · Library Hi Tech News · doi:10.1108/lhtn-01-2023-0009 · Cited by 1,355
Qualitative and descriptive research: Data type versus data analysis
Nassaji, H. (2015) · Language Teaching Research · doi:10.1177/1362168815572747 · Cited by 1,260
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00819-6 · Cited by 1,099
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022) · doi:10.1101/2022.12.23.521610 · Cited by 1,099 · cited 5×
“These results are consistent with Gao et al. (2023) findings about the effectiveness of GPT-2 Output Detector Demo in detecting ChatGPT-generated abstracts and OpenAI’s in-house experiments on using GPT-2-based detector to detect its generation (OpenAI, n.d.-b).”
BackgroundMethodResultInfluentialChatting about ChatGPT: How May AI and GPT Impact Academia and Libraries?Preprint
Lund, B. D., & Wang, T. (2023) · Social Science Research Network · doi:10.2139/ssrn.4333415 · Cited by 1,093
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · cited 2×
“OpenAI used Common Crawl to train the model on a dataset of a trillion words of internet texts (175 billion parameters) in supervised training (Brown, 2020) and then fine-tuned the model using reinforcement learning from human feedback approaches (Christiano et al., 2017) to improve output alignment and consistency wi…”
MethodPavlik, J. V. (2023) · Journalism & Mass Communication Educator · doi:10.1177/10776958221149577 · Cited by 972 · cited 1×
“1 As a generative language model (Pavlik, 2023), GPT is a machine-learning neural network trained to analyze and interpret natural language texts and model the linguistic system of a language (MacNeil et al., 2022).”
BackgroundDeng, X., Liu, Q., Deng, Y., & Mahadevan, S. (2016) · Information Sciences · doi:10.1016/j.ins.2016.01.033 · Cited by 615
A Conversation on Artificial Intelligence, Chatbots, and Plagiarism in Higher Education
King, M. R., & chatGPT (2023) · Cellular and Molecular Bioengineering · doi:10.1007/s12195-022-00754-8 · Cited by 602
Defending Against Neural Fake News
Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1905.12616 · Cited by 569
GLTR: Statistical Detection and Visualization of Generated Text
Gehrmann, S., Strobelt, H., & Rush, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/p19-3019 · Cited by 470 · cited 1×
“An example of a zero-shot classifier is Gehrmann et al. (2019)’s Giant Language Model Test Room (GLTR), which utilizes BERT (Devlin et al., 2018) and GPT-2 (Radford et al., 2019) models to discriminate between machine-generated and human-written texts based on the probability of word distribution…”
MethodGLTR: Statistical Detection and Visualization of Generated Text
Gehrmann, S., Strobelt, H., & Rush, A. M. (2019) · arXiv · doi:10.48550/arxiv.1906.04043 · Cited by 460
Transfer Learning in Collaborative Filtering for Sparsity Reduction
Pan, W., Xiang, E., Liu, N., & Yang, Q. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7578 · Cited by 375
ChatGPT: The End of Online Exam Integrity?
Susnjak, T. (2022) · arXiv · doi:10.48550/arxiv.2212.09292 · Cited by 370 · cited 4×
“Specifically, with the ability of ChatGPT to generate coherent texts (Cotton et al., 2023) and respond to critical questions (Susnjak, 2022), there is a growing fear that it could be widely utilized by college students to complete essay assignments (Francke & Benett, 2019; King & ChatGPT, 2023;…”
BackgroundInfluentialAutomatic Detection of Machine Generated Text: A Critical Survey
Jawahar, G., Abdul-Mageed, M., & Lakshmanan, V. S. L. (2020) · International Conference on Computational Linguistics · doi:10.48550/arxiv.2011.01314 · Cited by 327
Release Strategies and the Social Impacts of Language Models
Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J. W., Kreps, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1908.09203 · Cited by 292 · cited 19×
“Simple classifiers are basic machine-learning algorithms trained from scratch to discriminate between human-written and machine-generated texts (Solaiman et al., 2019).”
BackgroundMethodResultInfluentialAutomatic Detection of Generated Text is Easiest when Humans are Fooled
Ippolito, D., Duckworth, D., Callison-Burch, C., & Eck, D. (2019) · arXiv · doi:10.48550/arxiv.1911.00650 · Cited by 264 · cited 3×
“…2022), especially since human raters have difficulty distinguishing between AI-generated and human-written texts (Brown et al., 2020; Hovy, 2016; Ippolito et al, 2019; Solaiman et al., 2019) and traditional plagiarism detection software fail to detect AI-generated texts (Khalil & Er, 2023).”
BackgroundAutomatic Detection of Generated Text is Easiest when Humans are Fooled
Ippolito, D., Duckworth, D., Callison-Burch, C., & Eck, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.164 · Cited by 264 · cited 3×
“…2022), especially since human raters have difficulty distinguishing between AI-generated and human-written texts (Brown et al., 2020; Hovy, 2016; Ippolito et al, 2019; Solaiman et al., 2019) and traditional plagiarism detection software fail to detect AI-generated texts (Khalil & Er, 2023).”
BackgroundPlagiarism in second-language writing
Pecorari, D., & Petrić, B. (2014) · Language Teaching · doi:10.1017/s0261444814000056 · Cited by 211
Lee, C., Panda, P., Srinivasan, G., & Roy, K. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00435 · Cited by 205
Haque, M. U., Dharmadasa, I., Sworna, Z. T., Rajapakse, R. N., & Ahmad, H. (2022) · arXiv · doi:10.48550/arxiv.2212.05856 · Cited by 203
Generating Diverse Code Explanations using the GPT-3 Large Language Model
MacNeil, S., Tran, A., Mogil, D., Bernstein, S., Ross, E., & Huang, Z. (2022) · Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 2 · doi:10.1145/3501709.3544280 · Cited by 200
The death of the short-form physics essay in the coming AI revolution
Yeadon, W., Inyang, O.-O., Mizouri, A., Peach, A., & Testrow, C. P. (2023) · Physics Education · doi:10.1088/1361-6552/acc5cf · Cited by 199 · cited 4×
“…agent by performing a variety of NLP tasks, including translating texts, generating texts, summarizing content, gathering information, telling stories, composing music, or developing computer code (Gao et al., 2023; Khail & Er, 2023; Cotton et al., 2023; Lund & Wang, 2023; Yeadon et al., 2022).”
BackgroundInfluentialThe Death of the Short-Form Physics Essay in the Coming AI Revolution
Yeadon, W., Inyang, O.-O., Mizouri, A., Peach, A., & Testrow, C. (2022) · arXiv · doi:10.48550/arxiv.2212.11661 · Cited by 199
GPT Takes the Bar ExamPreprint
Bommarito, M., & Katz, D. (2022) · Social Science Research Network · doi:10.48550/arxiv.2212.14402 · Cited by 195
A Watermark for Large Language Models
Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T. (2023) · arXiv · doi:10.48550/arxiv.2301.10226 · Cited by 194 · cited 2×
“…NLP has explored different approaches to detecting AI-generated texts (e.g., Bakhtin et al., 2019), including watermarking AI-generated texts (e.g., Kirchenbauer et al., 2023), detecting traces of language modeling in generated texts (e.g., Tay et al., 2020), and training AI models to…”
BackgroundMethodTweepFake: about detecting deepfake tweets
Fagni, T., Falchi, F., Gambini, M., Martella, A., & Tesconi, M. (2021) · PLOS ONE · doi:10.1371/journal.pone.0251415 · Cited by 185 · cited 2×
“…classifiers based on RoBERTA model (Liu et al., 2019), a fine-tuned AI detector trained on the output of existing large language models (e.g., GPT-2), has demonstrated higher detection accuracy compared to other classifiers (Solaiman et al., 2019; Jawahar et al., 2020; Fagni et al., 2021).”
MethodBommarito, M. J., & Katz, D. M. (2022) · doi:10.2139/ssrn.4314839 · Cited by 181
The state of critical thinking today
Paul, R. (2005) · New Directions for Community Colleges · doi:10.1002/cc.193 · Cited by 179
Automatic Detection of Machine Generated Text: A Critical Survey
Jawahar, G., Abdul-Mageed, M., & Lakshmanan, V. S. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.208 · Cited by 156
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., & Finn, C. (2023) · arXiv · doi:10.48550/arxiv.2301.11305 · Cited by 151
Why do students cheat? Perceptions, evaluations, and motivations
Waltzer, T., & Dahl, A. (2023) · Ethics & Behavior · doi:10.1080/10508422.2022.2026775 · Cited by 87
Real or Fake? Learning to Discriminate Machine from Human Generated Text
Bakhtin, A., Gross, S., Ott, M., Deng, Y., Ranzato, M., & Szlam, A. (2019) · arXiv · doi:10.48550/arxiv.1906.03351 · Cited by 77
Will ChatGPT get you caught? Rethinking of Plagiarism Detection
Khalil, M., & Er, E. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.04335 · Cited by 49
Hovy, D. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p16-2057 · Cited by 44
Essential Statistics for Applied Linguistics
Lowie, W. M., & Seton, B. J. (2013) · doi:10.1007/978-1-137-28490-7 · Cited by 41
Plagiarism: To What Extent it is Understood?
Ali, W. Z. W., Ismail, H., & Cheat, T. T. (2012) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2012.09.320 · Cited by 39
Reverse Engineering Configurations of Neural Text Generation ModelsPreprint
Tay, Y., Bahri, D., Zheng, C., Brunk, C., Metzler, D., & Tomkins, A. (2020) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2004.06201 · Cited by 39
How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks
Chen, X., Ye, J., Zu, C., Xu, N., Zheng, R., Peng, M., Zhou, J., Gui, T., Zhang, Q., & Huang, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.00293 · Cited by 35
A Overview of Descriptive Research
Kramer, R. F. (1985) · Journal of Pediatric Oncology Nursing · doi:10.1177/104345428500200208 · Cited by 20
Reverse Engineering Configurations of Neural Text Generation Models
Tay, Y., Bahri, D., Zheng, C., Brunk, C., Metzler, D., & Tomkins, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.25 · Cited by 19
Research works printed but not linked: 3
- Baker, C. (2017). Quantitative research designs: Experimental, quasi-experimental, and descriptive. In H. Hall and L. Roussel (Eds.), Evidence-based practice: An integrative approach to research, administration, and practice, (2nd Ed., pp.155–183). Jones & Bartlett Learning.
- Francke, E., & Bennett, A. (2019). The Potential Influence of Artificial Intelligence on Plagiarism: A Higher Education Perspective. In P. Griffiths and M. N. Kabir (Eds.), European Conference on the Impact of Artificial Intelligence and Robotics (ECIAIR 2019) (pp. 131–140). Academic Conferences and Publishing Limited. DOI: https://doi.org/10.34190/ECIAIR.19.043 · doi:10.34190/eciair.19.043
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1(8), 9.
Cited by
Lo, C. K., Yu, P. L. H., Xu, S., Ng, D. T. K., & Jong, M. S.-Y. (2024) · Smart Learning Environments · doi:10.1186/s40561-024-00342-5 · Cited by 138
Li, B., Tan, Y. L., Wang, C., & Lowell, V. L. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100445 · Cited by 287
(2024) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2024.7.1.14 · Cited by 55
Nelson, A. S., Santamaría, P. V., Javens, J. S., & Ricaurte, M. (2025) · Education Sciences · doi:10.3390/educsci15050611 · Cited by 79
Giray, L., Sevnarayan, K., & Ranjbaran, F. (2025) · Internet Reference Services Quarterly · doi:10.1080/10875301.2024.2437174 · Cited by 78
Metadata
- Authors
- Ibrahim, K.
- Affiliations
- Gulf University for Science & Technology (KW)
- Corresponding author
- Ibrahim, Karim
- ORCID
- Ibrahim, Karim
- Venue
- Language Testing in Asia
- Published
- 2023-10-16
- Volume
- 13
- Issue
- 1
- ISSN
- 2229-0443 (ISSN-L)
- OpenAlex
- W4387671615
- Cited by
- 66
- References
- 40
- Citations by year
- 2026: 14 · 2025: 28 · 2024: 22 · 2023: 2
- FWCI
- 16.55
- Type
- journal article
- Language
- en
- Update notices
- correction (2023-10-31)
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Language Testing in Asia (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- NA
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- ChatGPT, ESL writing, writing assessment, AI-generated text detection, RoBERTa
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.