The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text
Open accessAbstract
The proliferation of artificial intelligence (AI)-generated content, particularly from models like ChatGPT, presents potential challenges to academic integrity and raises concerns about plagiarism. This study investigates the capabilities of various AI content detection tools in discerning human and AI-authored content. Fifteen paragraphs each from ChatGPT Models 3.5 and 4 on the topic of cooling towers in the engineering process and five human-witten control responses were generated for evaluation. AI content detection tools developed by OpenAI, Writer, Copyleaks, GPTZero, and CrossPlag were used to evaluate these paragraphs. Findings reveal that the AI detection tools were more accurate in identifying content generated by GPT 3.5 than GPT 4. However, when applied to human-written control responses, the tools exhibited inconsistencies, producing false positives and uncertain classifications. This study underscores the need for further development and refinement of AI content detection tools as AI-generated content becomes more sophisticated and harder to distinguish from human-written text.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“detection accuracy varies across writing styles and languages”
From Designing AI-resilient assessment in higher education: a four-pillar conceptual framework (2026) · cites this paper as Elkhatat et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“vendor accuracy claims routinely exceed observed performance under controlled testing”
From ARE AI DETECTORS FAIR TO EFL LEARNERS? A LIBRARY RESEARCH ON DETECTION BIAS AND ACADEMIC INTEGRITY (2026) · cites this paper as Elkhatat et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“many commercially available tools fail to achieve 80% accuracy despite exceptions among a few premium platforms”
From ARE AI DETECTORS FAIR TO EFL LEARNERS? A LIBRARY RESEARCH ON DETECTION BIAS AND ACADEMIC INTEGRITY (2026) · cites this paper as Elkhatat et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the nature of Generative AI was such that it would be almost impossible to reliably detect AI-generated writing.”
From Generative Artificial Intelligence: Implications and Considerations for Higher Education Practice (2023) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, post hoc detection systems can themselves be computationally expensive to run, and their practical usage is limited by their inconsistent performance16.”
From Scalable watermarking for identifying large language model outputs (2024) · cites this paper as [16]
Experimental — accuracy not yet measured
20 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022) · doi:10.1101/2022.12.23.521610 · Cited by 1,099
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00819-6 · Cited by 1,099
ChatGPT in higher education: considerations for academic integrity and student learning
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.17 · Cited by 853
Perkins, M. (2023) · Journal of University Teaching and Learning Practice · doi:10.53761/1.20.02.07 · Cited by 839 · cited 2×
“Academic misconduct in undergraduate education using ChatGPT has been widely studied (Crawford et al. 2023; King & chatGpt 2023; Lee 2023; Perkins 2023; Sullivan; et al. 2023).”
A Conversation on Artificial Intelligence, Chatbots, and Plagiarism in Higher Education
King, M. R., & chatGPT (2023) · Cellular and Molecular Bioengineering · doi:10.1007/s12195-022-00754-8 · Cited by 602
The rise of ChatGPT : Exploring its potential in medical education
Lee, H. (2024) · Anatomical Sciences Education · doi:10.1002/ase.2270 · Cited by 596 · cited 1×
“Academic misconduct in undergraduate education using ChatGPT has been widely studied (Crawford et al. 2023; King & chatGpt 2023; Lee 2023; Perkins 2023; Sullivan; et al. 2023).”
MethodCrawford, J., Cowling, M., & Allen, K.-A. (2023) · Journal of University Teaching and Learning Practice · doi:10.53761/1.20.3.02 · Cited by 547
Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education
Qadir, J. (2022) · doi:10.36227/techrxiv.21789434.v1 · Cited by 264
Academic Plagiarism Detection: A Systematic Literature Review
Foltýnek, T., Meuschke, N., & Gipp, B. (2020) · ACM Computing Surveys · doi:10.1145/3345317 · Cited by 238
Nhu, V.-H., Mohammadi, A., Shahabi, H., Ahmad, B. B., Al-Ansari, N., Shirzadi, A., Clague, J. J., Jaafari, A., Chen, W., & Nguyen, H. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17144933 · Cited by 162
Research works printed but not linked: 6
- Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877–1901
- Fishman T (2009) “We know it when we see it” is not good enough: toward a standard definition of plagiarism that transcends theft, fraud, and copyright 4th Asia Pacific Conference on Educational Integrity, University of Wollongong NSW Australia
- Foltýnek T, Meuschke N, Gipp B (2020) Academic Plagiarism Detection. ACM Comput Surv 52(6):1–42. https://doi.org/10.1145/3345317 · DOI · Paper page
- Frye BL (2022) Should Using an AI Text Generator to Produce Academic Writing Be Plagiarism? Fordham Intellectual Property, Media & Entertainment Law Journal. https://ssrn.com/abstract=4292283
- Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I (2019) Language models are unsupervised multitask learners. OpenAI Blog 1(8):9
- Radford A, Narasimhan K, Salimans T, Sutskever I (2018) Improving language understanding by generative pre-training
Cited by
Testing of detection tools for AI-generated text
Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023) · International Journal for Educational Integrity · doi:10.1007/s40979-023-00146-z · Cited by 582
“In the tests conducted by Compilatio, the detection tools for AI-generated text detected human-written text with reliability in the range of 78–98% and AI-generated text in the range of 56–88%.”
MethodGenerative Artificial Intelligence: Implications and Considerations for Higher Education Practice
Farrelly, T., & Baker, N. (2023) · Education Sciences · doi:10.3390/educsci13111109 · Cited by 414
“Research on the efficacy of a wide range of GAI detection tools is now beginning to become available (although most are still in pre-print or as yet not peer-reviewed at the time of writing, e.g., [13–18]).”
BackgroundGenerative AI in Higher Education: Balancing Innovation and Integrity
Francis, N. J., Jones, S., & Smith, D. P. (2025) · British Journal of Biomedical Science · doi:10.3389/bjbs.2024.14048 · Cited by 295
“Such concerns regarding academic integrity in the context of GenAI are supported by more recent data highlighting challenges in distinguishing between student-generated and AI-generated work [11].”
Wang, H., Dang, A., Wu, Z., & Mac, S. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100326 · Cited by 272
“Last but not least, considering the current GenAI detection tools are not reliable and supported to determine plagia-rism by most universities (Elkhatat et al., 2023; Weber-Wulff et al., 2023), teachers should adopt multifaceted evaluation strategies to assess students’ work.”
BackgroundPerkins, M., Furze, L., Roe, J., & MacVaugh, J. (2024) · Journal of University Teaching and Learning Practice · doi:10.53761/q3azde36 · Cited by 291
Metadata
- Authors
- Elkhatat, A. M., Elsaid, K., & Almeer, S.
- Affiliations
- Qatar University (QA)
- Texas A&M University at Qatar (QA)
- Corresponding author
- Elkhatat, Ahmed M.
- Venue
- International Journal for Educational Integrity
- Published
- 2023-09-01
- Volume
- 19
- Issue
- 1
- ISSN
- 1833-2595 (ISSN-L)
- OpenAlex
- W4386317922
- Cited by
- 392
- References
- 17
- Citations by year
- 2026: 130 · 2025: 132 · 2024: 119 · 2023: 10
- FWCI
- 98.09
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- International Journal for Educational Integrity (publishedVersion, cc-by)
- Qatar University QSpace (Qatar University) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Qatar National Library
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- AI-generated text detection, ChatGPT, GPT-3.5, GPT-4, academic integrity, plagiarism, human vs AI text classification, false positives, GPTZero, cooling towers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text. https://app.cochat.ai/papers/10.1007/s40979-023-00140-5
- The paper
- Elkhatat, A. M., Elsaid, K., & Almeer, S. (2023). Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text. International Journal for Educational Integrity. https://doi.org/10.1007/s40979-023-00140-5
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.