Evaluation of AI content generation tools for verification of academic integrity in higher education
The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
This research evaluates the capabilities of some highly mentioned AI detection tools to separate reality from their hyperbolic claims. For this purpose, eight AI engines have been tested on four different types of data, which cover the different ways of using ChatGPT. These types are Original, Paraphrased by AI, 100% AI generated and 100% AI generated with Contextual Information. The AI index recorded by these tools against the datasets was evaluated as an indicator of their performance. Findings: The resulting figures of cumulative mean validate that these tools excel at identifying human generated content (1.71% AI content) and perform reasonably well in labelling AI generated content (76.85% AI content). However, they are perplexed by the scenarios where the content is either paraphrased by the AI (39.42% AI content) or generated by giving a precise context for the output (60.1% AI content).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., et al. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102642 · Cited by 4,584
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519
What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024
A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development
Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q.-L., & Tang, Y. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123618 · Cited by 1,452
Haleem, A., Javaid, M., & Singh, R. P. (2022) · BenchCouncil Transactions on Benchmarks, Standards and Evaluations · doi:10.1016/j.tbench.2023.100089 · Cited by 694
Abstracts written by ChatGPT fool scientists
Else, H. (2023) · Nature · doi:10.1038/d41586-023-00056-7 · Cited by 611
Evaluating the Performance of ChatGPT in Ophthalmology
Antaki, F., Touma, S., Milad, D., El-Khoury, J., & Duval, R. (2023) · Ophthalmology Science · doi:10.1016/j.xops.2023.100324 · Cited by 563
Exploring Students’ Perceptions of ChatGPT: Thematic Analysis and Follow-Up Survey
Shoufan, A. (2023) · IEEE Access · doi:10.1109/access.2023.3268224 · Cited by 499
Cited by
Kangwa, D., Msafiri, M. M., & Fute, A. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70109 · Cited by 22
Aluko, H. A., Aluko, A., Offiah, G. A., Ogunjimi, F., Aluko, A. O., Alalade, F. M., Ukeje, I. O., & Nwani, C. H. (2025) · International journal of organizational analysis · doi:10.1108/ijoa-07-2024-4632 · Cited by 41
Embrace, Don’t Avoid: Reimagining Higher Education with Generative Artificial Intelligence
Noviandy, T. R., Maulana, A., Idroes, G. M., Zahriah, Z., Paristiowati, M., Emran, T. B., Ilyas, M., & Idroes, R. (2024) · Journal of Educational Management and Learning · doi:10.60084/jeml.v2i2.233 · Cited by 33
Bora, B. Y., & Kölemen, C. Ş. (2025) · Contemporary Educational Technology · doi:10.30935/cedtech/16433 · Cited by 28
Kabanda, M. (2025) · EIKI Journal of Effective Teaching Methods · doi:10.59652/jetm.v3i1.404 · Cited by 28
“Thus, AI detection tools are continually updated to address these challenges, providing a dynamic solution to an ever-changing problem (Saqib & Zia, 2024; Weber-Wulff et al., 2023).”
Metadata
- Authors
- Saqib, M. B., & Zia, S.
- Affiliations
- University of Lahore (PK)
- Corresponding author
- Saqib, Muhammad Bilal
- Venue
- Journal of Applied Research in Higher Education
- Publisher
- Emerald
- Published
- 2024-07-12
- Volume
- 17
- Issue
- 4
- Pages
- 1430-1440
- ISSN
- 1758-1184 (ISSN-L)
- 2050-7003
- OpenAlex
- W4400571644
- Semantic Scholar
- 3789596cfc9d4ba168c4c4b213988aa6c3da70db
- Cited by
- 18
- References
- 29
- Citations by year
- 2026: 10 · 2025: 7 · 2024: 1
- FWCI
- 7.46
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.emerald.com/insight/site-policies (tdm)
- Locations
- Journal of Applied Research in Higher Education (publishedVersion)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- AI-generated text detection, generative AI, academic integrity, ChatGPT, AI detection tools, originality verification, higher education, research integrity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluation of AI content generation tools for verification of academic integrity in higher education. https://app.cochat.ai/papers/10.1108/jarhe-10-2023-0470
- The paper
- Saqib, M. B., & Zia, S. (2025). Evaluation of AI content generation tools for verification of academic integrity in higher education. Journal of Applied Research in Higher Education. https://doi.org/10.1108/jarhe-10-2023-0470
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.