CoChat

Evaluation of AI content generation tools for verification of academic integrity in higher education

Saqib, Muhammad Bilal, Zia, Saba · 2025 · Journal of Applied Research in Higher Education

The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.

Abstract

This research evaluates the capabilities of some highly mentioned AI detection tools to separate reality from their hyperbolic claims. For this purpose, eight AI engines have been tested on four different types of data, which cover the different ways of using ChatGPT. These types are Original, Paraphrased by AI, 100% AI generated and 100% AI generated with Contextual Information. The AI index recorded by these tools against the datasets was evaluated as an indicator of their performance. Findings: The resulting figures of cumulative mean validate that these tools excel at identifying human generated content (1.71% AI content) and perform reasonably well in labelling AI generated content (76.85% AI content). However, they are perplexed by the scenarios where the content is either paraphrased by the AI (39.42% AI content) or generated by giving a precise context for the output (60.1% AI content).

Overview

Integrity signals

Registry

Crossref · 10.1108/jarhe-10-2023-0470

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 30 references

Cited by

See all 18

Metadata

Authors
Saqib, M. B., & Zia, S.
Affiliations
University of Lahore (PK)
Corresponding author
Saqib, Muhammad Bilal
Venue
Journal of Applied Research in Higher Education
Publisher
Emerald
Published
2024-07-12
Volume
17
Issue
4
Pages
1430-1440
ISSN
  • 1758-1184 (ISSN-L)
  • 2050-7003
OpenAlex
W4400571644
Cited by
18
References
29
Citations by year
2026: 10 · 2025: 7 · 2024: 1
FWCI
7.46
Type
journal article
Language
en
Open access
closed
Licence
www.emerald.com/insight/site-policies (tdm)
Locations
Journal of Applied Research in Higher Education (publishedVersion)
Topics
  • Social Sciences › Education › Academic integrity and plagiarism
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
Keywords
AI-generated text detection, generative AI, academic integrity, ChatGPT, AI detection tools, originality verification, higher education, research integrity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Evaluation of AI content generation tools for verification of academic integrity in higher education. https://app.cochat.ai/papers/10.1108/jarhe-10-2023-0470
The paper
Saqib, M. B., & Zia, S. (2025). Evaluation of AI content generation tools for verification of academic integrity in higher education. Journal of Applied Research in Higher Education. https://doi.org/10.1108/jarhe-10-2023-0470

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.