Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.

Detection of GPT-4 Generated Text in Higher Education: Combining Academic Judgement and Software to Identify Generative AI Tool Misuse

Open access
Perkins, Mike, Roe, Jasper, Postma, Darius, McGaughran, James, Hickerson, Don · 2024 · Journal of Academic Ethics

Published version of a preprint

Abstract

This study explores the capability of academic staff assisted by the Turnitin Artificial Intelligence (AI) detection tool to identify the use of AI-generated content in university assessments. 22 different experimental submissions were produced using Open AI’s ChatGPT tool, with prompting techniques used to reduce the likelihood of AI detectors identifying AI-generated content. These submissions were marked by 15 academic staff members alongside genuine student submissions. Although the AI detection tool identified 91% of the experimental submissions as containing AI-generated content, only 54.8% of the content was identified as AI-generated, underscoring the challenges of detecting AI content when advanced prompting techniques are used. When academic staff members marked the experimental submissions, only 54.5% were reported to the academic misconduct process, emphasising the need for greater awareness of how the results of AI detectors may be interpreted. Similar performance in grades was obtained between student submissions and AI-generated content (AI mean grade: 52.3, Student mean grade: 54.4), showing the capabilities of AI tools in producing human-like responses in real-life assessment situations. Recommendations include adjusting the overall strategies for assessing university students in light of the availability of new Generative AI tools. This may include reducing the overall reliance on assessments where AI tools may be used to mimic human writing, or by using AI-inclusive assessments. Comprehensive training must be provided for both academic staff and students so that academic integrity may be preserved.

Overview

Integrity signals

Registry

Crossref · 10.1007/s10805-023-09492-6

Cites

62 of 62 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

1 retracted

Show all 51 references

Research works printed but not linked: 3

  1. Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T. (2023). A watermark for large language models (arXiv:2301.10226). arXiv. http://arxiv.org/abs/2301.10226 · arXiv
  2. Kumar, R., Mindzak, M., Eaton, S. E., & Morrison, R. (2022). AI & AI: Exploring the contemporary intersections of artificial intelligence and academic integrity. Canadian Society for the Study of Higher Education Annual Conference, Online. Werklund School of Education. https://tinyurl.com/ycknz8fd.
  3. OpenAI (2023a). GPT-4 Technical Report (arXiv:2303.08774). arXiv. http://arxiv.org/abs/2303.08774. · arXiv

Cited by

See all 236

Metadata

Authors
Perkins, M., Roe, J., Postma, D., McGaughran, J., & Hickerson, D.
Affiliations
  • British University Vietnam (VN)
  • James Cook University Singapore (SG)
Corresponding author
Perkins, Mike
Venue
Journal of Academic Ethics
Published
2023-10-31
Volume
22
Issue
1
Pages
89-113
ISSN
  • 1570-1727 (ISSN-L)
  • 1572-8544
OpenAlex
W4388108480
Cited by
234
References
51
Citations by year
2026: 94 · 2025: 82 · 2024: 49 · 2023: 9
FWCI
58.60
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Academic Ethics (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
  • ResearchOnline at James Cook University (James Cook University) (submittedVersion)
  • Durham Research Online (Durham University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Education › Academic integrity and plagiarism
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
Keywords
generative AI detection, AI-generated text, academic integrity, academic misconduct, university evaluation, ChatGPT, prompting techniques, higher education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Detection of GPT-4 Generated Text in Higher Education: Combining Academic Judgement and Software to Identify Generative AI Tool Misuse. https://app.cochat.ai/papers/10.1007/s10805-023-09492-6
The paper
Perkins, M., Roe, J., Postma, D., McGaughran, J., & Hickerson, D. (2024). Detection of GPT-4 Generated Text in Higher Education: Combining Academic Judgement and Software to Identify Generative AI Tool Misuse. Journal of Academic Ethics. https://doi.org/10.1007/s10805-023-09492-6

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.