CoChat

Simple techniques to bypass GenAI text detectors: implications for inclusive education

Open access
Perkins, Mike, Roe, Jasper, Vu, Binh H., Postma, Darius, Hickerson, Don, McGaughran, James, Khuat, Huy Q. · 2024 · International Journal of Educational Technology in Higher Education

Published version of a preprint

The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index; 13 of the 40 sources its claims cite were checked for support.

Abstract

This study investigates the efficacy of six major Generative AI (GenAI) text detectors when confronted with machine-generated content modified to evade detection (n = 805). We compare these detectors to assess their reliability in identifying AI-generated text in educational settings, where they are increasingly used to address academic integrity concerns. Results show significant reductions in detector accuracy (17.4%) when faced with simple techniques to manipulate the AI generated content. The varying performances of GenAI tools and detectors indicate they cannot currently be recommended for determining academic integrity violations due to accuracy limitations and the potential for false accusation which undermines inclusive and fair assessment practices. However, these tools may support learning and academic integrity when used non-punitively. This study aims to guide educators and institutions in the critical implementation of AI text detectors in higher education, highlighting the importance of exploring alternatives to maintain inclusivity in the face of emerging technologies.

Overview

CoChat Summary

TL;DR

This experimental study tested seven AI text detectors on 114 samples, including 15 AI-generated texts, 89 adversarially modified AI texts, and 10 human controls. The detectors performed poorly overall and became substantially less accurate after simple adversarial techniques, raising concerns about fairness, false accusations, and the use of such tools in inclusive educational assessment.

Key findings

  • Across unmanipulated AI-generated content, the mean detector accuracy was 39.5%; for human-written control samples, accuracy was 67%.
  • After adversarial techniques were applied, mean detector accuracy fell to 22.14%; the paper also reports an overall 17.4% mean drop in accuracy.
  • Some techniques were especially effective: adding spelling errors and increasing burstiness were described as highly effective, with manipulated-output accuracies of 12.9% and 15.9%, respectively; increase complexity had only a 2% drop in detectability and was the least effective.
  • Detector performance varied widely: Copyleaks was the most accurate at 73.9% on non-manipulated output and 58.7% on manipulated output, while GPT Kit was the least accurate at 6% and 4.5%, respectively; Turnitin had the largest accuracy drop at 42.1%.
  • Error analysis found that four of seven detectors did not misclassify any human-written samples, but Copyleaks misidentified 50% of human samples as AI-written; for undetected machine-generated cases, six of seven detectors had a UDC ratio above 50%, and Copyleaks still failed to identify 39% of AI-generated cases.

Methods

  • Experimental design using three GenAI tools (GPT-4, Claude 2, Bard) to create 15 short text samples from five standardized prompts; those samples were then modified with six adversarial techniques to produce 89 additional AI-generated samples, alongside 10 human-written controls.
  • All 114 samples were tested against seven AI text detectors: Turnitin, GPTZero, ZeroGPT, Copyleaks, Crossplag, GPT-2 Output Detector, and GPTKit; there were 805 tests in total and 797 valid tests.
  • Accuracy was calculated using three methods—binary, semi-binary, and logarithmic—and the paper also conducted error analysis for false accusations and undetected cases.

Limitations

  • The paper explicitly states that the study used a relatively small number of samples, only three GenAI tools and seven detectors, so the findings do not cover the full range of possible tools, styles, or manipulations.
  • The authors say the results are only a snapshot in time because both GenAI tools and detectors evolve quickly; they may not remain stable for long.
  • The paper notes that 8 samples could not be tested with some tools because of proprietary software issues; it also says some failures may have been due to text length limits, formatting incompatibilities, or temporary server issues.
  • Evident from the design: the study used short, controlled samples and did not mirror the iterative real-world writing/editing process, which the authors say may mean real-world detectability could be even lower.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s41239-024-00487-w

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

10 Valid · 2 Not valid · 20 Not checked · 91 Own results and interpretations

53 more claims · 1 with their cited sources checked

Create a free account to see all 123 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 41
  • It may state two separate claims as one. 38

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

1 of 1

Citations

Cites

Show all 68 references

Research works printed but not linked: 3

  1. Originality.AI. (2023). AI Content Detector Accuracy Review + Open Source Dataset and Research Tool – Originality.AI. https://originality.ai
  2. Roe, J. (2024). AI and the Anthropological Imagination: Rethinking Education in the Digital Age. Open Anthropology Research Repository. https://openanthroresearch.org/index.php/oarr/preprint/view/399
  3. Sweenor, D., & Ramanathan, K. (2023). The CIO’s Guide to Adopting Generative AI: Five Keys to Success. TinyTechMedia LLC.

Cited by

See all 117

Metadata

Authors
Perkins, M., Roe, J., Vu, B. H., Postma, D., Hickerson, D., McGaughran, J., & Khuat, H. Q.
Affiliations
  • British University Vietnam (VN)
  • James Cook University Singapore (SG)
Corresponding author
Perkins, Mike
Venue
International Journal of Educational Technology in Higher Education
Published
2024-09-09
Volume
21
Issue
1
ISSN
2365-9440 (ISSN-L)
OpenAlex
W4402340204
Cited by
117
References
53
Citations by year
2026: 64 · 2025: 47 · 2024: 6
FWCI
48.31
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Educational Technology in Higher Education (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
  • ResearchOnline - JCU (James Cook University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Durham Research Online (Durham University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Education › Academic integrity and plagiarism
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
Keywords
academic integrity, false accusation, inclusive assessment, higher education, assessment fairness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Simple techniques to bypass GenAI text detectors: implications for inclusive education. https://app.cochat.ai/papers/10.1186/s41239-024-00487-w
The paper
Perkins, M., Roe, J., Vu, B. H., Postma, D., Hickerson, D., McGaughran, J., & Khuat, H. Q. (2024). Simple techniques to bypass GenAI text detectors: implications for inclusive education. International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-024-00487-w

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.