CoChat

Enhancing the Robustness of AI-Generated Text Detectors: A Survey

Open access
Liu, Xin, Li, Yang, Li, Kan · 2025 · Mathematics

The DOI matches its Crossref record; 150 of the 150 references it cites were found in a registry or an index; 13 of the 44 sources its claims cite were checked for support.

Abstract

Experiments show that text perturbations, out-of-distribution text, and AI-human hybrid text all affect detector performance.

Overview

CoChat Summary

TL;DR

This survey reviews methods for enhancing the robustness of AI-generated text (AIGT) detectors, categorizing them into text perturbation robustness, out-of-distribution (OOD) robustness, and AI-human hybrid text (AHT) detection robustness. Its experiments with eight detectors show all are affected by perturbations, OOD text, and AHT, with SCRN relatively robust under perturbations but AHT recall still low.

Key findings

  • The survey organizes robustness enhancement into three areas: text perturbation robustness, OOD robustness, and AHT detection robustness; it claims to be the first review focused specifically on the robustness of AIGT detection.
  • Under character-level perturbations, all evaluated detectors degrade; SCRN is relatively strong, with accuracy 99.87 normal, 96.93 under Swap 2-char, 97.80 under Drop 2-char, and 97.87 under Key 2-char; Rank is low at 63.93 normal and 14.27 under Swap 2-char.
  • On OOD text, most detectors’ accuracy drops noticeably; Log-likelihood achieves the best OOD results with relatively high accuracy and F1 (exact values not included in the provided text).
  • On AHT, performance is far from satisfactory, with some metrics below 50%; recall(AI) ranges from 18.87 (GLTR) to 65.20 (Rank), with SCRN 60.87 and RoBERTa 49.67; F1 ranges from 31.74 (GLTR) to 78.93 (Rank), with SCRN 75.67.

Methods

  • Systematic survey and taxonomy of robustness-enhancement methods for AIGT detectors, covering text perturbation, OOD, and AHT robustness, plus benchmark datasets, evaluation methods, and metrics.
  • Experiments evaluate Log-likelihood, Rank, Log-Rank, Entropy, GLTR, BERT, RoBERTa, and SCRN; GPT-2 is used as the surrogate model for statistical detectors, BERT and RoBERTa are fine-tuned, and SCRN uses a RoBERTa encoder.
  • Perturbation setup: 5000 balanced HC3 training samples and 1500 balanced test samples; four character-level perturbations (Swap, Drop, Keyboard, Add) at 1-char and 2-char levels, applying only one perturbation per word. OOD uses 1500 balanced TruthfulQA samples; AHT uses 1500 MIXSET samples, all treated as AIGT.

Limitations

  • The paper states current AIGT detectors lack mathematical theory, benchmarks often reflect ideal testing environments, AHT detection and short-text detection need more progress, and method categories have limitations (e.g., adversarial training is computationally expensive and may overfit; noise filtering requires accurate noise identification; text correction may blur AIGT/HWT boundaries; paraphrase methods add computational overhead).
  • Evident from the text: the experimental evaluation is limited to eight detectors and specific datasets (HC3, TruthfulQA, MIXSET), with 1500 samples for OOD and AHT; the AHT test set contains only AIGT, so precision(AI) is 1 and accuracy equals recall(AI), limiting interpretation.
  • Evident from the text: perturbation evaluation applies only one perturbation per word rather than the greedy all-potential-character strategy, which the authors say improves computational efficiency but may not fully reflect worst-case robustness.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/math13132145

Cites

150 of 150 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Taxonomy of the focus on robustness issues in AIGT detection.
Figure 1 Taxonomy of the focus on robustness issues in AIGT detection.CC-BY · From the PDF

Verification

Claims the paper makes

10 Valid · 6 Not valid · 25 Not checked · 52 Own results and interpretations

32 more claims · 3 with their cited sources checked

Create a free account to see all 93 claims and to verify the 10 references still open.

See all claims — free account
  • It may state two separate claims as one. 37
  • Probably not a substantial claim. 40

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

1 of 1

Citations

Cites

Show all 150 references

Research works printed but not linked: 4

  1. Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B.D., and Schockaert, S. (2025, January 19–24). Unveiling Fake News with Adversarial Arguments Generated by Multimodal Large Language Models. Proceedings of the 31st International Conference on Computational Linguistics (COLING 2025), Abu Dhabi, United Arab Emirates.
  2. Capkun, S., and Roesner, F. (2020, January 12–14). TextShield: Robust Text Classification Based on Multimodal Embedding and Neural Machine Translation. Proceedings of the 29th USENIX Security Symposium (USENIX Security 2020), Boston, MA, USA.
  3. Radford, 2019, Open and Efficient, OpenAI Blog, 1, 9
  4. Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B.D., and Schockaert, S. (2025, January 19–24). HLU: Human Vs LLM Generated Text Detection Dataset for Urdu at Multiple Granularities. Proceedings of the 31st International Conference on Computational Linguistics (COLING 2025), Abu Dhabi, United Arab Emirates.

Cited by

Metadata

Authors
Liu, X., Li, Y., & Li, K.
Affiliations
Beijing Institute of Technology (CN)
Corresponding author
Li, Kan
Venue
Mathematics
Publisher
MDPI AG
Published
2025-06-30
Volume
13
Issue
13
Pages
2145
ISSN
2227-7390 (ISSN-L)
OpenAlex
W4411802553
Cited by
10
References
91
Citations by year
2026: 7 · 2025: 3
FWCI
4.04
Type
journal article
Language
en
Open access
Locations
  • Mathematics (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Beijing Natural Science Foundation (4222037, L181010, 202306030179)
  • China Scholarship Council (4222037, L181010, 202306030179)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
Keywords
AI-generated text detection, robustness, out-of-distribution robustness, benchmark dataset

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.