Enhancing the Robustness of AI-Generated Text Detectors: A Survey
Open accessThe DOI matches its Crossref record; 150 of the 150 references it cites were found in a registry or an index; 13 of the 44 sources its claims cite were checked for support.
Abstract
Experiments show that text perturbations, out-of-distribution text, and AI-human hybrid text all affect detector performance.
Overview
CoChat Summary
TL;DR
This survey reviews methods for enhancing the robustness of AI-generated text (AIGT) detectors, categorizing them into text perturbation robustness, out-of-distribution (OOD) robustness, and AI-human hybrid text (AHT) detection robustness. Its experiments with eight detectors show all are affected by perturbations, OOD text, and AHT, with SCRN relatively robust under perturbations but AHT recall still low.
Key findings
- The survey organizes robustness enhancement into three areas: text perturbation robustness, OOD robustness, and AHT detection robustness; it claims to be the first review focused specifically on the robustness of AIGT detection.
- Under character-level perturbations, all evaluated detectors degrade; SCRN is relatively strong, with accuracy 99.87 normal, 96.93 under Swap 2-char, 97.80 under Drop 2-char, and 97.87 under Key 2-char; Rank is low at 63.93 normal and 14.27 under Swap 2-char.
- On OOD text, most detectors’ accuracy drops noticeably; Log-likelihood achieves the best OOD results with relatively high accuracy and F1 (exact values not included in the provided text).
- On AHT, performance is far from satisfactory, with some metrics below 50%; recall(AI) ranges from 18.87 (GLTR) to 65.20 (Rank), with SCRN 60.87 and RoBERTa 49.67; F1 ranges from 31.74 (GLTR) to 78.93 (Rank), with SCRN 75.67.
Methods
- Systematic survey and taxonomy of robustness-enhancement methods for AIGT detectors, covering text perturbation, OOD, and AHT robustness, plus benchmark datasets, evaluation methods, and metrics.
- Experiments evaluate Log-likelihood, Rank, Log-Rank, Entropy, GLTR, BERT, RoBERTa, and SCRN; GPT-2 is used as the surrogate model for statistical detectors, BERT and RoBERTa are fine-tuned, and SCRN uses a RoBERTa encoder.
- Perturbation setup: 5000 balanced HC3 training samples and 1500 balanced test samples; four character-level perturbations (Swap, Drop, Keyboard, Add) at 1-char and 2-char levels, applying only one perturbation per word. OOD uses 1500 balanced TruthfulQA samples; AHT uses 1500 MIXSET samples, all treated as AIGT.
Limitations
- The paper states current AIGT detectors lack mathematical theory, benchmarks often reflect ideal testing environments, AHT detection and short-text detection need more progress, and method categories have limitations (e.g., adversarial training is computationally expensive and may overfit; noise filtering requires accurate noise identification; text correction may blur AIGT/HWT boundaries; paraphrase methods add computational overhead).
- Evident from the text: the experimental evaluation is limited to eight detectors and specific datasets (HC3, TruthfulQA, MIXSET), with 1500 samples for OOD and AHT; the AHT test set contains only AIGT, so precision(AI) is 1 and accuracy equals recall(AI), limiting interpretation.
- Evident from the text: perturbation evaluation applies only one perturbation per word rather than the greedy all-potential-character strategy, which the authors say improves computational efficiency but may not fully reflect worst-case robustness.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
10 Valid · 6 Not valid · 25 Not checked · 52 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Even minor modifications, such as replacing a word with its abbre-viation [35] or simply inserting an extra space [39], can significantly impact the detection results.”
§ Enhancing the Robustness of AI-Generated Text Detectors: A Survey > 3. Text Perturbation Robustness
What the sources say
- Not matchedcites [35] Ku, L., Martins, A., and Srikumar, V. (2024, January 11–16). Are AI-Generated Text Detectors Robust to Adversarial Perturbations?. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), Bangkok, Thailand. Volume 1: Long Papers.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [39] Cai, S., and Cui, W. (2023). Evade ChatGPT Detectors via A Single Space. arXiv. source
The Introduction reports that adding a single space before a random comma significantly reduced AI-text detection rates. The Conclusion says the effect was verified across multiple benchmarks and detectors. The abbreviation example is attached to a different citation, so it is not attributed to this source. (LLM, conf=0.95)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
Every source the paper cites for this claim was checked and supports it.
In the paper
“This ability to generate diverse expressions while maintaining the original meaning makes paraphrase a key strategy for evading AIGT detection [42] has shown that recursive paraphrase can significantly reduce detection rates.”
§ Enhancing the Robustness of AI-Generated Text Detectors: A Survey > 3. Text Perturbation Robustness
What the sources say
- Supportedcites [42] Sadasivan, V.S., Kumar, A., Balasubramanian, S., Wang, W., and Feizi, S. (2023). Can AI-Generated Text be Reliably Detected?. arXiv. source
Section 2.3 (p. 7) reports that a recursive paraphrase attack reduced a watermark detector’s detection rate to below 20% after five rounds. Section 2.4 (p. 10) also reports that a retrieval-based detector’s accuracy fell below approximately 60% after five rounds. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Recent studies have demonstrated that current AIGT detectors are highly vulnerable to text perturbations.”
§ Enhancing the Robustness of AI-Generated Text Detectors: A Survey > 3. Text Perturbation Robustness
What the sources say
- Not matchedcites [35] Ku, L., Martins, A., and Srikumar, V. (2024, January 11–16). Are AI-Generated Text Detectors Robust to Adversarial Perturbations?. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), Bangkok, Thailand. Volume 1: Long Papers.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [39] Cai, S., and Cui, W. (2023). Evade ChatGPT Detectors via A Single Space. arXiv. source
The Introduction reports that adding a single space before a comma sharply reduced detection rates for both white-box and black-box detectors. The Conclusion says the strategy was effective across a variety of benchmarks and detectors, supporting the claim that current detectors are vulnerable to text perturbations. (LLM, conf=0.91)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“However, the high similarity between AIGT and HWT makes the accuracy of human judgment in AIGT detection only marginally better than random guessing [10].”
§ Enhancing the Robustness of AI-Generated Text Detectors: A Survey > 1. Introduction
What the sources say
- Could not verifycites [10] Wu, J., Yang, S., Zhan, R., Yuan, Y., Wong, D.F., and Chao, L.S. (2023). A Survey on LLM-generated Text Detection: Necessity, Methods, and Future Directions. arXiv. sourceExperimental — accuracy not yet measured
The Introduction and Related Work discuss methods for distinguishing LLM-generated from human-written text, but do not report the accuracy of unaided human judgment. Searches for human detection accuracy, random guessing, and textual similarity found no passage supporting the claimed near-chance performance. (LLM, conf=0.75)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“However, to date, no comprehensive review has systematically compiled this body of research.”
§ Enhancing the Robustness of AI-Generated Text Detectors: A Survey > 1. Introduction
What the sources say
- Could not verifycites [10] Wu, J., Yang, S., Zhan, R., Yuan, Y., Wong, D.F., and Chao, L.S. (2023). A Survey on LLM-generated Text Detection: Necessity, Methods, and Future Directions. arXiv. sourceExperimental — accuracy not yet measured
The passages on multi-feature detection and detector limitations discuss robustness and generalization, not whether a comprehensive review of robustness research exists. Searches for a systematic or comprehensive review found no passage in our copy addressing that claim; the cited sections have no page numbers in the index. (LLM, conf=0.72)
- Not checked — no full text of the sourcecites [30] Yang, 2025, The Imitation Game revisited: A comprehensive survey on recent advances in AI-generated text detection, Expert Syst. Appl. source
- Could not verifycites [31] Valiaiev, D. (2024). Detection of Machine-Generated Text: Literature Survey. arXiv. sourceExperimental — accuracy not yet measured
The discussion of detection challenges and the survey’s future directions addresses improving detectors, but it does not say whether a comprehensive review of research on detector robustness existed. Searches for prior reviews, systematic surveys, and robustness research found no passage establishing the claimed gap. (LLM, conf=0.72)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
32 more claims · 3 with their cited sources checked
Create a free account to see all 93 claims and to verify the 10 references still open.
- It may state two separate claims as one. 37
- Probably not a substantial claim. 40
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“Rank [12] 62.27 83.46 61.64 43.07 91.47 73.64”
§ Enhancing the Robustness of AI-Generated Text Detectors: A Survey > Model Accuracy Precision (H.) Precision (AI) Recall (H.) Recall (AI) F1 Score
Reported 62.27 · recomputed 67.27
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Momentum Contrast for Unsupervised Visual Representation LearningPreprint
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. B. (2019) · Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1911.05722 · Cited by 16,321
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602
Research works printed but not linked: 4
- Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B.D., and Schockaert, S. (2025, January 19–24). Unveiling Fake News with Adversarial Arguments Generated by Multimodal Large Language Models. Proceedings of the 31st International Conference on Computational Linguistics (COLING 2025), Abu Dhabi, United Arab Emirates.
- Capkun, S., and Roesner, F. (2020, January 12–14). TextShield: Robust Text Classification Based on Multimodal Embedding and Neural Machine Translation. Proceedings of the 29th USENIX Security Symposium (USENIX Security 2020), Boston, MA, USA.
- Radford, 2019, Open and Efficient, OpenAI Blog, 1, 9
- Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B.D., and Schockaert, S. (2025, January 19–24). HLU: Human Vs LLM Generated Text Detection Dataset for Urdu at Multiple Granularities. Proceedings of the 31st International Conference on Computational Linguistics (COLING 2025), Abu Dhabi, United Arab Emirates.
Cited by
An AI for Robust MCDM Ranking in a Large Number of Criteria
Garcia-Gastelum, T. S., Uzeta-Obregon, C. R., Carrillo, P. A. Á., & León‐Castro, E. (2025) · Mathematics · doi:10.3390/math13172789 · Cited by 3
AI-Generated Text in Orthopaedic Articles: A Cross-Sectional Analysis
Sweeney, P., Yuro, M. R., & Berberian, W. S. (2025) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.25.00971 · Cited by 3
Theodorakopoulos, L., & Theodoropoulou, A. (2026) · Electronics · doi:10.3390/electronics15071405 · Cited by 1
Vaidya, S., & Saini, J. R. (2025) · International Conference for Advancement in Technology (ICONAT) · doi:10.1109/iconat66879.2025.11362514
Hu, C., Zhou, W., & Zhu, Y. (2026) · Research Square · doi:10.21203/rs.3.rs-9070095/v1
Metadata
- Authors
- Liu, X., Li, Y., & Li, K.
- Affiliations
- Beijing Institute of Technology (CN)
- Corresponding author
- Li, Kan
- Venue
- Mathematics
- Publisher
- MDPI AG
- Published
- 2025-06-30
- Volume
- 13
- Issue
- 13
- Pages
- 2145
- ISSN
- 2227-7390 (ISSN-L)
- OpenAlex
- W4411802553
- Cited by
- 10
- References
- 91
- Citations by year
- 2026: 7 · 2025: 3
- FWCI
- 4.04
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Mathematics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Beijing Natural Science Foundation (4222037, L181010, 202306030179)
- China Scholarship Council (4222037, L181010, 202306030179)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- AI-generated text detection, robustness, out-of-distribution robustness, benchmark dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
