CoChat
Sign Up
Cited by 84Adversarial Robustness in Machine LearningAuthorship Attribution and ProfilingSpam and Phishing Detection

RADAR: Robust AI-Text Detection via Adversarial Learning

Open access
Hu, Xiaomeng, Chen, Pin-Yu, Ho, Tsung-Yi·2023·Advances in Neural Information Processing Systems 36
DOI 10.52202/075280-0662
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent advances in large language models (LLMs) and the intensifying popularity of ChatGPT-like applications have blurred the boundary of high-quality text generation between humans and machines. However, in addition to the anticipated revolutionary changes to our technology and society, the difficulty of distinguishing LLM-generated texts (AI-text) from human-generated texts poses new challenges of misuse and fairness, such as fake content generation, plagiarism, and false accusations of innocent writers. While existing works show that current AI-text detectors are not robust to LLM-based paraphrasing, this paper aims to bridge this gap by proposing a new framework called RADAR, which jointly trains a robust AI-text detector via adversarial learning. RADAR is based on adversarial training of a paraphraser and a detector. The paraphraser's goal is to generate realistic content to evade AI-text detection. RADAR uses the feedback from the detector to update the paraphraser, and vice versa. Evaluated with 8 different LLMs (Pythia, Dolly 2.0, Palmyra, Camel, GPT-J, Dolly 1.0, LLaMA, and Vicuna) across 4 datasets, experimental results show that RADAR significantly outperforms existing AI-text detection methods, especially when paraphrasing is in place. We also identify the strong transferability of RADAR from instruction-tuned LLMs to other LLMs, and evaluate the improved capability of RADAR via GPT-3.5-Turbo.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.52202/075280-0662

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Adversarial paraphrasing significantly reduces RADAR's detection.

    “However, adversarial paraphrasing significantly reduces RADAR's detection.”

    From Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text (2025) · cites this paper as [8]

    The Introduction states that RADAR's detector is not weakened by paraphrasing. Section 4.2 says its detection performance is stable across paraphrasing schemes and that RADAR mitigates the performance drop seen in other methods. These passages contradict the claim that adversarial paraphrasing significantly reduces RADAR's detection. (LLM, conf=0.90)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • More robust detectors [8, 14, 36, 1] require recursive paraphrasing to be effectively bypassed.

    “more robust detectors [8, 14, 36, 1] require recursive paraphrasing to be effectively bypassed.”

    From Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text (2025) · cites this paper as [8]

    Section 4.4 (page not available in our copy) reports that RADAR remained robust to multi-round paraphrasing and that, in general, multiple rounds did not increase the difficulty of AI-text detection. This contradicts the claim that recursive paraphrasing is required to effectively bypass robust detectors. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Adversarial paraphrasing guided by RADAR yields the lowest AUC and T@1%F values post-attack.

    “The results indicate that adversarial paraphrasing guided by RADAR yields the lowest AUC and T@1%F values post-attack.”

    From Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text (2025) · cites this paper as [8]

    Section 4.2 reports that paraphrasing substantially lowers the AUROC of most existing detectors, supporting the general post-attack degradation. It does not establish that RADAR-guided paraphrasing yields the lowest AUC across attacks, and the paper does not report T@1%F. Section 4.2 instead says RADAR's detector performs best under paraphrasing. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • RADAR proves to be the most effective guidance detector in the watermark stealing comparison setting.

    “Notably, RADAR [8] proves to be the most effective guidance detector in this setting.”

    From Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text (2025) · cites this paper as [8]

    The Introduction reports that RADAR outperformed six existing AI-text detectors when detecting text from an unseen paraphraser. It does not evaluate RADAR as a guidance detector in a watermark-stealing comparison, so it cannot establish that RADAR was the most effective detector in that setting. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Some advanced detectors, especially those trained on paraphrased AI outputs [8], may exhibit greater resilience to paraphrasing evasion techniques.

    “However, more recent work suggests that some advanced detectors - especially those trained on paraphrased AI outputs [8] - may exhibit greater resilience to such evasion techniques.”

    From Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text (2025) · cites this paper as [8]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Still being checked.

Research works printed but not linked: 36

  1. Jack Brassil, Steven H. Low, Nicholas F. Maxemchuk, and Lawrence O'Gorman. Electronic marking and identification techniques to discourage document copying. IEEE J. Sel. Areas Commun., 13(8):1495-1504, 1995.
  2. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In NeurIPS, 2020.
  3. Souradip Chakraborty, Amrit Singh Bedi, Sicheng Zhu, Bang An, Dinesh Manocha, and Furong Huang. On the possibilities of ai-generated text detection. CoRR, abs/2304.04736, 2023. · arXiv
  4. Minhao Cheng, Jinfeng Yi, Pin-Yu Chen, Huan Zhang, and Cho-Jui Hsieh. Seq2sick: Evaluating the robustness of sequence-to-sequence models with adversarial examples. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 3601-3608, 2020.
  5. Pedro Colon-Hernandez, Henry Lieberman, Yida Xin, Claire Yin, Cynthia Breazeal, and Peter Chin. Adversarial transformer language models for contextual commonsense inference. CoRR, abs/2302.05406, 2023. · arXiv
  6. Long Dai, Jiarong Mao, Xuefeng Fan, and Xiaoyi Zhou. Deephider: A multi-module and invisibility watermarking scheme for language model. CoRR, abs/2208.04676, 2022. · arXiv
  7. Cyprien de Masson d'Autume, Shakir Mohamed, Mihaela Rosca, and Jack W. Rae. Training language gans from scratch. In NeurIPS, pages 4302-4313, 2019.
  8. Sebastian Gehrmann, Hendrik Strobelt, and Alexander M. Rush. GLTR: statistical detection and visualization of generated text. In ACL, pages 111-116, 2019.
  9. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139-144, 2020.
  10. Tom Hosking and Mirella Lapata. Factorising meaning and form for intent-preserving para- phrasing. In ACL/IJCNLP, pages 1405-1418, 2021.
  11. Tom Hosking, Hao Tang, and Mirella Lapata. Hierarchical sketch induction for paraphrase generation. In ACL, pages 2489-2501, 2022.
  12. Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. Automatic detection of generated text is easiest when humans are fooled. In ACL, pages 1808-1822, 2020.
  13. Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In ICLR, 2017.
  14. Mohan S. Kankanhalli and K. F. Hau. Watermarking of electronic text documents. Electron. Commer. Res., 2(1-2):169-187, 2002.
  15. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. CoRR, abs/2301.10226, 2023. · arXiv
  16. Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. Para- phrasing evades detectors of ai-generated text, but retrieval is an effective defense. CoRR, abs/2303.13408, 2023. · arXiv
  17. Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. Para- phrasing evades detectors of ai-generated text, but retrieval is an effective defense. CoRR, abs/2303.13408, 2023. · arXiv
  18. Thomas Lavergne, Tanguy Urvoy, and François Yvon. Detecting fake content with relative en- tropy scoring. In Proceedings of the ECAI'08 Workshop on Uncovering Plagiarism, Authorship and Social Software Misuse, Patras, Greece, July 22, 2008, volume 377 of CEUR Workshop Proceedings, 2008.
  19. Qi Lei, Lingfei Wu, Pin-Yu Chen, Alex Dimakis, Inderjit S Dhillon, and Michael J Witbrock. Discrete adversarial attacks and submodular optimization with applications to text classification. Proceedings of Machine Learning and Systems, 1:146-165, 2019.
  20. Weixin Liang, Mert Yüksekgönül, Yining Mao, Eric Wu, and James Zou. GPT detectors are biased against non-native english writers. CoRR, abs/2304.02819, 2023. · arXiv
  21. Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. In ICLR, 2017.
  22. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. CoRR, abs/2301.11305, 2023. · arXiv
  23. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In NeurIPS, 2022.
  24. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR, 2020.
  25. Juan Rodriguez, Todd Hay, David Gros, Zain Shamsi, and Ravi Srinivasan. Cross-domain detection of gpt-2-generated technical text. In NAACL, pages 1213-1233, 2022.
  26. Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can ai-generated text be reliably detected? CoRR, abs/2303.11156, 2023. · arXiv
  27. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, abs/1707.06347, 2017. · arXiv
  28. Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203, 2019. · arXiv
  29. Hong Sun and Ming Zhou. Joint learning of a dual SMT system for paraphrase generation. In ACL, pages 38-42, 2012.
  30. Umut Topkara, Mercan Topkara, and Mikhail J. Atallah. The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions. In MM&Sec, pages 164-174, 2006.
  31. Honai Ueoka, Yugo Murawaki, and Sadao Kurohashi. Frustratingly easy edit-based linguistic steganography with a masked language model. In NAACL, pages 5486-5492, 2021.
  32. Qingyang Wu, Lei Li, and Zhou Yu. Textgail: Generative adversarial imitation learning for text generation. In AAAI, pages 14067-14075, 2021.
  33. Jingjing Xu, Xuancheng Ren, Junyang Lin, and Xu Sun. Diversity-promoting GAN: A cross- entropy based generative adversarial network for diversified text generation. In EMNLP, pages 3940-3949, 2018.
  34. Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, and Taylor Berg-Kirkpatrick. Unsupervised text style transfer using language models as discriminators. In NeurIPS, pages 7298-7309, 2018.
  35. Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In AAAI, pages 2852-2858, 2017.
  36. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. In NeurIPS, pages 9051-9062, 2019.

Cited by

Sorted by citations

Related

Authors
Hu, X., Chen, P.-Y., & Ho, T.-Y.
ORCID
Chen, Pin-Yu
Venue
Advances in Neural Information Processing Systems 36
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Published
2023
Pages
15077-15095
ISSN
1049-5258 (ISSN-L)
DOI
10.52202/075280-0662
OpenAlex
W7133226986
Cited by
62
References
0
Citations by year
2026: 22 · 2025: 23 · 2024: 17
FWCI
10.99
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
  • Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
Keywords
AI-generated text detection, adversarial learning, text detection, generative AI detection, robustness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review