CoChat
Sign Up
PreprintCited by 9Authorship Attribution and ProfilingAdversarial Robustness in Machine LearningDigital Media Forensic Detection

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

A Practical Examination of AI-Generated Text Detectors for Large Language Models

Open access
Tufts, Brian, Zhao, Xuandong, Li, Lei·2024·arXiv
DOI 10.48550/arxiv.2412.05139arXiv 2412.05139
Open this paper in CoChat — free accountOpen PDF

Abstract

The proliferation of large language models has raised growing concerns about their misuse, particularly in cases where AI-generated text is falsely attributed to human authors. Machine-generated content detectors claim to effectively identify such text under various conditions and from any language model. This paper critically evaluates these claims by assessing several popular detectors (RADAR, Wild, T5Sentinel, Fast-DetectGPT, PHD, LogRank, Binoculars) on a range of domains, datasets, and models that these detectors have not previously encountered. We employ various prompting strategies to simulate practical adversarial attacks, demonstrating that even moderate efforts can significantly evade detection. We emphasize the importance of the true positive rate at a specific false positive rate (TPR@FPR) metric and demonstrate that these detectors perform poorly in certain settings, with TPR@.01 as low as 0%. Our findings suggest that both trained and zero-shot detectors struggle to maintain high sensitivity while achieving a reasonable true positive rate.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2412.05139

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 1 references

Research works printed but not linked: 58

  1. Yiran Chen, Zhenqiao Song, Xianze Wu, Danqing Wang, Jingjing Xu, Jiaze Chen, Hao Zhou, and Lei Li. 2022. MTG: A benchmark suite for multilin- gual text generation. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2508-2527.
  2. Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh, and Bhiksha Raj. 2023. Token prediction as implicit classification to identify LLM-generated text. In Proc. of EMNLP, pages 13112-13120.
  3. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anasta- sios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. Chatbot arena: An open platform for evaluating llms by human prefer- ence. Preprint, arXiv:2403.04132. · arXiv
  4. Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018. A discourse-aware attention model for abstractive summarization of long documents. In Proc. of NAACL-HLT, pages 615-621.
  5. Kohinoor Darda, Marion Carre, and Emily Cross. 2023. Value attributed to text-based archives generated by artificial intelligence. Royal Society Open Science, 10(2):220915.
  6. Maxime De Bruyn, Ehsan Lotfi, Jeska Buhmann, and Walter Daelemans. 2021. MFAQ: a multilingual FAQ dataset. In Proceedings of the 3rd Workshop on Ma- chine Reading for Question Answering, pages 1-13.
  7. Angela Fan, Yacine Jernite, Ethan Perez, David Grang- ier, Jason Weston, and Michael Auli. 2019. ELI5: Long form question answering. In Proc. of ACL, pages 3558-3567.
  8. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proc. of ACL, pages 889-898.
  9. Shangbin Feng, Herun Wan, Ningnan Wang, Zhaoxuan Tan, Minnan Luo, and Yulia Tsvetkov. 2024. What does the bot say? opportunities and risks of large language models in social media bot detection. ArXiv preprint, abs/2402.00371. · arXiv
  10. Xiachong Feng, Xiaocheng Feng, and Bing Qin. 2022. MSAMSum: Towards benchmarking multi-lingual dialogue summarization. In Proceedings of the Sec- ond DialDoc Workshop on Document-grounded Dia- logue and Conversational Question Answering, pages 1-12.
  11. Kathleen C. Fraser, Hillary Dawkins, and Svetlana Kir- itchenko. 2024. Detecting ai-generated text: Fac- tors influencing detectability with current methods. Preprint, arXiv:2406.15583. · arXiv
  12. Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. GLTR: Statistical detection and visu- alization of generated text. In Proc. of ACL, pages 111-116.
  13. Soumya Suvra Ghosal, Souradip Chakraborty, Jonas Geiping, Furong Huang, Dinesh Manocha, and Am- rit Singh Bedi. 2023. Towards possibilities & im- possibilities of ai-generated text detection: A survey. ArXiv preprint, abs/2310.15264. · arXiv
  14. Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Alek- sander Wawer. 2019. SAMSum corpus: A human- annotated dialogue dataset for abstractive summariza- tion. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pages 70-79.
  15. Mandy Guo, Zihang Dai, Denny Vrandečić, and Rami Al-Rfou. 2020. Wiki-40B: Multilingual language model dataset. In Proceedings of the Twelfth Lan- guage Resources and Evaluation Conference, pages 2440-2452.
  16. Felix Hamborg, Norman Meuschke, Corinna Breitinger, and Bela Gipp. 2017. news-please. In Everything Changes, Everything Stays the Same? Understanding Information Spaces. Proceedings of the 15th Interna- tional Symposium of Information Science (ISI 2017), pages 218-223. · DOI
  17. Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. Spotting llms with binoculars: Zero-shot detection of machine-generated text. ArXiv preprint, abs/2401.12070. · arXiv
  18. Dan Hendrycks, Steven Basart, Saurav Kadavath, Man- tas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. 2021. Measuring coding challenge competence with apps. ArXiv preprint, abs/2105.09938. · arXiv
  19. Karl Moritz Hermann, Tomáš Kočiský, Edward Grefen- stette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Proceedings of the 28th Interna- tional Conference on Neural Information Processing Systems -Volume 1, NIPS'15, page 1693-1701, Cam- bridge, MA, USA. MIT Press.
  20. Dirk Hovy. 2016. The enemy in your own camp: How well can we detect statistically-generated fake re- views -an adversarial study. In Proc. of ACL, pages 351-356.
  21. Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. 2023. RADAR: robust ai-text detection via adversarial learning. In Proc. of NeurIPS.
  22. Daphne Ippolito, Daniel Duckworth, Chris Callison- Burch, and Douglas Eck. 2020. Automatic detec- tion of generated text is easiest when humans are fooled. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1808-1822, Online. Association for Computational Linguistics. · DOI · Paper page
  23. Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks Lakshmanan, V.S. 2020. Automatic detection of machine generated text: A critical survey. In Pro- ceedings of the 28th International Conference on Computational Linguistics, pages 2296-2309.
  24. Albert Q Jiang, Alexandre Sablayrolles, Arthur Men- sch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guil- laume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. ArXiv preprint, abs/2310.06825. · arXiv
  25. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Co- hen, and Xinghua Lu. 2019. PubMedQA: A dataset for biomedical research question answering. In Proc. of EMNLP, pages 2567-2577.
  26. Dongyeop Kang, Waleed Ammar, Bhavana Dalvi, Madeleine van Zuylen, Sebastian Kohlmeier, Eduard Hovy, and Roy Schwartz. 2018. A dataset of peer reviews (PeerRead): Collection, insights and NLP applications. In Proc. of NAACL-HLT, pages 1647- 1661.
  27. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. A watermark for large language models. In Proc. of ICML, volume 202 of Proceedings of Machine Learning Research, pages 17061-17084.
  28. Tharindu Kumarage, Paras Sheth, Raha Moraffah, Joshua Garland, and Huan Liu. 2023. How reli- able are AI-generated-text detectors? an assessment framework using evasive soft prompts. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1337-1349.
  29. Thomas Lavergne, Tanguy Urvoy, and François Yvon. 2008. Detecting fake content with relative entropy scoring. Pan, 8(27-31):4.
  30. Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. 2023. Who wrote this code? watermarking for code generation. ArXiv preprint, abs/2305.15060. · arXiv
  31. Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. 2024. MAGE: Machine-generated text detec- tion in the wild. In Proc. of ACL, pages 36-53.
  32. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. Preprint, arXiv:1907.11692. · arXiv
  33. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detec- tion using probability curvature. In Proc. of ICML, volume 202 of Proceedings of Machine Learning Research, pages 24950-24962.
  34. Nafise Sadat Moosavi, Andreas Rücklé, Dan Roth, and Iryna Gurevych. 2021. Learning to reason for text generation from scientific tables. ArXiv preprint, abs/2104.08296. · arXiv
  35. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. A cor- pus and cloze evaluation for deeper understanding of commonsense stories. In Proc. of NAACL-HLT, pages 839-849.
  36. Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don't give me the details, just the summary! topic-aware convolutional neural networks for ex- treme summarization. In Proc. of EMNLP, pages 1797-1807.
  37. Mike Perkins. 2023. Academic integrity considerations of ai large language models in the post-pandemic era: Chatgpt and beyond. Journal of university teaching & learning practice, 20(2):07.
  38. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9.
  39. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proc. of EMNLP, pages 2383-2392.
  40. Vinu Sankar Sadasivan, Aounon Kumar, Sriram Bala- subramanian, Wenxiao Wang, and Soheil Feizi. 2023. Can ai-generated text be reliably detected? ArXiv preprint, abs/2303.11156. · arXiv
  41. Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI Maga- zine, 29(3):93. · DOI · Paper page
  42. Zhouxing Shi, Yihan Wang, Fan Yin, Xiangning Chen, Kai-Wei Chang, and Cho-Jui Hsieh. 2024. Red team- ing language model detectors with language models. Transactions of the Association for Computational Linguistics, 12:174-189.
  43. Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. 2019. Release strategies and the social impacts of language models. ArXiv preprint, abs/1908.09203. · arXiv
  44. Chenhao Tan, Vlad Niculae, Cristian Danescu- Niculescu-Mizil, and Lillian Lee. 2016. Winning ar- guments: Interaction dynamics and persuasion strate- gies in good-faith online discussions. In Proceedings of the 25th International Conference on World Wide Web, WWW 2016, Montreal, Canada, April 11 -15, 2016, pages 613-624.
  45. Ruixiang Tang, Yu-Neng Chuang, and Xia Hu. 2023. The science of detecting llm-generated texts. ArXiv preprint, abs/2303.07205. · arXiv
  46. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open founda- tion and fine-tuned chat models. ArXiv preprint, abs/2307.09288. · arXiv
  47. Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavskii, Sergey I. Nikolenko, Evgeny Burnaev, Serguei Barannikov, and Irina Piontkovskaya. 2023. Intrinsic dimension estimation for robust detection of ai-generated texts. In Proc. of NeurIPS.
  48. Saranya Venkatraman, Adaku Uchendu, and Dongwon Lee. 2024. GPT-who: An information density-based machine-generated text detector. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 103-115.
  49. Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. 2024. Ghostbuster: Detecting text ghostwrit- ten by large language models. In Proceedings of the 2024 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pa- pers), pages 1702-1717.
  50. Yichen Wang, Shangbin Feng, Abe Bohan Hou, Xiao Pu, Chao Shen, Xiaoming Liu, Yulia Tsvetkov, and Tianxing He. 2024. Stumbling blocks: Stress testing the robustness of machine-generated text detectors under attacks. ArXiv preprint, abs/2402.11638. · arXiv
  51. Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Derek F Wong, and Lidia S Chao. 2023. A sur- vey on llm-gernerated text detection: Necessity, methods, and future directions. ArXiv preprint, abs/2310.14724. · arXiv
  52. Xianjun Yang, Wei Cheng, Linda Petzold, William Yang Wang, and Haifeng Chen. 2023a. Dna-gpt: Divergent n-gram analysis for training-free detection of gpt- generated text. ArXiv preprint, abs/2305.17359. · arXiv
  53. Xianjun Yang, Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Petzold, William Yang Wang, and Wei Cheng. 2023b. A survey on detec- tion of llms-generated content. ArXiv preprint, abs/2310.15654. · arXiv
  54. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019a. HellaSwag: Can a machine really finish your sentence? In Proc. of ACL, pages 4791-4800.
  55. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019b. Defending against neural fake news. In Proc. of NeurIPS, pages 9051-9062.
  56. Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text clas- sification. In Proc. of NeurIPS, pages 649-657.
  57. Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. 2023. Provable robust water- marking for ai-generated text. ArXiv preprint, abs/2306.17439. · arXiv
  58. Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Neil Zhenqiang Gong, Yue Zhang, et al. 2023. Promptbench: Towards evaluating the robustness of large language models on adversarial prompts. ArXiv preprint, abs/2306.04528. · arXiv

Cited by

Sorted by citations

Metadata

Authors
Tufts, B., Zhao, X., & Li, L.
Affiliations
  • Carnegie Mellon University (US)
  • University of California, Berkeley (US)
ORCID
Li, Lei
Venue
arXiv
Publisher
arXiv
Published
2024
Pages
4824-4841
DOI
10.48550/arxiv.2412.05139
arXiv
2412.05139
OpenAlex
W4411120583
Cited by
9
References
1
Citations by year
2026: 8 · 2025: 1
FWCI
8.58
Type
conference paper
Language
en
Open access
Licence
Locations
  • Findings of the Association for Computational Linguistics: NAACL (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, public-domain)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
Keywords
AI-generated text detection, large language models, detection evasion, adversarial prompting, radar, PhD, binoculars, authorship attribution, detector performance evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review