CoChat
Sign Up
PreprintCited by 61Advanced Steganography and Watermarking TechniquesAdversarial Robustness in Machine LearningPhysical Unclonable Functions (PUFs) and Hardware Security

The DOI matches its DataCite record.

Mark My Words: Analyzing and Evaluating Language Model Watermarks

Open access
Piet, Julien, Sitawarin, Chawin, Fang, Vivian, Mu, Norman, Wagner, David·2023·arXiv
DOI 10.48550/arxiv.2312.00273arXiv 2312.00273
Open this paper in CoChat — free accountOpen PDF

Abstract

The capabilities of large language models have grown significantly in recent years and so too have concerns about their misuse. It is important to be able to distinguish machine-generated text from human-authored content. Prior works have proposed numerous schemes to watermark text, which would benefit from a systematic evaluation framework. This work focuses on LLM output watermarking techniques - as opposed to image or model watermarks - and proposes Mark My Words, a comprehensive benchmark for them under different natural language tasks. We focus on three main metrics: quality, size (i.e., the number of tokens needed to detect a watermark), and tamper resistance (i.e., the ability to detect a watermark after perturbing marked text). Current watermarking techniques are nearly practical enough for real-world use: Kirchenbauer et al. [33]'s scheme can watermark models like Llama 2 7B-chat or Mistral-7B-Instruct with no perceivable loss in quality on natural language tasks, the watermark can be detected with fewer than 100 tokens, and their scheme offers good tamper resistance to simple perturbations. However, they struggle to efficiently watermark code generations. We publicly release our benchmark (https://github.com/wagner-group/MarkMyWords).

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2312.00273

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 43

  1. Canyu Chen and Kai Shu. Can LLM-generated misin- formation be detected? arXiv preprint arXiv:2309.13788, 2023. · arXiv
  2. Cheng-Han Chiang and Hung-yi Lee. Can large language models be an alternative to human evaluations? In ACL 2023.
  3. Tianshuo Cong, Xinlei He, and Yang Zhang. SSLGuard: A watermarking scheme for self-supervised learning pre- trained encoders. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security.
  4. Ingemar J. Cox, Ton Kalker, Georg Pakura, and Mathias Scheel. Information transmission and steganography. In Mauro Barni, Ingemar Cox, Ton Kalker, and Hyoung- Joong Kim, editors, Digital Watermarking. Springer Berlin Heidelberg, 2005.
  5. Tom Fawcett. An introduction to ROC analysis. Pattern Recognition Letters, 2006.
  6. Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. GLTR: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, 2019.
  7. Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang. Protecting intellectual property of language generation apis with lexical watermark. In Proceedings of the AAAI Conference on Artificial Intelligence, 2022.
  8. Xuanli He, Qiongkai Xu, Yi Zeng, Lingjuan Lyu, Fangzhao Wu, Jiwei Li, and Ruoxi Jia. Cater: Intellectual property protection on text generation apis via conditional watermarks. Advances in Neural Information Processing Systems, 2022.
  9. Zhiwei He, Binglin Zhou, Hongkun Hao, Aiwei Liu, Xing Wang, Zhaopeng Tu, Zhuosheng Zhang, and Rui Wang. Can watermarks survive translation? on the cross-lingual consistency of text watermark for large language models, 2024. URL https://arxiv.org/abs/2402.14007. · arXiv
  10. Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. Measuring Coding Challenge Competence With APPS. In NeurIPS, 2021.
  11. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring Massive Multitask Language Understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
  12. Karl Moritz Hermann, Tomas Kocisky, Edward Grefen- stette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching Machines to Read and Com- prehend. In Advances in Neural Information Processing Systems, 2015.
  13. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In International Conference on Learning Representations, 2020.
  14. Nicholas Hopper, Luis von Ahn, and John Langford. Provably secure steganography. IEEE Transactions on Computers, 2009.
  15. Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. Unbiased watermark for large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=uWVC5FVidc.
  16. Nurul Shamimi Kamaruddin, Amirrudin Kamsin, Lip Yee Por, and Hameedur Rahman. A review of text watermark- ing: Theory, methods, and applications. IEEE Access, 2018.
  17. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In Proceedings of the 40th International Conference on Machine Learning, 2023.
  18. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. OpenNMT: Open-source toolkit for neural machine translation. In Proceedings of ACL 2017, System Demonstrations, Vancouver, Canada. Association for Computational Linguistics.
  19. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In SOSP, 2023.
  20. Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. Who wrote this code? watermarking for code generation, 2024. URL https://arxiv.org/abs/2305.15060. · arXiv
  21. Aiwei Liu, Leyi Pan, Xuming Hu, Shu'ang Li, Lijie Wen, Irwin King, and Philip S. Yu. An Unforgeable Publicly Verifiable Watermark for Large Language Models, 2024. URL https://arxiv.org/abs/2307.16230. · arXiv
  22. Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. A semantic invariant robust watermark for large language models, 2024. URL https://arxiv.org/abs/ 2310.06356.
  23. Yijian Lu, Aiwei Liu, Dianzhi Yu, Jingjing Li, and Irwin King. An entropy-based text watermarking detection method, 2024. URL https://arxiv.org/abs/2403.13485. · arXiv
  24. Mohammed A. Majeed, Rossilawati Sulaiman, Zarina Shukur, and Mohammad K. Hasan. A review on text steganography techniques. Mathematics, 2021.
  25. George A. Miller. WordNet: A lexical database for english. In Human Language Technology: Proceedings of a Workshop Held at Plainsboro, New Jersey, 1994.
  26. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christo- pher D. Manning, and Chelsea Finn. DetectGPT: Zero- shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning, 2023.
  27. Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, et al. MarkLLM: An open-source toolkit for LLM watermarking. arXiv preprint arXiv:2405.10051, 2024. · arXiv
  28. Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. Mauve: Measuring the gap between neural text and human text using divergence frontiers. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 4816-4828.
  29. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/ file/260c2432a0eecc28ce03c10dadc078a4-Paper.pdf.
  30. Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers. In NeurIPS, 2021.
  31. Stefano Giovanni Rizzo, Flavio Bertini, Danilo Montesi, and Carlo Stomeo. Text watermarking in social media. In Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017.
  32. Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto, Eric Schulz, and Zeynep Akata. In-context impersonation reveals large language models' strengths and biases. NeurIPS, 2024.
  33. Abigail See, Peter J. Liu, and Christopher D. Manning. Get To The Point: Summarization with Pointer-Generator Networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017.
  34. Milad Taleby Ahvanooey, Hassan Dana Mazraeh, and Seyed Tabasi. An innovative technique for web text watermarking (aitw). Information Security Journal: A Global Perspective, 2016.
  35. Adaku Uchendu, Thai Le, Kai Shu, and Dongwon Lee. Authorship attribution for neural text generation. In Pro- ceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.
  36. Christoforos Vasilatos, Manaar Alam, Talal Rahwan, Yasir Zaki, and Michail Maniatakos. HowkGPT: Investigating the detection of ChatGPT-generated university student homework through context-aware perplexity analysis. arXiv preprint arXiv:2305.18226, 2023. · arXiv
  37. Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Di- nan, Kyunghyun Cho, and Jason Weston. Neural text generation with unlikelihood training. In International Conference on Learning Representations, 2020.
  38. Thomas Wolf et al. Transformers: State-of-the-art nat- ural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2020.
  39. Jiaying Wu, Jiafeng Guo, and Bryan Hooi. Fake news in sheep's clothing: Robust fake news detection against LLM-empowered style attacks. In SIGKDD, 2024.
  40. Yihan Wu, Zhengmian Hu, Junfeng Guo, Hongyang Zhang, and Heng Huang. A resilient and accessible distribution-preserving watermark for large language models, 2024. URL https://arxiv.org/abs/2310.07710. · arXiv
  41. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. In Advances in Neural Information Processing Systems, 2019.
  42. Xuandong Zhao, Yu-Xiang Wang, and Lei Li. Protecting language generation models via invisible watermarking. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 42187-42199. PMLR, 23-29 Jul 2023. URL https://proceedings.mlr.press/v202/ zhao23i.html.
  43. Zachary Ziegler, Yuntian Deng, and Alexander Rush. Neural linguistic steganography. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- IJCNLP), 2019.

Cited by

Sorted by citations

See all 61

Metadata

Authors
Piet, J., Sitawarin, C., Fang, V., Mu, N., & Wagner, D.
Affiliations
University of California, Berkeley (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
Pages
68-91
DOI
10.48550/arxiv.2312.00273
arXiv
2312.00273
OpenAlex
W4410608986
Cited by
61
References
0
Citations by year
2026: 17 · 2025: 23 · 2024: 19 · 2023: 2
FWCI
14.45
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
National Science Foundation (2229876)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Hardware and Architecture › Physical Unclonable Functions (PUFs) and Hardware Security
Keywords
benchmark, watermark detection, tamper resistance, AI-generated text detection, LLM misuse

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review