The DOI matches its DataCite record; 54 of the 64 references it cites were found in a registry or an index.
A Watermark for Large Language Models
Open accessAbstract
Potential harms of large language models can be mitigated by watermarking model output, i.e., embedding signals into generated text that are invisible to humans but algorithmically detectable from a short span of tokens. We propose a watermarking framework for proprietary language models. The watermark can be embedded with negligible impact on text quality, and can be detected using an efficient open-source algorithm without access to the language model API or parameters. The watermark works by selecting a randomized set of "green" tokens before a word is generated, and then softly promoting use of green tokens during sampling. We propose a statistical test for detecting the watermark with interpretable p-values, and derive an information-theoretic framework for analyzing the sensitivity of the watermark. We test the watermark using a multi-billion parameter model from the Open Pretrained Transformer (OPT) family, and discuss robustness and security.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“more robust detectors [8, 14, 36, 1] require recursive paraphrasing to be effectively bypassed.”
From Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text (2025) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, watermarking presents challenges for proprietary LLMs and third-party applications due to the necessity of accessing the language model logits [Kirchenbauer et al., 2023].”
From Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges (2025) · cites this paper as Kirchenbauer et al., 2023
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Preliminary evidence points to the possibility of model collapse, with the models losing diversity and converging to a state with low variance 162 .”
From Machine culture (2023) · cites this paper as [162]
Experimental — accuracy not yet measuredIn the citing paper
“Despite efforts to watermark machine-created content 161 , machine-made and human-made content will increasingly intertwine 162 .”
From Machine culture (2023) · cites this paper as [162]
In the citing paper
“The human element in this cycle may prove crucial-for instance, in preventing 'model collapse', a dynamic where repeated training on machine-generated data narrows its outputs to very few traits 162 .”
From Machine culture (2023) · cites this paper as [162]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“…via hierarchical language models in Wilson et al. (2014) still required human interactions, the emergence of modern neural language models (Vaswani et al., 2017; Devlin et al., 2019) also allowed for improved watermarking/steganography (Fang et al., 2017; Ziegler et al., 2019; Dai & Cai,…”
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“(2014) still required human interactions, the emergence of modern neural language models (Vaswani et al., 2017; Devlin et al., 2019) also allowed for improved watermarking/steganography (Fang et al.”
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 2×
“Typical vocabularies contain |V| = 50 , 000 tokens or more (Radford et al., 2019; Liu et al., 2019).”
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 2×
“3B (Zhang et al., 2022) using prompts from the C4 dataset’s RealNewsLike subset (Raffel et al., 2019).”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“A watermark is a hidden pattern in text that is imperceptible to humans, while making the text algorithmically identifiable as synthetic.”
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117 · cited 1×
“Note that instruction finetuning is already common (for example in ChatGPT) for other categories of malicious prompts, using reinforcement learning protocols (RLHF) (Christiano et al., 2017; Ouyang et al., 2022; Bai et al., 2022).”
Petitcolas, F., Anderson, R. J., & Kuhn, M. (1999) · Proceedings of the IEEE · doi:10.1109/5.771065 · Cited by 2,996 · cited 1×
“Watermarking is considered easier for continuous-valued data, where watermarks can be encoded with a variety of well-studied strategies (Petitcolas et al., 1999; Zhu et al., 2018; Lu et al., 2021; Boenisch, 2021).”
Research works printed but not linked: 7
- Foltýnek, T., Meuschke, N., and Gipp, B. Academic Plagiarism Detection: A Systematic Literature Review. ACM Computing Surveys, 52(6):112:1-112:42, October 2019. ISSN 0360-0300. doi: 10.1145/3345317. URL https://doi.org/10.1145/3345317. · DOI · Paper page
- Katzenbeisser, S. and Petitcolas, F. A. Information Hiding Techniques for Steganography and Digital Watermarking. Artech House, Inc., USA, 1st edition, 2000. ISBN 978-1- 58053-035-4.
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language Models are Unsupervised Multi- task Learners. OpenAI, pp. 24, 2019.
- Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv e-prints, 2019.
- Tian, E. Gptzero update v1, January 2023. URL https: //gptzero.substack.com/p/gptzero- update-v1. · DOI
- Venugopal, A., Uszkoreit, J., Talbot, D., Och, F., and Ganitkevitch, J. Watermarking the Outputs of Struc- tured Prediction with an application in Statistical Ma- chine Translation. In Proceedings of the 2011 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pp. 1363-1372, Edinburgh, Scotland, UK., July 2011. Association for Computational Linguistics. URL https://aclanthology.org/D11-1126.
- Curran Associates, Inc., 2019. URL https://proceedings. neurips.cc/paper/2019/hash/ 3e9f0fc9b2f89e043bc6233994dfcf76- Abstract.html.
Cited by
Sorted by citations
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
Regulating ChatGPT and other Large Generative AI Models
Hacker, P., Engel, A., & Mauer, M. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594067 · Cited by 466
“Importantly, more interdisciplinary research is necessary to develop markings that are easy to use and recognize, but hard to remove by average users [139].”
BackgroundA Categorical Archive of ChatGPT Failures
Borji, A. (2023) · doi:10.21203/rs.3.rs-2895792/v1 · Cited by 514
Can Generative AI improve social science?
Bail, C. A. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314021121 · Cited by 369
AI Deception: A Survey of Examples, Risks, and Potential Solutions
Park, P. S., Goldstein, S., O'Gara, A., Chen, M., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2308.14752 · Cited by 243
Artificial intelligence risk management framework :
National Institute of Standards and Technology (US) (2024) · NIST Trustworthy and Responsible AI · doi:10.6028/nist.ai.600-1 · Cited by 274
AI deception: A survey of examples, risks, and potential solutions
Park, P. S., Goldstein, S., O’Gara, A., Chen, M., & Hendrycks, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100988 · Cited by 243
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212
Can AI-Generated Text be Reliably Detected?
Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., & Feizi, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11156 · Cited by 810
“In this section, we discuss paraphrasing attacks that can degrade type-II errors of state-of-the-art AI text detectors such as soft watermarking [1], zero-shot detectors [2], trained neural network-based detectors [3], and retrieval-based detectors [4].”
BackgroundMethodInfluentialAdversarial machine learning :
Vassilev, A., Oprea, A., Fordyce, A. J., & Anderson, H. S. (2024) · doi:10.6028/nist.ai.100-2e2023 · Cited by 230
A Survey on the Detection and Impacts of Deepfakes in Visual, Audio, and Textual Formats
Mubarak, R., Alsboui, T., Alshaikh, O., Inuwa-Dutse, I., Khan, S., & Parkinson, S. (2023) · IEEE Access · doi:10.1109/access.2023.3344653 · Cited by 132
“This approach was recently revolutionised by Kirchenbauer et al. [255] In 2023, they introduced a novel approach by developing a watermarking method tailored for Large Language Models (LLMs).”
MethodScalable watermarking for identifying large language model outputs
Dathathri, S., See, A., Ghaisas, S., Huang, P.-S., McAdam, R., Welbl, J., Bachani, V., Kaskasoli, A., Stanforth, R., Matejovicova, T., et al. (2024) · Nature · doi:10.1038/s41586-024-08025-4 · Cited by 132
Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132
“Selection occurs when certain cultural traits are more likely to be adopted by individuals due to factors such as their usefulness,”
BackgroundThe Science of Detecting LLM-Generated Text
Tang, R., Chuang, Y.-N., & Hu, X. (2024) · Communications of the ACM · doi:10.1145/3624725 · Cited by 125
Language Model Behavior: A Comprehensive Survey
Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105
“posed “watermarking” model-generated text by slightly increasing the probabilities of “whitelist” tokens during text generation (Kirchenbauer et al. 2023), or by explicitly replacing some tokens with whitelist tokens (He et al.”
BackgroundHadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v4 · Cited by 193
Distinguishing Reality from AI: Approaches for Detecting Synthetic Content
Ghiurău, D., & Popescu, D. E. (2024) · Computers · doi:10.3390/computers14010001 · Cited by 122
Self-consuming generative models go MAD
Casco-Rodriguez, J., Alemohammad, S., Luzi, L., Imtiaz, A., Babaei, H., LeJeune, D., Siahkoohi, A., & Baraniuk, R. (2023) · LatinX in AI at Neural Information Processing Systems Conference 2023 · doi:10.52591/lxai202312101 · Cited by 83
“of watermarking of data generated by LLMs [75] and novel methods for watermarking LLMs [76], diffusion models [77–80], andn GANs [81] are currently active areas of research.”
MethodMAGE: Machine-generated Text Detection in the Wild
Li, Y., Li, Q., Cui, L., Bi, W., Wang, Z., Wang, L., Yang, L., Shi, S., & Zhang, Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.3 · Cited by 79
The Gradient of Generative AI Release: Methods and Considerations
Solaiman, I. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3593981 · Cited by 90
“Research strives ensure these watermarks are invisible to the human-eye [107], do not affect output quality [46], and tamper-proof from model attacks and alterations like fine-tuning via methods such as embedding noise as watermarks [106].”
Friend or foe? Exploring the implications of large language models on the science system
Fecher, B., Hebing, M., Laufer, M., Pohle, J., & Sofsky, F. (2025) · AI & SOCIETY · doi:10.1007/s00146-023-01791-1 · Cited by 78
Veselovsky, V., Ribeiro, M. H., & West, R. (2023) · arXiv · doi:10.48550/arxiv.2306.07899 · Cited by 73
“These concerns have led to work on watermarking LLM output (Kirchenbauer et al., 2023) by slightly altering the odds of each token during sampling, and to further work on methods for improving the detection of synthetically generated text (Yu et al., 2023; Verma et al., 2023; Diwan et al., 2021).”
Anatomy of an AI-powered malicious social botnet
Yang, K., & Menczer, F. (2024) · Journal of Quantitative Description: Digital Media · doi:10.51685/jqd.2024.icwsm.7 · Cited by 71
“White-box methods, on the other hand, require LLM owners to embed specific signals or watermarks (e.g., altered word frequencies) into generated content for subsequent identification (Kirchenbauer et al., 2023; Zhao et al., 2023).”
BackgroundLiang, W., Izzo, Z., Zhang, Y., Lepp, H., Cao, H., Zhao, X., Chen, L., Ye, H., Liu, S., Huang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07183 · Cited by 73
“Kirchenbauer et al. (2023) split the vocabulary into red-green lists based on hash values of previous n-grams and then increase the logits of green to-kens to embed the watermark.”
Ibrahim, K. (2023) · Language Testing in Asia · doi:10.1186/s40468-023-00260-2 · Cited by 66
“…NLP has explored different approaches to detecting AI-generated texts (e.g., Bakhtin et al., 2019), including watermarking AI-generated texts (e.g., Kirchenbauer et al., 2023), detecting traces of language modeling in generated texts (e.g., Tay et al., 2020), and training AI models to…”
BackgroundMethodInitial policy considerations for generative artificial intelligence
Lorenz, P., Perset, K., & Berryhill, J. (2023) · OECD artificial intelligence papers · doi:10.1787/fae2d1e6-en · Cited by 123
Can Generative AI Improve Social Science?
Bail, C. A. (2023) · doi:10.31235/osf.io/rwtzs · Cited by 369
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text
Su, J., Zhuo, T. Y., Wang, D., & Nakov, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.827 · Cited by 192
Identifying and Mitigating the Security Risks of Generative AI
Barrett, C., Boyd, B., Bursztein, E., Carlini, N., Chen, B., Choi, J., Chowdhury, A. R., Christodorescu, M., Datta, A., Feizi, S., et al. (2023) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000041 · Cited by 57
“A recent line of work tries this specifically for AI-generated text [47, 46, 103, 21, 50].”
BackgroundInfluentialPeople cannot distinguish GPT-4 from a human in a Turing test
Jones, C. R., Rathi, I., Taylor, S., & Bergen, B. K. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732108 · Cited by 76
Xie, Y., Wu, S., & Chakravarty, S. (2023) · Conference on Information Technology Education · doi:10.1145/3585059.3611449 · Cited by 54
“Embedding subtle fingerprints to text parts of programming code using similar strategies described in [46] could be a starting point to explore.”
MethodHow can artificial intelligence transform the training of medical students and physicians?
Ning, Y., Ong, J. C. L., Cheng, H., Wang, H., Ting, D. S. W., Tham, Y. C., Wong, T. Y., & Liu, N. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100900 · Cited by 62
Detecting AI-Generated Code Assignments Using Perplexity of Large Language Models
Xu, Z., & Sheng, V. S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i21.30361 · Cited by 59
SeqXGPT: Sentence-Level AI-Generated Text Detection
Wang, P., Li, L., Ren, K., Jiang, B., Zhang, D., & Qiu, X. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.73 · Cited by 53
Koike, R., Kaneko, M., & Okazaki, N. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30120 · Cited by 77
Giray, L., Sevnarayan, K., & Ranjbaran, F. (2025) · Internet Reference Services Quarterly · doi:10.1080/10875301.2024.2437174 · Cited by 62
“…of texts and perform thresholding to detect AI, may not perform well on texts with varying complexity or style (Solaiman et al., 2019), and watermarking approaches (imprinting patterns in the text) may not be robust to paraphrasing or other attacks (Kirchenbauer et al., 2023; Zhao et al., 2023).”
GPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · arXiv · doi:10.48550/arxiv.2304.02819 · Cited by 40
Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges
Huang, B., Chen, C., & Shu, K. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3715073.3715076 · Cited by 42
Robust Multi-bit Natural Language Watermarking through Invariant Features
Yoo, K., Ahn, W., Jang, J., & Kwak, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.117 · Cited by 43
“Similarly, Kirchenbauer et al. (2023) propose to watermark outputs of language models at decoding time in a zero-bit manner to distinguish machine-generated texts from human-written text.”
BackgroundDianova, V. G., & Schultz, M. D. (2023) · Industry and Higher Education · doi:10.1177/09504222231199989 · Cited by 48
Transformers: A Security Perspective
Latibari, B. S., Nazari, N., Chowdhury, M. A., Gubbi, K. I., Fang, C., Ghimire, S., Hosseini, E., Sayadi, H., Homayoun, H., Salehi, S., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3509372 · Cited by 43
“Authors in [82] introduced a watermarking framework for FIGURE 14.”
Jacovi, A., Caciularu, A., Goldman, O., & Goldberg, Y. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.308 · Cited by 36
Jacovi, A., Caciularu, A., Goldman, O., & Goldberg, Y. (2023) · arXiv · doi:10.48550/arxiv.2305.10160 · Cited by 36
Dual use concerns of generative AI and large language models
Grinbaum, A., & Adomaitis, L. (2024) · Journal of Responsible Innovation · doi:10.1080/23299460.2024.2304381 · Cited by 47
“Risks arising from fraud or manipulation, which comprise a large spectrum of societal risks, imply that users should not mistake an output produced by a machine for an output created by a human author (Aaronson 2022; Grinbaum and Adomaitis 2022b; Kirchenbauer et al. 2023).”
BackgroundIntrinsic Dimension Estimation for Robust Detection of AI-Generated Texts
Tulchinskii, E., Kuznetsov, K., Kushnareva, L., Cherniavskii, D., Barannikov, S., Piontkovskaya, I., Nikolenko, S., & Burnaev, E. (2023) · arXiv · doi:10.48550/arxiv.2306.04723 · Cited by 43
“Another idea, known as watermarking , is to inject some detectable artifacts into model generations; for instance, Kirchenbauer et al. [2023] propose to intentionally inject a statistical skew that can be detected in a text sample.”
BackgroundModelShield: Adaptive and Robust Watermark Against Model Extraction Attack
Pang, K., Qi, T., Wu, C., Bai, M., Jiang, M., & Huang, Y. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3530691 · Cited by 29
LLMDet: A Third Party Large Language Models Generated Text Detection Tool
Wu, K., Pang, L., Shen, H., Cheng, X., & Chua, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.139 · Cited by 64
“The current state-of-the-art approach, as proposed by (Kirchenbauer et al., 2023), partitions the model’s vocabulary into whitelist and blacklist tokens when predicting the next token given a prompt.”
MethodInfluentialWhen deep learning meets watermarking: A survey of application, attacks and defenses
Chen, H., Liu, C., Zhu, T., & Zhou, W. (2024) · Computer Standards & Interfaces · doi:10.1016/j.csi.2023.103830 · Cited by 28
Three Bricks to Consolidate Watermarks for Large Language Models
Fernandez, P., Chaffin, A., Tit, K., Chappelier, V., & Furon, T. (2023) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs58808.2023.10374576 · Cited by 63
“Empirical Validation of FPR with Z-Scores So far, the FPR was checked on small scale experiments: around 500 negative samples [18], [19], [21].”
BackgroundMethodInfluentialPromptCARE: Prompt Copyright Protection by Watermark Injection and Verification
Yao, H., Lou, J., Qin, Z., & Ren, K. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00209 · Cited by 32
“Language model watermarking [17], [18], [19], [20], [31], [32], [33], [34] is a technique used to embed a unique signature into the generated output of a language model.”
BackgroundMethodM4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection
Wang, Y., Mansurov, J., Ivanov, P., Su, J., Shelmanov, A., Tsvigun, A., Mohammed Afzal, O., Mahmoud, T., Puccetti, G., Arnold, T., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.218 · Cited by 25
A Pathway Towards Responsible AI Generated Content
Lyu, L. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/803 · Cited by 107
ProMark: Proactive Diffusion Watermarking for Causal Attribution
Asnani, V., Collomosse, J. P., Bui, T., Liu, X., & Agarwal, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01027 · Cited by 31
“[19] propose a watermarking method for language models by pre-selecting random tokens and subtly influencing their use during word generation.”
Fu, Y., Xiong, D., & Dong, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29756 · Cited by 37
Data Authenticity, Consent, and Provenance for AI Are All Broken: What Will It Take to Fix Them?
Longpre, S., Mahari, R., Obeng-Marnu, N., Brannon, W., South, T., Kabbara, J., & Pentland, A. (2024) · doi:10.21428/e4baedd9.a650f77d · Cited by 28
Detection of AI-Generated Text Using Large Language Model
Prajapati, M., Baliarsingh, S. K., Dora, C., Bhoi, A., Hota, J., & Mohanty, J. P. (2024) · International Conference on Emerging Systems and Intelligent Computing (ESIC) · doi:10.1109/esic60604.2024.10481602 · Cited by 25
“Various methodologies have been developed to detect AI-generated text, including binary classification models, zero-shot classifiers [6], neural network-based detectors like OpenAI’s RoBERTa [7], and soft watermarking [8].”
MethodTowards Trustworthy AI: A Review of Ethical and Robust Large Language Models
Ferdaus, M. M., Abdelguerfi, M., Ioup, E., Niles, K. N., Pathak, K., & Sloan, S. D. (2025) · ACM Computing Surveys · doi:10.1145/3777382 · Cited by 106
Reclaiming the Digital Commons: A Public Data Trust for Training Data
Chan, A., Bradley, H., & Rajkumar, N. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604658 · Cited by 29
“may add noise or watermarks to their deployed models to ensure that others cannot copy the model easily [13, 44, 60].”
BackgroundM4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection
Wang, Y., Mansurov, J., Ivanov, P., Su, J., Shelmanov, A., Tsvigun, A., Whitehouse, C., Afzal, O. M., Mahmoud, T., Sasaki, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.14902 · Cited by 19
“…posthoc lexical substitutions and synonym replacement over generated outputs (Szyller et al., 2021; He et al., 2022), and soft watermarking (Kirchenbauer et al., 2023) using green and red token lists, to hidden-space operations, such as injecting secret signals into the probability vector…”
BackgroundHow to Protect Copyright Data in Optimization of Large Language Models?
Chu, T., Song, Z., & Yang, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29741 · Cited by 41
Watermarking for Large Language Models: A Survey
Yang, Z., Zhao, G., & Wu, H. (2025) · Mathematics · doi:10.3390/math13091420 · Cited by 27
AI-generated text detection: A comprehensive review of methods, datasets, and applications
Kehkashan, T., Riaz, R. A., Al‐Shamayleh, A. S., Akhunzada, A., Ali, N., Hamza, M., & Akbar, F. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100793 · Cited by 24
AI Usage Cards: Responsibly Reporting AI-Generated Content
Wahle, J. P., Ruas, T., Mohammad, S. M., Meuschke, N., & Gipp, B. (2023) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl57899.2023.00060 · Cited by 28
“While automatic detection and watermarking methods exist [9, 11], they do not address integrity and accountability issues.”
BackgroundYang, Z., Feng, Z., Huo, R., Lin, H., Zheng, H., Nie, R., & Chen, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126694 · Cited by 17
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ac4sp · Cited by 143
Mosca, E., Abdalla, M. H. I., Basso, P., Musumeci, M., & Groh, G. (2023) · Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023) · doi:10.18653/v1/2023.trustnlp-1.17 · Cited by 14
“Alternative approaches explore using the probability curvature of the generative model itself (Mitchell et al., 2023) or watermarking machinegenerated text to facilitate detection (Kirchenbauer et al., 2023).”
BackgroundMethodLongo, L., Brčić, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.19775 · Cited by 14
Publicly-Detectable Watermarking for Language Models
Fairoze, J., Garg, S., Jha, S., Mahloujifar, S., Mahmoody, M., & Wang, M. (2025) · IACR Communications in Cryptology · doi:10.62056/ahmpdkp10 · Cited by 63
Large Language Models in Healthcare Current Development and Future DirectionsPreprint
Jiang, X., Yan, L., Vavekanand, R., & Hu, M. (2024) · Preprints.org · doi:10.20944/preprints202407.0923.v1 · Cited by 13
Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection
Peng, X., Zhou, Y., He, B., Sun, L., & Sun, Y. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.644 · Cited by 16
“Current AI-generated text detection methods can be in general categorized into three categories (Lu et al., 2023): statistical (Gehrmann et al., 2019), watermarking (Kirchenbauer et al., 2023), and training-based (Gallé et al., 2021).”
BackgroundA Comparison Study on AI Language Detector
Singh, A. (2023) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc57344.2023.10099219 · Cited by 12
“This approach involves using techniques such as plagiarism detection [7], language model fine-tuning, watermarking [9] and data masking to detect and prevent misuse.”
MethodUnderstanding the effects of human-written paraphrases in LLM-generated text detection
Lau, H. T., & Zubiaga, A. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100151 · Cited by 13
Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models
Zhang, H., Edelman, B. L., Francati, D., Venturi, D., Ateniese, G., & Barak, B. (2023) · arXiv · doi:10.48550/arxiv.2311.04378 · Cited by 12
A Framework for Detecting AI-Generated Text in Research Publications
Sarzaeim, P., Doshi, A. M., & Mahmoud, Q. H. (2023) · Proceedings of the International Conference on Advanced Technologies · doi:10.58190/icat.2023.28 · Cited by 11
Watermarking LLMs with Weight Quantization
Li, L., Jiang, B., Wang, P., Ren, K., Yan, H., & Qiu, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.220 · Cited by 32
On the Zero-Shot Generalization of Machine-Generated Text Detectors
Xiao, P., Zhang, J., Han, X., Tsvetkov, Y., & He, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.318 · Cited by 24
On the Societal Impact of Open Foundation Models
Kapoor, S., Bommasani, R., Klyman, K., Longpre, S., Ramaswami, A., Cihon, P., Hopkins, A., Bankston, K., Biderman, S., Bogen, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07918 · Cited by 103
How Reliable Are AI-Generated-Text Detectors? An Assessment Framework Using Evasive Soft Prompts
Kumarage, T., Sheth, P., Moraffah, R., Garland, J., & Liu, H. (2023) · arXiv · doi:10.48550/arxiv.2310.05095 · Cited by 11
Muthukumar, K. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1680552 · Cited by 10
“Human responses are usually more random and varied, yet technological improvements make distinguishing between humans and AI more difficult (Kirchenbauer et al., 2023).”
SrcMarker: Dual-Channel Source Code Watermarking via Scalable Code Transformations
Yang, B., Li, W., Xiang, L., & Li, B. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00097 · Cited by 15
A Brief Survey of Watermarks in Generative AI
Hwang, J., & Oh, S. (2023) · Information and Communication Technology Convergence · doi:10.1109/ictc58733.2023.10392465 · Cited by 11
“According to The New York Times [6], OpenAI appears to have adopted a watermarking approach similar to the one outlined in the paper [5], although specific details have not been publicly disclosed.”
MethodAI Usage Cards: Responsibly Reporting AI-generated ContentPreprint
Wahle, J. P., Ruas, T., Mohammad, S. M., Meuschke, N., & Gipp, B. (2023) · doi:10.22541/au.168323192.20543041/v1 · Cited by 10
Enhancing the Robustness of AI-Generated Text Detectors: A Survey
Liu, X., Li, Y., & Li, K. (2025) · Mathematics · doi:10.3390/math13132145 · Cited by 10
An explainable framework for assisting the detection of AI-generated textual content
Yan, S., Wang, Z., & Dobolyi, D. G. (2025) · Decision Support Systems · doi:10.1016/j.dss.2025.114498 · Cited by 12
Gutub, A. A. A., & Almehmadi, E. (2023) · Journal of Engineering Research · doi:10.1016/j.jer.2023.100099 · Cited by 9
Pudasaini, S., Miralles-Pechuán, L., Lillis, D., & Salvador, M. L. (2024) · arXiv · doi:10.48550/arxiv.2407.13105 · Cited by 8
“Along with these results, Zhao et al. [104] also proposed another watermarking solution called Unigram-Watermark, which outperformed the previous solution proposed by Kirchenbauer et al. [103] for paraphrasing attacks.”
MethodNational Institute of Standards and Technology (US) (2024) · doi:10.6028/nist.ai.100-4 · Cited by 9
GhostCite: A Large-Scale Analysis of Citation Validity in the Age of Large Language Models
Xu, Z., Qiu, Y., Sun, L., Miao, F., Wu, F., Li, X., Wang, X., Lu, H., Zhang, Z., Hu, Y., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.06718 · Cited by 3
“Detection efforts have emerged: GPTZero reported hallucinated citations in both ICLR 2026 submissions and NeurIPS 2025 accepted papers [9], [10], while watermarking [32] and zero-shot detection based on probability curvature [33] offer complementary strategies.”
LLMs beyond the lab: the ethics and epistemics of real-world AI research
Mollen, J. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09819-w · Cited by 11
“…for example, which protocols or normative frameworks can best guide a responsible and effective use of the AI system or what addi - tions might be necessary to secure responsible embedding, such as digital watermarks to algorithmically identify AI-generated content (Kirchenbauer et al., 2023).”
How Reliable Are AI-Generated-Text Detectors? An Assessment Framework Using Evasive Soft Prompts
Kumarage, T., Sheth, P., Moraffah, R., Garland, J., & Liu, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.94 · Cited by 27
AIRABIC: Arabic Dataset for Performance Evaluation of AI Detectors
Alshammari, H., & EI-Sayed, A. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00127 · Cited by 8
“In response to the potential misuse of the AI generative models, some recently proposed algorithms identify AI-generated text, such as [9, 10].”
Background(2023) · doi:10.21203/rs.3.rs-3226684/v1 · Cited by 8
GPTs Don’t Keep Secrets: Searching for Backdoor Watermark Triggers in Autoregressive Language Models
Lucas, E., & Havens, T. C. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.21 · Cited by 7
“Watermarking could be used to show that text was machine generated or to prove ownership of a model; this is generally done in a way that is subtle and not apparent to a human observer (Topkara et al., 2006; Kirchenbauer et al., 2023; Grinbaum and Adomaitis, 2022).”
BackgroundAuthorship Verification in software engineering education
Rios, S., Zhang, Y., & de Oliveira, E. A. (2023) · ASCILITE Publications · doi:10.14742/apubs.2023.559 · Cited by 7
“Another avenue to AI text generation is “watermarking” the text to ease detection by incorporating hidden patterns in the text that are unnoticeable and do not affect text quality, but algorithmically identify the text as synthetic (Kirchenbauer et al., 2023).”
BackgroundMethodWhere Am I From? Identifying Origin of LLM-generated Content
Li, L., Bai, Y., & Cheng, M. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.681 · Cited by 7
Robustness of generative AI detection: adversarial attacks on black-box neural text detectors
Fishchuk, V., & Braun, D. (2024) · International Journal of Speech Technology · doi:10.1007/s10772-024-10144-2 · Cited by 7
A Multimodal Approach for Detecting AI Generated Content using BERT and CNN
Vora, E. A. V. (2023) · International Journal on Recent and Innovation Trends in Computing and Communication · doi:10.17762/ijritcc.v11i9.8861 · Cited by 6
HANSEN: Human and AI Spoken Text Benchmark for Authorship Analysis
Tripto, N., Uchendu, A., Le, T., Setzu, M., Giannotti, F., & Lee, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.916 · Cited by 6
CurveMark: Detecting AI-Generated Text via Probabilistic Curvature and Dynamic Semantic Watermarking
Zhang, Y., Jiang, X., Sun, H., Zhang, Y., & Tong, D. (2025) · Entropy · doi:10.3390/e27080784 · Cited by 6
Metadata
- Authors
- Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T.
- Affiliations
- University of Maryland, College Park (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2301.10226
- OpenAlex
- W4318149317
- Cited by
- 194
- References
- 64
- Citations by year
- 2026: 4 · 2025: 73 · 2024: 64 · 2023: 53
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2212182, 2229885)
- Open Philanthropy Project
- Defense Advanced Research Projects Agency (HR00112020007)
- Multidisciplinary University Research Initiative
- Office of Naval Research (N000142112557)
- Air Force Office of Scientific Research
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Hardware and Architecture › Physical Unclonable Functions (PUFs) and Hardware Security
- Keywords
- watermarking, large language models, watermark detection, hypothesis testing, information-theoretic analysis, watermark robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
