A Watermark for Large Language Models
Open accessAbstract
Potential harms of large language models can be mitigated by watermarking model output, i.e., embedding signals into generated text that are invisible to humans but algorithmically detectable from a short span of tokens. We propose a watermarking framework for proprietary language models. The watermark can be embedded with negligible impact on text quality, and can be detected using an efficient open-source algorithm without access to the language model API or parameters. The watermark works by selecting a randomized set of "green" tokens before a word is generated, and then softly promoting use of green tokens during sampling. We propose a statistical test for detecting the watermark with interpretable p-values, and derive an information-theoretic framework for analyzing the sensitivity of the watermark. We test the watermark using a multi-billion parameter model from the Open Pretrained Transformer (OPT) family, and discuss robustness and security.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
More robust detectors [8, 14, 36, 1] require recursive paraphrasing to be effectively bypassed.
“more robust detectors [8, 14, 36, 1] require recursive paraphrasing to be effectively bypassed.”
From Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text (2025) · cites this paper as [14]
The paper’s “Paraphrasing Attacks” section says paraphrasing can defeat watermarks, while noting that automated paraphrasing trades reduced watermark strength for reduced fluency. “Details of the T5 Span Attack” reports more successful watermark removal at a larger replacement budget. Neither passage says recursive paraphrasing is required to bypass the detector. (LLM, conf=0.82)
Experimental — accuracy not yet measuredWatermarking presents challenges for proprietary LLMs and third-party applications due to the necessity of accessing the language model logits.
“Additionally, watermarking presents challenges for proprietary LLMs and third-party applications due to the necessity of accessing the language model logits [Kirchenbauer et al., 2023].”
From Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges (2025) · cites this paper as Kirchenbauer et al., 2023
Section 3 ('A more sophisticated watermark'; page unavailable) says embedding this watermark changes the model’s logits during generation, which supports the access concern for a third party trying to add it to a proprietary model. But the Introduction says third parties can detect the watermark without model parameters or API access, so the claimed difficulty does not apply to third-party detection; the paper also proposes its framework for proprietary models. (LLM, conf=0.88)
Experimental — accuracy not yet measuredDespite efforts to watermark machine-created content, machine-made and human-made content will increasingly intertwine.
“Despite efforts to watermark machine-created content 161 , machine-made and human-made content will increasingly intertwine 162 .”
From Machine culture (2023) · cites this paper as [162]
The human element in this cycle may prove crucial-for instance, in preventing 'model collapse', a dynamic where repeated training on machine-generated data narrows its outputs to very few traits.
“The human element in this cycle may prove crucial-for instance, in preventing 'model collapse', a dynamic where repeated training on machine-generated data narrows its outputs to very few traits 162 .”
From Machine culture (2023) · cites this paper as [162]
Preliminary evidence points to the possibility of model collapse, with the models losing diversity and converging to a state with low variance.
“Preliminary evidence points to the possibility of model collapse, with the models losing diversity and converging to a state with low variance 162 .”
From Machine culture (2023) · cites this paper as [162]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1× · see citing works
“…via hierarchical language models in Wilson et al. (2014) still required human interactions, the emergence of modern neural language models (Vaswani et al., 2017; Devlin et al., 2019) also allowed for improved watermarking/steganography (Fang et al., 2017; Ziegler et al., 2019; Dai & Cai,…”
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · cited 2× · see citing works
“(2014) still required human interactions, the emergence of modern neural language models (Vaswani et al., 2017; Devlin et al., 2019) also allowed for improved watermarking/steganography (Fang et al.”
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 2× · see citing works
“Typical vocabularies contain |V| = 50 , 000 tokens or more (Radford et al., 2019; Liu et al., 2019).”
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 2× · see citing works
“3B (Zhang et al., 2022) using prompts from the C4 dataset’s RealNewsLike subset (Raffel et al., 2019).”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1× · see citing works
“A watermark is a hidden pattern in text that is imperceptible to humans, while making the text algorithmically identifiable as synthetic.”
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347 · see citing works
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117 · cited 1× · see citing works
“Note that instruction finetuning is already common (for example in ChatGPT) for other categories of malicious prompts, using reinforcement learning protocols (RLHF) (Christiano et al., 2017; Ouyang et al., 2022; Bai et al., 2022).”
Petitcolas, F., Anderson, R. J., & Kuhn, M. (1999) · Proceedings of the IEEE · doi:10.1109/5.771065 · Cited by 2,996 · cited 1× · see citing works
“Watermarking is considered easier for continuous-valued data, where watermarks can be encoded with a variety of well-studied strategies (Petitcolas et al., 1999; Zhu et al., 2018; Lu et al., 2021; Boenisch, 2021).”
Robust Speech Recognition via Large-Scale Weak Supervision
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2022) · arXiv · doi:10.48550/arxiv.2212.04356 · Cited by 1,953 · cited 1× · see citing works
“Furthermore, the proliferation of synthetic data on the web complicates future dataset creation efforts, as synthetic data is often inferior to human content and must be detected and excluded before model training (Radford et al., 2022).”
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · see citing works
HiDDeN: Hiding Data With Deep Networks
Zhu, J., Kaplan, R., Johnson, J., & Fei-Fei, L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_40 · Cited by 1,186 · cited 1× · see citing works
“Watermarking is considered easier for continuous-valued data, where watermarks can be encoded with a variety of well-studied strategies (Petitcolas et al., 1999; Zhu et al., 2018; Lu et al., 2021; Boenisch, 2021).”
Eadie, J. (2001) · doi:10.1300/j159v02n01_02 · Cited by 1,063 · cited 1×
“Large language models (LLMs), such as the recently developed ChatGPT, can write documents, create executable code, and answer questions, often with human-like capabilities (Schulman et al., 2022).”
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · cited 1× · see citing works
“Note that instruction finetuning is already common (for example in ChatGPT) for other categories of malicious prompts, using reinforcement learning protocols (RLHF) (Christiano et al., 2017; Ouyang et al., 2022; Bai et al., 2022).”
Defending Against Neural Fake News
Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1905.12616 · Cited by 569 · cited 1× · see citing works
“…to watermarking is to develop detection models that perform a post-hoc analysis of machine-generated text, for example using language model features or finetuning existing large language models to behave as detectors (Zellers et al., 2019; Tan et al., 2020), see an overview in Jawahar et al. (2020).”
Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by BackdooringPreprint
Adi, Y., Baum, C., Cissé, M., Benny Pinkas, B., & Keshet, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04633 · Cited by 395 · cited 1× · see citing works
“3 , the average PPL of attacked sequences increases by 3 × in addition to requiring more model calls.”
Crosslingual Generalization through Multitask Finetuning
Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Le Scao, T., Bari, M. S., Shen, S., Yong, Z. X., Schoelkopf, H., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.891 · Cited by 361 · cited 1× · see citing works
“Consider the following process for creating the lists.”
Large-capacity Image Steganography Based on Invertible Neural Networks
Lu, S.-P., Wang, R., Zhong, T., & Rosin, P. L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01067 · Cited by 299 · see citing works
Natural Language Watermarking and Tamperproofing
Atallah, M., Raskin, V., Hempelmann, C. F., et al. (2002) · Information Hiding · doi:10.1007/3-540-36415-3_13 · Cited by 222
Natural Language Watermarking: Design, Analysis, and a Proof-of-Concept Implementation
Atallah, M. J., Raskin, V., Crogan, M., Hempelmann, C. F., Kerschbaum, F., Mohamed, D., & Naik, S. (2001) · Lecture notes in computer science · doi:10.1007/3-540-45496-9_14 · Cited by 201 · cited 1× · see citing works
“3 , the average PPL of attacked sequences increases by 3 × in addition to requiring more model calls.”
Machine-Generated Text: A Comprehensive Survey of Threat Models and Detection Methods
Crothers, E. N., Japkowicz, N., & Viktor, H. L. (2023) · IEEE Access · doi:10.1109/access.2023.3294090 · Cited by 196 · cited 1× · see citing works
“For many reasons, the ability to detect and audit the usage of machine-generated text becomes a key principle of harm reduction for large language models (Crothers et al., 2022; Grinbaum & Adomaitis, 2022).”
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods
Crothers, E., Japkowicz, N., & Viktor, H. (2022) · arXiv · doi:10.48550/arxiv.2210.07321 · Cited by 196 · see citing works
UL2: Unifying Language Learning ParadigmsPreprint
Tay, Y., Dehghani, M., Tran, V. Q., Garcia, X., Wei, J., Wang, X., Chung, H. W., Shakeri, S., Bahri, D., Schuster, T., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.05131 · Cited by 194 · see citing works
Neural Linguistic Steganography
Ziegler, Z. M., Deng, Y., & Rush, A. M. (2019) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d19-1115 · Cited by 189 · cited 1× · see citing works
“…via hierarchical language models in Wilson et al. (2014) still required human interactions, the emergence of modern neural language models (Vaswani et al., 2017; Devlin et al., 2019) also allowed for improved watermarking/steganography (Fang et al., 2017; Ziegler et al., 2019; Dai & Cai, 2019).”
Generating Steganographic Text with LSTMs
Fang, T., Jaggi, M., & Argyraki, K. (2017) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/p17-3017 · Cited by 161 · cited 4× · see citing works
“Their preliminary method is based only on biasing of the LM output, as opposed to complete determination as in Fang et al. (2017).”
InfluentialAutomatic Detection of Machine Generated Text: A Critical Survey
Jawahar, G., Abdul-Mageed, M., & Lakshmanan, V. S. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.208 · Cited by 156 · cited 1× · see citing works
“…to watermarking is to develop detection models that perform a post-hoc analysis of machine-generated text, for example using language model features or finetuning existing large language models to behave as detectors (Zellers et al., 2019; Tan et al., 2020), see an overview in Jawahar et al. (2020).”
Gabrilovich, E., & Gontmakher, A. (2002) · CACM · doi:10.1145/503124.503156 · Cited by 153 · cited 1× · see citing works
“The effect of tokenization attacks can be multiplied through homoglyph attacks (Gabrilovich & Gontmakher, 2002).”
The hiding virtues of ambiguity
Topkara, U., Topkara, M., & Atallah, M. J. (2006) · ACM workshop on Multimedia and security · doi:10.1145/1161366.1161397 · Cited by 152 · see citing works
Natural language watermarking via morphosyntactic alterations
Meral, H. M., Sankur, B., Sumru Özsoy, A., Güngör, T., & Sevinç, E. (2009) · Computer Speech & Language · doi:10.1016/j.csl.2008.04.001 · Cited by 113 · cited 2× · see citing works
“Rule-based watermarks were further improved in a series of works (Chiang et al., 2004; Topkara et al., 2006a;b; Meral et al., 2009) with variants also embedding watermarks based on synonym tables instead of only parse trees.”
CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks
He, X., Xu, Q., Zeng, Y., Lyu, L., Wu, F.-Z., Li, J.-W., & Jia, R. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2209.08773 · Cited by 106 · cited 1×
“, 2019) also allowed for improved watermarking/steganography (Fang et al., 2017; Ziegler et al., 2019; Dai & Cai, 2019; He et al., 2022).”
Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding
Abdelnabi, S., & Fritz, M. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00083 · Cited by 105 · cited 1× · see citing works
“Finally, Abdelnabi & Fritz (2021) design an end-to-end system where both encoding and decoding are handled by text-to-text language models that are trained adversarially for this purpose.”
Bad Characters: Imperceptible NLP Attacks
Boucher, N., Shumailov, I., Anderson, R., & Papernot, N. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833641 · Cited by 102 · cited 1× · see citing works
“Like homoglyphs, these characters must be removed through canonicalization (Pajola & Conti, 2021; Boucher et al., 2022).”
The Threat of Offensive AI to Organizations
Mirsky, Y., Demontis, A., Kotak, J., Shankar, R., Gelei, D., Yang, L., Zhang, X., Pintor, M., Lee, W., Elovici, Y., et al. (2023) · Computers & Security · doi:10.1016/j.cose.2022.103006 · Cited by 97 · cited 1× · see citing works
“As these systems become more pervasive, there is increasing risk that they may be used for malicious purposes (Bergman et al., 2022; Mirsky et al., 2023).”
A Systematic Review on Model Watermarking for Neural Networks
Boenisch, F. (2021) · Frontiers in Big Data · doi:10.3389/fdata.2021.729663 · Cited by 91 · cited 3× · see citing works
“This would not be used to watermark model output (as in this work), but to defend against model stealing (Adi et al., 2018; Boenisch, 2021).”
BackgroundMethodTowards Near-imperceptible Steganographic Text
Dai, F. Z., & Cai, Z. J. (2019) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1422 · Cited by 82 · cited 1× · see citing works
“…via hierarchical language models in Wilson et al. (2014) still required human interactions, the emergence of modern neural language models (Vaswani et al., 2017; Devlin et al., 2019) also allowed for improved watermarking/steganography (Fang et al., 2017; Ziegler et al., 2019; Dai & Cai, 2019).”
A Review of Digital Watermarking Techniques for Text Documents
Jalil, Z., & Mirza, A. M. (2009) · International Conference on Information and Multimedia Technology · doi:10.1109/icimt.2009.11 · Cited by 81 · cited 1× · see citing works
“Early developments were summarized in Jalil & Mirza (2009), but strong watermarks significantly degraded the text quality due to the limited capacity for natural language understanding at the time.”
Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News
Tan, R., Plummer, B. A., & Saenko, K. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.163 · Cited by 62 · cited 1× · see citing works
“…to watermarking is to develop detection models that perform a post-hoc analysis of machine-generated text, for example using language model features or finetuning existing large language models to behave as detectors (Zellers et al., 2019; Tan et al., 2020), see an overview in Jawahar et al. (2020).”
Meteor: Cryptographically Secure Steganography for Realistic Distributions
Kaptchuk, G., Jois, T. M., Green, M., & Rubin, A. D. (2021) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3460120.3484550 · Cited by 58 · see citing works
Protecting Intellectual Property of Language Generation APIs with Lexical Watermark
He, X., Xu, Q., Lyu, L., Wu, F., & Wang, C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21321 · Cited by 56 · cited 1× · see citing works
“3 , the average PPL of attacked sequences increases by 3 × in addition to requiring more model calls.”
Association for Computing Machinery
Babatope, L. O. (2016) · The SAGE Encyclopedia of Online Education · doi:10.4135/9781483318332.n41 · Cited by 52 · see citing works
The Ethical Need for Watermarks in Machine-Generated LanguagePreprint
Grinbaum, A., & Adomaitis, L. (2022) · arXiv.org · doi:10.48550/arxiv.2209.03118 · Cited by 45 · cited 1× · see citing works
“For many reasons, the ability to detect and audit the usage of machine-generated text becomes a key principle of harm reduction for large language models (Crothers et al., 2022; Grinbaum & Adomaitis, 2022).”
Natural Language Watermarking Using Semantic Substitution for Chinese Text
Chiang, Y.-L., Chang, L.-P., Hsieh, W.-T., & Chen, W. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24624-4_10 · Cited by 40 · cited 1× · see citing works
“Rule-based watermarks were further improved in a series of works (Chiang et al., 2004; Topkara et al., 2006a;b; Meral et al., 2009) with variants also embedding watermarks based on synonym tables instead of only parse trees.”
Frustratingly Easy Edit-based Linguistic Steganography with a Masked Language Model
Ueoka, H., Murawaki, Y., & Kurohashi, S. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.433 · Cited by 35 · cited 1× · see citing works
“Other approaches, such as Ueoka et al. (2021) use mask-infilling models such as BERT to edit already-generated text for the purpose of steganography.”
Natural language watermarking: Challenges in building a practical system
Topkara, M., Riccardi, G., Hakkani-Tür, D. Z., & Atallah, M. J. (2006) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.643560 · Cited by 35 · cited 1× · see citing works
“Rule-based watermarks were further improved in a series of works (Chiang et al., 2004; Topkara et al., 2006a;b; Meral et al., 2009) with variants also embedding watermarks based on synonym tables instead of only parse trees.”
Fall of Giants: How popular text-based MLaaS fall against a simple evasion attack
Pajola, L., & Conti, M. (2021) · European Symposium on Security and Privacy · doi:10.1109/eurosp51992.2021.00023 · Cited by 33 · cited 1× · see citing works
“Like homoglyphs, these characters must be removed through canonicalization (Pajola & Conti, 2021; Boucher et al., 2022).”
Crosslingual Generalization through Multitask Finetuning
Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Scao, T. L., Bari, M. S., Shen, S., Yong, Z.-X., Schoelkopf, H., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.01786 · Cited by 29 · see citing works
Linguistic steganography on Twitter: hierarchical language modeling with manual interaction
Wilson, A., Blunsom, P., & Ker, A. D. (2014) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2039213 · Cited by 27 · see citing works
Reverse Engineering Configurations of Neural Text Generation Models
Tay, Y., Bahri, D., Zheng, C., Brunk, C. A., Metzler, D., & Tomkins, A. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.25 · Cited by 21 · cited 1× · see citing works
“Implementation details, such as sampling strategies, can be reverse-engineered from text (Tay et al., 2020).”
On pushing DeepFake Tweet Detection capabilities to the limits
Gambini, M., Fagni, T., Falchi, F., & Tesconi, M. (2022) · 14th ACM Web Science Conference 2022 · doi:10.1145/3501247.3531560 · Cited by 19 · cited 1× · see citing works
“However, detection approaches are slowly losing ground as LM capabilities increase, for example Gambini et al. (2022) note that a range of detection strategies for GPT-2 already struggle with GPT-3.”
Attacking Neural Text Detectors
Wolff, M., & Wolff, S. (2020) · arXiv · doi:10.48550/arxiv.2002.11768 · Cited by 18 · cited 1× · see citing works
“Further, known detectors are also vulnerable to adversarial attacks that degrade their functionality (Wolff & Wolff, 2022).”
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design
Bergman, A. S., Abercrombie, G., Spruit, S. L., Hovy, D., Dinan, E., Boureau, Y.-L., & Rieser, V. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.4 · Cited by 16 · cited 1× · see citing works
“As these systems become more pervasive, there is increasing risk that they may be used for malicious purposes (Bergman et al., 2022; Mirsky et al., 2023).”
Watermarking Pre-trained Language Models with Backdooring
Gu, C., Huang, C., Zheng, X., Chang, K.-W., & Hsieh, C.-J. (2022) · arXiv · doi:10.48550/arxiv.2210.07543 · Cited by 15 · cited 1× · see citing works
“An attacker could make small alterations, adding additional whitespaces, or misspelling a few words to impact the computation of the hash.”
Dual canonicalization: An answer to the homograph attack
Helfrich, J. N., & Neff, R. (2012) · 2012 eCrime Researchers Summit · doi:10.1109/ecrime.2012.6489517 · Cited by 8 · cited 1× · see citing works
“Security against Homoglyph and tokenization attacks can be maintained using input normalization before the text is tested for watermarks, for example via canonicalization as in Helfrich & Neff (2012).”
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
10 of the 64 references couldn’t be matched to a known paper.
Research works printed but not linked: 7
- Foltýnek, T., Meuschke, N., and Gipp, B. Academic Plagiarism Detection: A Systematic Literature Review. ACM Computing Surveys, 52(6):112:1-112:42, October 2019. ISSN 0360-0300. doi: 10.1145/3345317. URL https://doi.org/10.1145/3345317. · DOI · Paper page
- Katzenbeisser, S. and Petitcolas, F. A. Information Hiding Techniques for Steganography and Digital Watermarking. Artech House, Inc., USA, 1st edition, 2000. ISBN 978-1- 58053-035-4.
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language Models are Unsupervised Multi- task Learners. OpenAI, pp. 24, 2019.
- Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv e-prints, 2019.
- Tian, E. Gptzero update v1, January 2023. URL https: //gptzero.substack.com/p/gptzero- update-v1. · DOI
- Venugopal, A., Uszkoreit, J., Talbot, D., Och, F., and Ganitkevitch, J. Watermarking the Outputs of Struc- tured Prediction with an application in Statistical Ma- chine Translation. In Proceedings of the 2011 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pp. 1363-1372, Edinburgh, Scotland, UK., July 2011. Association for Computational Linguistics. URL https://aclanthology.org/D11-1126.
- Curran Associates, Inc., 2019. URL https://proceedings. neurips.cc/paper/2019/hash/ 3e9f0fc9b2f89e043bc6233994dfcf76- Abstract.html.
Cited by
Sorted by citations
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · see citing works
Regulating ChatGPT and other Large Generative AI Models
Hacker, P., Engel, A., & Mauer, M. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594067 · Cited by 466 · see citing works
“Importantly, more interdisciplinary research is necessary to develop markings that are easy to use and recognize, but hard to remove by average users [139].”
BackgroundA Categorical Archive of ChatGPT Failures
Borji, A. (2023) · doi:10.21203/rs.3.rs-2895792/v1 · Cited by 514 · see citing works
Can Generative AI improve social science?
Bail, C. A. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314021121 · Cited by 369 · see citing works
AI Deception: A Survey of Examples, Risks, and Potential Solutions
Park, P. S., Goldstein, S., O'Gara, A., Chen, M., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2308.14752 · Cited by 243 · see citing works
Artificial intelligence risk management framework :
National Institute of Standards and Technology (US) (2024) · NIST Trustworthy and Responsible AI · doi:10.6028/nist.ai.600-1 · Cited by 274 · see citing works
AI deception: A survey of examples, risks, and potential solutions
Park, P. S., Goldstein, S., O’Gara, A., Chen, M., & Hendrycks, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100988 · Cited by 243 · see citing works
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212 · see citing works
Can AI-Generated Text be Reliably Detected?
Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., & Feizi, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11156 · Cited by 161 · see citing works
“In this section, we discuss paraphrasing attacks that can degrade type-II errors of state-of-the-art AI text detectors such as soft watermarking [1], zero-shot detectors [2], trained neural network-based detectors [3], and retrieval-based detectors [4].”
BackgroundMethodInfluentialAdversarial machine learning :
Vassilev, A., Oprea, A., Fordyce, A. J., & Anderson, H. S. (2024) · doi:10.6028/nist.ai.100-2e2023 · Cited by 194 · see citing works
A Survey on the Detection and Impacts of Deepfakes in Visual, Audio, and Textual Formats
Mubarak, R., Alsboui, T., Alshaikh, O., Inuwa-Dutse, I., Khan, S., & Parkinson, S. (2023) · IEEE Access · doi:10.1109/access.2023.3344653 · Cited by 132 · see citing works
“This approach was recently revolutionised by Kirchenbauer et al. [255] In 2023, they introduced a novel approach by developing a watermarking method tailored for Large Language Models (LLMs).”
MethodBrinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132 · see citing works
“Selection occurs when certain cultural traits are more likely to be adopted by individuals due to factors such as their usefulness,”
BackgroundThe Science of Detecting LLM-Generated Text
Tang, R., Chuang, Y.-N., & Hu, X. (2024) · Communications of the ACM · doi:10.1145/3624725 · Cited by 125 · see citing works
Language Model Behavior: A Comprehensive Survey
Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105 · see citing works
“posed “watermarking” model-generated text by slightly increasing the probabilities of “whitelist” tokens during text generation (Kirchenbauer et al. 2023), or by explicitly replacing some tokens with whitelist tokens (He et al.”
BackgroundHadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v4 · Cited by 173 · see citing works
Distinguishing Reality from AI: Approaches for Detecting Synthetic Content
Ghiurău, D., & Popescu, D. E. (2024) · Computers · doi:10.3390/computers14010001 · Cited by 108 · see citing works
Self-consuming generative models go MAD
Casco-Rodriguez, J., Alemohammad, S., Luzi, L., Imtiaz, A., Babaei, H., LeJeune, D., Siahkoohi, A., & Baraniuk, R. (2023) · LatinX in AI at Neural Information Processing Systems Conference 2023 · doi:10.52591/lxai202312101 · Cited by 83 · see citing works
“of watermarking of data generated by LLMs [75] and novel methods for watermarking LLMs [76], diffusion models [77–80], andn GANs [81] are currently active areas of research.”
MethodMAGE: Machine-generated Text Detection in the Wild
Li, Y., Li, Q., Cui, L., Bi, W., Wang, Z., Wang, L., Yang, L., Shi, S., & Zhang, Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.3 · Cited by 79 · see citing works
The Gradient of Generative AI Release: Methods and Considerations
Solaiman, I. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3593981 · Cited by 90 · see citing works
“Research strives ensure these watermarks are invisible to the human-eye [107], do not affect output quality [46], and tamper-proof from model attacks and alterations like fine-tuning via methods such as embedding noise as watermarks [106].”
Veselovsky, V., Ribeiro, M. H., & West, R. (2023) · arXiv · doi:10.48550/arxiv.2306.07899 · Cited by 73 · see citing works
“These concerns have led to work on watermarking LLM output (Kirchenbauer et al., 2023) by slightly altering the odds of each token during sampling, and to further work on methods for improving the detection of synthetically generated text (Yu et al., 2023; Verma et al., 2023; Diwan et al., 2021).”
Anatomy of an AI-powered malicious social botnet
Yang, K., & Menczer, F. (2024) · Journal of Quantitative Description: Digital Media · doi:10.51685/jqd.2024.icwsm.7 · Cited by 71 · see citing works
“White-box methods, on the other hand, require LLM owners to embed specific signals or watermarks (e.g., altered word frequencies) into generated content for subsequent identification (Kirchenbauer et al., 2023; Zhao et al., 2023).”
BackgroundLiang, W., Izzo, Z., Zhang, Y., Lepp, H., Cao, H., Zhao, X., Chen, L., Ye, H., Liu, S., Huang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07183 · Cited by 73 · see citing works
“Kirchenbauer et al. (2023) split the vocabulary into red-green lists based on hash values of previous n-grams and then increase the logits of green to-kens to embed the watermark.”
Ibrahim, K. (2023) · Language Testing in Asia · doi:10.1186/s40468-023-00260-2 · Cited by 66 · see citing works
“…NLP has explored different approaches to detecting AI-generated texts (e.g., Bakhtin et al., 2019), including watermarking AI-generated texts (e.g., Kirchenbauer et al., 2023), detecting traces of language modeling in generated texts (e.g., Tay et al., 2020), and training AI models to…”
BackgroundMethodInitial policy considerations for generative artificial intelligence
Lorenz, P., Perset, K., & Berryhill, J. (2023) · OECD artificial intelligence papers · doi:10.1787/fae2d1e6-en · Cited by 105 · see citing works
Can Generative AI Improve Social Science?
Bail, C. A. (2023) · doi:10.31235/osf.io/rwtzs · Cited by 369 · see citing works
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text
Su, J., Zhuo, T. Y., Wang, D., & Nakov, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.827 · Cited by 105 · see citing works
Identifying and Mitigating the Security Risks of Generative AI
Barrett, C., Boyd, B., Bursztein, E., Carlini, N., Chen, B., Choi, J., Chowdhury, A. R., Christodorescu, M., Datta, A., Feizi, S., et al. (2023) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000041 · Cited by 57 · see citing works
“A recent line of work tries this specifically for AI-generated text [47, 46, 103, 21, 50].”
BackgroundInfluentialPeople cannot distinguish GPT-4 from a human in a Turing test
Jones, C. R., Rathi, I., Taylor, S., & Bergen, B. K. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732108 · Cited by 76 · see citing works
Xie, Y., Wu, S., & Chakravarty, S. (2023) · Conference on Information Technology Education · doi:10.1145/3585059.3611449 · Cited by 54 · see citing works
“Embedding subtle fingerprints to text parts of programming code using similar strategies described in [46] could be a starting point to explore.”
MethodHow can artificial intelligence transform the training of medical students and physicians?
Ning, Y., Ong, J. C. L., Cheng, H., Wang, H., Ting, D. S. W., Tham, Y. C., Wong, T. Y., & Liu, N. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100900 · Cited by 52 · see citing works
Detecting AI-Generated Code Assignments Using Perplexity of Large Language Models
Xu, Z., & Sheng, V. S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i21.30361 · Cited by 54 · see citing works
SeqXGPT: Sentence-Level AI-Generated Text Detection
Wang, P., Li, L., Ren, K., Jiang, B., Zhang, D., & Qiu, X. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.73 · Cited by 53 · see citing works
Koike, R., Kaneko, M., & Okazaki, N. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30120 · Cited by 55 · see citing works
Giray, L., Sevnarayan, K., & Ranjbaran, F. (2025) · Internet Reference Services Quarterly · doi:10.1080/10875301.2024.2437174 · Cited by 62 · see citing works
“…of texts and perform thresholding to detect AI, may not perform well on texts with varying complexity or style (Solaiman et al., 2019), and watermarking approaches (imprinting patterns in the text) may not be robust to paraphrasing or other attacks (Kirchenbauer et al., 2023; Zhao et al., 2023).”
GPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · arXiv · doi:10.48550/arxiv.2304.02819 · Cited by 40 · see citing works
Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges
Huang, B., Chen, C., & Shu, K. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3715073.3715076 · Cited by 42 · see citing works
Robust Multi-bit Natural Language Watermarking through Invariant Features
Yoo, K., Ahn, W., Jang, J., & Kwak, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.117 · Cited by 43 · see citing works
“Similarly, Kirchenbauer et al. (2023) propose to watermark outputs of language models at decoding time in a zero-bit manner to distinguish machine-generated texts from human-written text.”
BackgroundDianova, V. G., & Schultz, M. D. (2023) · Industry and Higher Education · doi:10.1177/09504222231199989 · Cited by 43 · see citing works
Transformers: A Security Perspective
Latibari, B. S., Nazari, N., Chowdhury, M. A., Gubbi, K. I., Fang, C., Ghimire, S., Hosseini, E., Sayadi, H., Homayoun, H., Salehi, S., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3509372 · Cited by 41 · see citing works
“Authors in [82] introduced a watermarking framework for FIGURE 14.”
Jacovi, A., Caciularu, A., Goldman, O., & Goldberg, Y. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.308 · Cited by 36 · see citing works
Jacovi, A., Caciularu, A., Goldman, O., & Goldberg, Y. (2023) · arXiv · doi:10.48550/arxiv.2305.10160 · Cited by 36 · see citing works
Dual use concerns of generative AI and large language models
Grinbaum, A., & Adomaitis, L. (2024) · Journal of Responsible Innovation · doi:10.1080/23299460.2024.2304381 · Cited by 35 · see citing works
“Risks arising from fraud or manipulation, which comprise a large spectrum of societal risks, imply that users should not mistake an output produced by a machine for an output created by a human author (Aaronson 2022; Grinbaum and Adomaitis 2022b; Kirchenbauer et al. 2023).”
BackgroundIntrinsic Dimension Estimation for Robust Detection of AI-Generated Texts
Tulchinskii, E., Kuznetsov, K., Kushnareva, L., Cherniavskii, D., Barannikov, S., Piontkovskaya, I., Nikolenko, S., & Burnaev, E. (2023) · arXiv · doi:10.48550/arxiv.2306.04723 · Cited by 43 · see citing works
“Another idea, known as watermarking , is to inject some detectable artifacts into model generations; for instance, Kirchenbauer et al. [2023] propose to intentionally inject a statistical skew that can be detected in a text sample.”
BackgroundModelShield: Adaptive and Robust Watermark Against Model Extraction Attack
Pang, K., Qi, T., Wu, C., Bai, M., Jiang, M., & Huang, Y. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3530691 · Cited by 29 · see citing works
LLMDet: A Third Party Large Language Models Generated Text Detection Tool
Wu, K., Pang, L., Shen, H., Cheng, X., & Chua, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.139 · Cited by 45 · see citing works
“The current state-of-the-art approach, as proposed by (Kirchenbauer et al., 2023), partitions the model’s vocabulary into whitelist and blacklist tokens when predicting the next token given a prompt.”
MethodInfluentialWhen deep learning meets watermarking: A survey of application, attacks and defenses
Chen, H., Liu, C., Zhu, T., & Zhou, W. (2024) · Computer Standards & Interfaces · doi:10.1016/j.csi.2023.103830 · Cited by 27 · see citing works
Three Bricks to Consolidate Watermarks for Large Language Models
Fernandez, P., Chaffin, A., Tit, K., Chappelier, V., & Furon, T. (2023) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs58808.2023.10374576 · Cited by 39 · see citing works
“Empirical Validation of FPR with Z-Scores So far, the FPR was checked on small scale experiments: around 500 negative samples [18], [19], [21].”
BackgroundMethodInfluentialPromptCARE: Prompt Copyright Protection by Watermark Injection and Verification
Yao, H., Lou, J., Qin, Z., & Ren, K. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00209 · Cited by 29 · see citing works
“Language model watermarking [17], [18], [19], [20], [31], [32], [33], [34] is a technique used to embed a unique signature into the generated output of a language model.”
BackgroundMethodM4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection
Wang, Y., Mansurov, J., Ivanov, P., Su, J., Shelmanov, A., Tsvigun, A., Mohammed Afzal, O., Mahmoud, T., Puccetti, G., Arnold, T., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.218 · Cited by 25 · see citing works
A Pathway Towards Responsible AI Generated Content
Lyu, L. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/803 · Cited by 77 · see citing works
ProMark: Proactive Diffusion Watermarking for Causal Attribution
Asnani, V., Collomosse, J. P., Bui, T., Liu, X., & Agarwal, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01027 · Cited by 26 · see citing works
“[19] propose a watermarking method for language models by pre-selecting random tokens and subtly influencing their use during word generation.”
Fu, Y., Xiong, D., & Dong, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29756 · Cited by 27 · see citing works
Data Authenticity, Consent, and Provenance for AI Are All Broken: What Will It Take to Fix Them?
Longpre, S., Mahari, R., Obeng-Marnu, N., Brannon, W., South, T., Kabbara, J., & Pentland, A. (2024) · doi:10.21428/e4baedd9.a650f77d · Cited by 23 · see citing works
Detection of AI-Generated Text Using Large Language Model
Prajapati, M., Baliarsingh, S. K., Dora, C., Bhoi, A., Hota, J., & Mohanty, J. P. (2024) · International Conference on Emerging Systems and Intelligent Computing (ESIC) · doi:10.1109/esic60604.2024.10481602 · Cited by 24 · see citing works
“Various methodologies have been developed to detect AI-generated text, including binary classification models, zero-shot classifiers [6], neural network-based detectors like OpenAI’s RoBERTa [7], and soft watermarking [8].”
MethodTowards Trustworthy AI: A Review of Ethical and Robust Large Language Models
Ferdaus, M. M., Abdelguerfi, M., Ioup, E., Niles, K. N., Pathak, K., & Sloan, S. D. (2025) · ACM Computing Surveys · doi:10.1145/3777382 · Cited by 38 · see citing works
Reclaiming the Digital Commons: A Public Data Trust for Training Data
Chan, A., Bradley, H., & Rajkumar, N. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604658 · Cited by 23 · see citing works
“may add noise or watermarks to their deployed models to ensure that others cannot copy the model easily [13, 44, 60].”
BackgroundM4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection
Wang, Y., Mansurov, J., Ivanov, P., Su, J., Shelmanov, A., Tsvigun, A., Whitehouse, C., Afzal, O. M., Mahmoud, T., Sasaki, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.14902 · Cited by 19 · see citing works
“…posthoc lexical substitutions and synonym replacement over generated outputs (Szyller et al., 2021; He et al., 2022), and soft watermarking (Kirchenbauer et al., 2023) using green and red token lists, to hidden-space operations, such as injecting secret signals into the probability vector…”
BackgroundHow to Protect Copyright Data in Optimization of Large Language Models?
Chu, T., Song, Z., & Yang, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29741 · Cited by 21 · see citing works
Watermarking for Large Language Models: A Survey
Yang, Z., Zhao, G., & Wu, H. (2025) · Mathematics · doi:10.3390/math13091420 · Cited by 21 · see citing works
AI-generated text detection: A comprehensive review of methods, datasets, and applications
Kehkashan, T., Riaz, R. A., Al‐Shamayleh, A. S., Akhunzada, A., Ali, N., Hamza, M., & Akbar, F. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100793 · Cited by 21 · see citing works
AI Usage Cards: Responsibly Reporting AI-Generated Content
Wahle, J. P., Ruas, T., Mohammad, S. M., Meuschke, N., & Gipp, B. (2023) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl57899.2023.00060 · Cited by 20 · see citing works
“While automatic detection and watermarking methods exist [9, 11], they do not address integrity and accountability issues.”
BackgroundYang, Z., Feng, Z., Huo, R., Lin, H., Zheng, H., Nie, R., & Chen, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126694 · Cited by 17 · see citing works
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · doi:10.31219/osf.io/ac4sp · Cited by 51 · see citing works
Mosca, E., Abdalla, M. H. I., Basso, P., Musumeci, M., & Groh, G. (2023) · Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023) · doi:10.18653/v1/2023.trustnlp-1.17 · Cited by 14 · see citing works
“Alternative approaches explore using the probability curvature of the generative model itself (Mitchell et al., 2023) or watermarking machinegenerated text to facilitate detection (Kirchenbauer et al., 2023).”
BackgroundMethodLongo, L., Brčić, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.19775 · Cited by 14 · see citing works
Publicly-Detectable Watermarking for Language Models
Fairoze, J., Garg, S., Jha, S., Mahloujifar, S., Mahmoody, M., & Wang, M. (2025) · IACR Communications in Cryptology · doi:10.62056/ahmpdkp10 · Cited by 29 · see citing works
Large Language Models in Healthcare Current Development and Future DirectionsPreprint
Jiang, X., Yan, L., Vavekanand, R., & Hu, M. (2024) · Preprints.org · doi:10.20944/preprints202407.0923.v1 · Cited by 13 · see citing works
Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection
Peng, X., Zhou, Y., He, B., Sun, L., & Sun, Y. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.644 · Cited by 16 · see citing works
“Current AI-generated text detection methods can be in general categorized into three categories (Lu et al., 2023): statistical (Gehrmann et al., 2019), watermarking (Kirchenbauer et al., 2023), and training-based (Gallé et al., 2021).”
BackgroundA Comparison Study on AI Language Detector
Singh, A. (2023) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc57344.2023.10099219 · Cited by 12 · see citing works
“This approach involves using techniques such as plagiarism detection [7], language model fine-tuning, watermarking [9] and data masking to detect and prevent misuse.”
MethodUnderstanding the effects of human-written paraphrases in LLM-generated text detection
Lau, H. T., & Zubiaga, A. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100151 · Cited by 13 · see citing works
Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models
Zhang, H., Edelman, B. L., Francati, D., Venturi, D., Ateniese, G., & Barak, B. (2023) · arXiv · doi:10.48550/arxiv.2311.04378 · Cited by 12 · see citing works
A Framework for Detecting AI-Generated Text in Research Publications
Sarzaeim, P., Doshi, A. M., & Mahmoud, Q. H. (2023) · Proceedings of the International Conference on Advanced Technologies · doi:10.58190/icat.2023.28 · Cited by 11 · see citing works
Watermarking LLMs with Weight Quantization
Li, L., Jiang, B., Wang, P., Ren, K., Yan, H., & Qiu, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.220 · Cited by 17 · see citing works
On the Zero-Shot Generalization of Machine-Generated Text Detectors
Xiao, P., Zhang, J., Han, X., Tsvetkov, Y., & He, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.318 · Cited by 18 · see citing works
On the Societal Impact of Open Foundation Models
Kapoor, S., Bommasani, R., Klyman, K., Longpre, S., Ramaswami, A., Cihon, P., Hopkins, A., Bankston, K., Biderman, S., Bogen, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07918 · Cited by 11 · see citing works
How Reliable Are AI-Generated-Text Detectors? An Assessment Framework Using Evasive Soft Prompts
Kumarage, T., Sheth, P., Moraffah, R., Garland, J., & Liu, H. (2023) · arXiv · doi:10.48550/arxiv.2310.05095 · Cited by 11 · see citing works
Muthukumar, K. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1680552 · Cited by 10 · see citing works
“Human responses are usually more random and varied, yet technological improvements make distinguishing between humans and AI more difficult (Kirchenbauer et al., 2023).”
SrcMarker: Dual-Channel Source Code Watermarking via Scalable Code Transformations
Yang, B., Li, W., Xiang, L., & Li, B. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00097 · Cited by 15 · see citing works
A Brief Survey of Watermarks in Generative AI
Hwang, J., & Oh, S. (2023) · Information and Communication Technology Convergence · doi:10.1109/ictc58733.2023.10392465 · Cited by 11 · see citing works
“According to The New York Times [6], OpenAI appears to have adopted a watermarking approach similar to the one outlined in the paper [5], although specific details have not been publicly disclosed.”
MethodAI Usage Cards: Responsibly Reporting AI-generated ContentPreprint
Wahle, J. P., Ruas, T., Mohammad, S. M., Meuschke, N., & Gipp, B. (2023) · doi:10.22541/au.168323192.20543041/v1 · Cited by 10 · see citing works
Enhancing the Robustness of AI-Generated Text Detectors: A Survey
Liu, X., Li, Y., & Li, K. (2025) · Mathematics · doi:10.3390/math13132145 · Cited by 10 · see citing works
An explainable framework for assisting the detection of AI-generated textual content
Yan, S., Wang, Z., & Dobolyi, D. G. (2025) · Decision Support Systems · doi:10.1016/j.dss.2025.114498 · Cited by 10 · see citing works
Gutub, A. A. A., & Almehmadi, E. (2023) · Journal of Engineering Research · doi:10.1016/j.jer.2023.100099 · Cited by 9 · see citing works
Pudasaini, S., Miralles-Pechuán, L., Lillis, D., & Salvador, M. L. (2024) · arXiv · doi:10.48550/arxiv.2407.13105 · Cited by 8 · see citing works
“Along with these results, Zhao et al. [104] also proposed another watermarking solution called Unigram-Watermark, which outperformed the previous solution proposed by Kirchenbauer et al. [103] for paraphrasing attacks.”
MethodNational Institute of Standards and Technology (US) (2024) · doi:10.6028/nist.ai.100-4 · Cited by 9 · see citing works
GhostCite: A Large-Scale Analysis of Citation Validity in the Age of Large Language Models
Xu, Z., Qiu, Y., Sun, L., Miao, F., Wu, F., Li, X., Wang, X., Lu, H., Zhang, Z., Hu, Y., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.06718 · Cited by 3 · see citing works
“Detection efforts have emerged: GPTZero reported hallucinated citations in both ICLR 2026 submissions and NeurIPS 2025 accepted papers [9], [10], while watermarking [32] and zero-shot detection based on probability curvature [33] offer complementary strategies.”
LLMs beyond the lab: the ethics and epistemics of real-world AI research
Mollen, J. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09819-w · Cited by 9 · see citing works
“…for example, which protocols or normative frameworks can best guide a responsible and effective use of the AI system or what addi - tions might be necessary to secure responsible embedding, such as digital watermarks to algorithmically identify AI-generated content (Kirchenbauer et al., 2023).”
How Reliable Are AI-Generated-Text Detectors? An Assessment Framework Using Evasive Soft Prompts
Kumarage, T., Sheth, P., Moraffah, R., Garland, J., & Liu, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.94 · Cited by 15 · see citing works
AIRABIC: Arabic Dataset for Performance Evaluation of AI Detectors
Alshammari, H., & EI-Sayed, A. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00127 · Cited by 8 · see citing works
“In response to the potential misuse of the AI generative models, some recently proposed algorithms identify AI-generated text, such as [9, 10].”
Background(2023) · doi:10.21203/rs.3.rs-3226684/v1 · Cited by 8 · see citing works
GPTs Don’t Keep Secrets: Searching for Backdoor Watermark Triggers in Autoregressive Language Models
Lucas, E., & Havens, T. C. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.21 · Cited by 7 · see citing works
“Watermarking could be used to show that text was machine generated or to prove ownership of a model; this is generally done in a way that is subtle and not apparent to a human observer (Topkara et al., 2006; Kirchenbauer et al., 2023; Grinbaum and Adomaitis, 2022).”
BackgroundAuthorship Verification in software engineering education
Rios, S., Zhang, Y., & de Oliveira, E. A. (2023) · ASCILITE Publications · doi:10.14742/apubs.2023.559 · Cited by 7 · see citing works
“Another avenue to AI text generation is “watermarking” the text to ease detection by incorporating hidden patterns in the text that are unnoticeable and do not affect text quality, but algorithmically identify the text as synthetic (Kirchenbauer et al., 2023).”
BackgroundMethodWhere Am I From? Identifying Origin of LLM-generated Content
Li, L., Bai, Y., & Cheng, M. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.681 · Cited by 7 · see citing works
Robustness of generative AI detection: adversarial attacks on black-box neural text detectors
Fishchuk, V., & Braun, D. (2024) · International Journal of Speech Technology · doi:10.1007/s10772-024-10144-2 · Cited by 7 · see citing works
A Multimodal Approach for Detecting AI Generated Content using BERT and CNN
Vora, E. A. V. (2023) · International Journal on Recent and Innovation Trends in Computing and Communication · doi:10.17762/ijritcc.v11i9.8861 · Cited by 6 · see citing works
HANSEN: Human and AI Spoken Text Benchmark for Authorship Analysis
Tripto, N., Uchendu, A., Le, T., Setzu, M., Giannotti, F., & Lee, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.916 · Cited by 6 · see citing works
CurveMark: Detecting AI-Generated Text via Probabilistic Curvature and Dynamic Semantic Watermarking
Zhang, Y., Jiang, X., Sun, H., Zhang, Y., & Tong, D. (2025) · Entropy · doi:10.3390/e27080784 · Cited by 6 · see citing works
Emotion Classification using Generative Pre-trained Embedding and Machine Learning
Pattun, G., & Kumar, P. (2023) · IEEE International Conference on Machine Learning and Applied Network Technologies (ICMLANT) · doi:10.1109/icmlant59547.2023.10372980 · Cited by 6 · see citing works
“LLMs play crucial roles in medical diagnosis, education, creative writing, and legal research, offering versatile solutions[8], [11], [13], [14].”
BackgroundCooman, J. D. (2023) · Law Technology and Humans · doi:10.5204/lthj.2787 · Cited by 5 · see citing works
Detecting Artificial Intelligence: A New Cyberarms Race Begins
Campbell, M., & Jovanovic, M. (2023) · Computer · doi:10.1109/mc.2023.3279446 · Cited by 4 · see citing works
- Authors
- Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T.
- Affiliations
- University of Maryland, College Park (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2301.10226
- OpenAlex
- W4318149317
- Cited by
- 194
- References
- 64
- Citations by year
- 2026: 4 · 2025: 73 · 2024: 64 · 2023: 53
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2212182, 2229885)
- Open Philanthropy Project
- Defense Advanced Research Projects Agency (HR00112020007)
- Multidisciplinary University Research Initiative
- Office of Naval Research (N000142112557)
- Air Force Office of Scientific Research
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Hardware and Architecture › Physical Unclonable Functions (PUFs) and Hardware Security
- Keywords
- watermarking, large language models, watermark detection, hypothesis testing, information-theoretic analysis, watermark robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
