The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 11 of the 26 sources its claims cite were checked for support.
Scalable watermarking for identifying large language model outputs
Open accessAbstract
Large language models (LLMs) have enabled the generation of high-quality synthetic text, often indistinguishable from human-written content, at a scale that can markedly affect the nature of the information ecosystem 1–3 . Watermarking can help identify synthetic text and limit accidental or deliberate misuse 4 , but has not been adopted in production systems owing to stringent quality, detectability and computational efficiency requirements. Here we describe SynthID-Text, a production-ready text watermarking scheme that preserves text quality and enables high detection accuracy, with minimal latency overhead. SynthID-Text does not affect LLM training and modifies only the sampling procedure; watermark detection is computationally efficient, without using the underlying LLM. To enable watermarking at scale, we develop an algorithm integrating watermarking with speculative sampling, an efficiency technique frequently used in production systems 5 . Evaluations across multiple LLMs empirically show that SynthID-Text provides improved detectability over comparable methods, and standard benchmarks and human side-by-side ratings indicate no change in LLM capabilities. To demonstrate the feasibility of watermarking in large-scale-production systems, we conducted a live experiment that assessed feedback from nearly 20 million Gemini 6 responses, again confirming the preservation of text quality. We hope that the availability of SynthID-Text 7 will facilitate further development of watermarking and responsible use of LLM systems.
Overview
CoChat Summary
TL;DR
SynthID-Text is a production-ready generative text watermarking scheme that modifies only the LLM sampling procedure, using a novel Tournament sampling algorithm to preserve text quality while improving detectability over comparable methods. It was deployed in Gemini and Gemini Advanced and evaluated on nearly 20 million live responses, with negligible latency overhead.
Key findings
- Non-distortionary SynthID-Text preserved quality in a live Gemini experiment over approximately 20 million watermarked and unwatermarked responses: thumbs-up rate differed by 0.01% (watermarked higher) and thumbs-down rate by 0.02% (watermarked lower), both statistically insignificant.
- Non-distortionary SynthID-Text provided better detectability than Gumbel sampling for the same text length, measured as true-positive rate at 1% false-positive rate on Gemma 7B-IT with temperature 0.7 and ELI5 prompts; distortionary SynthID-Text offered a more favourable detectability/quality trade-off than Soft Red List.
- Computational impact was minimal: Gemma 7B-IT on 4 v5e TPUs generated text at 15.527 ms per token, increasing to 15.615 ms per token with 30-layer Tournament sampling, a latency increase of only 0.57% (Gumbel 0.26%, Soft Red List 0.28%).
- A controlled human preference test on 3,000 ELI5 questions comparing watermarked versus unwatermarked Gemma 7B-IT responses found no significant difference across grammaticality/coherence, relevance, correctness, helpfulness, and overall quality.
Methods
- SynthID-Text uses generative watermarking: a random seed generator, a sampling algorithm, and a scoring function. It uses Tournament sampling, with a sliding-window random seed generator hashing the last H = 4 tokens plus a watermarking key, m = 30 tournament layers, Bernoulli(0.5) g-values, and a Bayesian scoring function; detection requires only tokenized text, the key, and the seed generator, not the LLM.
- Evaluations used instruction-tuned Gemma 2B and Gemma 7B, and Mistral 7B-IT, with top-k = 100 and temperatures 0.5, 0.7, and 1.0, prompted by ELI5 questions. Non-distortionary experiments used 10,000 disjoint prompts each for test and development; distortionary experiments used 1,500 ELI5 prompts for the test set. Detectability was reported as true-positive rate at a fixed false-positive rate.
- The live production experiment routed a random fraction of Gemini queries to a watermarked model and an equivalent number to an unwatermarked counterpart, analysing approximately 20 million responses via thumbs-up and thumbs-down feedback. Additional automatic evaluations checked perplexity and standard benchmarks.
Limitations
- The paper states that generative watermarks such as SynthID-Text do not offer a complete solution to AI text detection; they require coordination between actors running LLM services, are difficult to enforce on decentralized open-source models, and are vulnerable to stealing, spoofing, and scrubbing attacks. They are also weakened by edits such as LLM paraphrasing.
- The paper states that no text detection method is foolproof and that watermarking is complementary to other approaches; detecting text from actors who do not implement watermarking requires other methods such as post hoc detection.
- Evident from the text: detection depends on text length and LLM distribution entropy, performing better with more entropy and less effectively in low-entropy settings; the non-distortionary configuration has some reduction to inter-response diversity, and the distortionary configuration trades text quality for higher detectability.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
11 Valid · 4 Not valid · 12 Not checked · 60 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Compared with the distortionary Soft Red List watermark23, distortionary SynthID-Text offers a more favourable trade-off, with substantially higher detection rates for the same effect on text quality as measured by log(perplexity).”
- Also stated in: § SynthID-Text provides better detectability than existing watermarks · plain
“In the distortionary category, we compare the trade-off of detectability and text quality for distortionary SynthID-Text and Soft Red List. Both methods have a strength parameter that controls this trade-off; in Fig. 3c, we see that the trade-off is more favourable for distortionary SynthID-Text.”
§ Scalable watermarking for identifying large language model outputs > Evaluation > SynthID-Text provides better detectability than existing watermarks
What the sources say
- Supportedcites [23] Kirchenbauer, J. et al. A watermark for large language models. In Proc. 40th International Conference on Machine Learning 17061–17084 (PMLR, 2023). source
The superscript is attached to the noun phrase 'the distortionary Soft Red List watermark', so what is attributed to this source is the identification of that method and its distortionary nature. Our copy introduces exactly that: Section 3 / Algorithm 2 ('A more sophisticated watermark', pp. n/a) defines the soft red list rule that adds a constant delta to green-list logits, and Section 4.2 ('Impact on quality of generated text') plus the 'Strength vs Text Quality' paragraph (Figure 2, left) state that the soft watermark does impact perplexity for moderate-entropy tokens and plots the trade-off between watermark strength (z-score) and text quality (perplexity). The head-to-head comparison against SynthID-Text and the log(perplexity) axis are the citing paper's own measurement, not a claim this source makes — our copy never mentions SynthID (searched: 'SynthID', 'comparison with other watermarking methods'). (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § SynthID-Text provides better detectability than existing watermarks · plain
Every source the paper cites for this claim was checked and supports it.
In the paper
“Furthermore, we provide an algorithm to combine generative watermarking with speculative sampling5—a frequently used technique to increase LLM text generation speed—allowing for the integration of SynthID-Text into large-scale production systems with negligible additional computational overhead.”
§ Scalable watermarking for identifying large language model outputs > Main
What the sources say
- Supportedcites [5] Chen, C. et al. Accelerating large language model decoding with speculative sampling. Preprint at https://arxiv.org/abs/2302.01318 (2023). source
The citation identifies speculative sampling as a technique for speeding LLM text generation. The Introduction presents it as an algorithm to accelerate transformer sampling, and the Conclusion reports speedups on benchmark tasks. The statement about integrating SynthID-Text is the citing authors’ own contribution, not a claim attributed to this source. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“To enable watermarking at scale, we develop an algorithm integrating watermarking with speculative sampling, an efficiency technique frequently used in production systems5.”
§ Scalable watermarking for identifying large language model outputs > Abstract
What the sources say
- Partly supportedcites [5] Chen, C. et al. Accelerating large language model decoding with speculative sampling. Preprint at https://arxiv.org/abs/2302.01318 (2023). sourceExperimental — accuracy not yet measured
The Introduction describes speculative sampling as a practical method for reducing decoding latency and reports a 2–2.5× speedup, supporting its description as an efficiency technique. The cited paper does not establish that it is frequently used in production systems. The sentence says the citing authors develop the watermarking algorithm; it does not attribute that development to this source. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Furthermore, we provide an algorithm to combine generative watermarking with speculative sampling5—a frequently used technique to increase LLM text generation speed—allowing for the integration of SynthID-Text into large-scale production systems with negligible additional computational overhead.”
§ Scalable watermarking for identifying large language model outputs > Main
What the sources say
- Supportedcites [5] Chen, C. et al. Accelerating large language model decoding with speculative sampling. Preprint at https://arxiv.org/abs/2302.01318 (2023). source
The Abstract identifies speculative sampling as the paper’s algorithm for accelerating transformer decoding and reports a 2–2.5× speedup. In the citing sentence, “we provide an algorithm to combine” watermarking with speculative sampling describes the citing authors’ contribution; citation 5 identifies the speculative-sampling technique, not the combination. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We find no significant difference between non-distortionary SynthID-Text and the equivalent unwatermarked model in terms of perplexity or performance on automated benchmarks.”
§ Scalable watermarking for identifying large language model outputs > Evaluation > SynthID-Text preserves quality including in a large-scale-production system
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
45 more claims
Create a free account to see all 87 claims and to verify the 3 references still open.
- It may state two separate claims as one. 27
- Probably not a substantial claim. 16
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Technology like watermarking could help improve the accountability of world model usage [30].”
From Understanding World or Predicting Future? A Comprehensive Survey of World Models (2026) · cites this paper as [30]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A creator-centric how of AI attribution remains an open question – while algorithms for identifying and watermarking AI-generated content offer useful technological approaches [20, 28, 36], they may not encompass the extent and nuance of creators’ attribution needs.”
From Which Contributions Deserve Credit? Perceptions of Attribution in Human-AI Co-Creation (2025) · cites this paper as [20]
Citations
Cites
A learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_7 · Cited by 3,402
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1082 · Cited by 1,266
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.04833 · Cited by 1,242
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11805 · Cited by 990
Self-citationAI models collapse when trained on recursively generated data
Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07566-y · Cited by 793 · cited 1×
“Given the widespread use of LLMs in education, software development and web content generation, identification and attribution of LLM text is critical to ensure safe and responsible use of the technology 8–11 .”
MethodSelf-citationGPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · Patterns · doi:10.1016/j.patter.2023.100779 · Cited by 673 · cited 1×
“In particular, they are known to perform poorly on out-of-domain data and may have higher false-positive rates for certain groups, such as non-native speakers 17 .”
BackgroundKöbis, N., & Mossink, L. D. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106553 · Cited by 406
Elkhatat, A. M., Elsaid, K., & Almeer, S. (2023) · International Journal for Educational Integrity · doi:10.1007/s40979-023-00140-5 · Cited by 392 · cited 1×
“However, post hoc detection systems can themselves be computationally expensive to run, and their practical usage is limited by their inconsistent performance 16 .”
BackgroundHuman heuristics for AI-generated language are flawed
Jakesch, M., Hancock, J. T., & Naaman, M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2208839120 · Cited by 342
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. D. L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.06825 · Cited by 340
Research works printed but not linked: 7
- Taori, R. & Hashimoto, T. Data feedback loops: model-driven amplification of dataset biases. In Proc. 40th International Conference on Machine Learning 33883–33920 (JMLR, 2023).
- Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D. & Finn, C. DetectGPT: zero-shot machine-generated text detection using probability curvature. In Proc. 40th International Conference on Machine Learning 24950–24962 (JMLR, 2023).
- Kuditipudi, R., Thickstun, J., Hashimoto, T. & Liang, P. Robust distortion-free watermarks for language models. Trans. Mach. Learn. Res. https://openreview.net/pdf?id=FpaCL1MO2C (2024).
- Christ, M., Gunn, S. & Zamir, O. Undetectable watermarks for language models. In Proc. Thirty Seventh Conference on Learning Theory 1125–1139 (PMLR, 2024).
- Jovanović, N., Staab, R. & Vechev, M. Watermark stealing in large language models. In Proc. 41st International Conference on Machine Learning 22570–22593 (PMLR, 2024).
- Zhang, H. et al. Watermarks in the sand: impossibility of strong watermarking for language models. In Proc. 41st International Conference on Machine Learning 58851–58880 (PMLR, 2024).
- Holtzman, A., Buys, J., Du, L., Forbes, M. & Choi, Y. The curious case of neural text degeneration. In Proc. Eighth International Conference on Learning Representations (ICLR, 2020).
Cited by
Generative AI in Marketing and Principles for Ethical Design and Deployment
Hermann, E., & Puntoni, S. (2024) · Journal of Public Policy & Marketing · doi:10.1177/07439156241309874 · Cited by 133
“…digital watermarking (i.e., embedding nonvi-sible but identi fi able markers to trace the origin of generated content and differentiate it from human-created content), risk management and veracity checks, and content restoration measures and processes (Dathathri et al. 2024; Epstein et al. 2023).”
Suchikova, Y., Tsybuliak, N., Teixeira da Silva, J. A., & Nazarovets, S. (2026) · Accountability in Research · doi:10.1080/08989621.2025.2544331 · Cited by 70
Understanding World or Predicting Future? A Comprehensive Survey of World Models
Ding, J., Zhang, Y., Shang, Y., Zhang, Y., Zong, Z., Feng, J., Yuan, Y., Su, H., Li, N., Sukiennik, N., et al. (2026) · ACM Computing Surveys · doi:10.1145/3746449 · Cited by 169
“Technology like watermarking could help improve the accountability of world model usage [41].”
Khosravi, B., Purkayastha, S., Erickson, B. J., Trivedi, H., & Gichoya, J. W. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100890 · Cited by 62
Ullah, M., Naeem, S. B., & Boulos, M. N. K. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8120194 · Cited by 69
Metadata
- Authors
- Dathathri, S., See, A., Ghaisas, S., Huang, P.-S., McAdam, R., Welbl, J., Bachani, V., Kaskasoli, A., Stanforth, R., Matejovicova, T., Hayes, J., Vyas, N., Merey, M. A., Brown-Cohen, J., Bunel, R., Balle, B., Cemgil, T., Ahmed, Z., Stacpoole, K., Shumailov, I., Baetu, C., Gowal, S., Hassabis, D., & Kohli, P.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Google (United States) (US)
- Corresponding author
- Dathathri, Sumanth
- Kohli, Pushmeet
- ORCID
- Venue
- Nature
- Published
- 2024-10-23
- Volume
- 634
- Issue
- 8035
- Pages
- 818-823
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC11499265
- OpenAlex
- W4403680773
- Cited by
- 132
- References
- 24
- Citations by year
- 2026: 63 · 2025: 62 · 2024: 7
- FWCI
- 18.43
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- DeepMind
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- text watermarking, large language models, synthetic text detection, watermark detection, Gemini, responsible AI use
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Scalable watermarking for identifying large language model outputs. https://app.cochat.ai/papers/10.1038/s41586-024-08025-4
- The paper
- Dathathri, S., See, A., Ghaisas, S., Huang, P.-S., McAdam, R., Welbl, J., Bachani, V., Kaskasoli, A., Stanforth, R., Matejovicova, T., Hayes, J., Vyas, N., Merey, M. A., Brown-Cohen, J., Bunel, R., Balle, B., Cemgil, T., Ahmed, Z., Stacpoole, K., . . . Kohli, P. (2024). Scalable watermarking for identifying large language model outputs. Nature. https://doi.org/10.1038/s41586-024-08025-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.