The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index; 2 of the 7 sources its claims cite were checked for support.
AI models collapse when trained on recursively generated data
Open accessAbstract
) demonstrated high performance across a variety of language tasks. ChatGPT introduced such language models to the public. It is now clear that generative artificial intelligence (AI) such as large language models (LLMs) is here to stay and will substantially change the ecosystem of online text and images. Here we consider what may happen to GPT-{n} once LLMs contribute much of the text found online. We find that indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear. We refer to this effect as 'model collapse' and show that it can occur in LLMs as well as in variational autoencoders (VAEs) and Gaussian mixture models (GMMs). We build theoretical intuition behind the phenomenon and portray its ubiquity among all learned generative models. We demonstrate that it must be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web. Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of LLM-generated content in data crawled from the Internet.
Overview
CoChat Summary
TL;DR
Indiscriminate use of model-generated content in training causes “model collapse,” a degenerative process in which generative models forget the true data distribution, beginning with the tails and eventually converging to a low-variance point estimate. The authors demonstrate this theoretically and empirically for Gaussian mixture models, variational autoencoders, and large language models.
Key findings
- Model collapse occurs across generative model families: in discrete distributions with exact approximation, the process converges to a delta function with probability 1; in the Gaussian setting, Theorem 3.1 states that the Wasserstein-2 distance between the true distribution and its approximation diverges arbitrarily far while variance collapses to zero as generations increase.
- In LLM experiments, an OPT-125m model fine-tuned on wikitext2 reaches 34 mean perplexity from a zero-shot baseline of 115. With five epochs and no original training data, performance degrades “from 20 to 28 perplexity points”; with ten epochs and 10% of original training data preserved, there is “only minor degradation of performance.”
- Generated data over generations contain more sequences the original model would produce with higher likelihood, but also much longer tails of sequences the original model would never produce, indicating accumulated errors.
- An ablation with a repeating penalty of 2.0 causes models to produce lower-score continuations to avoid repeats, and “causes the perplexity to double compared with the original”; models remain as susceptible to model collapse, if not more.
Methods
- Theoretical analysis of “learning with generational data”: a discrete distribution case with exact approximation, analyzed as a Markov chain, and a multidimensional Gaussian approximation using unbiased sample mean and variance estimators, with Wasserstein-2 distance as the divergence measure.
- LLM experiments fine-tune the OPT-125m causal language model from Meta via Hugging Face on wikitext2. Data generation uses five-way beam search; training sequences are blocked to 64 tokens, and for each token sequence the model predicts the next 64 tokens, producing an artificial dataset of the same size. Each experiment is run five times with different randomness seeds, and the best-performing model on the original task is used as the base for subsequent generations.
- Two LLM settings are tested: five epochs with no original training data retained, and ten epochs with a random 10% of original data points sampled for each new generation. VAEs and GMMs are covered in the Supplementary Materials.
Limitations
- The paper states that theoretical claims are mostly presented in the limit of generations going to infinity, whereas experiments use a finite, usually small, number of generations; complete collapse may or may not occur even after a small number of steps, and convergence rates are highly related to the specifics of the problem.
- The paper states that larger language models in non-fine-tuning settings were not run because training a single moderately large model produces twice the American lifetime’s worth of CO2; the LLM experiments are a fine-tuning proof of concept, and even those took weeks to run.
- Evident from the text: the LLM evidence is limited to OPT-125m, wikitext2, five runs, and fine-tuning; no full retraining from scratch is tested, and VAE/GMM details are relegated to the Supplementary Materials.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
1 Valid · 0 Not valid · 3 Not checked · 40 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The repeating problem has been observed in nearly all text-generation models17,18”
§ AI models collapse when trained on recursively generated data > Model collapse in language models > Ablation: Repetitions
What the sources say
- Partly supportedcites [17] Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C. & Socher, R. CTRL: a conditional transformer language model for controllable generation. Preprint at https://arxiv.org/abs/1909.05858 (2019). sourceExperimental — accuracy not yet measured
The Related Work section (p. 10) identifies repetition as a problem in text generation and discusses methods for reducing it. It does not say the problem has been observed in nearly all text-generation models; that prevalence claim is unsupported. (LLM, conf=0.86)
- Could not verifycites [18] Shumailov, I. et al. in Proc. 2021 IEEE European Symposium on Security and Privacy (EuroS&P) 212–231 (IEEE, 2021). sourceExperimental — accuracy not yet measured
Section 5.2 discusses attacks that increase output length, energy use, and latency, but not repetitive generated text. Searches for “repeating problem,” “repeated output,” and “repeat” found no passage saying that repetition has been observed in nearly all text-generation models; page numbers were unavailable in our copy. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Here, over the generations, models tend to produce more sequences that the original model would produce with the higher likelihood.”
§ AI models collapse when trained on recursively generated data > Model collapse in language models
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We find that data generated by language models in our experiments end up containing a large number of repeating phrases, as in Example 1.”
§ AI models collapse when trained on recursively generated data > Model collapse in language models > Ablation: Repetitions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we can show that the probability of losing information (that is, sampling no data at some generation) is equal to 1 − q”
§ AI models collapse when trained on recursively generated data > Theoretical intuition > Discrete distributions with exact approximation
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We discover that indiscriminately learning from data produced by other models causes ‘model collapse’—a degenerative process whereby, over time, models forget the true underlying data distribution, even in the absence of a shift in the distribution over time.”
§ AI models collapse when trained on recursively generated data > Main
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
11 more claims
Create a free account to see all 44 claims and every source verdict.
- It may state two separate claims as one. 22
- The quote only partly states this claim. 1
- Probably not a substantial claim. 10
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Second, LLMs are frequently trained on the same open-access corpora we examine, creating a feedback loop in which tomorrow's models will learn today's hallucinations [34].”
From LLM hallucinations in the wild: Large-scale evidence from non-existent citations (2026) · cites this paper as [34]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Successive training cycles cause LLMs to “forget” the statistical long-tail of rare but important patterns, resulting in homogenized, impoverished, and epistemically diminished outputs.”
From 4E cognition and the coevolution of human–AI interaction (2025) · cites this paper as [61]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AI-generated content now saturates the internet, reducing reliability and contaminating training data for future models (Paredes et al., 2025;Shumailov et al., 2024).”
From Purpose before policy: academic integrity, generative AI, and rhetorical stance (2026) · cites this paper as Shumailov et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As successive generations of models are trained on data increasingly contaminated by historical synthetic outputs, the system progressively erodes distributional diversity and collapses functional modes—a degenerative process that remains robust to scale and is highly sensitive to even minimal synthetic fractions [14,15].”
From Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement (2026) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The accumulation of synthetic data may affect model output quality, reinforce existing biases, and alter information feedback dynamics, thereby posing potential challenges to the stability of the digital knowledge ecosystem (Shumailov et al., 2024).”
From The Attention Mismatch: Mapping the Structural Academic Governance Deficit in the Age of Generative AI (2026) · cites this paper as Shumailov et al., 2024
Experimental — accuracy not yet measured
19 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ AI models collapse when trained on recursively generated data > Abstract
“GPT-2 (ref. 1), GPT-3(.5) (ref. 2) and GPT-4 (ref. 3)”
§ AI models collapse when trained on recursively generated data > Model collapse in language models
“BERT4, RoBERTa5 or GPT-2 (ref. 2)”
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
5th International Conference on Learning Representations (ICLR 17)
Havrylov, S., & Titov, I. (2017) · Cited by 3,503
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
The Implicit Bias of Gradient Descent on Separable DataPreprint
Soudry, D., Hoffer, E., Nacson, M. S., Gunasekar, S., & Srebro, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10345 · Cited by 829
CTRL: A Conditional Transformer Language Model for Controllable Generation
Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751
Opt: Open pre-trained transformer language models
Zhang, S. (2025) · TIB Data Manager · doi:10.57702/dn0pmcbs · Cited by 405
Aljundi, R., Kelchtermans, K., & Tuytelaars, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01151 · Cited by 324 · cited 1×
“We also briefly mention two close concepts to model collapse from the existing literature: cata-strophic forgetting arising in the framework of task-free continual learning 7 and data poisoning 8,9 maliciously leading to unintended behaviour.”
Black Swans and the Domains of Statistics
Taleb, N. N. (2007) · The American Statistician · doi:10.1198/000313007x219996 · Cited by 267
The Curse of Recursion: Training on Generated Data Makes Models Forget
Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Papernot, N., & Anderson, R. (2023) · arXiv · doi:10.48550/arxiv.2305.17493 · Cited by 151
Self-citation
Research works printed but not linked: 7
- Radford, A. et al. Language models are unsupervised multitask learners. OpenAI blog 1, 9 (2019).
- Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. in Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (eds Burstein, J., Doran, C. & Solorio, T.) 4171–4186 (Association for Computational Linguistics, 2019).
- Carlini, N. & Terzis, A. in Proc. Tenth International Conference on Learning Representations (ICLR, 2022).
- Carlini, N. et al. in Proc. 2024 IEEE Symposium on Security and Privacy (SP) 179 (IEEE, 2024).
- Mousavi-Hosseini, A., Park, S., Girotti, M., Mitliagkas, I. & Erdogdu, M. A. in Proc. Eleventh International Conference on Learning Representations (ICLR, 2023).
- Gu, Y., Dong, L., Wei, F. & Huang, M. in Proc. Twelfth International Conference on Learning Representations (ICLR, 2024).
- Strubell, E., Ganesh, A. & McCallum, A. in Proc. 57th Annual Meeting of the Association for Computational Linguistics (eds Korhonen, A., Traum, D. & Màrquez, L.) 3645–3650 (Association for Computational Linguistics, 2019).
Cited by
A Survey of Large Language Models in Medicine: Progress, Application, and Challenge
Zhou, H., Liu, F., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S. S., Zhou, P., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05112 · Cited by 240
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Wong, D. F., & Chao, L. S. (2023) · arXiv · doi:10.48550/arxiv.2310.14724 · Cited by 212
Taeihagh, A. (2025) · Policy and Society · doi:10.1093/polsoc/puaf001 · Cited by 190
“Moreover, recent research shows that indiscriminate use of synthetic content from the models in training can eventually cause model collapse (Shumailov et al., 2024).”
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212
Artificial Intelligence for Materials Discovery, Development, and Optimization
Madika, B., Saha, A., Kang, C., Buyantogtokh, B., Agar, J., Wolverton, C. M., Voorhees, P., Littlewood, P., Kalinin, S., & Hong, S. (2025) · ACS Nano · doi:10.1021/acsnano.5c04200 · Cited by 171
Metadata
- Authors
- Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y.
- Affiliations
- University of Oxford (GB)
- University of Cambridge (GB)
- Imperial College London (GB)
- University of Toronto (CA)
- Corresponding author
- Shumailov, Ilia
- Shumaylov, Zakhar
- Gal, Yarin
- Venue
- Nature
- Published
- 2024-07-24
- Volume
- 631
- Issue
- 8022
- Pages
- 755-759
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC11269175
- OpenAlex
- W4400949264
- Semantic Scholar
- 603d3f90fc40f79ff51258f0295de3ec5107f73e
- Cited by
- 793
- References
- 11
- Citations by year
- 2026: 407 · 2025: 330 · 2024: 56
- FWCI
- 116.53
- Type
- journal article
- Language
- en
- Update notices
- correction (2025-03-21)
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Apollo (University of Cambridge) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- model collapse, large language models, self-training, generative AI training data, variational autoencoder, Gaussian mixture model, distribution shift, web scraping, ChatGPT, generative models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: AI models collapse when trained on recursively generated data. https://app.cochat.ai/papers/10.1038/s41586-024-07566-y
- The paper
- Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024). AI models collapse when trained on recursively generated data. Nature. https://doi.org/10.1038/s41586-024-07566-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.