What exactly is model collapse?
Model collapse is a degenerative process where a generative AI model, such as a large language model, loses its ability to produce diverse and high-quality outputs after being trained on data that includes content generated by other AI models. Think of it like a photocopy of a photocopy: each generation loses detail and introduces errors, until the final copy is a blurry, useless mess. The landmark study in Nature [1] defines it as 'irreversible defects' in which the 'tails of the original content distribution disappear'—meaning the model forgets rare but important information, like niche vocabulary or unusual facts.
The same study [1] demonstrated this effect across multiple types of models, including LLMs, variational autoencoders (VAEs), and Gaussian mixture models (GMMs), showing it is a universal problem for generative AI. They built a theoretical framework to explain why it happens, and warned that the value of data from genuine human interactions will become increasingly valuable as more AI-generated content floods the internet.
How much synthetic data is dangerous? Even a tiny amount.
You might think that as long as most of the training data is human-written, a little AI-generated text won't hurt. The evidence says otherwise. A 2024 study [5] found that even a fraction as small as 1% synthetic data in the training set can lead to 'strong model collapse'—a critical performance degradation where larger and larger training sets no longer improve the model. This is a striking finding because it contradicts the usual scaling law assumption that more data always helps.
The same study [5] also investigated whether making models bigger—the current trend with models like GPT-4—helps or hurts. In a simplified regime, they found that larger models can actually amplify the collapse, making the problem worse. However, they noted that beyond a certain 'interpolation threshold' (which can be extremely high for very large datasets), larger models might mitigate the collapse slightly, though they cannot prevent it entirely. This suggests that simply scaling up is not a solution.
Can we prevent model collapse? Yes, but it requires careful strategies.
The good news is that researchers are developing ways to avoid model collapse. One simple but effective approach, proposed in a 2025 study [3], is to carefully control the ratio of synthetic to human-generated data during training. They showed mathematically that by maintaining a proper balance, you can ensure the model reaches a stable, high-quality equilibrium rather than degenerating.
Another promising strategy is to use verification—having a separate system (or a human) check the quality of AI-generated data before using it for training. A 2024 study [4] provided a theoretical characterization of this approach using Gaussian mixtures and linear classifiers, and then tested it on two practical tasks: computing matrix eigenvalues with transformers and news summarization with LLMs. They found that even imperfect verifiers can prevent model collapse, and they developed a measurable proxy to predict when verification will work. This is important because it's often easier to check if an output is good than to generate a good output from scratch.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2024 to 2025, 5 from 2024 or later, 1 in Q1 journals, collectively cited 429 times — selected as the most relevant from 7 studies that passed quality screening, drawn from 32 papers retrieved from a database of over 500 million.
Sources used in this answer
AI models collapse when trained on recursively generated data
This Nature paper is the foundational study on model collapse, showing it causes irreversible defects in LLMs, VAEs, and GMMs when trained on recursively generated data, with tails of the original distribution disappearing.
A Lifecycle-Oriented Survey of Emerging Threats and Vulnerabilities in Large Language Models
This 2025 survey identifies model collapse as an emerging threat in the LLM lifecycle, categorizing it alongside 16 other vulnerabilities and calling for more research into its causes and mitigations.
Preventing Model Collapse when Training LLMs with Synthetic Data
This 2025 paper proposes a simple strategy to prevent model collapse by controlling the ratio of synthetic to human-generated data, proving mathematically that a stable equilibrium exists under the right conditions.
Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification
This 2024 ICLR paper shows that using verifiers on synthesized data can prevent model collapse, even with imperfect verifiers, and provides a theoretical framework with experiments on transformers and LLMs.
Strong Model Collapse
This 2024 ICLR paper establishes 'strong model collapse,' showing that as little as 1% synthetic data can cause collapse, and that larger models can amplify the problem, though beyond a high interpolation threshold they may slightly mitigate it.