The use of artificial intelligence (AI) is rapidly growing. In the past year, Google searches for AI have reached 92 percent of their highest levels ever. However, new research warns that AI’s rapid rise might also lead to serious problems.
A team of researchers from Cambridge and Oxford universities decided to investigate what happens when AI tools use content created by other AI systems. Their findings are a cause for concern.
Dr. Ilia Shumailov from the University of Oxford and his research team found that when generative AI software depends only on content created by other AI, the quality of its responses begins to decline. This discovery was detailed in a study published last month in the journal Nature.
AI responses begin to degrade after the first two prompts
Researchers observed that after the first two prompts, the AI’s responses began to miss the point. By the fifth attempt, the quality dropped significantly. By the ninth consecutive query, the answers became complete nonsense.
They called this decline “model collapse.” This happens when AI repeatedly relies on content generated by other AI, leading to a steady deterioration in its responses. Over time, this cycle pollutes training data, resulting in output that is a distorted and useless version of reality.
Shumailov noted the surprising speed at which model collapse begins and how difficult it can be to detect. Initially, it impacts less common data, which is not well represented. Then, it starts reducing AI output diversity, making it more uniform.
Occasionally, there might be a slight improvement in the majority of data, which might mask the decline in performance on less common data. Shumailov warned that model collapse could lead to serious problems. He shared his insight on this in an email exchange.
More than half of all text on the internet is AI generated or translated
This issue is significant because about 57 percent of all text on the internet is either generated or translated by AI, according to a separate study by Amazon Web Services researchers published in June.
If human-created content online is rapidly being replaced by AI-generated material and if Shumailov’s findings are accurate, it raises the alarming possibility that AI might be undermining its own effectiveness—and potentially harming the internet itself.
The team confirmed model collapse by starting with a pre-trained AI-powered wiki. They then allowed the AI to update itself based on its own generated outputs. Over time, this self-referencing led to the original, accurate information being corrupted. The content gradually became meaningless.
For example, after nine cycles of this process, a study’s wiki article on 14th-century English church steeples had absurdly transformed into a jumbled thesis about the colors of jack-tailed rabbits. This shift clearly demonstrated how the data had degraded into nonsense.