A rambling about AI, model collapse, and where new thoughts come from.

What happens if humans only listen to the AI-generated echo chamber, and their new original content is actually echoes of the model collapse?

I have been thinking about AI, writing, and other deep thoughts while I sit in the reeds, listen to the swamp, and watch life drift by. There have been a number of concepts that I have run into while I ponder, and as my mind works, I have tried to mesh them together, see how they fit, and if they have a greater significance. And I keep coming back to this main question: what happens if human creativity slows down while AI continues to grind away? What happens if fewer people create something genuinely new because it is easier to ask AI to create it for them?

What if the echo chamber eventually becomes an AI echo chamber—one increasingly filled not with new, original human thought, but with AI-generated content and human-created content that was itself shaped by what AI had already produced?

What happens when both the machine and the human begin learning from the same echo? That's When We Become the Echo.

Let me explain what I have come up with. This is what I am going to term the Echo Chamber. This is an older idea, but an echo chamber is when you sit and all that you experience is the echo of the same ideas over and over again. I am going to mix the common idea of an echo chamber with a literal echo in a canyon. Basically like you are yelling into a canyon and you hear the echo back. But each time you hear the echo, it is a fragment or only part of what it was originally; it becomes quieter and less full each time. The chamber can never restore the original because it eventually does not have the original, it cannot create a new echo by itself, and it has to have a new echo introduced into the chamber. It is kind of like the "Second Law of Thermodynamics." I am using it as an analogy, not an AI mechanism. If you simply recycle an output indefinitely, it is eventually lost, or something akin to entropy occurs.

Let's walk down a path together; let's get our feet wet and enter the swamps. AI right now is being trained on content. Companies are spending millions of dollars to purchase human-created content, books, magazines, and art. That human content is then fed into training models. AI learns patterns from it and can then generate new content based on what it has learned. People, instead of reading the original content, use the summaries created by AI. They rely on the content fed to them by AI. What happens if AI content becomes the main echo in our chamber, and eventually the new original content is the oddity, not the AI content being the oddity. Then humans train themselves on the AI-generated content and feed that back into the echo chamber. AI then trains itself on that new human echoes. Eventually all that is in the chamber is echoes of echoes.

As the chamber builds content from the echoes in the chamber, we see what I would liken to Entropy. The content starts out rich, diverse, and full of original human content. It makes its rounds and AI trains and outputs the content. That content is then used to train AI again, either by humans spitting it out as content or AI picking up other AI content. As the cycle continues, the diversity can decrease, the unusual becomes harder to hear, and eventually you can have model collapse. The dominant patterns remain while some of the edges begin to fade.

There is actually a term for something similar to this happening with AI: model collapse. Researchers have found that when AI models are repeatedly trained using content generated by other AI models, some of the information contained in the original training data can begin to disappear. The unusual things, the less common things, the things out on the edges can become less represented while the more common patterns remain. The things that are human and not AI in nature. Feed those results back into another generation, and the representation can become narrower again. The echo starts losing parts of the original human voice.

But model collapse is looking at what happens to the machine. That is not really the part that has me sitting in the reeds thinking. I started wondering what happens when you put humans into that same loop. What if we increasingly read, watch, and learn from what AI creates? Then even when we sit down and create something ourselves, how much of our new human-created content is actually influenced by the AI-generated content we have been consuming? It may have been written by a human, but is it a new voice entering the chamber, or is the human now repeating the echo?

It is hard for me to explain what I am thinking. Here is an infographic that explains it visually.

As AI becomes better, people are less likely to generate new content but rely on what is fed to them by the echo chamber. This is a major concern currently in the writers' world. If you do a search for AI and writing, you will see what I am talking about. There are articles everywhere about the evils of AI and what some writers have classified as AI slop. AI content is everywhere. What happens if people give up on writing? How can they compete with the speed and polished articles produced by AI? What happens if the echo chamber no longer gets original, rich, messy, unpredictable human-created content?

This future is a possibility, but only a possibility, and I would say not very probable. Because, like me and others, we are thinking, the world is full of thinkers. People who are willing to not just listen to the echo chamber but to question it and to add original content. I personally don't think that the echo chamber will ever become the echo chamber of only AI. I think that humanity and socieity will not allow it to happen. Even if the human content that starts its echo in the chamber is not polished, syntax perfected, it will keep the chamber fresh.