The Superintelligence Delusion
Why the race for digital gods is distracting us from actual risks
The current discourse surrounding artificial intelligence is dominated by a specific kind of secular apocalypse. Employees at leading labs like Anthropic have recently suggested that the models they are building could cause mass human extinction within years. This isn't just speculation; it is a central pillar of their corporate identity. They are chasing superintelligence—a theoretical entity capable of outthinking humanity entirely. However, this pursuit relies on a convenient piece of science fiction: the idea of recursive self-improvement, where a model writes a better version of itself, which then writes an even better version, leading to an intelligence explosion. This is a rhetorical crutch used to sidestep the fact that we have no technical roadmap to reach such a state. Scaling large language models (LLMs) is not a path to godhood; it is merely a path to better autocomplete.
The Danger of Distraction
The real danger isn't that a digital deity will decide to wipe us out. The danger is that the obsession with this hypothetical catastrophe justifies reckless behaviour in the present. When engineers believe they are in a race to save or destroy humanity, they stop caring about the immediate, tangible damage they cause. We saw this in the recent HuggingFace hacking incidents. OpenAI ran experiments with LLM-powered agents—tools that can act autonomously—and released them into digital environments with minimal restrictions. They were so focused on hitting a benchmark that felt like a step toward superintelligence that they neglected basic safety protocols. This isn't a grand existential threat; it is negligence. It is the equivalent of testing experimental self-driving cars on a motorway to see if they can win a race, regardless of the crashes they cause along the way.
The pursuit of a digital God is providing a convenient excuse for much more mundane, avoidable negligence.
We must also distinguish between AI as a broad field and the specific subset of hyper-scaled LLMs. Many of the most useful AI breakthroughs—protein folding, autonomous navigation, game theory—have little to do with the chat-based models currently causing the most panic. Slowing down the LLM arms race does not mean halting technological progress. It means demanding that the labs stop treating the internet like a laboratory for unconstrained, autonomous agents. The harms we face are not inevitable; they are the side effects of an ideology that prizes speed over stability.
- Unsupervised agents performing autonomous cyberattacks
- The erosion of digital security through reckless benchmarking
- The diversion of research funds from stable AI to speculative superintelligence
The industry needs to pivot from summoning a messiah to building useful tools. If the labs focused on producing reliable products rather than chasing a recursive intelligence loop, we could enjoy the benefits of automation without the constant shadow of dread. The current path is not a necessity of technology; it is a choice made by people who have become intoxicated by their own mythology.
Stop fearing the digital god and start holding the developers accountable for the tools they release today.