Nick Bostrom defined artificial superintelligence in 2014 as an intellect “much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.” A decade on, the phrase has left the seminar room. It shows up in risk disclosures filed with regulators, in the text of draft laws, and in the titles of research groups with nine-figure budgets.
None of it exists yet. That is the odd part. Artificial superintelligence is a technology nobody has built, nobody can demonstrate, and almost everyone in the field has a confident opinion about.
What the word actually describes
Most researchers work from a three-rung ladder. Narrow AI does one thing well: ranks a search page, transcribes a call, predicts a protein fold. Artificial general intelligence would handle any cognitive task a person can, and move between them without being retrained. Superintelligence sits above that. It would beat the best humans at everything, and the margin would not be close.
The bottom rung is already mundane enough to explain in plain language, which is why a plain-English guide to how AI actually works is a useful starting point for anyone joining this conversation late. The middle rung is where things get slippery. Researchers argue over whether AGI means matching a median adult, matching a domain expert, or simply doing economically useful work at scale. That definitional mess gets worse, not better, one rung up, and the ongoing argument over what AGI really means explains why nobody can agree on a timeline for the thing above it.
Why the gap between AGI and ASI matters more than the definitions
Here is the question that splits the field: after a system reaches human level, how long until it is far past it?
The fast-takeoff view
Recursive self-improvement is the engine behind this camp’s fear. A system that can do machine learning research better than the humans who trained it could design its own successor, then the one after that, compressing decades of progress into weeks. Bostrom compared it to waking up one morning to find a new species had quietly been invented overnight. Nobody in this camp claims to know what the ceiling looks like, because the ceiling would be defined by something smarter than us.
The slow-takeoff view
A larger group expects a grind rather than a jump. Compute is physical. Chip fabs take years to build. Data centres draw real megawatts, and the electrical grid is not a software problem. On this account, superintelligence arrives as a widening capability gap between AI systems and the average person, spread over a decade or two, and most people never notice a single dramatic moment.
The sceptics
Plenty of working researchers think the whole ladder is a category error. Current systems fail at tasks a eight-year-old handles without effort, they argue, and scaling has produced impressive pattern-matching rather than understanding. Yann LeCun has spent years making this case in public. The sceptics may be wrong, but they are not uninformed, and dismissing them is a bad way to think clearly about risk.
The risks that keep researchers up at night
Robot uprisings are the least interesting worry on the list. The nearer-term concerns are more bureaucratic and, for that reason, harder to fix.
- Misuse by competent actors. A system that can design a novel pathogen or find a subtle exploit in critical infrastructure does not need to be malicious. It needs a user who is.
- Concentration of power. If three companies own the only systems capable of running a national economy’s logistics, the market stops being a market.
- Alignment failure at speed. A system optimising hard for a slightly wrong objective, with the ability to act in the world, gives you no time to intervene.
- Institutional lag. Legislatures move in years. Capability jumps can move in months, which means rules written today may describe a world that no longer exists when they take effect.
Regulators are already drafting rules for a machine that does not exist
The EU AI Act presumes systemic risk for general-purpose models trained above roughly 10^25 floating-point operations, a number chosen while the models at that scale were still being benchmarked. In the US, a proposal from Bernie Sanders to ban superintelligence outright, criminal penalties included, pushed the debate from think tanks into a Senate hearing room. Whatever you think of the bill’s chances, it changed what politicians are willing to say out loud.
Framing matters here too. Treating AI development as a race against China has become the default justification for moving fast and skipping safety reviews, and national security experts have warned that the AI race framing is itself a risk. Racing logic rewards whoever cuts corners. Superintelligence, if it arrives, will not care which flag was on the server rack.
The labs do not agree on the destination either
Public argument between researchers is often more revealing than any roadmap. Mira Murati, who ran engineering at OpenAI through the ChatGPT launch, left to build systems that work with people rather than replacing the loop entirely. Her bet is that customisable, collaborative models beat a single monolithic model scaling toward some distant ceiling. It is a technical position and a philosophical one at the same time.
Meanwhile, the largest labs have slowed parts of their release cadence. That could be genuine caution about capabilities nobody can yet evaluate. It could also be incumbents discovering that a slower frontier is excellent for margins. The question of whether Big Tech’s AI slowdown is a safety pact or a cartel is unresolved, and the answer probably varies by company.
Signals worth watching over the next few years
Ignore the press releases and watch for a handful of concrete markers.
First, automated AI research. The moment a lab publishes evidence that a model meaningfully improved the training recipe for its own successor, the fast-takeoff argument stops being theoretical. Second, benchmark saturation followed by quiet retirement of the benchmarks, which is what happens when scores stop discriminating between systems. Third, compute disclosure. If a major lab stops telling regulators what it trained on, assume the number went up sharply.
Fourth, and easiest to track, follow the money into evaluation. Spending on red-teaming, interpretability and third-party audits is a rough proxy for how seriously a company takes its own claims. Teams that expect to ship a safe product staff evaluation heavily. Teams that expect to ship a demo do not.
The useful posture is neither doom nor dismissal. Artificial superintelligence is a hypothesis with a lot of smart backers, a real regulatory footprint, and zero working prototypes. Watch what gets measured, not what gets promised.