☀️ AI Morning Minute: Foom
The scenario AI safety researchers argue about most. And hope never happens.
Foom is shorthand for a specific AI scenario: a system that can improve its own intelligence starts doing so recursively, each improvement making the next one faster, until it goes from roughly human-level to superintelligence in a very short time. Not years. Possibly hours. The word sounds absurd. The possibility is what keeps some researchers up at night.
What it means
The term comes from Eliezer Yudkowsky, an AI safety researcher who used it to describe a “hard takeoff”: an intelligence explosion so fast that humans can’t meaningfully respond. You’ll sometimes see it written as an acronym — Fast Onset of Overwhelming Mastery — though whether that’s the origin or a backronym the community attached later isn’t entirely settled. It was the subject of a famous public debate with economist Robin Hanson in the early 2010s.
Hanson argued AI capability would grow gradually, giving society time to adapt. Yudkowsky argued that once a system could meaningfully improve itself, the jump to superintelligence wouldn’t be gradual. It would be nearly instant.
The opposite position is called “slow takeoff”: capability grows over years, with time to regulate and course-correct. Most current AI researchers lean toward slow takeoff. But “more likely” isn’t “impossible,” and the consequences of being wrong are asymmetric.
Why it matters
Foom makes the Alignment problem far harder. If an AI transitions to superintelligence over years, there’s time to identify and fix misaligned goals. If it happens in hours, there isn’t, which is why alignment researchers argue for getting values right before capability crosses certain thresholds, not after.
Today’s AI systems don’t self-improve in this way — they don’t rewrite their own weights or redesign their own architecture. But the foom debate shaped how AI safety research gets funded and prioritized, pushing it into mainstream labs well before hard takeoff is anywhere close to plausible.
The word has become a signal. Researchers who take hard takeoff seriously are sometimes called “foomers” by skeptics. The underlying disagreement — fast versus slow takeoff — is still live among people who study what happens when AI gets very good very fast.
Simple example
Compound interest is the closest everyday analogy. A dollar earning 10% annually grows slowly at first, then the curve bends sharply upward. Foom is what happens if an AI’s self-improvement compounds not annually but hourly.
Each smarter version is better at making the next version smarter. By the time the curve goes vertical, the window for doing anything about it has already closed.