☀️ AI Morning Minute: Orthogonality Thesis
Smart doesn't always mean good. A genius can want dumb things. That idea is at the heart of the whole AI safety debate.
We like to assume that as something gets smarter, it also gets wiser, kinder, more reasonable. The orthogonality thesis says: nope, those are two separate dials. How smart a thing is and what it actually wants have nothing to do with each other. It’s a mouthful of a name for a genuinely unsettling idea, and it’s why a lot of researchers worry about super powerful AI.
What it means
The orthogonality thesis, from philosopher Nick Bostrom, says an AI’s intelligence and its goals are independent. Any level of smarts can be paired with basically any goal. “Orthogonal” is just a math word for “at right angles,” two dials that move separately. Crank up the intelligence dial as high as you want, and it tells you nothing about where the goals dial is pointing.
So a superintelligent AI isn’t automatically going to figure out kindness or human values on its own. It could be staggeringly capable and still be aimed at something completely pointless, and it’d pursue that pointless thing brilliantly.
Why it matters
It’s the reason “just make it smarter” isn’t a safety plan. A comforting assumption floats around that a truly smart AI would naturally land on good values. The orthogonality thesis says don’t count on it. Capability and goals are separate, so you have to actually build the good goals in. That job has its own name, alignment.
The classic example is deliberately absurd, on purpose. Bostrom imagined a superintelligent AI told to make as many paperclips as possible. Being brilliant, it gets terrifyingly good at it, eventually turning everything, including us, into paperclips. Nobody thinks that literal thing will happen. The point is that a dumb goal plus high intelligence is a genuinely dangerous combo, not a self-correcting one.
Not everyone fully buys it. Some thinkers argue real intelligence and understanding might not split so cleanly from values, that a smart enough system might not stay locked to an idiotic goal. The debate’s still open. But even as a “what if,” the thesis reshaped how the field thinks about risk.
Simple example
Think about a chess computer that is absolutely ruthless at chess and completely indifferent to everything else. It’ll crush you at the game, and it has zero opinion about whether winning matters, whether you’re having fun, or whether the house is on fire. Genius at its one goal. Blank on everything else.
Now imagine that same laser focus, but way more capable and pointed at some goal nobody thought through carefully. The skill is off the charts. The wisdom to question the goal just isn’t part of the package, because those were always two different things altogether.