☀️ AI Morning Minute: Overfitting
When AI aces the practice test and bombs the real one
You’d think an AI that scores 100% on its training data would be the best model you could build. It’s not. It’s often the worst. The model didn’t learn the subject. It memorized the answers. And the moment it sees a question it hasn’t seen before, it falls apart.
What it means
Overfitting is what happens when an AI model learns the specific details and noise in its training data so precisely that it loses the ability to handle new, unseen data. Instead of learning general patterns (”dogs have four legs and fur”), the model memorizes specific examples (”this exact photo of this exact golden retriever at this exact angle is a dog”).
It performs really well on data it’s seen before and terribly on everything else. The opposite problem, underfitting, is when a model is too simple to capture any useful patterns at all. The goal is the sweet spot in between: general enough to work on new data, specific enough to be accurate.
Why it matters
It’s one of the oldest and most common failures in machine learning. Every data scientist has shipped a model that looked great in testing and collapsed in production. The training metrics said 99% accuracy. The real-world accuracy was 60%. The gap between those two numbers is overfitting.
It explains why more data doesn’t always mean better results. A model trained on a small, biased dataset will overfit to whatever quirks that dataset contains. If every photo of a cat in your training set was taken indoors, the model might learn that “cat” means “indoor lighting” rather than “animal with whiskers.” More data helps, but only if the data is diverse enough to teach general patterns instead of specific ones.
The fixes are counterintuitive. You make the model worse on purpose. Techniques like dropout (randomly turning off parts of the neural network during training), regularization (penalizing complexity), and early stopping (ending training before the model has fully memorized the data) all work by preventing the model from getting too good at the training set. You’re intentionally adding imperfection so the model stays flexible.
Simple example
A student studies for a history exam by memorizing every answer on last year’s test word for word. Last year’s test asked “What year did the Civil War end?” and the answer was 1865. This year’s test asks “What event ended in 1865?” Same knowledge, different question.
The student who memorized can’t answer it.
The student who understood the material can. Overfitting is the AI version of memorizing without understanding. The scores look perfect until the questions change.