☀️ AI Morning Minute: Model Degradation
AI models don’t stay sharp forever. The world moves, and they don’t always move with it.
You build an AI tool, it works great, and six months later people are complaining it feels off. The model didn’t change. But everything around it did: the products it references, the prices it quotes, the policies it explains.
That gap between what the model learned and what’s true right now is model degradation. It’s one of the most common and least glamorous problems in AI deployment.
What it means
Model degradation is the gradual decline in an AI model’s accuracy or usefulness over time. It takes two main forms. The first is data drift, where the real world changes but the model doesn’t. A model trained on data from two years ago has no knowledge of events, products, or terminology that emerged since then. The second is fine-tuning degradation, sometimes called catastrophic forgetting, where updates or additional training on new data cause the model to lose capabilities it had before. A model retrained to handle a new product line might start giving worse answers about the original one. Both forms are real, both are common, and neither announces itself until users start noticing.
Why it matters
Degradation is often invisible until it causes a problem. A customer service bot trained before a major policy change keeps giving customers the old policy. A legal research tool trained before new case law drops answers that are technically accurate but no longer current. By the time someone catches it, the damage is done.
The cost of monitoring for degradation is real, but the cost of ignoring it is higher. A 2024 study found that model performance on production tasks dropped an average of 20% within six months of deployment without active monitoring and retraining. That’s not a theoretical risk.
Fixing degradation isn’t always as simple as retraining the model. Retraining takes time and compute, and as noted above, it can introduce new degradation in areas that were working fine. Teams managing production AI systems increasingly treat degradation monitoring as an ongoing operational task, not a one-time launch checklist item.
Simple example
A mortgage calculator tool built on an AI model trained in early 2024 was quoting interest rate ranges that made sense when it launched. By late 2025, rates had shifted enough that its estimates were consistently off.
No one changed the model. No one changed the tool. The world just moved, and the model didn’t follow. Users started trusting it less without knowing exactly why. That’s model degradation in the form most organizations actually encounter it.