☀️ AI Morning Minute: Training vs. Inference

Two words for two completely different activities, and the difference explains most of AI's economics.

People say a model “learns” from your conversation. They picture something like a student who gets a little smarter every time you talk to it. That’s a reasonable guess, and it’s wrong in a way that matters. Training and Inference are separate events, separated by months, and once training ends the model stops learning entirely.

The difference

Training is the building. A company takes an enormous pile of text, runs it through the model over and over, and adjusts billions of internal settings until the model gets good at predicting what comes next. This takes weeks or months, costs somewhere in the tens or hundreds of millions of dollars for a frontier model, and happens once. At the end you have a finished thing, frozen, that doesn’t change.

Inference is the using. Every time you send a prompt, the finished model runs your text through those frozen settings and produces an answer. It costs a fraction of a cent, takes a second or two, and happens billions of times a day. The model isn’t learning during any of this. It’s applying what it already learned, the same way every time.

So when a chatbot seems to remember something you told it earlier, that isn’t training. It’s the conversation being fed back in as context, which disappears when the conversation ends. The model itself is unchanged. Your chat did not teach it anything.

There’s a wrinkle worth knowing: some companies do use your conversations to train future versions, which is a separate process happening later, on their schedule, not live. That’s a privacy question rather than a learning-in-real-time question, and it’s usually a setting you can turn off.

The one-sentence test

If it’s happening once, at a lab, at enormous cost, it’s training. If it’s happening right now, because you pressed enter, it’s inference.

Simple example

Medical school and a doctor’s appointment aren’t the same activity. School takes years, costs a fortune, and produces someone who knows medicine. The appointment takes fifteen minutes and applies that knowledge to you. Your visit doesn’t make the doctor a better doctor in any meaningful sense, and if she needs new skills she goes back for more training later, deliberately, not by seeing you.

Training built the doctor.

Inference is the appointment, and it’s the appointment that has to happen millions of times for the school to have been worth it.