Every time you send a message to an AI model, it doesn’t just read your new question. It rereads everything that came before it: the instructions, the document you uploaded, the whole chat so far. For a long conversation, that’s a lot of rereading.
Prompt caching is the shortcut. The AI saves its work on the part that didn’t change, so it only has to process what’s new.
What it means
Prompt caching means storing the processed version of the opening chunk of a prompt so it can be reused. Anthropic’s docs describe it as reusing “a previously processed prompt prefix.” The prefix is the stuff at the front that stays the same from message to message, like a system prompt or a 200-page contract.
The first time, the model does the full work and saves it. The next request that starts with that exact same chunk skips straight to the new part. Cheaper, and faster too.
Why it matters
The discount is big. Anthropic, OpenAI and Google all offer some version of it, and cached input usually costs around a tenth of the normal price. On a long document, that’s most of your bill gone. (Saving to the cache usually costs a little extra the first time, so it pays off on the second and third question, not the first.)
It doesn’t last long. Anthropic’s default cache lives for about 5 minutes and resets each time it’s used, with a pricier one-hour option. OpenAI’s is on automatically and sticks around longer on its newer models. Wander off for lunch mid-project and you’re probably paying full price again.
Agents lean on it heavily. An AI agent might send the same long instructions and tool list dozens of times while working through one task. Without caching, it pays to reread all of that on every single step. It’s a big part of why agent costs have dropped, even as agents run longer. (Your Inference Cost card is where this really shows up.)
Simple example
You call a support line and spend ten minutes explaining your problem. They put you on hold and transfer you, and the next person asks you to explain it all again from the beginning.
A good support desk writes it down on your ticket. The next person reads the note and just picks up from there.
Prompt caching is the ticket note. The catch is it gets thrown out if nobody touches it for a few minutes.

