Most of your AI coding bill is cache, and how to stop breaking it
When your quota vanishes twice as fast as it should, the culprit is almost never how much you generated. It is how often you broke the cache without noticing.
If you use an AI coding agent seriously, most of what you pay for is not the code it writes. It is the same context, sent again and again, turn after turn: the tool definitions, your project instructions, the files already in the conversation. Every message you send re-submits that whole pile so the model can respond in context. Left naive, that is staggeringly wasteful, and the thing that saves you from the bill is a mechanism most people never think about directly. It is also the thing they quietly sabotage all day.
That mechanism is prompt caching, and understanding it is the difference between a quota that lasts and one that evaporates.
How the cache actually works
The idea is simple. Instead of reprocessing your entire context from scratch on every turn, the model caches the stable front portion and reuses it. Anthropic prices this to make the deal obvious: writing something into the cache the first time costs a small premium, 1.25 times normal input for the short-lived cache, and reading it back on every later turn costs one tenth of normal input. Read that again. A cache hit is ninety percent off. In a long agent session where the same context rides along for dozens of turns, nearly all of your tokens are cache reads, which is exactly why the whole thing stays affordable.
But there is a catch built into the word “prefix,” and it is where all the trouble lives. Caching is prefix matching. The model caches your context up to a point, and the cache is keyed on the exact bytes up to that point. Change anything before it, a single character, even a stray space in your instructions, and the entire cached portion after that change is invalidated and has to be rewritten from scratch.
Once you see the cache as a fragile prefix, the ways people burn their quota stop looking like separate problems and start looking like one problem with five faces.
The five ways people break it
A bloated, frequently edited instructions file. Your project instructions sit near the front of the prefix, so they get sent on every single turn. A five-hundred-line file is not just harder for the model to follow, it is expensive dead weight riding along forever. Worse, if you keep tweaking it mid-project, every edit invalidates the cache behind it and forces a full rewrite. Short and stable is not a style preference here. It is money.
Too many tool integrations. Every tool server you connect injects its full schema into the context, thousands of tokens each, and it goes in near the front where it is always present. Wire up five of them and you have spent a meaningful slice of your window, and your cache, before you have typed a word. Connect the ones the task needs, not the ones you might someday want.
Letting a session go cold. The short-lived cache lasts about five minutes. Step away for a meeting and come back half an hour later, and the cache has expired. Your next message does not get the ninety-percent discount, it pays to rebuild the entire context from scratch. A long idle gap is one of the most expensive things you can do, and it is invisible because nothing looks different when you return.
Switching models mid-conversation. Each model keeps its own separate cache. Hop from one model to another partway through a session and you get a full cache miss, because the cache you built belongs to the model you left. The cost of rebuilding the new model’s cache usually dwarfs whatever you were trying to save by switching.
The kitchen-sink session. Running five unrelated tasks in one long-lived conversation piles up a context full of files and history that no longer matter. The cache still technically hits, but you are now paying to carry twenty stale files on every turn, and the model’s attention is split across all of them. This one costs you twice: tokens and answer quality.
The discipline is small
None of the fixes are hard. Keep the instructions file short and mostly frozen. Connect only the tools this work needs. Do not leave sessions idle across long breaks, and if you have to, expect the next turn to be expensive. Pick a model and stay on it. And clear the context between unrelated tasks instead of letting one session sprawl.
What ties all of it together is a single mental model: your context has a cheap, stable front and an expensive, changing tail, and your job is to keep the front stable and small. Do that and the cache does its work quietly, most of your tokens cost a tenth of list price, and your quota lasts as long as it should. Fight it, usually without realizing you are fighting it, and you pay full freight to rebuild the same context over and over while wondering where the month went.
The bill was never really about how much the model wrote. It was about how well you kept the cache intact.