The Cache Miss You Pay For Every Turn

Here is a request that is 99.92% identical to the one before it. By Anthropic’s documented rules, it reads nothing from the cache and pays to write all of it again. On the next turn it does the same.

There is no error. The response is correct. The only symptom is a bill, and the bill arrives later and addressed to someone else.

Two rules do all the damage#

The cache is a prefix match. Requests render as tools, then system, then messages, and a change at any point orphans everything after it.

The second rule is the easy one to miss: an entry only exists where a breakpoint was written. The docs are blunt about it: the lookback “does not find stable content behind your breakpoint and cache it. It finds entries that prior requests already wrote.” Stable bytes are not enough. Something has to have marked that exact spot on an earlier request.

Then there is the price. On Opus 5.5 a read costs $0.20 per million tokens, plain input costs $4, and a write costs $5. So a marker on a prefix that never repeats is not a neutral mistake. It is a 25% surcharge for the privilege of caching something nobody will ever read.

What each mistake costs#

The same session each time, about 23,000 tokens of context by the last turn, with a marker placed automatically on the last block. The dashed line is what that session costs with no marker at all:

The key-order bar depends on how often the shuffle lands on the same order by luck, so read it as “bad”, not as 19%. The trim bar’s session is also smaller, because trimming removes tokens, which is why it beats no marker on price despite the rewrites.

Everything past the dashed line is worse than not caching. That is the part worth pinning to a wall: the three longest bars pay 25% more than the people who never heard of the feature.

The reminder is the good one#

The timestamp is the famous example, and you can spot it in review. The reminder is harder, because it looks like tidiness.

An agent loop adds a line after the latest tool result: “3 of 12 parts read.” Next turn it deletes that line and adds a new one, so the history stays clean. Almost nothing changed. But the automatic marker sits on the last block, and the last block was the reminder. Every entry ever written ends in a reminder that no longer exists.

Here is the whole session:

The top chart is what healthy looks like: the read grows, the write stays small. The bottom one is what it looks like when the write grows instead. Same conversation.

The fix costs one line: stop deleting. The reminder that stays in the transcript takes the session from 5.9x back to 1.2x. If you must delete it, a second marker on the system prompt at least keeps the tools and system cached, at 4.0x.

A test that costs nothing#

Every row above has the same cause, and you can catch it without a key or a bill. The previous request has to survive, intact, as the start of the next one:

import json


def without_markers(value):
    if isinstance(value, dict):
        return {k: without_markers(v) for k, v in value.items() if k != "cache_control"}
    if isinstance(value, list):
        return [without_markers(v) for v in value]
    return value


def as_blocks(system):
    return [{"type": "text", "text": system}] if isinstance(system, str) else system


def flatten(request):
    blocks = [(f"tools[{i}]", t) for i, t in enumerate(request.get("tools", []))]
    blocks += [(f"system[{i}]", b) for i, b in enumerate(as_blocks(request.get("system", [])))]
    for m, message in enumerate(request["messages"]):
        for b, block in enumerate(as_blocks(message["content"])):
            blocks.append((f"messages[{m}].content[{b}]", {"role": message["role"], **block}))
    return [(path, json.dumps(without_markers(block))) for path, block in blocks]


def first_break(previous, following):
    for setting in ("model", "thinking", "output_config"):
        if previous.get(setting) != following.get(setting):
            return setting
    next_blocks = flatten(following)
    for index, (path, block) in enumerate(flatten(previous)):
        if index >= len(next_blocks) or next_blocks[index][1] != block:
            return path
    return None

Markers are stripped first, because a marker that moves is supposed to move. Then point it at two consecutive requests from your own prompt builder:

def test_each_turn_only_appends():
    assert first_break(build_request(turn=3), build_request(turn=4)) is None

It does not just fail, it names the block. Run between the third and fourth turn of each scenario above, it returned:

only appends          None
reminder kept         None
deletes reminder      messages[4].content[1]
timestamp in system   system[0]
shuffled tools        tools[1]
shuffled JSON keys    messages[2].content[0]

A check like this belongs next to the code that assembles the prompt, because that is where the regression comes from. Caching works on the day someone adds it. Then, months later, someone adds the current date to the system prompt, and nothing complains.

It catches changed bytes, not everything. A cache can also expire, or miss because a prefix is too short or parallel requests never saw each other’s write. With a key, cache_read_input_tokens is the only number that can tell you it actually worked.

The fixes, in order of how often you need them#

The second-to-last row deserves its number. Six questions asked one after another over the same context, with the automatic marker landing on each unique question, cost $0.45 and read nothing, against $0.36 with no marker at all. With an explicit marker at the end of the shared part: $0.09. Send them at the same moment and none of them can read what the others are still writing, so let the first response start before you fire the rest.

The rule#

A cache that fails loudly gets fixed on the day it breaks. This one fails by charging you slightly more than it would have cost to not have it, on a request that returned the right answer.

Assert the prefix, because your agent is append-only until the day someone tidies it.