Cutting LLM costs with prompt caching in the Claude API

Most LLM applications send the same long prefix on every request: a system prompt, tool definitions, a reference document, or the conversation so far. Prompt caching lets the API keep the processed form of that prefix for a few minutes, so later requests that start with the same bytes are billed at a fraction of the normal input price and usually return their first token sooner.
This post explains how the cache matches prompts, how to mark breakpoints in the Claude API, and how to check from the response that caching is working. Examples use the anthropic Python SDK (0.104) and claude-sonnet-4-5. I won’t quote prices here because they change; the pricing page lists cache write and cache read rates per model.
How prefix caching works
Section titled “How prefix caching works”The cache is keyed on an exact prefix of the request. The API renders a request in a fixed order:
toolssystemmessages
When you put a cache_control marker on a content block, you are saying “cache everything up to and including this block”. On a later request, if everything up to that point is byte-for-byte identical, the API reads it from the cache instead of processing it again. If one byte differs anywhere before the marker, it’s a miss, and everything after the change is processed (and written) again.
That has one practical consequence that matters more than anything else in this post: stable content first, variable content last. A timestamp at the top of a system prompt (“Today is 2026-05-24 16:03:11”) makes every request a cache miss, even if the 20,000 tokens after it never change.
There are three numbers to know, all documented in the prompt caching guide:
- Breakpoints. Up to 4
cache_controlmarkers per request. - Minimum length. A prefix shorter than the model’s minimum is not cached at all. For Claude Sonnet 4.5 the minimum is 1024 tokens; some other models need 2048 or 4096, so check the table in the guide for the model you use. A too-short prefix doesn’t cause an error: the cache fields in the usage just stay at zero.
- Lifetime (TTL). The default is 5 minutes, refreshed each time the entry is read. You can ask for a 1-hour TTL with
{"type": "ephemeral", "ttl": "1h"}. Writing a 1-hour entry costs more than writing a 5-minute one, so it pays off only when requests sharing the prefix are more than 5 minutes apart.
Writes cost more than normal input tokens and reads cost much less. So caching saves money when a prefix is read at least once or twice after being written, and costs a little extra when it never is.
A worked example: questions about a long document
Section titled “A worked example: questions about a long document”The classic case is a long document plus many questions about it. The document goes in the system prompt, with the breakpoint on it; the question goes in the user message, after the breakpoint.
# cached_qa.py: ask several questions about one long document, caching the documentimport osimport sysfrom pathlib import Path
import anthropic
MODEL = os.environ.get("CLAUDE_MODEL", "claude-sonnet-4-5")client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY
document = Path(sys.argv[1]).read_text(encoding="utf-8")
# Stable prefix: instructions + the document. The breakpoint goes on the last# block that is the same for every request.system = [ { "type": "text", "text": "You answer questions about the internal runbook below. " "Quote the section you used. If the runbook does not say, reply 'not covered'.", }, { "type": "text", "text": f"<runbook>\n{document}\n</runbook>", "cache_control": {"type": "ephemeral"}, # 5-minute TTL },]
count = client.messages.count_tokens( model=MODEL, system=system, messages=[{"role": "user", "content": "x"}])print(f"prefix is about {count.input_tokens} tokens")
def ask(question: str) -> str: response = client.messages.create( model=MODEL, max_tokens=1024, system=system, messages=[{"role": "user", "content": question}], # varies, so after the breakpoint ) u = response.usage print( f"write={u.cache_creation_input_tokens} " f"read={u.cache_read_input_tokens} " f"uncached={u.input_tokens} out={u.output_tokens}" ) return response.content[0].text
for q in [ "How do I rotate the database password?", "Who gets paged for a disk-full alert?", "What is the rollback procedure for a bad deploy?",]: print(q, "->", ask(q)[:80])Run it with uv run --with anthropic cached_qa.py runbook.md. The count_tokens call is a cheap way to confirm the prefix is over the minimum before you rely on caching; it doesn’t run the model.
Reading the usage fields
Section titled “Reading the usage fields”Each response’s usage splits the input into three parts:
| Field | What it counts |
|---|---|
cache_creation_input_tokens | Tokens written to the cache by this request (billed at the write rate) |
cache_read_input_tokens | Tokens served from the cache (billed at the read rate) |
input_tokens | Tokens after the last breakpoint, processed normally |
The total input for the request is the sum of the three. With a 6,000-token runbook, the pattern from the script above looks like this (illustrative numbers, not a recorded run):
write=6050 read=0 uncached=14 out=210write=0 read=6050 uncached=13 out=180write=0 read=6050 uncached=15 out=240The first request writes the prefix; the next two read it. If you see write on every request, something in the prefix changes between calls. If you see zeros in both cache fields, the prefix is under the minimum, or the cache_control marker isn’t where you think it is.
Log these three numbers in production, per endpoint. A cache hit rate (read / (read + write + uncached)) that drops after a deploy almost always means someone added something variable to the top of a prompt.
Caching a conversation
Section titled “Caching a conversation”In a chat, the whole history is the prefix. Put the breakpoint on the last block of the newest message, and each turn reads the previous turn’s cache and writes a slightly longer entry:
def chat_turn(history: list[dict], user_text: str) -> str: # Drop the marker from older messages so we stay under 4 breakpoints. for message in history: if message["role"] == "user": for block in message["content"]: block.pop("cache_control", None)
history.append({ "role": "user", "content": [{"type": "text", "text": user_text, "cache_control": {"type": "ephemeral"}}], }) response = client.messages.create( model=MODEL, max_tokens=1024, system=system, messages=history ) history.append({"role": "assistant", "content": response.content}) return response.content[0].textOn a hit, the API looks for the longest previously cached prefix by checking block boundaries before your breakpoint. That lookback has a limit of about 20 content blocks, so if a single turn adds many blocks (long runs of tool calls and results, for example), add a second breakpoint earlier in the turn, or the previous entry can fall out of reach and the whole conversation gets written again.
Ordering the prompt for cache hits
Section titled “Ordering the prompt for cache hits”Because the cache is a prefix match in tools → system → messages order, a change in an early section invalidates everything after it:
- Changing the tool list (adding, removing or reordering tools) invalidates the whole cache. Keep the tool set fixed per endpoint and sort it deterministically.
- Changing the system prompt invalidates the system and message caches, but not cached tools.
- Changing request settings such as
tool_choice, or turning extended thinking on or off, can invalidate the cached messages even when the text is the same.
Some habits that keep the prefix stable:
- Put dates, user names, request IDs and feature flags in the user message, not the system prompt. “The current date is …” can sit at the end of the latest user turn.
- Serialise JSON with sorted keys (
json.dumps(obj, sort_keys=True)). Python dicts keep insertion order, which can vary between code paths. - Keep retrieved documents (RAG results) after the breakpoint unless the same documents really are reused across requests.
- If several prompts share a long common part, put that part first in all of them, so they can share one entry.
Gotchas
Section titled “Gotchas”- Parallel cold starts don’t share. An entry is available only once the first response that writes it has started. If you fire 50 requests at once against a cold prefix, they all pay for a write. Send one request first, then fan out.
- Caches are scoped. Entries are never shared between organizations (on the Claude API they are isolated per workspace), and different models have separate caches. Switching model mid-conversation means a fresh write.
- The TTL counts from the start of a request. A long generation uses up part of the 5 minutes before the next request can read the entry.
- Short prompts gain nothing. Under the minimum length, a
cache_controlmarker is silently ignored. That is fine; just don’t count on savings. - Batch and caching combine. Message Batches requests can carry
cache_controltoo, but batch requests run in whatever order the service chooses, so hits are opportunistic rather than guaranteed.
Further reading
Section titled “Further reading”© 2026 David Li · Home · Blog · RSS ·Classic site · Built with Astro, Starlight and three.js · Logos:Font Awesome Free (CC BY 4.0)