Order your prompt so the cache can work
Caching matches on an exact prefix. Anything variable — a timestamp, a user name, a session ID — placed near the top invalidates everything after it. Put the stable material first (system prompt, tool definitions, fixed documents) and all variable content last. This single ordering change is often the difference between a 20% and a 90% hit rate.
Hit rate is a traffic-shape problem
Caches expire on a TTL, typically 5 minutes to an hour. Steady traffic keeps the cache warm; bursty or low-volume traffic means most requests pay the write premium instead. If you send fewer requests than one per TTL window, caching will cost you money — the calculator's break-even line shows exactly where.
- • Stable content first, variable content last.
- • Keep the prefix byte-identical — whitespace changes bust the cache.
- • Consolidate traffic rather than spreading it across prompt variants.
- • Watch TTL: a 5-minute cache needs a request at least every 5 minutes.
What is worth caching
Long system prompts, tool and function schemas, few-shot example blocks, and stable retrieved documents. Not worth caching: short prompts (under ~1,000 tokens the overhead rarely pays back), anything that changes per request, and workloads with very low request rates.
Related guides
Long-form playbooks on the same topic, written by the RevenueLab editorial team.
FAQ
How much does prompt caching save?
On prompt-heavy applications with a good hit rate, 40–70% of total spend. The saving scales with how large your fixed prefix is relative to variable content — a 30,000-token RAG prefix with a 90% hit rate can cut the input line by over 80%.
Does prompt caching ever cost more?
Yes. Cache writes typically bill at 100–125% of the normal input rate. If your hit rate is below the break-even point (often around 20–25%), you pay the write premium repeatedly without recovering it on reads.
What's a good cache hit rate?
Above 80% for steady production traffic. Below 40% suggests your prefix isn't stable, variable content is placed too early in the prompt, or traffic is too sparse to keep the cache warm within its TTL.
How long does a prompt cache last?
Typically 5 minutes to an hour depending on the provider and tier, refreshed on each hit. Sparse traffic means expiry between requests, which is the most common cause of a disappointing hit rate.
What's the minimum prompt size worth caching?
Providers usually enforce a minimum (commonly around 1,000–2,000 tokens) and below that the write premium rarely pays back. Cache when your stable prefix is comfortably into the thousands of tokens.
Can I cache conversation history?
Partially. The stable prefix of a growing conversation can be cached, but each new turn extends the prompt and requires a new cache write for the extended prefix. Pair caching with summarization to keep the growing part small.
How this calculator is built
Independently maintained
Written by Sam Doshi and the RevenueLab editorial team. We don't sell the data feeds this tool is built on.
Sourced from primary data
Benchmarks come from public AdSense / Stripe / IRS disclosures and reader-submitted data — never third-party "$X per view" claims. Full methodology.
Last editorial review
Reviewed on a rolling quarterly cycle. Dated reviews are published on the methodology record for each calculator.
Editorial standards
See our editorial policy and disclaimer. Results are estimates, not advice.