Coding agents and multi-turn assistants often resend the same system prompt, tool definitions, and conversation prefix. Caching can make those repeated input tokens cheaper, but only when the model, endpoint, and request shape support it. It does not discount generated output, and repeating similar text is not enough if the cache key changes. This guide separates the OpenAI-style automatic path from Claude’s explicit breakpoint behavior.
What changes on the bill?
A cached read is charged at the cached-input rate instead of the ordinary input rate. The GetVibe model catalog publishes cache support and cache pricing alongside model rates. On the catalog snapshot checked October 3, 2026, examples include $0.0055 input · $0.0274 output per 1M tokens for GPT-6 Luna and $0.1734 input · $0.8667 output per 1M tokens for Claude Sonnet 5.5. These price values are rendered from the live model pages. Check the cache details there too; do not assume every model uses the same multiplier or write policy.
A simple estimate for a repeated prefix is: repeated input tokens multiplied by normal input rate, versus repeated input tokens multiplied by cache-read rate. The difference is the possible input saving. Subtract any cache-write cost if that model charges one. Your observed request usage is required to turn this estimate into a measured result.
OpenAI-compatible models: keep the prefix stable
For supported OpenAI-family routes, caching is automatic. There is no cache-control field to add in a normal Chat Completions request. The repeated beginning must remain the same. Put the system instructions, stable examples, and tool definitions before user-specific values. Avoid placing timestamps, random IDs, or per-request feature flags at the start. Keep tool order deterministic.
Do not rewrite earlier conversation turns when continuing an agent session if you expect the same prefix to match. Append new turns. Switching models starts a different model cache. A request can be valid and still report no cached tokens because the prefix is below the model’s cache threshold or the cache has expired.
Claude: breakpoints and the Messages endpoint
Claude caching uses cache-control breakpoints. The GetVibe integration can insert a breakpoint when one is absent on supported Claude requests; if you manage breakpoints yourself, check the current caching documentation and keep the marker on a stable section. Claude Code uses the Anthropic Messages endpoint; the Claude Code integration guide covers its base URL and credentials.
curl https://getvibeapi.com/v1/messages \
-H "x-api-key: $GETVIBE_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-sonnet-5.5",
"max_tokens": 300,
"system": [{"type":"text","text":"Stable instructions shared by every request."}],
"messages": [{"role":"user","content":"A request-specific question."}]
}'This example shows a stable system section and a changing user message. Use the current docs for the exact breakpoint field supported by your client and model.
Verify a cache hit instead of assuming one
Send the same long prefix twice, changing only a short suffix. Inspect the response usage. OpenAI-shaped responses report cached input under prompt_tokens_details.cached_tokens; Anthropic-shaped responses expose cache read and cache creation fields. The account usage page can also show caching information. If those counts remain zero, check prefix equality, minimum cacheable length, model support, and whether you changed the selected route.
Use a sample large enough to exceed the model’s cache threshold. A one-sentence system prompt may not produce a visible difference. Compare several repeated calls, because cache lifetime and request scheduling affect whether a prefix is still warm.
A practical savings test
- Choose one real prompt with a sizeable reusable prefix.
- Record its ordinary input usage and current model-page rates.
- Repeat it with only the user suffix changed.
- Confirm the response reports cached input.
- Compare the ledger cost for the repeated requests.
Keep output length similar during the experiment so that output spend does not obscure the input change. Do not claim savings from a cache multiplier alone; verify the cache-hit count and billed usage.
Common cache misses
- A timestamp or request ID appears before the stable instructions.
- Tool definitions are serialized in a different order each request.
- The application changes whitespace or truncates history differently.
- The model changes between turns.
- The prefix is too short or the cache window elapsed.
When caching works, it can make long agent conversations more economical, but it is not a reason to send unnecessary repository context. Keep prompts focused and evaluate total task cost. For deeper endpoint details, read the caching implementation guide and the model’s live price page.
Place breakpoints where they can be reused
For a Messages-format prompt, keep stable instructions and reference material in the system block, then put the changing question in the user message. A cache breakpoint belongs at the end of the stable portion, not after a request-specific timestamp. If the context includes several large documents, place breakpoints in the portions reused across calls and verify the model’s supported breakpoint count.
Be careful with agents that summarize their own history. If each turn rewrites the prior conversation, the prefix changes and a cache hit may disappear even though the overall conversation looks similar to a person. Compare the serialized request body when debugging a miss, while keeping private content out of logs.
Account for cache writes and minimum lengths
Some models require a minimum prefix length before caching applies, and a first write can be billed differently from a later read. A short prompt may therefore show no saving. Check the current model’s cache read and write prices and cacheable token threshold. Then compare the first request with subsequent requests separately rather than averaging them into one unexplained number.
Cache availability is time-dependent. A prefix may expire after an idle period, and the cache is not a durable store for application data. Never rely on it to preserve conversation state; continue sending the context required for a correct answer.
Separate cache savings from prompt optimization
Reducing duplicated context and caching repeated context are different improvements. First remove irrelevant instructions, stale examples, and copied logs. Then cache the stable material that remains. This keeps the prompt smaller even on a cache miss and avoids paying repeatedly for unused context.
When evaluating a prompt change, hold the task examples fixed. A shorter prefix may reduce cost but also remove information the model needs. Measure output quality together with cached and uncached usage so the optimization does not trade away correctness invisibly.