Prompt caching in coding agents: where eight harnesses put the breakpoint

A coding agent sends almost the same request forty times in a row. The tool definitions do not change, the system prompt does not change, and the conversation only grows at the end. Every large API provider will skip the work for a prefix it has already seen and bill those tokens at a tenth of the price or less. So the order in which a harness lays out its request, and what it lets move, decides most of its input bill.

Arpit Bhayani put this well in a post on 17 October: "if anything changes early in the prompt, it invalidates the cache for everything after it", so a harness "has to order the prompt from the most stable content to the least stable content." He then described three harnesses: Claude Code (static system prompt and tools, then project memory, then session context, then messages), OpenCode ("caches the first 2 system messages and the last 2 non-system messages") and Pi ("marks the system prompt, the last user message, and the last tool").

I wanted to know whether that holds in code, and what the other open agents do. So I cloned eight harnesses with git clone --depth 1 on 17 October 2026, read where each one puts its cache markers and what it lets change mid-session, read the four providers' caching docs the same day, and priced a session. Nothing here was run against an API; every harness claim has a file and a line. The agent harness piece covers what a harness is; this one covers one thing it does to your bill.

What is being cached

During prefill a transformer computes a key and a value vector for every token at every layer (the attention and KV cache explainer draws this). Those vectors depend only on the tokens before them. Two requests that share their first 30,000 tokens therefore share the first 30,000 tokens' KV state exactly, and a server that kept it can start prefill at token 30,001.

That is automatic prefix caching as in vLLM and SGLang's radix tree, exposed as a billing feature. It is not the trained KV caching in flow models, nor the expert-weight cache misses in FreeToken.

Two properties of a prefix cache drive everything below.

1. The key is cumulative. Anthropic's docs say a cache write is "a hash of the prefix ending at that block", so "changing any block at or before the breakpoint produces a different hash on the next request." A one-character change at token 50 makes every token after it new.
2. The provider renders the request in a fixed order. For Anthropic that order is "tools, system, and messages (in that order)." A harness cannot put the tool list after the conversation even if it wanted to. Change a tool and, per the docs' invalidation table, the tools, system and messages caches all go.

The four providers' rules

I read each page on 17 October 2026. All numbers in this table are reported by the provider.

Sources: Anthropic's prompt caching guide, OpenAI's prompt caching guide and pricing page, DeepSeek's context caching guide and pricing page.

Gemini is the odd one out. Its caching page (last updated 2026-10-13) says implicit caching is on by default for Gemini 2.5 and newer with "no cost saving guarantee", with minimums of 2,048 to 4,096 tokens. Explicit caching is a separate cache object with a TTL that defaults to 1 hour and is billed for storage. It is not available on the Interactions API or on Gemini 4 Argon and later. I did not price Gemini below.

Three details in Anthropic's page matter for harnesses more than the prices do.

Writes happen only at breakpoints. Reads walk backward up to 20 blocks looking for an entry an earlier request wrote. They do not discover stable content behind a marker. The docs' own warning is a timestamp: put the breakpoint on a block that changes every request and "you pay for a fresh cache write on every request and never get a read."

Breakpoints are free; writes are not. "Adding more cache_control breakpoints doesn't increase your costs." That is why most harnesses below use three or four.

Lifetimes can be mixed, longest first. Anthropic's figure shows how a request is billed when it has both 1-hour and 5-minute markers: a read up to the highest hit, then 1-hour writes, then 5-minute writes.

expand

Eight harnesses, read from the code

opencode: Arpit's description is the old version

opencode moved from sst/opencode to anomalyco/opencode. Arpit's sentence matches the v1 line exactly. In packages/opencode/src/provider/transform.ts at tag v1.19.3, applyCaching takes msgs.filter((msg) => msg.role === "system").slice(0, 2) and msgs.filter((msg) => msg.role !== "system").slice(-2) (lines 358-360) and marks those four, unless the user has configured Anthropic's automatic caching instead (lines 467-483). Tools get no marker of their own.

The current tree (v2.0.32, and main at 8c173a8) is a rewrite. packages/ai/src/cache-policy.ts defines an AUTO policy of tools: true, system: true, messages: { tail: 1 } under a BREAKPOINT_CAP of 4. In practice that is the last tool definition, the first and last system part, and the final message (lines 17-24, 95-105, 173-185). The system parts are the agent prompt and one block of instructions rendered at session start (core/src/session/model-request.ts:100-118). core/src/session/context.ts:147-149 explains why the instruction block is ordered "from most to least shared across sessions so the baseline stays a reusable prompt-cache prefix," with the date and environment last.

The date is the interesting part. v1 wrote Today's date: ${new Date().toDateString()} into its environment block (session/system.ts:83 at v1.19.3), so a session that crossed midnight rewrote a system message. v2 makes the date an instruction with two renderings, Today's date: … initially and Today's date is now: … when it changes (core/src/instructions/builtins.ts:24-31). A change is committed as an InstructionsUpdated event and sent as a system message after the history (core/src/session/generate.ts:57-61), so the head never moves.

Two more lines worth copying. opencode skips cache markers entirely for routes that cache implicitly, and its comment explains why not to put a tail marker on Gemini through a gateway: "Gemini uses only the last breakpoint, so a conversation-tail breakpoint writes a new cache every step and costs more than none" (cache-policy.ts:58-60). It also sends the session id as promptCacheKey (model-request.ts:295).

Verdict on the claim: true of opencode through v1.19.3. It does not describe v2, where tools get a marker and only one message does.

pi: the claim holds, with one extra

pi's Anthropic path (packages/ai/src/api/anthropic-messages.ts at c268154) puts cache_control on the system prompt block (lines 1244-1252), on the last tool definition (1264-1275, applied at line 1659), and on the last block of the final message if it is a user, tool-result or system message (1529-1555). That is three markers, exactly as Arpit says. With an OAuth token it prepends a separate "You are Claude Code" identity block that gets its own marker, which makes four (1227-1243).

The prompt itself has no date. coding-agent/src/core/system-prompt.ts builds named sections: preamble, tools, rules, docs, project context, skills and the working directory. A later change becomes a diff of sections, not a rewrite. On models that accept mid-conversation system messages the update is appended after history. Tool changes become tool_addition and tool_removal blocks, so "the cached prefix survives every tool change" (anthropic-messages.ts:1265-1270). On models that do not, collapseSystemMessages folds every update back into the leading system message (utils/transcript.ts:104-120), which is a full invalidation. Anthropic documents both features: mid-conversation system messages on Claude Opus 5.5, Sonnet 5.5, Haiku 5.5 and others, and the inline-tools-2026-09-15 beta for tool changes.

pi also treats the cache as something to keep alive. core/cache-warmer.ts replays the last request with a one-token output cap at 90% of the TTL. It does this only when the expected saving is at least $0.05, with an idle continuation probability of 0.15 that the comment says was "measured from our own usage". core/cache-stats.ts counts per-turn misses above a 1,024-token noise floor and flags idle gaps longer than the 5-minute TTL as the likely cause.

Codex CLI: no markers, a key, and appends

Codex targets OpenAI, where caching is automatic, so it places no markers. What it controls is the bytes. codex-rs/core/src/client.rs at ce0323f sets prompt_cache_key to the session id (lines 646-658). It inserts the base instructions as the first input item, with an id derived from the thread id and the instruction text (975-1005), so the item serializes identically on every request.

Time does not go in the head. A CurrentTimeReminder is a developer-role item whose text is It is %Y-%m-%d %H:%M:%S UTC. (core/src/context/current_time_reminder.rs), and session/time_reminder.rs:192-199 records it into the conversation when one is due. Over its WebSocket transport Codex goes further and sends only the new items with previous_response_id. It falls back to a full request, and logs why, when the model, tools, tool choice, reasoning settings, service tier or cache key changed (client.rs:429-453).

One comment overstates. When compaction overflows the context window, compact.rs:361 drops the oldest history item "to preserve cache (prefix-based)". The instructions and tools stay cached; every history token after the dropped item misses.

Goose, Cline, Continue, Aider

Goose (bfcc6ee) marks the last tool, the system prompt and the last two user messages, four in all: "The last two user messages extend the cached prefix each turn" (crates/goose-provider-types/src/formats/anthropic.rs:525-545, 576-600). Its prompt manager rounds the timestamp to the hour "to balance user time accuracy and multi session prompt cache hits" (prompt_manager.rs:180-182) and sorts extensions by name because "stable tool ordering is important for multi session prompt caching" (line 110). In this commit the default system.md does not render that timestamp. Current time travels in a per-turn context block in the conversation instead, and "blocks from earlier turns stay in the conversation history" (moim.rs:18). Adding an extension mid-session still changes both the tool list and the system prompt.

Cline (a42df8c) has moved to an SDK. For Anthropic-compatible routes it marks one place: the last text part of the most recent user message (sdk/packages/llms/src/providers/ai-sdk.ts:427-436). With Anthropic's 20-block lookback that is enough for a conversation that grows by fewer than 20 blocks a request. It leaves no separate entry for tools and system, so any edit to history costs the whole prefix.

Continue (5522c6f) makes caching opt-in through cacheBehavior. cacheSystemMessage marks the system prompt, and cacheConversation marks the last two user messages (core/llm/llms/Anthropic.ts:417-443, openai-adapters/src/apis/AnthropicUtils.ts:99-120). Tool definitions get no marker.

Aider (5dc9490, last commit 22 May 2026) is off by default too: --cache-prompts defaults to False (aider/args.py:233-237). When on, it marks the system prompt or examples, the repo map with read-only files, and the chat files (aider/coders/chat_chunks.py:28-41). Its layout is the cleverest of the eight. History comes before the editable files (all_messages, lines 16-26), so editing a file invalidates only the file block and the turn. Turning caching on also switches --map-refresh from auto to files so the repo map stops reshuffling (main.py:954-955). --cache-keepalive-pings sends a max_tokens=1 request every 5 minutes less 5 seconds (base_coder.py:1340-1390). The cost is a date: get_platform_info writes - Current date: YYYY-MM-DD into the system prompt (base_coder.py:1143-1144), the first block of the request.

DeepSeek's dsh and Claude Code

dsh (687f282) places no markers. Its main DeepSeek adapter only maps cache_read_input_tokens into usage (packages/llm/llm-deepseek/src/translate.ts:36), because DeepSeek caches on disk without being asked. A second adapter, built on pi's pi-ai package, is labelled a "design-verification twin". What dsh does is treat cache reuse as a contract. 335 package READMEs carry a "KV Cache effect" section the repo's verifier requires (measured, by grep). Its compaction summarizer replays the conversation's own system prompt, tools and messages, then appends the instruction as a final user message, "so the provider's KV cache is reused instead of invalidated" (packages/compaction/compaction-basic/src/summarizer.ts:25-31). Its architecture doc says "capable routes append" prompt updates "after cached history" (docs/architecture.md:110). That is the same move as pi and opencode v2.

Claude Code is not open source, so Arpit's ordering for it cannot be checked against code. I am not counting it in either direction.

A session, priced

The prices make the stakes concrete. My model (reasoned, not measured):

• a 20,000-token prefix: 11,000 tokens of tool definitions, a 5,980-token system prompt, a 20-token date line and 3,000 tokens of project memory;
• 40 requests, each adding 1,500 tokens of user or tool-result input, with 500 tokens of reply that become history on the next request;
• one request a minute, so nothing expires unless I add a pause.

Request n sends $20{,}000 + 2{,}000(n-1) + 1{,}500$ tokens, 2,420,000 over the session. With a stable prefix and a marker on the last message, request 1 writes 21,500 tokens and every later request reads the previous request's whole input and writes 2,000 new tokens. That gives 2,320,500 tokens read and 99,500 written.

All reasoned. The prices in parentheses are base input, cache write and cache read per million tokens. Output is unchanged by caching ($0.30 on Sonnet 4.6 for 20,000 output tokens), so it is left out. gpt-5.5's reads are rounded down to its 2,048-token interval, which is why it lands at 15.2% rather than 14.7%.

Three results are worth stating plainly.

Caching is worth about 85% of the input bill. It is more on DeepSeek, which charges nothing to write and stores the reply as well as the input.

One early change costs one prefix. The date flipping at turn 21 turns 59,500 tokens of reads into writes. On Sonnet 4.6 that costs about 21 cents, or 19% more than the stable layout. With a separate marker after the tool block, as opencode, pi and Goose place one, the 11,000 tool tokens survive and the session costs $1.237. A 10-minute pause before request 25 costs Sonnet 4.6 the same kind of rewrite ($1.302), since its default lifetime is 5 minutes. The OpenAI and DeepSeek lifetimes outlast that pause.

A timestamp at the top is worse than no caching. If something early changes on every request, every request is a full write. Anthropic and GPT-5.6 bill writes at 1.25x, so the session costs 25% more than turning caching off ($9.075 against $7.26). Moving the same 20 tokens into the latest message costs $1.072, the stable price plus 20 tokens a turn. That is the whole argument for Codex's time reminders and Goose's turn context.

The simulator runs the same model. Its default is the date flip on Sonnet 4.6. Switch providers, move the date line, connect an MCP server at turn 15 (a tool-list change, which the Anthropic docs say invalidates everything), or compact at turn 30. Compaction lowers the total because the history shrinks to a 4,000-token summary, but the first request after it reads only the static prefix.

What moves a prefix

From the providers' invalidation tables and the code above, the things that change a coding agent's prefix mid-session are:

• time: a date or timestamp anywhere before the history (Aider's system prompt, opencode v1's environment block);
• tools: an MCP server connecting, a tool's description changing, or tools serialized in a different order (Goose sorts extensions for this reason);
• settings: the model, and on Anthropic the thinking and effort settings, which the docs say "always invalidates message blocks";
• history rewrites: compaction, or trimming the oldest turn (Codex's overflow path);
• idle time past the lifetime: 5 minutes on Anthropic by default.

The fix the newer harnesses converged on is to append instead of edit: opencode v2's instruction updates, pi's section diffs and tool_addition blocks, Codex's time reminders, Goose's turn context and dsh's prompt updates all land after the history. Each turns a full-prefix rewrite into a few new tokens at the tail.

Checking the post

One clarification, not a correction. On OpenAI the key is a routing hint for models before GPT-5.6, which the docs say should see about 15 requests per minute per key. On GPT-5.6 and later it is only for cost accounting. On Anthropic there is no key; isolation is per workspace.

What I did not check

I did not run any harness or call any API, so no hit rate here is measured. The cost table is arithmetic on published prices with an invented but ordinary session shape; it ignores DeepSeek's common-prefix detection and OpenAI's hidden system tokens. Gemini is not priced. Closed harnesses are out of scope. Line numbers are at the commits named above, and most of these repositories took commits the day I read them.

What the code does show is a convergence. The harnesses written or rewritten most recently (opencode v2, pi, Codex, Goose and dsh) all treat the request head as immutable for the life of a session and push every change to the tail. The harness effect piece reported that changing only the orchestration layer cut an agent's tokens by 38%. Cache layout is a quieter lever of the same kind: it does not change how many tokens are sent, only how many are billed at full price.