# Prompt caching in coding agents: where eight harnesses put the breakpoint

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/prompt-caching-harnesses
> date: 2026-10-17
> tags: agents, harness, kv-cache, pricing

A coding agent sends almost the same request forty times in a row. The tool definitions do not change, the system prompt does not change, and the conversation only grows at the end. Every large API provider will skip the work for a prefix it has already seen and bill those tokens at a tenth of the price or less. So the order in which a harness lays out its request, and what it lets move, decides most of its input bill.

Arpit Bhayani put this well in a [post on 17 October](https://x.com/arpit_bhayani/status/2111398785012342951): "if anything changes early in the prompt, it invalidates the cache for everything after it", so a harness "has to order the prompt from the most stable content to the least stable content." He then described three harnesses: Claude Code (static system prompt and tools, then project memory, then session context, then messages), OpenCode ("caches the first 2 system messages and the last 2 non-system messages") and Pi ("marks the system prompt, the last user message, and the last tool").

I wanted to know whether that holds in code, and what the other open agents do. So I cloned eight harnesses with `git clone --depth 1` on 17 October 2026, read where each one puts its cache markers and what it lets change mid-session, read the four providers' caching docs the same day, and priced a session. Nothing here was run against an API; every harness claim has a file and a line. The [agent harness](/articles/agent-harness) piece covers what a harness is; this one covers one thing it does to your bill.

## What is being cached

During prefill a transformer computes a key and a value vector for every token at every layer (the [attention and KV cache](/architectures/attention-kv) explainer draws this). Those vectors depend only on the tokens before them. Two requests that share their first 30,000 tokens therefore share the first 30,000 tokens' KV state exactly, and a server that kept it can start prefill at token 30,001.

That is automatic prefix caching as in [vLLM](/articles/vllm) and [SGLang](/articles/sglang)'s radix tree, exposed as a billing feature. It is not the trained KV caching in [flow models](/articles/kv-caching-flow-models), nor the expert-weight cache misses in [FreeToken](/articles/freetoken).

Two properties of a prefix cache drive everything below.

1. **The key is cumulative.** Anthropic's docs say a cache write is "a hash of the prefix ending at that block", so "changing any block at or before the breakpoint produces a different hash on the next request." A one-character change at token 50 makes every token after it new.
2. **The provider renders the request in a fixed order.** For Anthropic that order is "`tools`, `system`, and `messages` (in that order)." A harness cannot put the tool list after the conversation even if it wanted to. Change a tool and, per the docs' invalidation table, the tools, system and messages caches all go.

## The four providers' rules

I read each page on 17 October 2026. All numbers in this table are reported by the provider.

| | Anthropic | OpenAI, GPT-5.6 and later | OpenAI, GPT-5.5 and earlier | DeepSeek |
|---|---|---|---|---|
| How a prefix is marked | `cache_control` on up to 4 blocks, or one top-level `cache_control` that moves to the last block | implicit breakpoint at the end of the latest eligible message, or explicit `prompt_cache_breakpoint`; up to four writes a request | implicit, at model-dependent intervals (2,048 tokens on GPT-5.5) | automatic, at the end of each input and each output |
| Minimum prefix | 512 to 4,096 tokens by model | 1,024 visible tokens | varies by request settings | none stated |
| Lifetime | 5 minutes, or 1 hour on request | at least 30 minutes | `in_memory` 5 to 10 minutes, or up to 24 hours (`24h` is the only option on GPT-5.5) | cleared "within a few hours to a few days" |
| Write price | 1.25x base (5 min), 2x (1 h) | 1.25x base | no extra charge | no extra charge |
| Read price | 0.1x base; 0.05x on Opus 5.5 and Sonnet 5.5 | 0.1x base; 0.05x on GPT-6.1 Sol | model-dependent (gpt-5.5: \$0.50 against \$5) | V4 Pro at peak: \$0.044 against \$1.32 |

Sources: Anthropic's [prompt caching guide](https://platform.claude.com/docs/en/build-with-claude/prompt-caching), OpenAI's [prompt caching guide](https://developers.openai.com/api/docs/guides/prompt-caching) and [pricing page](https://developers.openai.com/api/docs/pricing), DeepSeek's [context caching guide](https://api-docs.deepseek.com/guides/kv_cache) and [pricing page](https://api-docs.deepseek.com/quick_start/pricing).

Gemini is the odd one out. Its [caching page](https://ai.google.dev/gemini-api/docs/caching) (last updated 2026-10-13) says implicit caching is on by default for Gemini 2.5 and newer with "no cost saving guarantee", with minimums of 2,048 to 4,096 tokens. Explicit caching is a separate cache object with a TTL that defaults to 1 hour and is billed for storage. It is not available on the Interactions API or on Gemini 4 Argon and later. I did not price Gemini below.

Three details in Anthropic's page matter for harnesses more than the prices do.

**Writes happen only at breakpoints.** Reads walk backward up to 20 blocks looking for an entry an earlier request wrote. They do not discover stable content behind a marker. The docs' own warning is a timestamp: put the breakpoint on a block that changes every request and "you pay for a fresh cache write on every request and never get a read."

**Breakpoints are free; writes are not.** "Adding more `cache_control` breakpoints doesn't increase your costs." That is why most harnesses below use three or four.

**Lifetimes can be mixed**, longest first. Anthropic's figure shows how a request is billed when it has both 1-hour and 5-minute markers: a read up to the highest hit, then 1-hour writes, then 5-minute writes.

<Figure
  src="https://media.thesatyajit.com/ai/articles/prompt-caching-harnesses/fig1.png?v=f13668dcb42609ff"
  alt="Three horizontal bars, each a request's input tokens. The first is split into a cache read, a 1-hour cache write, a 5-minute cache write and the rest of the prompt, with arrows marking positions A, B and C. In the second, A is zero and the request starts with a 1-hour write. In the third the whole prefix up to a 5-minute breakpoint is a cache read."
  caption="Billing positions for a request that mixes 1-hour and 5-minute breakpoints: A is the highest hit, B the last 1-hour breakpoint after it, C the last breakpoint. Tokens to A are reads, A to B are 1-hour writes, B to C are 5-minute writes. (Anthropic prompt caching guide, mixed TTL figure)."
/>

## Eight harnesses, read from the code

<BreakpointMap />

### opencode: Arpit's description is the old version

opencode moved from `sst/opencode` to `anomalyco/opencode`. Arpit's sentence matches the v1 line exactly. In `packages/opencode/src/provider/transform.ts` at tag v1.19.3, `applyCaching` takes `msgs.filter((msg) => msg.role === "system").slice(0, 2)` and `msgs.filter((msg) => msg.role !== "system").slice(-2)` (lines 358-360) and marks those four, unless the user has configured Anthropic's automatic caching instead (lines 467-483). Tools get no marker of their own.

The current tree (v2.0.32, and `main` at 8c173a8) is a rewrite. `packages/ai/src/cache-policy.ts` defines an `AUTO` policy of `tools: true, system: true, messages: { tail: 1 }` under a `BREAKPOINT_CAP` of 4. In practice that is the last tool definition, the first and last system part, and the final message (lines 17-24, 95-105, 173-185). The system parts are the agent prompt and one block of instructions rendered at session start (`core/src/session/model-request.ts:100-118`). `core/src/session/context.ts:147-149` explains why the instruction block is ordered "from most to least shared across sessions so the baseline stays a reusable prompt-cache prefix," with the date and environment last.

The date is the interesting part. v1 wrote `Today's date: ${new Date().toDateString()}` into its environment block (`session/system.ts:83` at v1.19.3), so a session that crossed midnight rewrote a system message. v2 makes the date an instruction with two renderings, `Today's date: …` initially and `Today's date is now: …` when it changes (`core/src/instructions/builtins.ts:24-31`). A change is committed as an `InstructionsUpdated` event and sent as a system message after the history (`core/src/session/generate.ts:57-61`), so the head never moves.

Two more lines worth copying. opencode skips cache markers entirely for routes that cache implicitly, and its comment explains why not to put a tail marker on Gemini through a gateway: "Gemini uses only the last breakpoint, so a conversation-tail breakpoint writes a new cache every step and costs more than none" (`cache-policy.ts:58-60`). It also sends the session id as `promptCacheKey` (`model-request.ts:295`).

<RepoCard repo="anomalyco/opencode" />

**Verdict on the claim:** true of opencode through v1.19.3. It does not describe v2, where tools get a marker and only one message does.

### pi: the claim holds, with one extra

pi's Anthropic path (`packages/ai/src/api/anthropic-messages.ts` at c268154) puts `cache_control` on the system prompt block (lines 1244-1252), on the last tool definition (1264-1275, applied at line 1659), and on the last block of the final message if it is a user, tool-result or system message (1529-1555). That is three markers, exactly as Arpit says. With an OAuth token it prepends a separate "You are Claude Code" identity block that gets its own marker, which makes four (1227-1243).

The prompt itself has no date. `coding-agent/src/core/system-prompt.ts` builds named sections: preamble, tools, rules, docs, project context, skills and the working directory. A later change becomes a diff of sections, not a rewrite. On models that accept mid-conversation system messages the update is appended after history. Tool changes become `tool_addition` and `tool_removal` blocks, so "the cached prefix survives every tool change" (`anthropic-messages.ts:1265-1270`). On models that do not, `collapseSystemMessages` folds every update back into the leading system message (`utils/transcript.ts:104-120`), which is a full invalidation. Anthropic documents both features: mid-conversation system messages on Claude Opus 5.5, Sonnet 5.5, Haiku 5.5 and others, and the `inline-tools-2026-09-15` beta for tool changes.

pi also treats the cache as something to keep alive. `core/cache-warmer.ts` replays the last request with a one-token output cap at 90% of the TTL. It does this only when the expected saving is at least \$0.05, with an idle continuation probability of 0.15 that the comment says was "measured from our own usage". `core/cache-stats.ts` counts per-turn misses above a 1,024-token noise floor and flags idle gaps longer than the 5-minute TTL as the likely cause.

<RepoCard repo="badlogic/pi-mono" />

### Codex CLI: no markers, a key, and appends

Codex targets OpenAI, where caching is automatic, so it places no markers. What it controls is the bytes. `codex-rs/core/src/client.rs` at ce0323f sets `prompt_cache_key` to the session id (lines 646-658). It inserts the base instructions as the first input item, with an id derived from the thread id and the instruction text (975-1005), so the item serializes identically on every request.

Time does not go in the head. A `CurrentTimeReminder` is a developer-role item whose text is `It is %Y-%m-%d %H:%M:%S UTC.` (`core/src/context/current_time_reminder.rs`), and `session/time_reminder.rs:192-199` records it into the conversation when one is due. Over its WebSocket transport Codex goes further and sends only the new items with `previous_response_id`. It falls back to a full request, and logs why, when the model, tools, tool choice, reasoning settings, service tier or cache key changed (`client.rs:429-453`).

One comment overstates. When compaction overflows the context window, `compact.rs:361` drops the oldest history item "to preserve cache (prefix-based)". The instructions and tools stay cached; every history token after the dropped item misses.

<RepoCard repo="openai/codex" />

### Goose, Cline, Continue, Aider

**Goose** (bfcc6ee) marks the last tool, the system prompt and the last two user messages, four in all: "The last two user messages extend the cached prefix each turn" (`crates/goose-provider-types/src/formats/anthropic.rs:525-545, 576-600`). Its prompt manager rounds the timestamp to the hour "to balance user time accuracy and multi session prompt cache hits" (`prompt_manager.rs:180-182`) and sorts extensions by name because "stable tool ordering is important for multi session prompt caching" (line 110). In this commit the default `system.md` does not render that timestamp. Current time travels in a per-turn context block in the conversation instead, and "blocks from earlier turns stay in the conversation history" (`moim.rs:18`). Adding an extension mid-session still changes both the tool list and the system prompt.

**Cline** (a42df8c) has moved to an SDK. For Anthropic-compatible routes it marks one place: the last text part of the most recent user message (`sdk/packages/llms/src/providers/ai-sdk.ts:427-436`). With Anthropic's 20-block lookback that is enough for a conversation that grows by fewer than 20 blocks a request. It leaves no separate entry for tools and system, so any edit to history costs the whole prefix.

**Continue** (5522c6f) makes caching opt-in through `cacheBehavior`. `cacheSystemMessage` marks the system prompt, and `cacheConversation` marks the last two user messages (`core/llm/llms/Anthropic.ts:417-443`, `openai-adapters/src/apis/AnthropicUtils.ts:99-120`). Tool definitions get no marker.

**Aider** (5dc9490, last commit 22 May 2026) is off by default too: `--cache-prompts` defaults to `False` (`aider/args.py:233-237`). When on, it marks the system prompt or examples, the repo map with read-only files, and the chat files (`aider/coders/chat_chunks.py:28-41`). Its layout is the cleverest of the eight. History comes *before* the editable files (`all_messages`, lines 16-26), so editing a file invalidates only the file block and the turn. Turning caching on also switches `--map-refresh` from `auto` to `files` so the repo map stops reshuffling (`main.py:954-955`). `--cache-keepalive-pings` sends a `max_tokens=1` request every 5 minutes less 5 seconds (`base_coder.py:1340-1390`). The cost is a date: `get_platform_info` writes `- Current date: YYYY-MM-DD` into the system prompt (`base_coder.py:1143-1144`), the first block of the request.

### DeepSeek's dsh and Claude Code

[dsh](/articles/deepseek-harness) (687f282) places no markers. Its main DeepSeek adapter only maps `cache_read_input_tokens` into usage (`packages/llm/llm-deepseek/src/translate.ts:36`), because DeepSeek caches on disk without being asked. A second adapter, built on pi's `pi-ai` package, is labelled a "design-verification twin". What dsh does is treat cache reuse as a contract. 335 package READMEs carry a "KV Cache effect" section the repo's verifier requires (measured, by grep). Its compaction summarizer replays the conversation's own system prompt, tools and messages, then appends the instruction as a final user message, "so the provider's KV cache is reused instead of invalidated" (`packages/compaction/compaction-basic/src/summarizer.ts:25-31`). Its architecture doc says "capable routes append" prompt updates "after cached history" (`docs/architecture.md:110`). That is the same move as pi and opencode v2.

**Claude Code** is not open source, so Arpit's ordering for it cannot be checked against code. I am not counting it in either direction.

## A session, priced

The prices make the stakes concrete. My model (reasoned, not measured):

- a 20,000-token prefix: 11,000 tokens of tool definitions, a 5,980-token system prompt, a 20-token date line and 3,000 tokens of project memory;
- 40 requests, each adding 1,500 tokens of user or tool-result input, with 500 tokens of reply that become history on the next request;
- one request a minute, so nothing expires unless I add a pause.

Request *n* sends $20{,}000 + 2{,}000(n-1) + 1{,}500$ tokens, 2,420,000 over the session. With a stable prefix and a marker on the last message, request 1 writes 21,500 tokens and every later request reads the previous request's whole input and writes 2,000 new tokens. That gives 2,320,500 tokens read and 99,500 written.

| Input cost of the 40 requests | No cache | Stable layout | Date flips at turn 21 | Timestamp every request |
|---|---|---|---|---|
| Claude Sonnet 4.6 (\$3 / \$3.75 / \$0.30) | \$7.260 | \$1.069 (14.7%) | \$1.275 | \$9.075 |
| OpenAI gpt-5.6-terra (\$2 / \$2.50 / \$0.20) | \$4.840 | \$0.713 (14.7%) | \$0.850 | \$6.050 |
| OpenAI gpt-5.5 (\$5 / none / \$0.50) | \$12.100 | \$1.833 (15.2%) | \$2.101 | \$12.100 |
| DeepSeek V4 Pro, peak (\$1.32 / none / \$0.044) | \$3.194 | \$0.209 (6.5%) | \$0.285 | \$3.194 |

All reasoned. The prices in parentheses are base input, cache write and cache read per million tokens. Output is unchanged by caching (\$0.30 on Sonnet 4.6 for 20,000 output tokens), so it is left out. gpt-5.5's reads are rounded down to its 2,048-token interval, which is why it lands at 15.2% rather than 14.7%.

Three results are worth stating plainly.

**Caching is worth about 85% of the input bill.** It is more on DeepSeek, which charges nothing to write and stores the reply as well as the input.

**One early change costs one prefix.** The date flipping at turn 21 turns 59,500 tokens of reads into writes. On Sonnet 4.6 that costs about 21 cents, or 19% more than the stable layout. With a separate marker after the tool block, as opencode, pi and Goose place one, the 11,000 tool tokens survive and the session costs \$1.237. A 10-minute pause before request 25 costs Sonnet 4.6 the same kind of rewrite (\$1.302), since its default lifetime is 5 minutes. The OpenAI and DeepSeek lifetimes outlast that pause.

**A timestamp at the top is worse than no caching.** If something early changes on every request, every request is a full write. Anthropic and GPT-5.6 bill writes at 1.25x, so the session costs 25% *more* than turning caching off (\$9.075 against \$7.26). Moving the same 20 tokens into the latest message costs \$1.072, the stable price plus 20 tokens a turn. That is the whole argument for Codex's time reminders and Goose's turn context.

<PromptLayoutSim />

The simulator runs the same model. Its default is the date flip on Sonnet 4.6. Switch providers, move the date line, connect an MCP server at turn 15 (a tool-list change, which the Anthropic docs say invalidates everything), or compact at turn 30. Compaction lowers the total because the history shrinks to a 4,000-token summary, but the first request after it reads only the static prefix.

## What moves a prefix

From the providers' invalidation tables and the code above, the things that change a coding agent's prefix mid-session are:

- **time**: a date or timestamp anywhere before the history (Aider's system prompt, opencode v1's environment block);
- **tools**: an MCP server connecting, a tool's description changing, or tools serialized in a different order (Goose sorts extensions for this reason);
- **settings**: the model, and on Anthropic the thinking and effort settings, which the docs say "always invalidates message blocks";
- **history rewrites**: compaction, or trimming the oldest turn (Codex's overflow path);
- **idle time** past the lifetime: 5 minutes on Anthropic by default.

The fix the newer harnesses converged on is to append instead of edit: opencode v2's instruction updates, pi's section diffs and `tool_addition` blocks, Codex's time reminders, Goose's turn context and dsh's prompt updates all land after the history. Each turns a full-prefix rewrite into a few new tokens at the tail.

## Checking the post

| Claim | Verdict | Receipt |
|---|---|---|
| Change something early and everything after it misses | Holds | Anthropic: the cache key is "a hash of the prefix ending at that block"; OpenAI: "the prefix after that change cannot match" |
| OpenCode caches the first 2 system and last 2 non-system messages | Held through v1.19.3; v2 marks the last tool, first and last system part, and one message | `transform.ts:358-360` at v1.19.3; `cache-policy.ts:17-24` at 8c173a8 |
| Pi marks the system prompt, the last user message and the last tool | Holds; a fourth marker on the identity block with OAuth | `anthropic-messages.ts:1227-1275, 1529-1555` |
| Claude Code's ordering | Not verifiable from public code | closed source |
| A stable key per session | Holds where a key exists: Codex and opencode both send the session id | `client.rs:646-658`; `model-request.ts:295` |

One clarification, not a correction. On OpenAI the key is a routing hint for models before GPT-5.6, which the docs say should see about 15 requests per minute per key. On GPT-5.6 and later it is only for cost accounting. On Anthropic there is no key; isolation is per workspace.

## What I did not check

I did not run any harness or call any API, so no hit rate here is measured. The cost table is arithmetic on published prices with an invented but ordinary session shape; it ignores DeepSeek's common-prefix detection and OpenAI's hidden system tokens. Gemini is not priced. Closed harnesses are out of scope. Line numbers are at the commits named above, and most of these repositories took commits the day I read them.

What the code does show is a convergence. The harnesses written or rewritten most recently (opencode v2, pi, Codex, Goose and dsh) all treat the request head as immutable for the life of a session and push every change to the tail. The [harness effect](/articles/harness-effect) piece reported that changing only the orchestration layer cut an agent's tokens by 38%. Cache layout is a quieter lever of the same kind: it does not change how many tokens are sent, only how many are billed at full price.