The harness effect: orchestration, not the model, sets your agent token bill

2026-07-18 · 10 min · agents · orchestration · token-economics · llm · explainer

The line I keep hearing is that per-token prices fall every quarter, so agent costs will sort themselves out. The invoices say otherwise. This paper — The Harness Effect from a Writer AI team (Muayad Sayed Ali et al., corresponding author Waseem AlShikh) — runs the clean experiment I wanted someone to run: hold the model fixed, change only the orchestration layer around it, and measure the bill. The result is that the orchestration layer — the harness — moves cost per task more than switching between the cheapest and most expensive model on the menu does.

One caveat up front, because it is load-bearing: the harness under test is Writer's own, and Writer ran the evaluation. I read the numbers as directional, not as a neutral benchmark. What makes it worth reading anyway is the mechanism — the paper formalizes why orchestration sets token economics, and the formalization is provider-agnostic.

Token maxing

The paper names the failure mode first. Token maxing is buying capability with tokens: longer reasoning traces, more agent turns, wider tool payloads, larger replayed contexts — so that tokens per task grow faster than task value. Falling per-token prices mask the pattern without fixing it. Total spend rises anyway.

The bill for one agentic task is a sum over its k turns:

C = \sum_{i=1}^{k}\left(p_{\text{in}}\,T^{\text{in}}_{i} + p_{\text{out}}\,T^{\text{out}}_{i}\right)

where $p_{\text{in}}, p_{\text{out}}$ are the input/output prices per token and $T^{\text{in}}_i, T^{\text{out}}_i$ the tokens at turn $i$ . The input side is the part the orchestration layer builds:

T^{\text{in}}_{i} = \underbrace{S_i}_{\text{system}} + \underbrace{H_i}_{\text{history}} + \underbrace{G_i}_{\text{tool schemas}} + \underbrace{R_i}_{\text{retrieval}} + \underbrace{U_i}_{\text{user turn}}

Here is the trap. If every turn replays the full transcript, the history term $H_i$ grows with $i$ , so the cumulative input over a task grows as $O(k^2)$ . A harness that compacts and caches history keeps it near $O(k)$ . The gap between those two curves is spend that buys no quality — and because the per-token price is falling the whole time, the total keeps climbing quietly. Drag the horizon and watch it happen:

token maxing · price falls, spend risesillustrative

tokens/task 5.7kprice $7.38/MTok ↓total spend $0.042 ↑

task horizon — more turns, wider context (drag)

Drag right. The per-token price keeps falling — the number everyone quotes when they say inference is getting cheaper. But tokens per task climb faster, so the product of the two, total spend, rises the whole way. That is token maxing: the falling headline price masks a rising bill.

Line chart of cumulative input tokens against agent turns k. A dark 'naive replay' curve grows quadratically as O(k squared); a blue 'harness-managed context' curve grows linearly as O(k). The shaded region between them is labelled the token maxing region. — Where token maxing comes from: full-history replay grows as O(k²), harness-managed context as O(k); the shaded gap is spend that buys no quality (Sayed Ali et al., 2026, Figure 1).

The bill, and the one price that actually moves

The lever the harness pulls hardest is prompt caching. Providers serve a previously seen prompt prefix from cache at roughly a tenth of the base input rate. If a fraction $h$ of input tokens are cache reads billed at multiplier $\kappa$ , the effective input price is

p^{\text{eff}}_{\text{in}} = p_{\text{in}}\left(1 - h\,(1-\kappa)\right), \qquad \kappa \approx 0.1

so a harness that keeps $h$ near 1 pays about a tenth of list price on the dominant input term. The point the paper makes well: $h$ is not a model property and not a provider favor. It is a function of how byte-stable your prompt prefix is across turns — which is set entirely by the orchestration layer. On an identical-prefix call the harness served 99.9% of prompt tokens as cache reads (7,876 of 7,886). That is the whole game: shape the prompt so the expensive term is almost always a cache hit.

The controlled swap

The experiment is deliberately boring, which is why it is convincing. Twenty-two locked evaluation tasks. Six foundation models — Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, Palmyra X6. Each model runs the tasks twice: once under a frozen conventional production loop, once under the Writer Agent Harness. Nothing else changes — same tasks, same judges, same price table. Only the orchestration layer swaps. Flip it:

controlled swap · model fixed, orchestration swapped6 models · 22 tasks

orchestration

same models · same tasks · same price table

Only the middle box changes. Flip it from the frozen production loop to the Writer Harness and, with the six models held constant, cost drops 41%, wall-clock 44%, and tokens 38% — while task-completion quality holds at parity (0.78 → 0.81). The lever is the orchestration layer, not the model.

Blended across all six models and 22 tasks, replacing the loop with the harness cuts cost per task 41% ($0.21 → $0.12), median wall-clock 44% (48s → 27s), and tokens per task 38% (14.2k → 8.8k) — with task-completion quality at parity (0.78 → 0.81, directional at this sample size).

Three grouped bar charts comparing a baseline production loop against the Writer harness on cost per task, wall-clock per task, and tokens per task. Cost falls from $0.21 to $0.12 (minus 41%), wall-clock from 48s to 27s (minus 44%), tokens from 14.2k to 8.8k (minus 38%). — Blended efficiency across six models and 22 tasks, models held constant: cost per task −41%, median wall-clock −44%, tokens per task −38% (Sayed Ali et al., 2026, Figure 3).

Two derived numbers make the parity concrete. Quality per dollar rises 82%. And throughput — task-completions per million tokens — nearly doubles:

task-completions per million tokens

Writer harness

production loop

54.9

050100

Everyone gets cheaper

The efficiency win is not a quirk of one model. Under the swap, every model's cost and latency fall — cost by 33% to 61%, latency by 33% to 55%. The effect is a property of the orchestration layer, not of any model.

cost cut from the harness, per model (%)

Flash 3.5

61%

Palmyra X6

52%

GLM 5.1

48%

Qwen 3.6

44%

Sonnet 4.6

38%

Gemini 3.1

32%

020406080

Harness leverage

Now the finding that made me want to write this up. Efficiency is model-invariant, but the quality gain from the same orchestration upgrade is not — it scales almost perfectly with a model's baseline strength. Plot each model's mean quality gain against its baseline capability and the points fall on a line: $r = 0.99$ over $n = 6$ . The paper calls it harness leverage. Stronger models extract more from the same harness. Scrub the models:

harness leverage · quality gain vs baseline strengthr = 0.99 · n = 6

Palmyra X6 · baseline 0.79quality +0.079cost still −52%

highlight model — weaker → stronger baseline (drag)

Two different stories on two axes. On cost every model wins — 32% to 61% cheaper — so efficiency is a property of the harness, not the model. On quality, the gain climbs the diagonal with baseline strength: Palmyra X6 and Sonnet 4.6 lead (+0.079, +0.073), while Qwen 3.6 actually regresses (−0.031) yet is still 44% cheaper. Stronger models extract more from the same orchestration upgrade — that is the leverage.

Scatter plot of quality gain from the harness on the y-axis against mean baseline capability on the x-axis, for six models. The points rise almost linearly along a dashed fit line: Qwen 3.6 is slightly negative, Flash 3.5 near zero, GLM 5.1 and Gemini 3.1 positive, Sonnet 4.6 and Palmyra X6 highest at plus 0.073 and plus 0.079. — Harness leverage: mean quality gain vs baseline strength. Stronger models gain more from the same orchestration upgrade — r = 0.99, n = 6 (Sayed Ali et al., 2026, Figure 6).

The honest edge of this: across 48 capability×model cells, 30 improve, 11 are flat, and 7 regress — all of them in the three smaller models, concentrated in orchestration-heavy capabilities (tool use over MCP, playbooks, presentations). Qwen 3.6 comes out net negative on quality (−0.031). It is still 44% cheaper. So the harness is a strict efficiency win everywhere, and a quality win that grows with the model you point it at.

Six mechanisms behind the effect

The paper decomposes the harness into six mechanism families. None of them is exotic — they are the unglamorous orchestration glue, which is exactly why they are easy to leave on the table:

Cache-shape discipline — the two-zone prompt. A byte-stable prefix (tool-schema catalog, stable system prompt, append-only transcript) carries the provider's cache breakpoints; everything volatile is confined to a tail that is rebuilt each turn and structurally excluded from caching. This is what pushes $h$ toward 1 in the effective-price equation.
Structured, incremental, cache-aware compaction. Shrink history without breaking the cache prefix — compact the middle, keep the front byte-identical.
Context offload. Tool outputs land in a store the model can reference, not in the prompt. Tokens the model never pays to re-read.
Zero-token waiting; durability as economics. Durable execution so a pause, retry, or long-running tool call does not replay the whole context to resume.
Failure-spend governance. Cap what a failing or looping run can burn before it is stopped. Most runaway bills are failures, not successes.
A model-agnostic floor. The five above set an efficiency floor under any model — which is what makes the savings a property of the layer, not the checkpoint.

How other harnesses compare

The paper also scores six widely used agent systems on the same axes — vendor-integrated clients, orchestration libraries, multi-agent conversation frameworks, and open personal harnesses — from public documentation rather than head-to-head runs. The pattern is that most frameworks implement some mechanisms and leave the rest "to the application to build and budget." Cache-shape discipline and failure-spend governance are the two most often missing, and they are two of the biggest levers. Treat that table as a design-time source study, not a measurement.

What it is worth at fleet scale

The reason this matters past a single task: the per-task delta multiplies by volume, and by every model you run. Apply the blended cost gap to monthly task volume and at one million agent tasks per month the harness is worth about $90k/month over the baseline — $1.08M/year — and the gap widens linearly with volume. An organization does not run one model; it runs a fleet, present and future. The harness is the one component whose efficiency multiplies across all of them.

The take

Strip the framing and the useful claim is narrow and testable: for agentic workloads, the orchestration layer is a first-class cost object, and most of the cost is in prompt shape, not model choice. The effective-input-price equation is the part I will actually use — it says the expensive input term is a cache hit if and only if your prompt prefix is byte-stable across turns, and that is an engineering property you control. Efficiency came out model-invariant (every model 33–61% cheaper); quality came out capability-dependent (r = 0.99 with baseline strength). I would want an independent harness and a second workload before trusting the exact percentages. But the direction matches what I see in production: the token bill is set less by which model you picked and more by how you assemble the context you hand it.

Source: "The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI" (Muayad Sayed Ali et al., Writer AI, 2026) — arXiv 2607.06906. Figures 1, 3, 4, and 6 are reproduced from the paper for commentary. Benchmark numbers are quoted as reported; the interactive diagrams illustrate the mechanisms and use the paper's headline values.