Recursive Harness Self-Improvement: beat your last harness, not a population of them

2026-08-03 · 9 min · agents · harness-optimization · information-theory · llm · explainer

Agent harnesses argued that the loop wrapped around a model — tools, context policy, control flow — matters as much as the model's own intelligence. The harness effect showed that orchestration, not the model, is what actually sets an agent's token bill. Both pieces treat the harness as a thing worth engineering carefully by hand. Recursive Harness Self-Improvement (Lee, Xu, Seely, Lee, Zaharia, Tang — Sakana AI and UC Berkeley) asks the next question: can the harness improve itself? Their answer treats the harness as a single text prompt and updates it using nothing but a comparison against its own immediately-previous version.

The idea, and the objective it can't afford

The harness a coding agent runs under — roles, instructions, and the workflow connecting them — is, in RHI's framing, just a string $H$ drawn from a space of harnesses $\mathcal H$ . Optimizing it against a broad population of competitors is the obvious move, and it's what most prior work does:

H^*_x \in \arg\max_{H \in \mathcal{H}} f_x(H), \qquad f_x(H) = \mathbb{E}_{H' \sim \mu,\; y \sim \mathcal{A}(H,x),\; y' \sim \mathcal{A}(H',x)} \big[\mathbf{1}\{y \succ y'\}\big]

$\mu$ is a distribution over competitor harnesses, $\mathcal A(H,x)$ is the agent running harness $H$ on task $x$ , and $y \succ y'$ means an LLM judge preferred output $y$ . The problem is cost: a population of size $m$ needs $m$ fresh agent executions and $\binom{m}{2}$ pairwise judgments per iteration — $\Theta(m^2)$ — before you can even take one optimization step. For a user continually specializing a harness to a new task, that's not a research inconvenience, it's prohibitive.

RHI's relaxation replaces the population with a point mass on the harness's own previous version:

\tilde{f}_x^{(i)}(H) = \mathbb{E}_{y \sim \mathcal{A}(H,x),\; y^- \sim \mathcal{A}(H_x^{(i-1)},x)} \big[\mathbf{1}\{y \succ y^-\}\big]

One new execution, one comparison, cached forever after. $\Theta(1)$ per iteration, independent of how large a population you'd otherwise have wanted.

search cost · population vs. trajectory-localC = N_trace + N_pair

m — population size a finite-population search would comparem = 10

population (m=10) 55 ops/iterationRHI 2 ops/iterationgap 27.5×

Every population-based harness search — ADAS, AFlow, GEPA, and the roughly a dozen others RHI cites — pays for a sampled population of size m: m fresh agent executions plus C(m,2)pairwise judgments per iteration, Θ(m²). RHI replaces the population with a point mass on the harness's own immediately-previous version: exactly one new execution and one comparison, Θ(1), independent of m. Drag m up and the flat RHI line stops looking like a simplification and starts looking like the whole point.

Why comparing to yourself is still principled

The obvious objection: isn't comparing only to your immediate predecessor a much weaker signal than comparing to a whole population? RHI's answer is a Bradley-Terry argument. Assume there's a latent task utility $u_x : \mathcal H \to \mathbb R$ and a link function $\sigma$ (strictly increasing, $\sigma(0) = \tfrac12$ ) such that $\Pr(H \succ H') = \sigma(u_x(H) - u_x(H'))$ — the standard pairwise-preference model. Then both objectives are monotone in the same latent utility:

f_x(H) = \mathbb{E}_{H' \sim \mu}\big[\sigma(u_x(H) - u_x(H'))\big], \qquad \tilde{f}_x^{(i)}(H) = \sigma\big(u_x(H) - u_x(H_x^{(i-1)})\big)

So any revision that beats $H^{(i-1)}_x$ with probability greater than one-half also increases the ideal, population-level objective. RHI performs noisy local ascent on the same utility ordering a much more expensive search would climb — it just takes a smaller, cheaper step each time, using the accumulated preference history as the only signal for which direction is up. There's no proof this converges, or how fast; it's a directional argument, not a guarantee.

The algorithm this licenses is short. At iteration $i$ : run the agent under $H^{(i)}$ , get an output. Compare it against the cached output from $H^{(i-1)}$ . Save the preference. Feed the accumulated preference history to an LLM harness optimizer, which writes $H^{(i+1)}$ .

Five-step diagram: a harness at iteration i runs a task through a group of coding agents to produce an output repo; the output is compared against the output repo from iteration i minus 1 to produce preference feedback; the feedback is saved into a self-comparison history alongside all prior preference feedback; the history is used to update the harness, which feeds back into the next iteration. — The RHI loop: run, self-compare against the immediately-previous output, save the preference, update the harness — one execution and one comparison per iteration (Lee et al., 2026, Figure 2).

Critically, the harness optimizer never sees the evaluation prompt $x_{eval}$ directly — only the preference history, which was itself generated by a judge conditioned on $x_{eval}$ . Alignment with the actual evaluation criteria happens indirectly, through the accumulated comparisons, not because the optimizer was told what's being graded.

What actually gets rewritten

RHI decomposes the harness into agent design (roles and instructions for each candidate agent) and agent workflow, which splits further into contracts — what information passes between subagents and the orchestrator — and hops — the interaction structure and control flow. The optimizer's own prompt is explicit about where to spend its edits: prioritize contracts and hops over roles and instructions.

Decomposition diagram: a harness splits into agent design (role and instruction, shown as one agent directing four subagents) and agent workflow, which splits into contract (an interface contract with a note icon, shown as bidirectional arrows between an orchestrator and subagents) and hop (an interaction structure, shown as numbered arrows setting the order agents run in). — RHI's harness decomposition: agent design (role, instruction) versus workflow, itself split into contract (what's exchanged) and hop (the interaction order) — the optimizer is told to prioritize the second pair (Lee et al., 2026, Figure 3).

The hypothesis behind that priority: a task-specific contract tells the orchestrator and subagents what to pass along instead of making them condition on the entire interaction history, which is "conceptually analogous to imposing a task-dependent sparsity pattern on inter-agent information flow" — sparse attention for agent communication, in effect. Better contracts should mean less redundant context, better cache efficiency, and lower cost, for free, alongside better task performance.

Does it work, and what does it cost

Across 30 synthetic ML-research tasks (finance, robotics, pharma), a few RHI iterations substantially raise the ceiling that test-time scaling alone can reach. With Opus-4.7, one iteration is enough to beat both xhigh and max reasoning-effort settings. With Opus-4.8, two iterations beat xhigh, max, and the provider's own built-in dynamic multi-agent harness, ultracode — a user-constructed, prompt-level harness beating a vendor's dynamic scaffold.

Six panels of pairwise-win bar charts across three models (sonnet-4.6, opus-4.7, opus-4.8) against baselines max, xhigh, and ultracode, each panel also plotting normalized cost as an orange line; in every panel the RHI-improved harness's win count and cost line rise together across iterations H[0] through H[4], crossing above 19.5 out of 30 wins (marked with a crown) while cost stays at or below the compared baseline's. — Few-shot RHI raises the ceiling of test-time scaling across all three model families; cost (orange line) tracks flat or falls even as win count climbs (Lee et al., 2026, Figure 1).

The gains aren't from longer outputs — normalized token usage stays roughly flat across iterations for Sonnet-4.6 and Opus-4.8 while win rate climbs (Opus-4.7's data can't separate the two hypotheses; only two iterations were run and its token count rose alongside performance, which the paper states plainly as inconclusive). What actually improves is cost, largely through less redundant cache read/write from better-managed context:

cost reduction from RHI-improved harness, vs. same-effort baseline

Opus-4.8 vs ultracode

60%

Opus-4.8 vs max

23%

Opus-4.7 vs max

18%

Sonnet-4.6 vs max

0204060

The 60% figure is the abstract's headline, and it's the comparison against the provider's own dynamic multi-agent harness — not against a same-family reasoning-effort setting. A companion ablation (Appendix A) found something the paper didn't have to report: the provider's built-in multi-agent mode scores a lower Elo than running single-agent, despite costing far more — the vendor's own dynamic scaffold failing to pay for itself on this benchmark, stated without softening.

The information-theoretic account

Section 6.3 goes further than "it works" and proposes why: RHI implicitly maximizes task information in the components it's told to prioritize (contracts, hops) while minimizing task-conditional redundancy across all components. Formalized,

J(g_i) = \underbrace{\sum_{hc \in \mathcal{C}_{ext}} \frac{1}{K^{hc}_{Xi}} \sum_{k=1}^{K^{hc}_{Xi}} I\big(z^{hc,(i)}_{Xk}; X\big)}_{f_{ext}} \;-\; \beta \underbrace{\text{TC}\big\{z^{hc,(i)}_{Xk}\big\} \big|X}_{f_{int}}, \qquad \beta > 0

$f_{ext}$ is mutual information between the externally-emphasized components (contracts, hops) and the task $X$ ; $f_{int}$ is the task-conditional total correlation — redundancy — across all four component types. The hypothesis: RHI is implicitly raising the first term and lowering the second, estimated here with canonical-correlation mutual information and total correlation over PCA-whitened sentence embeddings.

component drift · task mutual information I(z; task)debiased text-embedding-3-large

RHI iteration

redundancy (total correlation | task) 4.84 nats

RHI's optimizer prompt tells it to prioritize editing contracts and hops — the workflow — over roles and instructions. Toggle H[1] → H[4] and that priority shows up as measurement: hop and contract mutual information with the task rise, role falls, instruction stays flat — while redundancy across all four components drops from 4.84 to 3.63 nats. Read together, that is the paper's hypothesis for what RHI is implicitly optimizing: more task information in the components it is told to touch, less duplicated information everywhere. The paper is explicit that this is a correlational reading of an embedding proxy, not a proof of the optimizer's true objective.

The paper is careful about how much weight this deserves: it "does not prove that RHI optimizes a unique scalar objective," should be read as "an embedding-based proxy," and is explicitly "correlational rather than causal" — not a claim about what the optimizer LLM is actually doing internally, just a consistent pattern in what its edits produce.

warning

Two honesty gaps worth stating plainly, because the paper doesn't headline them. First, in the Opus-4.8 experiments, one of the two LLM judges is opus-4.8-xhigh itself — the same model family whose harness is being evaluated also serves as one of its own judges, scoring opus-4.8 runs against other opus-4.8 baselines. The paper does average across two judges (the second is an independent gpt-5.5-max), which dilutes the self-judging influence, but the paper does not discuss it as a potential bias source. Second, every agent tested belongs to one vendor — Sonnet-4.6, Opus-4.7, Opus-4.8 — with no cross-family test on GPT or Gemini as the agent under improvement (those families only ever appear as judges). And there is no empirical comparison against any of the roughly 15 directly competing methods the paper discusses in its own related work — Meta-Harness, Self-Harness, TTHE, ADAS, GPTSwarm, AFlow, GEPA, DSPy, and others. RHI is only benchmarked against same-family reasoning-effort scaling and the provider's built-in multi-agent mode, not against the alternatives it explicitly positions itself against.

The benchmark itself is also self-constructed: 30 tasks synthesized from real job postings by an LLM, evaluated by the same lab that designed the method being tested on them. None of this means the result is wrong — the plain admission that Opus-4.7's token-length claim is inconclusive, and the decision to run and report the ablation showing the vendor's own multi-agent mode underperforms single-agent, are both the kind of finding a paper trying to look better than it is would have left out.

The take

The part of RHI I'd actually reuse is the trajectory-local relaxation itself: comparing only against your immediately-previous version turns an intractable population search into something you can run continuously, cheaply, and the Bradley-Terry argument for why that's still principled — not just convenient — is genuinely nice. The information-theoretic account of why it lands on contracts and hops is a good hypothesis, stated with the right hedges. What I'd want before trusting the magnitude of any specific number: a comparison against even one of the population-based methods it explicitly argues against, and an evaluator that isn't sometimes the model being graded. This reads as the first half of a real idea — the paper says so itself, calling the harness-to-model feedback loop "the second half" left to future work — and the half that's here is worth having, with its gaps named rather than papered over.

Source: Recursive Harness Self-Improvement (Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Matei Zaharia, Yujin Tang — Sakana AI, UC Berkeley), arXiv:2607.15524. Figures 1, 2, and 3 are reproduced from the paper for commentary; the interactives are mine, built on the paper's own reported formulas and measured endpoints.