2026-08-03 · 5 min · agents · llm · context-management · explainer
Get the caveat out of the way first, because it changes how to read everything below: AutoCompact has no paper, no arXiv listing, and no released code. I checked the arXiv API by title and by author — zero hits. The project page is the only artifact, and the two charts that carry its headline claims (Figures 4–6) are unlabeled-axis line plots of "pass rate vs. inference-cost budget," not a results table. The one hard number in the whole post is "+10.6% RL gain on average on SWE-bench Verified," stated in prose, not shown as a table row you could check against a baseline.
So this is a research blog post, and I'm treating it as one. I still think the idea is worth explaining, because the mechanism — training a model to make a decision it doesn't naturally make, using an LLM judge to correct its trajectory rather than hand-labeling it — is a genuinely interesting instance of a pattern that shows up across post-training right now. Just don't read the rest of this as a verified result.
The problem: compaction is usually a clock, not a decision
Long-horizon coding agents run out of context. The standard fix — what the post says OpenAI uses for ChatGPT and Codex — is a fixed-threshold compaction: once the trajectory crosses some token count, everything before it gets replaced by a summary. It doesn't matter whether the agent is mid-hypothesis or about to write the fix; the clock doesn't know the difference.
AutoCompact's bet is that when to compact is a decision the agent should make about its own task state, not a number a harness enforces on it. The model gets a compact() tool call it can invoke at any point. When it does, the trajectory so far gets replaced by a generated summary — objective, the localized issue, files touched, what's verified, what's next — while the original task description and the most recent turns stay verbatim. Everything else (dead-end hypotheses, reverted edits, redundant file reads) gets dropped.
Drag the scrubber past 40k or 80k and the fixed-threshold lane fires mid-explore or mid-test — it only ever counts tokens, never task state. Past 55k or 90k the AutoCompact lane fires too, but always exactly at a phase transition (Explore→Implement, Test→Verify) — the same rough token budget, timed to a boundary the model itself decided was safe to summarize across. This trajectory and both thresholds are illustrative, not a measured trace; AutoCompact's post gives the mechanism but no exact numbers to reproduce here.
Training it: a judge corrects the trajectory, not the label
The harder problem is that models don't call compact() well without training — they don't reliably notice when a phase has ended. AutoCompact's answer is judge-guided correction rather than hand-authored demonstrations. At each step of a rollout, a judge (GPT-5.5-Codex) sees only the history visible so far, plus the fact that compact() exists, and makes one of three calls: leave the model's proposed action alone, replace it with a compact() call if this is a good moment to summarize, or repair a summary/continuation that's missing state or drifting off-track. That turns "teach the model to self-manage its own context" into "step-level correction under an annotation protocol" — something you can run at scale with a well-prompted LLM instead of a small army of human raters.
From 379 SWE-rebench tasks, after filtering malformed and off-track examples, this produces 1,052 SFT examples — a genuinely small cold-start set, split roughly 24% teaching when to trigger, 53% teaching what to preserve, 23% teaching how to continue after compaction. From that SFT checkpoint, GRPO reinforcement learning on SWE-Gym with a binary pass/fail reward pushes further: active compaction rate rises from 44.3% of tasks (SFT) to 58.5% (SFT+RL) — the RL stage doesn't just improve quality, it makes the model reach for compact() more often, because doing so is apparently what correlates with solving the task. Two more self-reported quality numbers from the post: generated summaries retain relevant state 99.8% of the time and specify a concrete next action 97.8% of the time.
This is also a context-management idea, which puts it in conversation with how a harness manages context more generally — Lilian Weng's point that durable state belongs on disk, not in an ever-growing prompt. AutoCompact is one layer higher: it's not asking where state should live, it's asking when the model itself should decide to shed it. Both are betting that context is the scarce resource and the harness (or the model, here) needs an explicit policy for spending it.
The take
The training method — judge-corrected trajectories teaching a behavior the base model doesn't do on its own — is a pattern worth knowing regardless of whether AutoCompact's specific numbers hold up: it's a cheap way to get supervision for a decision (when to compact, when to stop, when to ask) that's hard to hand-label at scale but easy for a strong model to critique step by step. What I can't tell you is how good the resulting agent actually is, because there's nothing outside one team's own charts to check it against. If code or a paper ships later, the interesting question is whether the qualitative "wins at every budget" story survives being reduced to a table.
Source: the AutoCompact project page (Xuan Zhang, Longtao Zheng, Cunxiao Du, Bo An, Xin Dong; July 30, 2026). No code or paper release exists at time of writing; all figures on the source page are JS-rendered widgets, not downloadable images — the timeline above is my own illustration of the mechanism, using an invented trajectory and thresholds, not a reproduction of anything on the page.