2026-08-25 · 11 min · agents · open-weights · benchmarks · harness · moe
Almost every agent release conflates two things. New weights land, they arrive wrapped in a new scaffold, and a single benchmark number reports the pair — which tells you the combination improved and nothing about which half did it. It is one of the more expensive ambiguities in the field, because the two halves have wildly different costs to reproduce.
Apodex-1.1 does something better, apparently without meaning to make a point of it. Their benchmark chart carries three rows per task: the previous model, the new model in a single-agent ReAct loop, and the new model in the Agent Team harness. Rows two and three are the same checkpoint. The gap between them is a measurement of a harness with the model held fixed, and it is bigger than I expected.
| Weights | Apodex-1.1-mini · 36B total (Qwen3.5-35B-A3B fine-tune) · Apache 2.0 · FP8, NVFP4, GPTQ-Int4 variants |
| Context | 262,144 tokens deployed · SGLang or vLLM, qwen3_coder tool parser, qwen3 reasoning parser |
| Harness | ApodexAI/FrontierAgent — Apache 2.0, ReAct and Agent Team, TUI + eval runner |
| Flagship scores | APEX-Agents 38.5 · GDPval 78.8 · FrontierFinance 54.3 · FrontierScience-Research 63.3 · BioMysteryBench 35.3 · HLE 56.1 |
| Where it leads | FrontierFinance and FrontierScience-Research — the only two of six where it passes the best competing system |
| The harness premium | ≈40% of the 1.0 → 1.1 gain, averaged over six benchmarks · +11.8 on BioMysteryBench alone |
| Report | arXiv:2608.23283 · tech blog |
- parameters
- 35.95B
- repo size
- 71.92 GB
- architecture
- Qwen3_5MoeForConditionalGeneration
- license
- apache-2.0
- downloads
- 7.6K
- likes
- 126
- files
- 28
The number nobody publishes
Rows two and three are the same checkpoint. Nothing about the weights differs between them — only the scaffolding the weights run inside, and whether the coordinator is allowed to fan work out to bounded sub-agents instead of doing it in one long loop. So the green segment is a rare thing: a measurement of a harness, with the model held fixed.
Across the six flagship benchmarks it averages about 40% of the total gain from 1.0 to 1.1. On BioMysteryBench it is +11.8 against +5.9 for the entire model upgrade — the harness contributes twice what the new weights do. Note also where the yellow markers sit: Agent Team passes the strongest competing system on two of six, and on GDPval and BioMysteryBench it is not close. This is a good open model with an unusually good harness, not a frontier model.
The green segment is the harness. Same weights above and below it; the only difference is whether the coordinator may decompose the request and dispatch bounded parallel assignments, or has to do the work in one long stateful loop.
Averaged over the six flagship benchmarks it is about 40% of everything gained from 1.0 to 1.1. On BioMysteryBench it is +11.8 against +5.9 for the model upgrade — the scaffolding contributes twice what the new weights do. On GDPval it is +9.3, against +10.2 for the model: near parity between a training run and a control-flow decision.
Two caveats before anyone over-reads that, and Apodex states the first themselves.
The first is that Agent Team is not free. It runs more sub-agents, which means more tokens and more wall-clock spend per task, and none of the charts report either. A harness that spends 4× the tokens to gain 10 points is a real result, but it is a different result from one that gains 10 points for free, and the published figures cannot distinguish them. This is the single biggest hole in the release.
The second is that these are Apodex's own evaluations of Apodex's own harness. They took an unusually good precaution — benchmark-hosting sites are blocked during evaluation, which addresses the most common way agentic scores get inflated — but the harness and the model were developed together, and the ReAct baseline is the one they chose to compare against rather than one an independent party picked.
Even discounted for both, the direction is the interesting part. We are two or three years into a period where the scaffold is a first-class contributor to capability, and it is almost never measured separately. Apodex measured it, arguably by accident, and the number is large.
Where the model actually places
Read the grey bars and the picture is more specific than "frontier-level performance across professional work, finance, scientific research, and general reasoning."
On FrontierFinance (54.3 vs Claude Fable 5 at 49.2) and FrontierScience-Research (63.3 vs DeepSeek V4 Flash at 55.0), Apodex-1.1 Agent Team leads. Those are the two Apodex names as wins, and they are wins.
On the other four it does not. GDPval 78.8 against Claude Opus 5's 89.4 is a ten-point gap. BioMysteryBench 35.3 against 49.4 is a fourteen-point gap. HLE 56.1 against 64.7. APEX-Agents 38.5 against 42.3.
That is a completely respectable position — an Apache-2.0 model that tops two research-heavy leaderboards and sits mid-pack against closed frontier systems on the rest. "Frontier-level" is doing some work in that sentence, but the underlying result does not need the adjective.
The mini checkpoint is the one you can actually run, and its story is the same shape at smaller scale. 36B total parameters on a Qwen3.5-35B-A3B base, so roughly 3B active per token — cheap to serve for what it does. FrontierFinance 50.2 leads the compared systems; APEX-Agent 27.7 is within 0.2 of Kimi K2.6's 27.9; FrontierScience-Research 51.7 trails DeepSeek V4 Flash's 55.0. And Agent Team beats ReAct on all three again: +3.5, +10.2, +6.7.
What the harness actually does
The mechanism behind the green segment is not exotic, and the diagram gives it away. The coordinator's state is a task board — pending, active, completed, blocked, cancelled — plus a pool of structured reports. It never holds the raw material the sub-agents worked through.
The ReAct bar is not merely bigger, it grows with the task. Every page read and every recovered error stays in the same context for the rest of the run, so a long job spends its back half reasoning over a transcript largely made of things it already finished with. Push the steps slider right and it crosses the window — at which point something has to be discarded, and the agent has no principled way to know which part it will need in an hour.
Fanning out does not make the total work smaller. It changes which agent has to hold it. Each sub-agent sees a bounded slice and returns a structured report; the coordinator sees k reports, never the raw observations. Both bars stay flat as long as you widen k with the task — which is also why the wall-clock tab barely resembles the context tab, and why the two arguments for the Team harness are genuinely separate. Raising k buys latency and buys headroom, and only one of those is what the benchmark deltas are measuring.
That is the argument in one picture. A ReAct agent accumulates: every page it read, every command it ran, every error it recovered from remains in the same context for the rest of the task, so a long job spends its back half reasoning over a transcript mostly composed of things it already finished with. Past the window, something must be discarded, and the agent has no principled way to know which part it will need in an hour.
Fanning out does not reduce the work. It changes which agent has to hold it — and both the worst sub-agent context and the coordinator context stay roughly flat in total task size, as long as the width grows with the job.
Worth separating carefully: that control has two tabs and they are two different arguments. Widening k buys latency and buys context headroom, and only the second one plausibly explains a benchmark delta, since a benchmark does not score you on wall clock. If the Agent Team premium were mostly parallelism, it would show up as faster, not better. It shows up as better, which points at the context argument — or at the verification pass, which is the other thing the Team configuration adds and the ReAct one does not have.
That verification layer deserves its own line, because Apodex describes it as Statement Review: key claims are independently checked against their supporting sources, data and computations before delivery, and when evidence is insufficient or citations do not match, the system flags it, corrects the affected conclusion, and keeps the review inspectable. On research benchmarks specifically — which is where the premium is largest — a second pass that catches unsupported claims is exactly the intervention that would move a score. It is possible the harness premium is mostly this, and not the fan-out at all. The published numbers cannot separate them.
The runtime you can actually use
FrontierAgent is Apache-2.0 and, unusually, is the same code that produced the benchmark numbers rather than a cleaned-up demo of it. The layering is deliberate:
frontier_agent/ generic loop, scheduling, registries, AgentBus, observers
plugins/tools/ web, shell, file, sandbox, and team tool implementations
workflows/ ReAct and Agent Team pipelines, profiles, prompts, observers
apodex/ terminal CLI/TUI, approvals, sessions, traces, Docker path
benchmarks/ public harness plus bundled FrontierSearchBench/FrontierChallengeThe boundary that matters is between workflows/ and everything else: the workflow engine that runs the TUI is the workflow engine that runs the evaluation, so a change in how the coordinator delegates shows up in both. A great many agent frameworks ship an eval harness that has quietly drifted from the product.
Two things in that table are worth more than the rest of it. The first is that rm -rf /workspace/tmp is governed by the same policy as a file write, because the shell tool and the file tools share one sandbox. A great many agent runtimes wrap file access carefully and then hand the model a shell that walks straight around it.
The second is the order of the checks. Switch to --yes and try to write to /inputs: the root policy decides before the approval mode gets a say, so the flag that removes the human does not remove the boundary. The failures are fail-closed — an unrecognised path is denied, not allowed, which is the only default that survives an agent inventing a path you did not anticipate. Behind all of it, sessions are checkpointed, mutations are journalled, and /revert exists — worth more in practice than any amount of care about what the agent was allowed to do.
The filesystem contract is the part I would look at first in any agent runtime, and this one is right. Three roots — /inputs read-only, /workspace read-write, /outputs for persistent deliverables — shared by the shell tool and the file tools alike, so rm -rf obeys the same policy as a file write instead of escaping through a subprocess. That single detail separates a sandbox from a gesture at one.
The rest of the operational story is the unglamorous stuff that decides whether you can leave a thing running. Mutations show a diff and require approval unless --yes. Sessions are checkpointed, actions are traced locally, /revert restores changes, --resume continues a saved run. Typing while an agent runs queues an instruction that is injected at the next safe turn boundary rather than tearing down the active run — and in Agent Team mode it steers the coordinator while already-running sub-agents are allowed to finish, which is the correct semantics and slightly fiddly to implement. On macOS and Docker, /outputs maps to .apodex/runs/<session-id>/outputs on the host alongside the checkpoint, trace, engine log and trajectories.
Running it takes an OpenAI-compatible endpoint, Python 3.12, and uv:
git clone https://github.com/ApodexAI/FrontierAgent.git
cd FrontierAgent && uv sync --python 3.12 --extra dev
cp .env.example .env # OPENAI_API_KEY / OPENAI_BASE_URL / OPENAI_MODEL
uv run frontier-agent --mode agent_team --cwd /path/to/projectServing the weights yourself is a Qwen3.5 deployment with two parsers attached:
vllm serve apodex/Apodex-1.1-mini --tensor-parallel-size 8 \
--max-model-len 262144 --enable-auto-tool-choice \
--tool-call-parser qwen3_coder --reasoning-parser qwen3One deployment note buried in the model card and easy to get wrong: pass tool schemas through the API's tools= parameter rather than inlining descriptions in the system prompt. The chat template renders them into the <tool_call><function=…> format the server-side parser expects, and hand-written descriptions in the prompt produce calls the parser cannot recover. The recommended sampling is temperature 1.0, top_p 0.95, repetition_penalty 1.05, max_tokens 32768 — a notably high temperature, consistent with a model meant to explore a long trajectory rather than produce one right answer.
The evaluation suite
Thirteen benchmarks ship with the runner: BrowseComp, xbench-DeepResearch, Humanity's Last Exam, SuperChem, FrontierScience-Research, FrontierScience-Olympiad, DeepSearchQA, WideSearch, FrontierSearchBench, OfficeQA, GDPval, APEX, OneMillion-Bench. It supports deterministic and model-based judges, resumable experiments, concurrent runs, progress inspection, and rerunning individual failures — which sounds like a list of features until you have tried to debug a hundred-task agentic eval where the only affordance is "run it all again."
uv run python -m benchmarks.public.runner.run_subprocess \
--benchmark browsecomp --pipeline stateful-react-agent \
--profile default --limit 1 --concurrency 1 --out ./results/smokeThe genuinely useful thing here is not that Apodex scores well on these. It is that the harness someone else's model needs in order to be compared fairly is now available, along with the evaluation code that produced the published numbers. If you want to know whether the ~40% harness premium reproduces on a model Apodex did not train, the apparatus to check is sitting in the repository.
The ledger
Real. An Apache-2.0 36B-total / ~3B-active checkpoint that leads FrontierFinance and comes within 0.2 of the best APEX-Agent score, with FP8, NVFP4 and GPTQ-Int4 variants published alongside. An Apache-2.0 runtime that is the same code as the eval harness, with a shared shell/file sandbox, fail-closed authorization, checkpoints, traces and revert. Thirteen bundled benchmarks. And a three-row benchmark chart that isolates the harness — the most useful thing in the release and the thing least likely to be noticed.
Unreported. Token cost and wall clock for Agent Team versus ReAct, which is the number that decides whether a 40% harness premium is a bargain or a bill. No ablation separating fan-out from Statement Review, so the mechanism behind the premium is inferred rather than shown. No independent replication of the harness delta on a model Apodex did not train — though, to their credit, the code to do it is public.
Overstated. "Frontier-level performance" across all four listed domains, when the published charts show it trailing Claude Opus 5 by ten points on GDPval and fourteen on BioMysteryBench. Leading two of six is a good result stated accurately, and the accurate statement is right there in the same paragraph.
The thing I will keep from this release is not the model. It is the shape of the chart. Three rows, one checkpoint held fixed across two of them, and a number for the scaffold — published almost in passing, by a team who mostly seem to want you to look at the top bar.