# Agents-A1: scaling the agent horizon, not the parameter count
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/agents-a1
> date: 2026-06-30
> tags: llm, agents, reinforcement-learning, distillation, explainer
Most of the frontier's agentic gains over the last year came from one move: make the model
bigger. Kimi-K2.6 and DeepSeek-V4-pro are *trillion*-parameter systems, and they top the
hard agent benchmarks. **Agents-A1**, from InternScience, makes the opposite bet. It's a
**35B Mixture-of-Experts** model with only **~3B active parameters per token** (it's
initialized from `Qwen3.5-35B-A3B`), and it lands in the same benchmark band as those 1T
models. Its thesis, verbatim from the tech report: *scaling the horizon, not the
parameters.*
The chart is the whole argument. Agents-A1 and `Qwen3.5-35B-A3B` are the **same model** —
same 35B size, same ~3B active params. The ~18-point vertical gap between them is *entirely*
the training recipe, and it lifts the 35B student into the band held by models 30× its size.
The question this article answers is: what is "the horizon," and how do you scale it instead
of the parameter count?
## Two axes of horizon
"Agent horizon" splits into two things you can grow independently, and Agents-A1 pushes both:
1. **Long-horizon trajectories** — how *far* a single agent run goes. Agents-A1 is trained on
agentic trajectories averaging **~45K tokens** (deep-research runs average 44K, coding 48K,
scientific reasoning 37K, general agentic 39K; only short instruction-following tasks pull
the mean down at ~3K). That's hundreds of think→act→observe steps per task, not a single
question-answer turn.
2. **Heterogeneous abilities** — how *many kinds* of agent the one model can be. Agents-A1
unifies **six domains**: long-horizon search, engineering, scientific research, instruction
following, general agentic tasks, and scientific agentic tasks.
A single long-horizon run looks like this — a chain of tool calls, each with an observation
and a **verifier outcome** that says whether the step actually worked:
The verifier is the load-bearing part. A raw transcript of an agent flailing is not training
data; a transcript where every step is *checked* — did the code converge, does the answer
match the cited evidence — is. That check is what turns a 45K-token trace into a sequence of
trainable targets. Which is the next problem: where do verified 45K-token trajectories at
scale come from?
## The knowledge-action graph
You can't hand-write 100K agent trajectories. Agents-A1's answer is a **knowledge-action
graph** (KAG) per domain — a typed four-tuple
$$
\mathcal{G}_d = (\mathcal{C}_d,\ \mathcal{A}_d,\ \mathcal{O}_d,\ \mathcal{V}_d)
$$
| symbol | what it holds |
|---|---|
| $\mathcal{C}_d$ — **corpus** | evidence chunks, entities, facts, constraints — the domain's grounded knowledge |
| $\mathcal{A}_d$ — **actions** | tool calls, retrieval queries, code edits and executions, reasoning steps |
| $\mathcal{O}_d$ — **observations** | tool returns, retrieved evidence, execution states, intermediate artifacts |
| $\mathcal{V}_d$ — **verifiers** | automatic checks over correctness, evidence support, constraint satisfaction, goal completion |
The graph is populated by a **proposer–solver–verifier self-play game**: a proposer policy
$\pi_P$ samples regions of the graph to pose constrained tasks, a solver $\pi_S$ attacks them
with retrieval and tools, and a verifier $\pi_V$ checks the answer, the evidence, the
execution result, and the trajectory for shortcut-taking. A candidate task is kept **only if**
it's verifiable, valid, process-informative, evidence-covering, and unambiguously specified.
Each accepted step is logged as a record $(s_t, a_t, o_t, v_t)$ — prior state, action,
observation, verifier outcome — and *that* tuple is the trainable target. The data engine and
the agent are the same machinery: the graph that grounds the agent's actions is the graph that
generates its training data.
## The three-stage recipe
With verified trajectories in hand, the model is built in three stages — broaden, specialize,
then re-unify:
Stages 1 and 2 are familiar: a **full-domain SFT** pass aligns the base model with broad agent
behavior across all domains (~100K trajectories, response-token cross-entropy, one epoch at up
to 131K sequence length), then a set of **domain-level teachers** is trained, each with its own
recipe — the search teacher with SFT then GRPO over web-search/read/code tools; the science
teacher with reasoning-enhanced then tool-augmented SFT; the instruction-following and
tool-calling teachers with their own GRPO setups and reward shaping. Each teacher goes deep
where a single generalist would be pulled thin.
Stage 3 is the interesting one, and it's the same idea three separate papers landed on this
same week ([MOPD and DOPD](/arxiv/2026-06-30)): **on-policy distillation** as the way to fuse
capabilities. The full name is a mouthful — *multi-teacher domain-routed on-policy distillation
with salient vocabulary alignment* — so here's what each piece means:
- **On-policy.** The *student* generates the rollout, and the teacher supervises the student's
own tokens — not a fixed teacher transcript. This kills exposure bias: the student learns to
recover from the states it actually visits, not the ones a teacher would have.
- **Domain-routed.** Routing is hard and per-sample: each trajectory carries a domain label, and
it's supervised *only* by that domain's teacher ($\theta_{t,i} = \theta_t^{d_i}$). No learned
per-token gate — the task picks the specialist.
- **Salient vocabulary alignment (SVA).** At each position, the distillation loss is computed
*only over the teacher's top-$k$ vocabulary* — the handful of tokens the teacher actually puts
probability on. Both distributions are renormalized onto that support and matched with a
forward-KL term. The long low-probability tail, which carries no decision information, is
dropped. You align where the capability lives.
- **Heterogeneity-aware.** Losses are averaged *within* a domain first, then *across* domains, so
a high-volume domain can't drown out a small one — each active domain gets comparable influence
on the update.
The result is one deployable 35B student that inherits all six specialists, with **no teacher
shipped at inference**. If you've read the [inference write-up](/articles/how-llm-inference-works),
this is the training-time mirror of the serving-time story: the whole game is getting frontier
behavior out of a model small enough to actually run.
## The numbers
The payoff is parity with — and on several benchmarks, victory over — models ~30× larger. The
sharpest case is **FrontierScience-Research**, where the gap to the trillion-parameter field is
not subtle:
The base model scores **2.5**; the trained 35B scores **40.0** — above GPT-5.5's 26.7 and more
than double the trillion-parameter Kimi and DeepSeek. On long-horizon search, it takes overall
SOTA on **Seal-0**, edging out frontier systems that are far larger:
And on instruction following it leads outright, which matters because it's the capability most
likely to *degrade* when you fuse many domains into one model:
Across the full suite, Agents-A1 takes overall SOTA on **six** benchmarks (Seal-0, HiPhO 46.4,
FrontierScience-Olympiad 79.0, FrontierScience-Research 40.0, IFBench 80.6, IFEval 94.8) and is
the best ~35B-class model on most of the rest (BrowseComp 75.5, GAIA 96.0, XBench-DS 86.0,
SciCode 44.3, HLE-w/-tools 47.6, MolBench-Bind 56.8). It does **not** win everywhere — GPT-5.5
still leads pure web search (BrowseComp 84.4) and engineering (SciCode 56.1, MLE-Lite 72.7), and
DeepSeek-V4-pro tops GAIA (98.1) and the general-agentic τ²-Bench. The honest summary is parity
in the frontier band, with clear leads in science and instruction following — from a model you
can serve on a fraction of the hardware.
## Running it
Agents-A1 is **Apache-2.0** and runs on the standard stack — Hugging Face Transformers, vLLM, or
SGLang with OpenAI-compatible endpoints, at a served context of **262K tokens**. The release
recommends specific sampling for the long-horizon behavior to hold up:
```python
# vLLM / SGLang OpenAI-compatible call
sampling = dict(
temperature=0.85,
top_p=0.95,
top_k=20,
min_p=0.0,
presence_penalty=1.1, # discourages the repetitive loops long agents fall into
)
```
The `presence_penalty` is the non-obvious one: long agent rollouts are prone to getting stuck
repeating a failing action, and a mild penalty keeps the trajectory exploring.
## The take
What I like about Agents-A1 is that it's an honest systems argument, not a parameter flex. The
recipe is the contribution: a knowledge-action graph that makes verified long-horizon data a
renewable resource, and an on-policy distillation stage that folds many specialists into one
small model without the capability erosion you'd expect. It converges with a clear 2026 theme —
[on-policy distillation](/arxiv/2026-06-30) is becoming the default way to *integrate*
capabilities rather than trade them off, and [horizon, not size](/articles/how-llm-inference-works),
is where the agentic gains are now coming from.
The caveats are the usual ones for a benchmark-led release: the expert count and MoE routing
aren't disclosed, the "trillion-parameter performance" framing rests on benchmark parity rather
than a fitted scaling law, and benchmark SOTA is not the same as robustness in a messy
production loop. But the direction is the point. If a 35B model with 3B active parameters can be
trained to sit in the frontier's agentic band, the interesting frontier stops being *how big*
and becomes *how far* — how long the horizon, how many the domains, how good the verifiers.
---
*Built on [Agents-A1: Reaching Trillion-Parameter Performance with a 35B Agent](https://arxiv.org/abs/2606.30616)
(InternScience, 2026), the [project page](https://internscience.github.io/Agents-A1/), and the
[model release](https://huggingface.co/InternScience/Agents-A1) (Apache-2.0). Benchmark figures
are quoted from the tech report and model card.*