# Collective is a distribution claim, and the benchmark disagrees with it

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/skill2env
> date: 2026-09-22
> tags: explainer, agents, reinforcement-learning, evaluation, llm
"Collective" in a paper about agents usually means one of three things: a shared
replay buffer, a shared skill library the agents read at run time, or a joint
policy update across a population. [*Reinforcing Agents with Collective
Skills*](https://www.alphaxiv.org/abs/2609.reinforcing-agents-collective-skills)
(Xu et al., NVIDIA, September 2026) means none of them.

It means the **training distribution** comes from what thousands of people
bothered to write down. A public Agent Skill is a folder rooted at a `SKILL.md`
— frontmatter with a name, a description and trigger conditions, then free-form
instructions, optionally with `scripts/`, `references/` and `assets/`. Since the
format appeared in late 2025, practitioners have published thousands of them.
The paper's claim is that read as data, that corpus is three things at once: a
sample of the tasks people actually want delegated, a set of pointers to real
artifacts, and a body of quality criteria that outcome tests cannot express.

So Skill2Env is a compiler. `SKILL.md` in, containerised reinforcement-learning
environments out. 3.4k Skills became **7,971 Harbor tasks** across thirteen
domains, and RL over them moved Qwen3.8-27B **+4.7 points on Terminal-Bench 2.1**
and **+4.3 on their own held-out set**.

That is the headline. The result I have spent longer on is the one that lost.

<Callout type="note">
Another piece here, [The lesson is not in the skill](/articles/agent-beacon),
covers Beacon — a memory layer that watches agent sessions and promotes
high-signal runs into reusable skills. The two are often described as the same
idea and they are not; they run the arrow in opposite directions. Beacon turns
agent *runs* into skills. Skill2Env turns human-written *skills* into
environments an agent is trained in. One is a distillation loop closing on
itself; the other is an import from outside the loop, which is exactly why the
word "collective" is load-bearing here and not there.
</Callout>

## The compiler, and the two gates that matter

<Figure
  src="/articles/skill2env/fig1.png"
  alt="Six-panel diagram of the Skill2Env pipeline. SkillHub holds web-crawled Agent Skills, labelled collective human expertise and preference. Plan uses Codex as an investigator to research the web and decompose a Skill into three workflows. Diversify samples six axes — archetype, verifier pattern, complexity, persona, tone, expertise — with an example combination of repair/debug, regression suite, hard, support engineer, ticket snippet, novice. Create uses Codex as a generator inside Docker to ground public assets and produce the task, environment, tests and rubrics. Verify applies static checks, requires the Oracle to pass and a no-op agent to fail, and pins the base image. The output is an RL environment in Harbor task format with instruction.md, environment/Dockerfile, tests/test.sh, rubric.md and solution/solve.sh."
  caption="The pipeline. Three of the six panels are model work; the one that decides what survives is Verify, and it involves no model at all (Xu et al., Figure 2)."
/>

<SkillCompiler />

Two things in that pipeline are worth taking rather than admiring.

**The ordering rule in stage three.** The creator agent builds the world, writes
the instruction, implements the tests and writes the rubric, and *only then*
writes the reference solution. The paper says why in one sentence:

> Tests and rubric are frozen before the solution exists so that the solution has
> to satisfy the grading contract rather than the other way round; a failing
> reference solution is treated as a solution bug, not a reason to weaken the
> verifier.

Anyone who has watched a model write both a test and the code it tests knows
which way that pressure runs by default.

**The two acceptance gates.** Every candidate is built and run twice in fresh
containers: the reference solution must earn full reward on **every** metric, and
a no-op agent must earn **zero** on every metric. Fail either and the task is
dropped. The first catches a verifier nothing can satisfy; the second catches a
verifier that passes for free, which is the most common way a synthesised RL task
is silently worthless. Neither gate involves a model, which is precisely why they
can be trusted about tasks that models wrote.

This is the same pair [Prime Intellect's taskset work](/articles/scaling-agentic-rl)
landed on independently — gold-patch and no-op validation on every re-upload,
across ~365,000 tasks. Two groups, two corpora, the same two assertions. At this
point the Oracle/NOP pair should be table stakes for anyone publishing a
synthesised environment set, and its absence should read as a missing section.

There is also a deliberate omission worth naming, because it cuts the other way
from most pipelines. Skill2Env does **not** run a validation pass where a teacher
model attempts each task and discards the ones it fails — the standard move.
Their reasoning: that caps task difficulty at the validator's ability and roughly
doubles generation cost, and group-based RL with online dynamic filtering already
discards prompts whose rollouts carry no advantage. Unsolvable and trivial tasks
get filtered on-policy instead of a priori. That is a defensible trade and it
means the corpus contains tasks nobody has solved, by design.

## What "collective" buys, measured two ways

The first measurement is coverage, and it is the cleanest evidence in the paper.

<Figure
  src="/articles/skill2env/fig2.png"
  alt="Pie charts of domain distribution. The large left chart, Skill2Env, is split across thirteen domains: software engineering and testing 22.5%, AI/ML and scientific computing 10.5%, business/finance/legal/HR 9.3%, marketing and sales 8.8%, research and education 7.6%, documents and media 7.3%, infrastructure and cloud 6.9%, product and user research 6.7%, visual and UI/UX design 6.1%, data engineering 5.2%, productivity and collaboration 4.0%, cybersecurity 3.4%, health and consumer life 1.7%. Six smaller charts compare S2EBench, TMax-15K, Endless-Terminals, Terminal-Bench 2.1, Terminal-Bench 4.0 and DeepSWE 1.1; DeepSWE is almost entirely one colour, and the others concentrate in three or four technical domains."
  caption="Skill2Env is the only corpus in the comparison covering all thirteen domains, and the only one where non-technical knowledge work is a large share. DeepSWE 1.1 on the bottom right is a single slice — that contrast is the whole argument for compiling Skills rather than mining repositories (Xu et al., Figure 3)."
/>

Software engineering is **22.5%** of Skill2Env, against pie charts to its right
that are one or two slices wide. Business, marketing, research, documents,
product, design, productivity and health together outweigh it. If you believe
that agentic RL generalises from the distribution it trains on, that chart is the
paper's thesis rendered as area.

The second measurement asks whether the compiled task still carries anything of
the Skill, and it is the kind of check I wish more data papers ran.

<SkillFidelity />

A TF-IDF retrieval probe over all 8k tasks: query with a task's instruction and
rubric, search the 3.4k original `SKILL.md` documents, and the true source is
top-1 for **73.2%** and top-10 for **94.6%**, against 0.03% chance. The number
that earns its place is the ablation: the **rubric alone** retrieves its Skill at
**68.5%** top-1. Generic agent advice would retrieve nothing in particular. A
rubric that fingerprints its own Skill at 2,300x chance is carrying that Skill's
specific methodology.

So "collective" is defensible as a distribution claim and as a fidelity claim.
The question is what happens when you actually train on the methodology half.

## The result that lost

Two runs. Identical data, identical 300 steps, identical DPPO recipe — group
advantages over G = 8 rollouts, the binary-KL token mask at δ = 0.05, no
reference model, no KL penalty, no entropy bonus. One term of difference in the
reward:

$$
r = r_V + \lambda\,\frac{s}{5}, \qquad \lambda = 0.2
$$

where $r_V \in [0,1]$ is the programmatic Harbor reward averaged over up to six
named test metrics, and $s \in [-5, 5]$ is an LLM judge's score for the trace,
given the task's `rubric.md` and a "Constitution" penalising aimless loops,
reward hacking and unresponsive answers. The judge can move the reward by at most
±0.2.

<RewardTension />

Outcome-only RL wins every benchmark. The rubric run gives back 4.0 points on
Terminal-Bench 2.1, 3.0 on S2EBench pass and 11.5 on S2EBench mean score. And on
the one measurement that asks whether the agent *worked the way the Skill says
to* — 200 paired trajectories, a judge holding the source `SKILL.md`, A/B order
randomised — the rubric checkpoint is preferred **73.0%** of the time against the
base model, where outcome-only manages **54.5%**.

The paper's diagnosis is careful and I think correct:

> The rubric run's programmatic reward sits between 0.5 and 0.6 for most of
> training, below the outcome-only run, while its judge term stays flat from the
> first update to the last. The policy gives up some test passes for behavior the
> rubric rewards, and the judge signal shows no trend it could be climbing.

A flat judge term across 300 updates is the detail to sit with. It is not that
the judge is broken — it assigns −2 or lower to 21% of traces and grades
evidenced reward hacking at −5, so it discriminates. It is that the policy never
found a direction that moved it. Meanwhile the additive form lets a task that
failed every test still earn positive reward, and the judge sees the agent's
messages but not the filesystem. The authors list all three as untuned choices
rather than defending them.

But they also name the possibility that is not a tuning problem, and it is the
sentence I would put on the cover:

> The methodology the Skills write down may simply not be the distribution that
> maximizes benchmark pass rates.

That is the honest statement of what "collective" costs. A corpus of human
methodology encodes how people think a job should be done — evidence traced,
citations reconciled, defaults respected. A benchmark encodes whether the tests
pass. Optimising the first at λ = 0.2 measurably degrades the second, and the
only instrument that sees the gain is a preference test whose judge is another
model reading the Skill.

## Scale, in the right units

<Figure
  src="/articles/skill2env/fig3.png"
  alt="Bar chart of Terminal-Bench 2.1 scores. Two bars on the left under the label local: 49.4 for the base model and 54.1 after RL, annotated RL +4.7. Eight bars on the right under cloud/frontier, rising from 51.0 through 62.7, 69.2, 69.7, 71.2, 71.5 and 72.8 to 84.9, each labelled with a provider logo."
  caption="The same +4.7 with the rest of the field in it. A 27B model you can run locally moves from 49.4 to 54.1; the top of the chart is 84.9. The gain is real, transfers to a benchmark the corpus was never tuned for, and closes about 13% of the gap (Xu et al., Figure 1)."
/>

I am glad the paper drew this one. Isolated, "+4.7 points from 300 RL steps"
sounds like a step change. In the frame, it is a 27B open model closing roughly an
eighth of the distance to the best cloud agent, from a corpus that cost over
**\$90,000 in API usage** to generate with GPT-5.6 Sol at xhigh reasoning effort.
Both facts are true and the second one is not in the abstract.

The transfer argument does hold up, and it is guarded: Terminal-Bench 2.1 shares
no task above 13-gram Jaccard similarity of 0.8 with the training corpus, and the
training set was never tuned toward it beyond picking a technical subset. A gain
that survives that check reads as a general improvement in planning, tool use and
finishing rather than a memorised task family.

One more negative result deserves repeating because it replicates somebody
else's. They rolled out every task twice with GLM-5.3 — 15,968 trajectories, mean
reward 0.74, median 19 model calls and 23 tool calls — and used them for SFT.
On Qwen3.8-27B it slightly raised S2EBench and lowered Terminal-Bench. On
Qwen3.5-4B it **collapsed the model**: 18.7 → 3.4 on Terminal-Bench, 5.8 → 0.0 on
S2EBench pass, with repeated thinking and tool-call loops. Their reading is that
the teacher's interleaved-thinking style overwrites patterns the student's own
post-training relies on without transferring the competence. TMax reported the
same shape with a weaker teacher. Two papers, two teachers, one lesson: SFT on a
better model's traces is not a free cold start for an already post-trained agent.

## What I would check before using it

**The released dataset is not the dataset the results came from.** Footnote 1,
and it is a real caveat rather than a formality:

> For legal reasons, the initial dataset released is generated with Kimi-K3-max
> under the same pipeline, dropping a small subset from source Skills with
> concerning influence.

The paper's 7,971 tasks and every number above come from a GPT-5.6 Sol corpus.
What you can download was regenerated with a different model and is missing some
Skills. Same pipeline, same gates, and the Oracle/NOP checks mean the released
tasks are at least valid — but nobody has published a number comparing the two
corpora, and "same pipeline" is not the same as "same data."

**S2EBench is private.** Seventy-nine hand-verified tasks, reviewed for
unambiguous instructions, faithfulness to the source Skill and well-rounded
tests. The review criteria are stated, the tasks are not released. The largest
single number in the paper — S2EBench mean score, 56.6 → 75.1 — is on a set
nobody outside NVIDIA can run. Terminal-Bench 2.1 is the checkable half, and it
moved 4.7.

**The rubric channel is unfinished and the paper says so.** A single λ, an
additive form that lets a failed task earn positive reward, and a judge blind to
the filesystem. Anyone building on this should read Section 5.2 as a list of
three experiments rather than a result.

## The take

The mechanism is a compiler with two model-free gates, and it is good. The
corpus is the broadest terminal-agent training distribution anyone has published,
and the domain chart is the argument. The retrieval probe is the right check and
it passes.

But the word in the title is doing something specific, and it is worth being
precise about. "Collective" is not a training architecture here. It is a claim
about where supervision comes from: not a taxonomy someone designed, not a
repository crawl, but the accumulated residue of people writing down how their
own jobs should be done. That is a genuinely different source of signal and this
is the first paper I have seen treat it as a dataset rather than as a prompt.

And then the corpus turns around and reports that optimising the part of itself
that encodes *how* costs you the part that measures *whether*. The authors could
have buried that in an appendix. They ran it as a headline ablation, showed the
judge term going flat for 300 steps, and wrote down the interpretation that hurts
their own framing. That is the most valuable thing in the paper, and it is a
result the field will have to deal with long after this particular corpus is
superseded: **collective preference and benchmark pass rate are two objectives,
and right now we can only optimise one of them at a time.**

<ChangeMyMind>

<Falsifier claim="&quot;Collective&quot; here means the task distribution, not shared replay, a shared skill library at train time, or joint policy updates.">
Read off the pipeline: Skills are consumed once, offline, to produce containerised
tasks, and RL runs on a single policy against those tasks. It falls if some part
of the system I missed lets a training agent read the Skill corpus at rollout
time, or shares experience across concurrent policies. Nothing in Sections 3 or 4
describes either, but I read the paper and the released code index, not the code.
</Falsifier>

<Falsifier claim="Adding the rubric channel costs benchmark points and buys Skill-alignment preference.">
One seed, one λ, one judge model, 300 steps. A λ sweep that finds a setting
improving all three benchmark metrics over outcome-only would retire the tension
outright, and the paper itself predicts this is possible. Equally: the preference
test's judge is the same model family used as the training judge, which is a
conflict the paper does not flag. A human preference study, or a different judge,
could move the 73.0% substantially.
</Falsifier>

<Falsifier claim="The Oracle-must-pass / no-op-must-fail pair is what makes a synthesised environment set trustworthy.">
Both gates test the verifier, not the task. A task can pass both and still be
trivially solvable by a shortcut the reference solution happens not to take, or
unfaithful to the Skill it came from. If someone hand-audits a random 100 of the
released tasks and finds a meaningful rate of shortcut-solvable environments, the
gates are necessary and clearly not sufficient, and my "table stakes" framing is
too generous.
</Falsifier>

<Falsifier claim="The rubric carries Skill-specific methodology, at 68.5% top-1 retrieval against 0.03% chance.">
TF-IDF is lexical. A rubric that merely repeats the Skill's distinctive nouns —
tool names, file extensions, domain jargon — would retrieve it without carrying
any methodology at all. An embedding-based probe, or a probe with the Skill's
proper nouns stripped from both sides, would separate shared vocabulary from
shared method. The paper runs the lexical version only.
</Falsifier>

<Falsifier claim="The +4.7 on Terminal-Bench 2.1 is a general gain, not a memorised task family.">
The decontamination check is 13-gram Jaccard at 0.8, which is a surface-form test.
Two tasks can share a solution strategy without sharing 13-grams. A held-out
family the corpus demonstrably never touches — a domain absent from all 3.4k
Skills — would be a stronger test, and S2EBench cannot be it, since it is drawn
from SkillHub too.
</Falsifier>

</ChangeMyMind>

---

*Source: Binfeng Xu, Jian Hu, Hao Zhang, Shaokun Zhang, Yifan Zhang, Michael
Demoret, Jan Kautz and Yi Dong,
[*Reinforcing Agents with Collective Skills*](https://www.alphaxiv.org/abs/2609.reinforcing-agents-collective-skills)
(NVIDIA, 21 September 2026). Code at
[NVlabs/Skill2Env](https://github.com/NVlabs/Skill2Env). Figures 1, 2 and 3 are
reproduced from the paper for commentary; every quotation is from its text. The
compiler, reward-tension and retrieval-probe interactives are my own
illustrations, built from the paper's Tables 1 and 2 and Section 3.3 — no numbers
in them are invented, and none are measured by me.*