Tev-Gym: build the answer first, then grade every step

md json mcp

2026-10-14 · 19 min · reinforcement-learning · rl-environments · reward-design · grpo · agents · synthetic-data · benchmarks

Why read this

Notabletop 60%

Latent-world synthesis and PPRS from first principles, a live GRPO group under both rewards, and every headline gain recounted as tasks out of 18 or 22.

Original analysis
A lasting reference
A new technique

Training & RLNeeds datacenter GPUsNon-commercial licenceResearch paper

How this was scored

Is it new?: 2 of 3: A real new idea, method or capability
Can I trust it?: 2 of 3: Measures key facts from files, code or configs
Can I run it?: 1 of 3: API-only, gated or restrictive licence
Will I understand it?: 2 of 3: Mechanism from first principles with figures
Can I act on it?: 2 of 3: A concrete recipe, numbers or comparison
Will it last?: 2 of 3: A reference for a year or more
Does it affect many?: 1 of 3: A specialist community
Only here?: 2 of 3: A teardown or measurement few others did

Score 63 of 100, ranked 274 of 577 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored

Wenbo Hu posted the thread on Tev-Gym this week: "1,170 envs with dense progress rewards," a 27B model that "matches GPT-6 Astra & Claude Opus 5" on a long-horizon exam at Pass@5, and a 320B model "only 0.02 behind GPT 5.6 sol" on Terminal-Bench 4.0. The paper is arXiv 2610.18868, from Adobe Research and UCLA, with a project page.

Two ideas carry it: write the correct answer first, as a program, and derive the messy task from it; then, knowing the right answer at every stage, grade the middle of a rollout and not only its end. The thread's arithmetic is right everywhere I checked. What it leaves out is how small the denominators are.

Why long-horizon RL starves

Group-relative policy optimisation (GRPO) samples $G$ rollouts of one task and weights each by its group-normalised advantage, the paper's Eq. 2:

\hat{A}_i = \frac{R_i - \operatorname{mean}(R_{1:G})}{\operatorname{std}(R_{1:G}) + \epsilon}

With a terminal reward $R_i = \mathbb{1}[\tau_i \text{ passes}]$ , a group where every rollout fails has all $R_i = 0$ . The mean is 0, the numerator is 0 for every rollout, and the task contributes nothing to the gradient. Those $G$ rollouts were compute spent for nothing.

If the policy solves a task with probability $p$ , and rollouts are independent, the chance that all $G$ fail is $(1-p)^G$ . The thread says "66% of groups at a 5% solve rate, G=8". Reasoned: $0.95^8 = 0.663$ . The paper's other figures check too: $0.99^8 = 0.923$ (it says 92%) and $0.95^{16} = 0.440$ (it says 44%).

Two details the thread does not carry. First, the paper trains with $G = 24$ , not 8. At $p = 0.05$ that gives $0.95^{24} = 0.292$ , so 29% of groups would be dead. Reasoned. But on Tev-Gym itself the untrained Qwen3.8-27B fully solves 3.2% of environments in single-rollout probes (reported), and at $p = 0.03$ the figure is $0.97^{24} = 0.481$ . Reasoned. About half the batch, at the start of training, when signal matters most.

Second, a group with exactly one success is barely better. The one pass gets a positive advantage and the other $G-1$ failures all get the same negative one. A rollout that built nine-tenths of the answer and one that never opened the input folder are indistinguishable to the gradient.

Mid-horizon pipelines discard all-fail groups and resample, and so does this paper's training. On long tasks that means many draws per usable batch.

Build the answer first

The usual way to make an agent task is prompt-first: ask a model for a task, an input, a checker and a solution, and hope the four describe the same problem. The paper calls the failure mode artifact drift: each piece looks valid alone, but together they disagree.

Tev-Gym inverts the order. Every environment comes from a seeded generator program, written once by a coding agent and reviewed by the authors, that runs these stages with no model in the loop:

World construction. From a seed and a scale tier, generate the latent world $W$ : the complete, correct state of the domain. For the paper's running example, a retail warehouse with 502 customers, 52 products and 6,210 transactions over six months. $W$ is checked against invariants (unique ids, valid joins, finite measures). The same step draws a defect plan: exactly which records or fields will be corrupted, and by which declared operator.
Workflow DAG. Compile the workflow into a directed acyclic graph of stages, built by code from the size of $W$ . For the warehouse: profile-sources → reconcile-facts / normalize-dims → load-facts → aggregate-validate → package-sidecars. The longest chain of dependent stages is the task's horizon.
Rendering. Project $W$ onto the files an agent would see: CSV, TSV, JSON, SQLite, safetensors shards, binaries, manifests. Rendering is a pure function of $W$ , and the generator asserts it round-trips: parse the rendered files and you get $W$ back exactly.
Controlled corruption. Apply the defect plan. For the warehouse that is three operators: schema drift, duplicates and nulls, and category drift. The corrupted tree $s_0$ is what the agent receives. The untouched $W$ is the ground truth.
Public contract and hidden suite. Emit the prompt (inputs, required outputs and schemas, precedence rules, allowed tools) and seal, out of the agent's sight, metamorphic transforms that must not change the grade and plausible-but-wrong stale outputs that must fail.

Latent-world-first synthesis on the warehouse environment. The world and the corruption plan come first; the DAG, the corrupted inputs and both graders are derived from them (Tev-Gym paper, Figure 3).

Why does this give ground truth at every step, and not just at the end? Because nothing in the environment is independent of $W$ . The paper writes an environment as

E = \bigl(x,\; s_0,\; \operatorname{ver}_W,\; \Phi_W\bigr),

the prompt, the input tree, the strict verifier and the progress potential, all functions of the same $W$ . Rendering round-trips and every corruption is declared and invertible, so $s_0$ determines $W$ : the task is solvable from what the agent sees, and the generator already holds the answer. Since the DAG says what the workspace should look like after each stage, every correct intermediate state can be rendered from $W$ and compared against. That is the "exact answer key for every step."

A concrete instance: a duplicate-records operator injects copies of rows that exist in $W$ , and the defect plan records which. Removing them restores $W$ , and the grader can check row by row whether the agent's fact table did exactly that. A prompt-first pipeline has only the agent-visible files and whatever checker the model wrote.

The caveat: "invertible" is something each generator declares; the paper does not prove it per operator, and "missing records" is only recoverable if the information survives elsewhere in the files. The check is empirical: a reference solver and an independently written alternative must produce identical output trees on every seed, or the environment is rejected. I cannot check any of this in code. The generators are not released yet: the project page says "Code soon / Data soon," and the thread says they are "in internal approval, out within 7 days."

The eight gates, and what a gaming probe is

An environment is admitted only if it passes eight automatic gates (Appendix A.6, reported): reward causality (each reward component must drop when its cause is mutated), exploit resistance, role independence (reference solver, alternative solver, verifier and progress scorer share no code), solver agreement, progress validity, metamorphic invariance, six-seed diversity, and public-contract hygiene (no answer material in the prompt or input tree).

The 14 gaming probes are the exploit-resistance gate. Each is a canned cheat applied to the public tree, and each must score exactly 0 under both the strict verifier and the progress potential. The paper names six of the 14: empty output, minimal schema-valid output, copied input, single-seed hard-coding, path traversal and private lookup. The other eight are "and others." If you have read the MiMo audit, where a fix commit sat in .git in most of the coding tasks, "private lookup" is the one to want a list of, and it is not in the paper.

Is 12.4 minutes "6x the longest prior corpus"?

Reported (Table 1): a median rollout on Tev-Gym takes Gemini 3.6 Flash 12.4 minutes of wall-clock agent time; the longest prior corpus is SWE-Smith at 2.0 minutes. Reasoned: $12.4 / 2.0 = 6.2$ . The project page names SWE-Smith explicitly. The other corpora sit at 0.5 to 1.9 minutes.

The evaluator is shared; the scaffold is not. Prior corpora ran under Harbor's terminus-2 with 60 turns, Tev-Gym in the paper's single-bash harness with a 300-tool-call budget. The paper argues the prior medians would not move under a larger budget, since 60 turns ends only 1 to 13% of their rollouts. That holds for medians.

The progress potential

Each environment ships a progress contract: a set of nodes $V$ , each a checkable state of the output workspace, with weights $w_v > 0$ summing to 1 and prerequisite sets $\mathrm{pre}(v)$ . A partial scorer $c_v(s) \in [0, 1]$ grades the workspace on node $v$ directly against $W$ , using set precision and recall for record-valued outputs and declared tolerances for numbers.

Credit is prerequisite-gated. $\tilde{c}_v(s)$ is the lowest partial score among $v$ and all of its direct and indirect prerequisites. The potential is the weighted sum, the paper's Eq. 3:

\Phi(s) = \sum_{v \in V} w_v \, \tilde{c}_v(s) \in [0, 1]

The gating is what makes this hard to farm. A correct aggregate computed on a wrong fact table earns nothing, because aggregates inherits the score of the facts node beneath it. For the warehouse the nine nodes and their weights, read off Figure 3, are facts 0.18, dimensions 0.15, schema 0.12, summary 0.12, normalization, relations, aggregates and quality_report at 0.10 each, and readable at 0.03. Reasoned: they sum to 1.00.

Three of the gates exist to make $\Phi$ safe to train on: it is anchored (the untouched tree and every gaming probe score 0; $\Phi = 1$ exactly when the strict verifier accepts), monotone (rises along the reference trajectory, falls on regression), and independent (its scorer shares no code with the verifier or the solvers). No step labels, no trained reward model, no LLM judge. The paper's own AI-use statement is explicit: "No generative model grades any task or produces any training reward."

PPRS, and whether it is policy-invariant

Let $s_0, \dots, s_T$ be the workspace after each turn, with $\Phi(s_0) = 0$ . Following potential-based shaping, the per-step reward is the change in potential, $r_t = \Phi(s_t) - \Phi(s_{t-1})$ , and the rollout reward is the paper's Eq. 4:

R(\tau) = \mathbb{1}[\tau \text{ passes}] + \lambda \sum_{t=1}^{T} r_t = \mathbb{1}[\tau \text{ passes}] + \lambda \, \Phi(s_T)

with $\lambda = 0.25$ in every run, and not tuned.

The reference this invites is Ng, Harada and Russell (1999). Their result: add $F = \gamma \Phi(s') - \Phi(s)$ to the reward and the optimal policy is unchanged, because along any trajectory the shaping terms telescope to a constant. That constancy needs the terminal potential pinned (to 0, or to the same value everywhere). PPRS keeps $\lambda \Phi(s_T)$ , which differs between trajectories. So PPRS is not policy-invariant in Ng's sense, and the paper says so in as many words (Appendix B): "PPRS (Eq. (4)) is not policy-invariant shaping in the sense of Ng et al. (1999)."

That is deliberate: zero the terminal potential and all-fail groups starve again. What PPRS preserves is the ranking of individual trajectories. A pass earns $1 + \lambda = 1.25$ and a failure at most $\lambda = 0.25$ , so no single rollout is rewarded for trading completion for partial progress. What it does not preserve is the ranking of stochastic policies in expectation. The paper bounds it: the shaped objective can prefer a policy with pass rate $p_1$ over one with $p_2 > p_1$ only if

p_2 - p_1 < \frac{\lambda (1 - p_1)}{1 + \lambda} \le \frac{\lambda}{1 + \lambda},

which is 0.2 at $\lambda = 0.25$ , and only when the worse policy's failures end nearly complete while the better one's end near empty. Reasoned: $0.25 / 1.25 = 0.2$ , so the bound is right. Since every number reported is a strict-verifier pass rate, a distortion would show up in the tables; the sparse-reward ablation is the control.

Because $\Phi$ grades the workspace alone, two routes to the same files earn the same reward, and a perfect workspace that runs out of context before submit earns $\lambda \Phi(s_T) = 0.25$ , not 1.25.

One group, scored both ways

The widget below draws one GRPO group and scores it twice. The plateaus a failing rollout can stop on are the ones the paper measured on the warehouse environment: 0.05, 0.26, 0.69, 0.78, 0.90 and 1. Which plateau a rollout reaches is a toy distribution that moves deeper as $p$ rises; the paper gives none. At $p = 5\%$ and $G = 8$ , most draws produce an all-fail group: the terminal column is flat zero while the PPRS column ranks the rollouts. Push $\lambda$ to 0 and PPRS collapses back to the terminal reward. Under PPRS a group goes silent only when every rollout stops on the same plateau.

One GRPO group, two rewardsplateaus from the paper · their odds are a toy

solve rate p5%shaping weight λ0.25

group size G

0 of 8 passed

#	where it stopped	Φ(s_T)	terminal Â	PPRS Â
1	deeper plateau	0.78	+0.00	+1.22
2	nearly complete	0.90	+0.00	+1.55
3	every table partly loaded	0.26	+0.00	−0.23
4	readable, schema partly there	0.05	+0.00	−0.81
5	readable, schema partly there	0.05	+0.00	−0.81
6	nothing usable	0.00	+0.00	−0.95
7	schema and facts complete	0.69	+0.00	+0.97
8	nothing usable	0.00	+0.00	−0.95

terminal reward 1[pass]

Every reward is the same, so the group std is 0 and every advantage is 0. This task adds nothing to the gradient.

all-fail chance (1−p)^G = 66.3% · zero-signal groups in 4,000 draws: 66.0%

PPRS 1[pass] + λ·Φ(s_T)

Failures are ranked by how far they got. A pass still outscores any failure: it earns 1 + λ, a failure at most λ.

zero-signal groups in 4,000 draws: 0.0%

The real version: two rollouts submit an incomplete warehouse, five run out of the 65k-token context, and one builds the complete warehouse and never submits.

Step plot of the progress potential against turn number for eight rollouts. A red dashed line at zero marks terminal-only reward for all eight. Rollouts climb to plateaus labelled every table partially loaded near 0.26, schema and facts complete near 0.69, and all nodes satisfied at 1. Two circles at about 0.69 near turn 20 mark incomplete submissions; crosses mark context exhaustion; one orange rollout reaches 1 at turn 28 without submitting. — Eight rollouts of the trained 27B model on one warehouse environment. Terminal reward is 0 for every one; the progress potential separates them (Tev-Gym paper, Figure 4).

Across 24 such groups (12 seeds, base and trained 27B, 192 rollouts) the paper reports terminal reward 0 on all 192, so all 24 groups have zero advantage, while every group has non-zero variance under $\Phi$ .

What training actually used

The thread motivates PPRS with GRPO. The runs use DPPO, a group-relative objective with centred advantages and a total-variation bound of 0.1 on the policy ratio, no KL term (reported, Table 11). Per step: 8 tasks, $G = 24$ rollouts each, 192 rollouts, a 65,536-token response budget with 16,384 per turn, at most 64 tool calls, one bash tool in a python:3.13-slim container. Credit is per turn: turn $t$ gets $\lambda r_t$ , the last turn also gets the terminal term, and the advantage is the return-to-go with $\gamma = 1$ minus the group mean at the same turn index.

Since all-fail groups are already refilled by active sampling, the honest statement of what PPRS buys is cost and feasibility, and the ablations (Table 5, on Qwen3.5-9B, reported) say so:

Reward	GPU-h / step	ALE-CM S@1	ALE-CM P@5
Sparse (terminal only)	16.7	0.25	0.06
PPRS, pooled normalisation	10.9	0.28	0.06
Direct additive $\Phi(s_t)$ , $\gamma = 0$	9.7	0.29	0.06
PPRS, per-turn normalisation (Tev-9B)	9.0	0.29	0.11

Reasoned: sparse reward costs $16.7 / 9.0 = 1.86\times$ the GPU-hours per step. At 27B the paper says the sparse run "fails to complete a single optimizer step": active sampling never finds eight groups with any variance. That is the strongest evidence in the paper for the reward, and it is a sentence, not a table.

The data ablations make the case for the latent world. A public-view verifier (an LLM writes the checker from the agent's view, everything else kept) discards 88% of sampled groups and lands at S@1 0.24; prompt-first generation lands at 0.23; the base model is 0.25 (reported). Neither beats doing nothing.

The numbers, checked

The 27B base is Qwen3.8-27B, the 9B base is Qwen3.5-9B, and Tev-320B is GLM-5.3-Flash (the model in this site's GLM-5.3-Flash walkthrough). Measured from the Hugging Face metadata: gordonhu/Tev-27B holds 26,895,998,464 parameters and declares base_model: Qwen/Qwen3.8-27B; gordonhu/Tev-320B holds 321,323,031,390, with 288 routed experts and 8 per token in its config.json, and declares zai-org/GLM-5.3-Flash. Both are cc-by-nc-4.0. The same base was post-trained the same week for knowledge work, with an LLM judge and rubrics instead of verifiable rewards: Ultramarine 1 Flash.

Every relative gain in the thread recomputes from the paper's tables (reasoned on reported inputs):

Claim	Table value	Recomputed
ALE-CM Pass@5, Tev-27B, +65%	0.17 → 0.28	+64.7%
TB 3.0-CM Pass@1, Tev-27B, +56%	0.09 → 0.14	+55.6%
TB 4.0-CM Pass@1, Tev-27B, 2.5x	0.04 → 0.10	2.5x
TB 4.0-CM Pass@1, Tev-320B, +28%	0.29 → 0.37	+27.6%
TB 3.0-CM Pass@1, Tev-320B, +35%	0.20 → 0.27	+35.0%
TB 2.1 Pass@1, Tev-9B, +28%	0.178 → 0.227	+27.5%
"0.02 behind GPT 5.6 sol"	0.39 vs 0.37	0.02

Now the denominators. ALE-CM, the computing-and-math split of Agents' Last Exam, is 18 tasks (8 Math & Science, 5 ML, 5 Coding & System), each run five times. Pass@5 is the fraction of tasks solved at least once in five tries, so it moves in steps of 1/18. Reasoned: $0.17 \times 18 = 3.06$ and $0.28 \times 18 = 5.04$ . The +65% is three tasks becoming five. "#1 on Math & Science: Pass@5 0.50 vs 0.38" is four of eight against three of eight. Every one of the ten frontier models scores exactly 0.38 there. One task is the whole lead.

On the same table, Tev-27B's Pass@5 of 0.28 ties GPT-6 Astra, GPT-5.6 (sol) and Claude Opus 5, and sits below Claude Fable 5 and Claude Sonnet 5 at 0.33. On mean score S@1 it is 0.40 against 0.50 to 0.62 for the frontier rows; the "0.40 vs 0.32" in the thread is against the open Qwen3.8-2.4T-A95B, which runs in the same harness. On ML, Pass@1 and Pass@5 are 0.00 before and after training. The gains are concentrated in Math & Science.

The Terminal-Bench subsets are also small. TB 3.0-CM is 22 tasks and TB 4.0-CM is 18, each with five trials. They were chosen by asking Claude whether each TB 3.0 task belongs to the computing-and-math domain "even if only slightly," then keeping those a CPU-only harness can run. The paper reports Pearson $r = 0.96$ and $0.97$ between subset and full-benchmark Pass@1 across leaderboard models, which is a reasonable defence of the subset as a ranking device. It does not remove the sampling noise: reasoned, Tev-27B's TB 4.0 move from 0.04 to 0.10 is roughly 4 to 9 successful trials out of 90, and because both values are rounded to two places, the true ratio could sit anywhere from about 2.1x to 3.0x.

Bar chart of Pass@1 on the Terminal-Bench 4.0 CM subset for seventeen models. Claude Opus 5 leads at 0.52, then Claude Fable 5 0.46, GPT-5.6 sol 0.39, Tev-320B 0.37 highlighted in orange with an arrow marked plus 28 percent from its base GLM-5.3-Flash at 0.29. Further right, Tev-27B at 0.10 with an arrow marked 2.5 times from its base Qwen3.8-27B at 0.04. Gemini 3.7 Flash, Gemini 3.6 Flash and GLM 5.2 are starred. — Every model on the 18-task TB 4.0-CM subset, ranked by Pass@1. Starred rows were run by the authors, not taken from the leaderboard (Tev-Gym paper, Figure 2).

Whose numbers are these? On TB 3.0 and 4.0, frontier rows are the official leaderboard's; GLM 5.2 on TB 4.0 and both Gemini Flash models were run by the authors (the asterisks). On ALE-CM the frontier models ran in their native harnesses (Codex, Claude Code, Gemini CLI), every Qwen model in the training harness, Vanillux2. On TB 2.1 (89 tasks, leaderboard frontier rows), Tev-27B's 0.794 against Claude Opus 4.8's 0.789 is a 0.030 gain over a base already at 0.764.

In-domain versus zero-shot matters here. ALE-CM is in-domain by design: the generators "target ALE-CM," built around its seven computing-and-math subdomains, and the ethics statement says "part of the training mix is built against the interfaces of ALE-CM." Decontamination is lexical: no environment shares an 8-gram or a 13-gram with any evaluation task, and nearest-neighbour prompt Jaccard to ALE-CM is 0.22 median against 0.28 for ALE-CM tasks compared with each other (reported). That rules out copying, not targeting. The Terminal-Bench results are the zero-shot ones, and they are the more interesting evidence that the training generalises.

Finally, the 320B result is mostly reliability. Tev-320B's Pass@5 on TB 4.0-CM is 0.56 before and after; on TB 3.0-CM it moves 0.50 to 0.55. The paper says it plainly: training "mainly makes the model solve the tasks it can already reach more reliably rather than unlocking new ones."

Left: grouped bars comparing Tev and the Qwen base. ALE-CM 27B Pass@1 0.12 to 0.16 and Pass@5 0.17 to 0.28. TB 3.0-CM 27B Pass@1 0.09 to 0.14 and Pass@5 0.24 to 0.27. TB 4.0-CM 27B Pass@1 0.04 to 0.10 and Pass@5 0.19 to 0.25. TB 2.1 9B Pass@1 0.18 to 0.23 and Pass@5 0.29 to 0.35. Below, all models on ALE-CM ranked by Pass@5, with Tev-27B at 0.28 alongside four frontier models. Right: a summary of latent-world-first synthesis, from latent world to task environment to progress reward. — Headline results across ALE-CM and the three Terminal-Bench evaluations, and the synthesis pipeline in brief (Tev-Gym paper, Figure 1).

What is released

The weights are out, both under CC BY-NC 4.0, so research use only:

gordonhu/Tev-27B@60c36fd · snapshot 2026-10-14

parameters

26.90B

repo size

53.81 GB

finetuneQwen/Qwen3.8-27B

architecture: Qwen3_5ForConditionalGeneration
task: text-generation
library: transformers
license: cc-by-nc-4.0
safetensors: 12 shards
largest file: 4.97 GB
files: 28
downloads: 2
likes: 9
languages: en

parameters by dtype

BF1626.90B

agentlong-horizonterminalreinforcement-learningtev

repo last modified 2026-10-14

gordonhu/Tev-320B@08addaf · snapshot 2026-10-14

parameters

321.32B

repo size

642.67 GB

finetunezai-org/GLM-5.3-Flash

architecture: Glm5NextForConditionalGeneration
task: text-generation
library: transformers
license: cc-by-nc-4.0
safetensors: 49 shards
largest file: 14.87 GB
files: 62
downloads: 2
likes: 10
languages: en

parameters by dtype

BF16321.32BF32295.5K

agentlong-horizonterminalreinforcement-learningmoetev

repo last modified 2026-10-14

The generators, the 1,170 environments, the progress scorers and the training code are not. Until they are, the latent-world construction, the gates and the 14 probes are descriptions I can check for internal consistency but not run. When the code lands, the first thing worth reading is the list of gaming probes, and the second is whether every corruption operator is really invertible from the rendered files alone.

Where it fits

Most environment work on this site is about scale and containment: Prime Intellect's 365,000 environments behind one contract, Karotte keeping the answer key in the sandbox, Skill2Env, PortSimEnv grading against a proven optimum. Tev-Gym works upstream of that. Generate the answer before the question, and the answer key stops being something you guard at the end and becomes a grader you consult every turn.

The cost is authoring: one reviewed generator per workflow. Once it exists, an environment costs a median 30 s of CPU and no model calls (reported).

The transferable lesson is narrower than the thread's headline. A terminal reward on a task your policy solves 3% of the time leaves about half your groups silent at $G = 24$ . If you can say what a correct workspace looks like halfway through, $\lambda \Phi(s_T)$ gets that half back, at the price of an objective that is no longer exactly the one you care about. For why GRPO's estimator behaves this way at the token level, see Qwen's first-order RL formulation.