# Tev-Gym: build the answer first, then grade every step > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.thesatyajit.com/articles/tev-gym > date: 2026-10-14 > tags: reinforcement-learning, rl-environments, reward-design, grpo, agents, synthetic-data, benchmarks Wenbo Hu [posted the thread](https://x.com/gordonhu608/status/2110400049515728911) on Tev-Gym this week: "1,170 envs with dense progress rewards," a 27B model that "matches GPT-6 Astra & Claude Opus 5" on a long-horizon exam at Pass@5, and a 320B model "only 0.02 behind GPT 5.6 sol" on Terminal-Bench 4.0. The paper is [arXiv 2610.18868](https://arxiv.org/abs/2610.18868), from Adobe Research and UCLA, with a [project page](https://tev-agent.github.io). Two ideas carry it: write the correct answer first, as a program, and derive the messy task from it; then, knowing the right answer at every stage, grade the middle of a rollout and not only its end. The thread's arithmetic is right everywhere I checked. What it leaves out is how small the denominators are. ## Why long-horizon RL starves Group-relative policy optimisation (GRPO) samples $G$ rollouts of one task and weights each by its group-normalised advantage, the paper's Eq. 2: $$ \hat{A}_i = \frac{R_i - \operatorname{mean}(R_{1:G})}{\operatorname{std}(R_{1:G}) + \epsilon} $$ With a terminal reward $R_i = \mathbb{1}[\tau_i \text{ passes}]$, a group where every rollout fails has all $R_i = 0$. The mean is 0, the numerator is 0 for every rollout, and the task contributes nothing to the gradient. Those $G$ rollouts were compute spent for nothing. If the policy solves a task with probability $p$, and rollouts are independent, the chance that all $G$ fail is $(1-p)^G$. The thread says "66% of groups at a 5% solve rate, G=8". **Reasoned:** $0.95^8 = 0.663$. The paper's other figures check too: $0.99^8 = 0.923$ (it says 92%) and $0.95^{16} = 0.440$ (it says 44%). Two details the thread does not carry. First, the paper trains with $G = 24$, not 8. At $p = 0.05$ that gives $0.95^{24} = 0.292$, so 29% of groups would be dead. **Reasoned.** But on Tev-Gym itself the untrained Qwen3.8-27B fully solves 3.2% of environments in single-rollout probes (**reported**), and at $p = 0.03$ the figure is $0.97^{24} = 0.481$. **Reasoned.** About half the batch, at the start of training, when signal matters most. Second, a group with exactly one success is barely better. The one pass gets a positive advantage and the other $G-1$ failures all get the *same* negative one. A rollout that built nine-tenths of the answer and one that never opened the input folder are indistinguishable to the gradient. Mid-horizon pipelines discard all-fail groups and resample, and so does this paper's training. On long tasks that means many draws per usable batch. ## Build the answer first The usual way to make an agent task is prompt-first: ask a model for a task, an input, a checker and a solution, and hope the four describe the same problem. The paper calls the failure mode *artifact drift*: each piece looks valid alone, but together they disagree. Tev-Gym inverts the order. Every environment comes from a seeded generator program, written once by a coding agent and reviewed by the authors, that runs these stages with no model in the loop: 1. **World construction.** From a seed and a scale tier, generate the latent world $W$: the complete, correct state of the domain. For the paper's running example, a retail warehouse with 502 customers, 52 products and 6,210 transactions over six months. $W$ is checked against invariants (unique ids, valid joins, finite measures). The same step draws a *defect plan*: exactly which records or fields will be corrupted, and by which declared operator. 2. **Workflow DAG.** Compile the workflow into a directed acyclic graph of stages, built by code from the size of $W$. For the warehouse: `profile-sources → reconcile-facts / normalize-dims → load-facts → aggregate-validate → package-sidecars`. The longest chain of dependent stages is the task's horizon. 3. **Rendering.** Project $W$ onto the files an agent would see: CSV, TSV, JSON, SQLite, safetensors shards, binaries, manifests. Rendering is a pure function of $W$, and the generator asserts it round-trips: parse the rendered files and you get $W$ back exactly. 4. **Controlled corruption.** Apply the defect plan. For the warehouse that is three operators: schema drift, duplicates and nulls, and category drift. The corrupted tree $s_0$ is what the agent receives. The untouched $W$ is the ground truth. 5. **Public contract and hidden suite.** Emit the prompt (inputs, required outputs and schemas, precedence rules, allowed tools) and seal, out of the agent's sight, metamorphic transforms that must not change the grade and plausible-but-wrong stale outputs that must fail.
Why does this give ground truth at every step, and not just at the end? Because nothing in the environment is independent of $W$. The paper writes an environment as $$ E = \bigl(x,\; s_0,\; \operatorname{ver}_W,\; \Phi_W\bigr), $$ the prompt, the input tree, the strict verifier and the progress potential, all functions of the same $W$. Rendering round-trips and every corruption is declared and invertible, so $s_0$ determines $W$: the task is solvable from what the agent sees, and the generator already holds the answer. Since the DAG says what the workspace should look like after each stage, every correct intermediate state can be rendered from $W$ and compared against. That is the "exact answer key for every step." A concrete instance: a duplicate-records operator injects copies of rows that exist in $W$, and the defect plan records which. Removing them restores $W$, and the grader can check row by row whether the agent's fact table did exactly that. A prompt-first pipeline has only the agent-visible files and whatever checker the model wrote. The caveat: "invertible" is something each generator *declares*; the paper does not prove it per operator, and "missing records" is only recoverable if the information survives elsewhere in the files. The check is empirical: a reference solver and an independently written alternative must produce identical output trees on every seed, or the environment is rejected. I cannot check any of this in code. The generators are not released yet: the project page says "Code soon / Data soon," and the thread says they are "in internal approval, out within 7 days." ### The eight gates, and what a gaming probe is An environment is admitted only if it passes eight automatic gates (Appendix A.6, **reported**): reward causality (each reward component must drop when its cause is mutated), exploit resistance, role independence (reference solver, alternative solver, verifier and progress scorer share no code), solver agreement, progress validity, metamorphic invariance, six-seed diversity, and public-contract hygiene (no answer material in the prompt or input tree). The 14 gaming probes are the exploit-resistance gate. Each is a canned cheat applied to the public tree, and each must score exactly 0 under both the strict verifier and the progress potential. The paper names six of the 14: empty output, minimal schema-valid output, copied input, single-seed hard-coding, path traversal and private lookup. The other eight are "and others." If you have read [the MiMo audit](/articles/mimo-env-reward-hacking), where a fix commit sat in `.git` in most of the coding tasks, "private lookup" is the one to want a list of, and it is not in the paper. ### Is 12.4 minutes "6x the longest prior corpus"? **Reported** (Table 1): a median rollout on Tev-Gym takes Gemini 3.6 Flash 12.4 minutes of wall-clock agent time; the longest prior corpus is SWE-Smith at 2.0 minutes. **Reasoned:** $12.4 / 2.0 = 6.2$. The project page names SWE-Smith explicitly. The other corpora sit at 0.5 to 1.9 minutes. The evaluator is shared; the scaffold is not. Prior corpora ran under Harbor's `terminus-2` with 60 turns, Tev-Gym in the paper's single-bash harness with a 300-tool-call budget. The paper argues the prior medians would not move under a larger budget, since 60 turns ends only 1 to 13% of their rollouts. That holds for medians. ## The progress potential Each environment ships a *progress contract*: a set of nodes $V$, each a checkable state of the output workspace, with weights $w_v > 0$ summing to 1 and prerequisite sets $\mathrm{pre}(v)$. A partial scorer $c_v(s) \in [0, 1]$ grades the workspace on node $v$ directly against $W$, using set precision and recall for record-valued outputs and declared tolerances for numbers. Credit is prerequisite-gated. $\tilde{c}_v(s)$ is the lowest partial score among $v$ and all of its direct and indirect prerequisites. The potential is the weighted sum, the paper's Eq. 3: $$ \Phi(s) = \sum_{v \in V} w_v \, \tilde{c}_v(s) \in [0, 1] $$ The gating is what makes this hard to farm. A correct aggregate computed on a wrong fact table earns nothing, because `aggregates` inherits the score of the `facts` node beneath it. For the warehouse the nine nodes and their weights, read off Figure 3, are facts 0.18, dimensions 0.15, schema 0.12, summary 0.12, normalization, relations, aggregates and quality_report at 0.10 each, and readable at 0.03. **Reasoned:** they sum to 1.00. Three of the gates exist to make $\Phi$ safe to train on: it is *anchored* (the untouched tree and every gaming probe score 0; $\Phi = 1$ exactly when the strict verifier accepts), *monotone* (rises along the reference trajectory, falls on regression), and *independent* (its scorer shares no code with the verifier or the solvers). No step labels, no trained reward model, no LLM judge. The paper's own AI-use statement is explicit: "No generative model grades any task or produces any training reward." ## PPRS, and whether it is policy-invariant Let $s_0, \dots, s_T$ be the workspace after each turn, with $\Phi(s_0) = 0$. Following potential-based shaping, the per-step reward is the change in potential, $r_t = \Phi(s_t) - \Phi(s_{t-1})$, and the rollout reward is the paper's Eq. 4: $$ R(\tau) = \mathbb{1}[\tau \text{ passes}] + \lambda \sum_{t=1}^{T} r_t = \mathbb{1}[\tau \text{ passes}] + \lambda \, \Phi(s_T) $$ with $\lambda = 0.25$ in every run, and not tuned. The reference this invites is Ng, Harada and Russell (1999). Their result: add $F = \gamma \Phi(s') - \Phi(s)$ to the reward and the optimal policy is unchanged, because along any trajectory the shaping terms telescope to a constant. That constancy needs the terminal potential pinned (to 0, or to the same value everywhere). PPRS keeps $\lambda \Phi(s_T)$, which differs between trajectories. So **PPRS is not policy-invariant in Ng's sense**, and the paper says so in as many words (Appendix B): "PPRS (Eq. (4)) is not policy-invariant shaping in the sense of Ng et al. (1999)." That is deliberate: zero the terminal potential and all-fail groups starve again. What PPRS preserves is the ranking of individual trajectories. A pass earns $1 + \lambda = 1.25$ and a failure at most $\lambda = 0.25$, so no single rollout is rewarded for trading completion for partial progress. What it does not preserve is the ranking of stochastic policies in expectation. The paper bounds it: the shaped objective can prefer a policy with pass rate $p_1$ over one with $p_2 > p_1$ only if $$ p_2 - p_1 < \frac{\lambda (1 - p_1)}{1 + \lambda} \le \frac{\lambda}{1 + \lambda}, $$ which is 0.2 at $\lambda = 0.25$, and only when the worse policy's failures end nearly complete while the better one's end near empty. **Reasoned:** $0.25 / 1.25 = 0.2$, so the bound is right. Since every number reported is a strict-verifier pass rate, a distortion would show up in the tables; the sparse-reward ablation is the control. Because $\Phi$ grades the workspace alone, two routes to the same files earn the same reward, and a perfect workspace that runs out of context before `submit` earns $\lambda \Phi(s_T) = 0.25$, not 1.25. ### One group, scored both ways The widget below draws one GRPO group and scores it twice. The plateaus a failing rollout can stop on are the ones the paper measured on the warehouse environment: 0.05, 0.26, 0.69, 0.78, 0.90 and 1. Which plateau a rollout reaches is a toy distribution that moves deeper as $p$ rises; the paper gives none. At $p = 5\%$ and $G = 8$, most draws produce an all-fail group: the terminal column is flat zero while the PPRS column ranks the rollouts. Push $\lambda$ to 0 and PPRS collapses back to the terminal reward. Under PPRS a group goes silent only when every rollout stops on the same plateau. The real version: two rollouts submit an incomplete warehouse, five run out of the 65k-token context, and one builds the complete warehouse and never submits.
Across 24 such groups (12 seeds, base and trained 27B, 192 rollouts) the paper **reports** terminal reward 0 on all 192, so all 24 groups have zero advantage, while every group has non-zero variance under $\Phi$. ## What training actually used The thread motivates PPRS with GRPO. The runs use DPPO, a group-relative objective with centred advantages and a total-variation bound of 0.1 on the policy ratio, no KL term (**reported**, Table 11). Per step: 8 tasks, $G = 24$ rollouts each, 192 rollouts, a 65,536-token response budget with 16,384 per turn, at most 64 tool calls, one bash tool in a `python:3.13-slim` container. Credit is per turn: turn $t$ gets $\lambda r_t$, the last turn also gets the terminal term, and the advantage is the return-to-go with $\gamma = 1$ minus the group mean at the same turn index. Since all-fail groups are already refilled by active sampling, the honest statement of what PPRS buys is *cost and feasibility*, and the ablations (Table 5, on Qwen3.5-9B, **reported**) say so: | Reward | GPU-h / step | ALE-CM S@1 | ALE-CM P@5 | |---|---|---|---| | Sparse (terminal only) | 16.7 | 0.25 | 0.06 | | PPRS, pooled normalisation | 10.9 | 0.28 | 0.06 | | Direct additive $\Phi(s_t)$, $\gamma = 0$ | 9.7 | 0.29 | 0.06 | | PPRS, per-turn normalisation (Tev-9B) | 9.0 | 0.29 | 0.11 | **Reasoned:** sparse reward costs $16.7 / 9.0 = 1.86\times$ the GPU-hours per step. At 27B the paper says the sparse run "fails to complete a single optimizer step": active sampling never finds eight groups with any variance. That is the strongest evidence in the paper for the reward, and it is a sentence, not a table. The data ablations make the case for the latent world. A *public-view verifier* (an LLM writes the checker from the agent's view, everything else kept) discards 88% of sampled groups and lands at S@1 0.24; *prompt-first* generation lands at 0.23; the base model is 0.25 (**reported**). Neither beats doing nothing. ## The numbers, checked The 27B base is Qwen3.8-27B, the 9B base is Qwen3.5-9B, and Tev-320B is GLM-5.3-Flash (the model in [this site's GLM-5.3-Flash walkthrough](/articles/glm-5-3-flash)). **Measured** from the Hugging Face metadata: `gordonhu/Tev-27B` holds 26,895,998,464 parameters and declares `base_model: Qwen/Qwen3.8-27B`; `gordonhu/Tev-320B` holds 321,323,031,390, with 288 routed experts and 8 per token in its `config.json`, and declares `zai-org/GLM-5.3-Flash`. Both are `cc-by-nc-4.0`. The same base was post-trained the same week for knowledge work, with an LLM judge and rubrics instead of verifiable rewards: [Ultramarine 1 Flash](/articles/ultramarine-1-flash). Every relative gain in the thread recomputes from the paper's tables (**reasoned** on **reported** inputs): | Claim | Table value | Recomputed | |---|---|---| | ALE-CM Pass@5, Tev-27B, +65% | 0.17 → 0.28 | +64.7% | | TB 3.0-CM Pass@1, Tev-27B, +56% | 0.09 → 0.14 | +55.6% | | TB 4.0-CM Pass@1, Tev-27B, 2.5x | 0.04 → 0.10 | 2.5x | | TB 4.0-CM Pass@1, Tev-320B, +28% | 0.29 → 0.37 | +27.6% | | TB 3.0-CM Pass@1, Tev-320B, +35% | 0.20 → 0.27 | +35.0% | | TB 2.1 Pass@1, Tev-9B, +28% | 0.178 → 0.227 | +27.5% | | "0.02 behind GPT 5.6 sol" | 0.39 vs 0.37 | 0.02 | Now the denominators. ALE-CM, the computing-and-math split of Agents' Last Exam, is **18 tasks** (8 Math & Science, 5 ML, 5 Coding & System), each run five times. Pass@5 is the fraction of tasks solved at least once in five tries, so it moves in steps of 1/18. **Reasoned:** $0.17 \times 18 = 3.06$ and $0.28 \times 18 = 5.04$. The +65% is **three tasks becoming five**. "#1 on Math & Science: Pass@5 0.50 vs 0.38" is **four of eight against three of eight**. Every one of the ten frontier models scores exactly 0.38 there. One task is the whole lead. On the same table, Tev-27B's Pass@5 of 0.28 ties GPT-6 Astra, GPT-5.6 (sol) and Claude Opus 5, and sits below Claude Fable 5 and Claude Sonnet 5 at 0.33. On mean score S@1 it is 0.40 against 0.50 to 0.62 for the frontier rows; the "0.40 vs 0.32" in the thread is against the open Qwen3.8-2.4T-A95B, which runs in the same harness. On ML, Pass@1 and Pass@5 are 0.00 before and after training. The gains are concentrated in Math & Science. The Terminal-Bench subsets are also small. TB 3.0-CM is 22 tasks and TB 4.0-CM is 18, each with five trials. They were chosen by asking Claude whether each TB 3.0 task belongs to the computing-and-math domain "even if only slightly," then keeping those a CPU-only harness can run. The paper reports Pearson $r = 0.96$ and $0.97$ between subset and full-benchmark Pass@1 across leaderboard models, which is a reasonable defence of the subset as a ranking device. It does not remove the sampling noise: **reasoned**, Tev-27B's TB 4.0 move from 0.04 to 0.10 is roughly 4 to 9 successful trials out of 90, and because both values are rounded to two places, the true ratio could sit anywhere from about 2.1x to 3.0x.
Whose numbers are these? On TB 3.0 and 4.0, frontier rows are the official leaderboard's; GLM 5.2 on TB 4.0 and both Gemini Flash models were run by the authors (the asterisks). On ALE-CM the frontier models ran in their native harnesses (Codex, Claude Code, Gemini CLI), every Qwen model in the training harness, `Vanillux2`. On TB 2.1 (89 tasks, leaderboard frontier rows), Tev-27B's 0.794 against Claude Opus 4.8's 0.789 is a 0.030 gain over a base already at 0.764. In-domain versus zero-shot matters here. ALE-CM is in-domain by design: the generators "target ALE-CM," built around its seven computing-and-math subdomains, and the ethics statement says "part of the training mix is built against the interfaces of ALE-CM." Decontamination is lexical: no environment shares an 8-gram or a 13-gram with any evaluation task, and nearest-neighbour prompt Jaccard to ALE-CM is 0.22 median against 0.28 for ALE-CM tasks compared with each other (**reported**). That rules out copying, not targeting. The Terminal-Bench results are the zero-shot ones, and they are the more interesting evidence that the training generalises. Finally, the 320B result is mostly reliability. Tev-320B's Pass@5 on TB 4.0-CM is 0.56 before and after; on TB 3.0-CM it moves 0.50 to 0.55. The paper says it plainly: training "mainly makes the model solve the tasks it can already reach more reliably rather than unlocking new ones."
## What is released The weights are out, both under CC BY-NC 4.0, so research use only: The generators, the 1,170 environments, the progress scorers and the training code are not. Until they are, the latent-world construction, the gates and the 14 probes are descriptions I can check for internal consistency but not run. When the code lands, the first thing worth reading is the list of gaming probes, and the second is whether every corruption operator is really invertible from the rendered files alone. ## Where it fits Most environment work on this site is about scale and containment: [Prime Intellect's 365,000 environments behind one contract](/articles/scaling-agentic-rl), [Karotte keeping the answer key in the sandbox](/articles/karotte-rl-environments), [Skill2Env](/articles/skill2env), [PortSimEnv grading against a proven optimum](/articles/portsimenv). Tev-Gym works upstream of that. Generate the answer before the question, and the answer key stops being something you guard at the end and becomes a grader you consult every turn. The cost is authoring: one reviewed generator per workflow. Once it exists, an environment costs a median 30 s of CPU and no model calls (**reported**). The transferable lesson is narrower than the thread's headline. A terminal reward on a task your policy solves 3% of the time leaves about half your groups silent at $G = 24$. If you can say what a correct workspace looks like halfway through, $\lambda \Phi(s_T)$ gets that half back, at the price of an objective that is no longer exactly the one you care about. For why GRPO's estimator behaves this way at the token level, see [Qwen's first-order RL formulation](/articles/first-order-rl).