Ultramarine 1 Flash: RL on a board memo, graded by 31 sentences

md json mcp

2026-10-14 · 18 min · reinforcement-learning · post-training · agents · rl-environments · glm · benchmarks · synthetic-data · lora

Why read this

Notabletop 60%

Scores the blog's real 31-criterion rubric in a widget, reads the open rubric and Feather code it trains with, and maps the Fable 5 claim row by row.

Original analysis
A new technique
Concrete numbers to act on

Training & RLAPI onlyPractitioner model

How this was scored

Is it new?: 2 of 3: A real new idea, method or capability
Can I trust it?: 2 of 3: Measures key facts from files, code or configs
Can I run it?: 1 of 3: API-only, gated or restrictive licence
Will I understand it?: 2 of 3: Mechanism from first principles with figures
Can I act on it?: 2 of 3: A concrete recipe, numbers or comparison
Will it last?: 1 of 3: Relevant for months
Does it affect many?: 1 of 3: A specialist community
Only here?: 2 of 3: A teardown or measurement few others did

Score 60 of 100, ranked 325 of 577 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored

Paper Instruments released Ultramarine 1 Flash today. It is GLM-5.3-Flash after a long run of reinforcement learning on knowledge work: diligence memos, board pre-reads, credit-agreement reviews, warehouse tickets. The launch post on X makes four claims. It says the model performs similarly to Claude Fable 5. It says it is 60x cheaper. It says it was "post-trained by a team of 5 for ~$300k all-in". And it calls the model open-source, with weights "releasing in 2 weeks".

The interesting part is the training signal. Coding RL has unit tests. A board memo has none. Ultramarine's reward is an LLM judge reading a weighted list of propositions, and most of the blog's engineering goes into keeping that signal honest. So I start there, with the blog's worked example and the open-source grader it uses. The claims come after.

Labels: measured means I computed it from a file, the released code or the blog's own data. Reported is Paper Instruments' number, not re-run. Reasoned is my arithmetic on the other two.


Base	`zai-org/GLM-5.3-Flash`, 320B total and 18B active, MIT (site teardown); the same base became Tev-320B this week, with verifiable per-step rewards instead of a judge
Method	Agentic RL, rubric rewards from an LLM judge, CISPO loss, LoRA rank 64 (reported)
Curriculum	4 domains, 19 practice areas, 88 environment types, all synthetic (reported; 88 measured from Table 1)
Harness	Feather: `read`, `edit`, `write`, `grep`, `glob`, `bash`, `todo_write` (reported, and in the released code)
Context, turns	131,072 tokens, up to 250 agent turns (reported)
Out today	API, the Feather harness and a reference training recipe (MIT), the `rubric` grader (MIT)
Not out	Weights, the task data, the snapshot builder, training length and compute

Why a memo is hard to reward

Policy-gradient RL needs one number per rollout. The model makes several attempts at a task, each gets a reward, and the update pushes up the attempts that beat their siblings. That number has to rank attempts the way an expert would.

For code the number is cheap and almost incorruptible: run the tests. For a board pre-read the only available grader is another model. That opens three failure modes.

The judge has tastes the task does not. Length, confident tone, headings and tables tend to read as quality. A policy optimised against a judge finds those tastes faster than it finds the substance.
The criterion is too tight. Write the reward from one golden answer and every valid alternative scores as wrong. The policy learns the phrasing of the reference rather than the job.
The criterion is too loose. "Provides a useful forecast update for the board" rewards anything that sounds like one, including the plausible-sounding wrong answer.

The blog's first figure draws the last two as boundaries over six candidate answers.

Six candidate answers to a board-forecast task. Answers A, B and C are complete: each says the board can approve the v7 forecast now and that counsel must still clear external wording. Answers D, E and F each give only one piece. An overfit criterion, which demands the phrase sole forecast basis and a fixed ordering, rejects valid answers B and C. A vague criterion, provides a useful forecast update, also accepts the incomplete D, E and F. Criterion 13 of the example task, which states the substantive separation between what the board can approve now and what needs counsel clearance, accepts exactly A, B and C. — An overfit criterion rejects valid answers; a vague one accepts incomplete ones. The criterion they train with is written to match the boundary (Paper Instruments, Ultramarine blog, criterion-boundary figure).

Their rule is that a criterion tests "one coherent proposition", and names the entities, dates and thresholds needed to check it. Independent accomplishments get separate criteria so that partial work earns partial credit. A judge then marks each one met or unmet. That is the Rubrics as Rewards idea, pushed into long agentic tasks.

How 31 sentences become one reward

The blog publishes a complete training task. An agent gets a deal-room filesystem of 31 files (measured, from the listed tree), email and calendar MCP servers, and an instruction to update the forecast section of a board pre-read .docx in place. The grader is 31 weighted criteria in six groups. I summed them: positive weights total +247, negative weights total −210 (measured). Eight of the 31 are negative: things a careless memo does, such as quoting the stale 11.7x multiple or telling management it may use the new numbers on the roadshow before counsel clears them. That one costs 50.

The grader is PerCriterionOneShotGrader from Paper Instruments' open rubric package, with a custom prompt they do not publish. I read the package. It asks the judge for a MET or UNMET verdict on every criterion in one call, then aggregates:

\text{raw} = \sum_{i} \mathbb{1}[\text{MET}_i]\, w_i, \qquad r = \operatorname{clip}\!\left(\frac{\text{raw}}{\sum_i \max(0, w_i)},\, 0,\, 1\right)

A negative criterion is phrased as the error, so MET means "the error is present" and its negative weight subtracts. The default prompt spends most of its length on that inversion, with examples such as "Avoid doing X because…" counting as not committing X.

One task, 31 criteria, one rewardweights from the blog’s worked example

raw (sum of MET)

164 / 247

reward, clipped 0-1

0.664

group mean (8)

0.407

advantage

+0.257

Factual accuracy

Analytical soundness

Completeness

Source and context fidelity

Communication

Instruction following

Load “Right numbers, one track”: every figure is correct, and the memo still loses 25 points by treating the board’s approval as counsel’s clearance, and 18 more for not separating them. The judge grades propositions, not prose. Then load “Stale v5 basis”: the raw sum goes negative and the reward floors at zero, the same as an empty file. Below the floor, a bad answer and a worse one are indistinguishable to the optimiser.

Load the presets and three properties fall out.

Correct numbers are not enough. The "right numbers, one track" memo gets every figure right and still scores 164 of 247, a reward of 0.664 (measured from the weights). It merges the board's approval with counsel's clearance, which costs 25, and so misses the 18-point criterion that rewards separating them. The reward is shaped around the judgement, not the arithmetic.
The floor erases information. The stale-basis memo has a raw score of −71, which clips to zero (measured). So does an empty file. Below the floor, the optimiser cannot tell a bad answer from a worse one. That is a defensible choice, because clipping stops one catastrophic criterion from dominating a group's variance. It is still information thrown away.
Criteria overlap. Criterion 14 ("v7 is the sole basis for Board, fairness, and transaction-valuation work", +10) and criterion 24 ("Uses Management Forecast v7 … as the forecast basis", +18) reward close to the same proposition. Criteria 11 and 16 both penalise presenting the stale v5 case. Getting the basis right is worth 28 points and getting it wrong costs 50 (reasoned, from the published rubric). The blog's own principle says independent accomplishments get separate criteria. These two pairs are not independent, so the rubric weights the forecast basis more heavily than the group totals suggest.

One quirk in the package matters for training. If the judge's JSON leaves a criterion out, the aggregator records it as UNMET (measured, per_criterion_one_shot_grader.py). For a positive criterion, that is a lost point. For a negative one, it is a penalty that silently never applies. A judge that drops items under a long context lets errors through. I cannot tell whether Paper Instruments' custom prompt or their retry logic guards against this. The released code does not show it.

The judge sees the deliverable "parsed and rendered as text". The post names neither the judge used in training nor its agreement with human experts; it points to a separate post, "Notes on Choosing a Rubric Judge", for the method.

Where 88 kinds of tasks come from

The blog rejects production traces, because a reward written from one user's request bakes in that user's unstated preferences ("top 10 medtech companies" by revenue, or by market cap?), and expert-written tasks, as expensive and narrow. It chose fully synthetic generation.

The curriculum spans finance (34 environment types), legal (15), consulting (28) and operations (11). That is 88, and the table lists exactly 88 named environments (measured from Table 1). Each practice area is a generation pipeline that samples a taxonomy: deliverable format, sources of complexity, stakeholders, conflicting sources, the stage of the deal. The blog puts each area's configuration space at 10⁷ to 10¹⁰ combinations (reported). An environment is a filesystem of real SEC filings and synthetic contracts and emails, plus live systems the agent changes (CRM, ERP, warehouse management), and a private ground-truth record only the task-proposer agents see.

The reward-hacking defence is mostly in how the tasks are made. Producing one task "involves several dozen independent agent loops and multiple rounds of review" (reported). Those rounds check grounding, realism, difficulty and whether the rubric matches the assignment. Classifiers screen for answer leakage and reward hacking, "iteratively added or refined in response to failures observed during training". A last filter drops tasks by base-model performance. Tasks the base model always solves or never solves give every rollout in a group the same reward, so their advantage is zero and they teach nothing.

The blog does not say how many tasks were generated, how many survived the filters, or what any of the hacking classifiers caught. Those are the numbers that would say whether the defence worked.

The harness is part of the reward

Ultramarine trains inside its own harness, Feather. Paper Instruments surveyed twelve coding harnesses (Claude Code, Codex, OpenCode, OpenHands, Qwen Code and others) and built one on their shared conventions instead of training on several. Three choices bear directly on reward integrity.

The agent loop runs outside the sandbox. Commands execute in an isolated E2B sandbox, but the loop, the credentials and the grader do not. Grading material arrives only after the rollout. In the blog's words, "nothing the model does in the sandbox can alter its own harness, credentials, or grader". This is the structural answer to the failures documented in MiMo's environments, where the answer was sitting in .git.
Environment failures are excluded, not zeroed. A malformed tool call is the policy's fault, so the error goes back to the model and costs a penalty. A lost sandbox says nothing about the policy, so the attempt is dropped and restarted. Scoring it as zero would add noise to every group it lands in.
The training prompt says almost nothing. The training system prompt is three sentences. I counted 42 words in system_prompt.md in the released code (measured). Feather's own production prompt, in the same repo, is 3,944 words (measured). The blog argues that prescriptive instructions raise test-time scores but narrow exploration, so production instructions came in "only … in final stages of training".

Feather's edit tools also fingerprint each file on read and refuse an edit if it changed since, and Office files are edited through Paper Instruments' MIT forks paper-docx, paper-pptx and paper-xlsx, which refuse operations they cannot do safely. A silently corrupted workbook would otherwise reach the judge as a quality failure.

The optimiser

The loss is a modified CISPO, from MiniMax-M1. For rollouts $i$ in a group $g$ on one task:

\mathcal{L}(\theta) = -\frac{1}{N}\sum_{i,t} m_{i,t}\,\operatorname{sg}\!\left[\min\!\left(\rho_{i,t}, 5\right)\right] A_i \log\pi_\theta(a_{i,t}\mid h_{i,t}), \qquad A_i = \tilde r_i - \frac{1}{|g|}\sum_{j\in g}\tilde r_j

Here $\rho_{i,t}$ is the probability ratio between the current policy and the policy that sampled the rollout. $m_{i,t}$ masks to model-generated tokens and $N$ counts them. $\operatorname{sg}$ is stop-gradient. Unlike PPO's clip, CISPO caps the importance weight and stops its gradient, so a token whose ratio runs high still receives a gradient. It just gets less weight. The released recipe defaults to the same cap: clip_high_threshold: 5.0 (measured).

The other pieces, all reported:

Mean-centred advantage, no standard-deviation division. Following Dr. GRPO, so a group whose rewards barely differ does not have its noise blown up to unit scale.
Zero-advantage filtering, from DAPO: groups where every rollout scored the same are dropped.
Rollout Routing Replay (R3). GLM-5.3-Flash is a mixture-of-experts model with 288 routed experts. If the inference engine and the trainer route a token to different experts, the importance ratio measures routing disagreement, not policy change. R3 replays the inference-time routes in training.
Asynchronous, PipelineRL-style updates. Finished groups stream into a buffer, and a full batch triggers exactly one optimiser step. There are no mini-steps and no separate "old policy" forward pass. Inference weights refresh mid-rollout. Their earlier synchronous setup with three policies is gone.
LoRA rank 64, AdamW, learning rate 5×10⁻⁵, batch 64×8, temperature 1, no top-p or top-k. A batch of 64 tasks with 8 rollouts each is 512 trajectories per step (reasoned).
Shaping. A capped penalty per invalid tool call, a turn factor that fades positive reward towards zero as turns approach the cap, and token penalties beyond an allowance. The token penalties apply only when the group's mean reward is positive.

What the table does not give is the length of training: no step count, no GPU type, no GPU-hours. "Through prolonged training" in the thread is all there is. So the "~$300k all-in" from the tweet, which is not in the blog, cannot be checked against anything (reasoned).

Checking "similar performance to Fable 5"

The blog's Table 8 has six knowledge-work rows and two coding rows "for reference". Every model ran at its highest reasoning setting.

Six horizontal bar charts. JobBench: Ultramarine 1 Flash 62.9, Gemini 3.8 Flash 60.6, DeepSeek V4.1 Flash 60.4, Claude Fable 5 57.4, GLM-5.3 Flash 56.3, Kimi K3 54.3, GPT-5.6 Sol 45.4. PI Knowledge Work (internal): Ultramarine 59.3, Fable 5 57.8, GPT-5.6 Sol 55.1, GLM-5.3 Flash 52.0, Kimi K3 51.2, Gemini 3.8 Flash 49.4, DeepSeek V4.1 Flash 47.9. FrontierFinance: Ultramarine 53.3, GLM-5.3 Flash 50.7, Gemini 50.0, Fable 5 49.2. AutomationBench: DeepSeek V4.1 Flash 51.3, Ultramarine 50.2, GLM-5.3 Flash 48.8, Fable 5 46.2. DiligenceBench: Gemini 53.1, Fable 5 52.9, Ultramarine 50.6, GLM-5.3 Flash 48.2. APEX-Agents 1.1: Gemini 64.3, Fable 5 63.6, Ultramarine 57.0, GLM-5.3 Flash 52.8. — The six knowledge-work benchmarks. Ultramarine leads four, but two of the six, PI Knowledge Work and DiligenceBench, are Paper Instruments' own (Paper Instruments, Ultramarine blog, Table 8 charts).

Ultramarine 1 Flash minus Claude Fable 5ahead on 4 of 6 knowledge-work rows · mean +1.0

·own = a benchmark Paper Instruments built and graded · ·code = reported “for reference”, not a target of the training · scores from Table 8 of the Ultramarine blog

Against Fable 5 the knowledge-work rows split four to two, and the mean gap is about one point. The two losses are not small: APEX-Agents by 6.6, DiligenceBench by 2.3. Hide PI’s own benchmarks and the tally is three of four, still one large loss. Switch to the base model and the picture changes shape: six of six knowledge-work rows go up, by 4.1 on average, and both coding rows go down by one to two points. That is what targeted post-training looks like.

Against Claude Fable 5, Ultramarine is ahead on four rows: JobBench (+5.5), PI Knowledge Work (+1.5), FrontierFinance (+4.1) and AutomationBench (+4.0). It is behind on four: DiligenceBench (−2.3), APEX-Agents 1.1 (−6.6), Terminal-Bench 2.1 (−1.1) and DeepSWE (−8.4). All measured from Table 8. Across the six knowledge-work rows the mean gap is +1.0 (reasoned). "Similar" is a fair word for that. "Fable-class", as the product page puts it, depends on which rows you read.

Three caveats sit in the table's footnotes.

Two of the six knowledge-work rows are Paper Instruments' own benchmarks. PI Knowledge Work is internal and judged by GPT-5.6 Terra. DiligenceBench is "our own benchmark", and Paper Instruments ran every model on it. The tweet's headline comparison, 59.3 against 57.8, is on the internal suite. Ultramarine was trained on the same four domains, by the same people, with the same kind of rubric grading. On the other four knowledge-work rows it leads Fable on three, by a mean of +1.75 (reasoned).
The comparators come from mixed sources. On JobBench, FrontierFinance and APEX-Agents, Fable's score comes from a public leaderboard. On AutomationBench it is "the public-set scores reported by their developers". Ultramarine and some comparators were run by Paper Instruments in the official harnesses. Same benchmark, different operators.
Fable's largest lead is bigger than any of Ultramarine's. APEX-Agents 1.1, run in Mercor's leaderboard harness and scored Pass@1 over three runs, goes to Fable by 6.6 points. Ultramarine's largest lead is 5.5, on JobBench.

Against its own base, the story is cleaner. Ultramarine improves on GLM-5.3-Flash on all six knowledge-work rows, by 4.1 points on average (reported; my recomputation from Table 8 gives 4.08). PI Knowledge Work gains the most (+7.3), AutomationBench the least (+1.4). Both coding rows fall slightly: Terminal-Bench 2.1 from 84.3 to 83.2, DeepSWE from 63.4 to 61.5 (measured from Table 8). The base-model figures for Terminal-Bench, DeepSWE and AutomationBench (84.3, 63.4, 48.8) match the numbers in the GLM-5.3-Flash card as this site's teardown recorded them (measured). Targeted RL moved the domains it targeted and cost a little elsewhere. That is the honest result, and the blog's table shows it rather than hiding it.

The blog also reports 6.5% fewer output tokens than the base across the six benchmarks: 30.5M against 32.7M. Those two rounded totals give 6.7% (reasoned), which is consistent within rounding.

Checking "60x cheaper"

The cost claim comes from one chart. It plots PI Knowledge Work score against cost per task at each developer's official API price as of October 2026.

Scatter plot of PI Knowledge Work score against cost per task on a log axis from one cent to ten dollars. Ultramarine 1 Flash at max reasoning is highlighted at 59.3 and 13 cents. A frontier line runs from GPT-6 Luna low at under a cent through GPT-6 Luna max, Ultramarine, GPT-6 Astra low and medium, GPT-6 Sol max, Claude Opus 5.5 high, GPT-6 Astra max and up to Claude Fable 5.1 max at 65.5 near ten dollars. Claude Fable 5 max sits at 57.8 near eight dollars, inside the shaded region of runs that cost more and score less than Ultramarine. — PI Knowledge Work, score against cost per task. The shaded region is every run that costs more and scores less than Ultramarine; this is Paper Instruments' internal suite (Paper Instruments, Ultramarine blog, cost-quality chart).

PI Knowledge Work · score vs cost per task (log)27 of 39 other runs cost more and score lower

selected

Claude Fable 5 (max)

cost vs Ultramarine

$8.19 / $0.13 = 63.0×

Ultramarine score gap

+1.5 points

● Ultramarine · ● Claude Fable 5 / 5.1 · ● GLM-5.3 Flash, the base · tap any dot · data: the blog’s chart labels

The “60x cheaper” is Fable 5 at max reasoning, $8.19 a task, over Ultramarine’s $0.13: 63×. It is a price ratio, and Ultramarine is priced like its base. The base itself is cheaper still per task, and the closest rival in the cheap corner is a closed model: GPT-6 Luna at max reasoning, 58.3 for $0.11. Select Fable 5.1 to see the run the tweet did not compare against.

$8.19 for Fable 5 at max reasoning divided by $0.13 for Ultramarine is 63x (reasoned). The "60x" is that, rounded down. Four things qualify it.

It is a price ratio, set partly by the seller. The Ultramarine API charges $0.15 per million input tokens, $0.03 cached and $0.50 output, "the same as GLM-5.3 Flash" (reported, product page). The ratio measures an 18B-active open MoE priced like one against a closed frontier model at its most expensive setting. GLM-5.3-Flash itself costs $0.10 a task on the same chart, which is 82x below Fable 5 at max (reasoned).
Ultramarine costs more per task than its base. $0.13 against $0.10 is 30% more (reasoned), at the same token prices. So on this suite Ultramarine used more tokens per task than GLM-5.3-Flash, even though it used 6.5% fewer across all six benchmarks. That is likely the per-task extra work it learned to do. The blog does not break it down.
Max is not Fable 5's best value point. On this suite Fable 5 scores 50.8 at low reasoning ($1.82) and 48.6 at medium ($2.85). So 57.8 at $8.19 is the only Fable 5 setting that outscores the base model. And Claude Fable 5.1 at max scores 65.5 for $8.64, 6.2 points above Ultramarine (reported, chart labels).
The nearest rival is cheap and closed. GPT-6 Luna at max reasoning scores 58.3 for $0.11 (reported), within a point of Ultramarine and a little cheaper.

The Pareto claim does hold on the published points. Of the 39 other runs, 27 cost more and score lower (reported, and measured from the chart labels). No run is both cheaper and better (measured). Ultramarine is on the frontier of this chart. The chart is Paper Instruments' benchmark, judged by a model they chose. The blog's one external anchor is a correlation of r = 0.81 between PI Knowledge Work and the Artificial Analysis Intelligence Index, over the 24 runs both measured (reported). That supports the suite as a general-capability ranking. It says nothing about whether a model trained on the suite's own domains is advantaged on it.

What "open-source" means today

I checked Hugging Face (api/models?search=ultramarine, and the obvious author names) and found no weights (measured, 2026-10-14). The thread says weights will be linked "within 2 weeks". What is public:

paper-instruments/feather@d864944 · snapshot 2026-10-14

tracked files

2,822

license: MIT
branch: main
tests: 130 files
source: 1.3 MB
commit date: 2026-10-13

source by language

Python1.3 MB(224)Shell7.6 kB(7)Dockerfile1.8 kB(4)

by size of tracked source at this commit, file counts in brackets; docs, data and vendored trees excluded

local clone, 2026-10-14 at d864944 — branch, commit, commitDate, fileCount, hasTests, languages, license, licenseFile, shallow, testFileCount

shallow clone: counts describe the pinned tree, not the history

Feather (paper-instruments/feather, MIT, 2,822 tracked files at commit d864944, 2026-10-13; measured). It holds the harness, a Harbor evaluation package and feather-train. That last one covers rollouts and grading on E2B snapshots, plus "a reference Tinker training integration through Verifiers". This is the "training code" the thread promises, and it is already out. It is a reference recipe rather than the run itself. The example config trains Qwen/Qwen3.6-27B at learning rate 1×10⁻⁵ for two steps, where the blog used 5×10⁻⁵ on GLM-5.3-Flash. Its documented length penalty is a simpler form than the blog's equation, with no turn-fade or extreme-length term (measured from grading.md and examples/tinker.json). It also "does not provide a snapshot-baking command", and the task graders are baked into those snapshots.
rubric (MIT). The licence file's copyright line reads "The LLM Data Company", and the README links that organisation's repository (measured). It is a small package: 31 tracked files, three graders.
paper-docx, paper-pptx, paper-xlsx. MIT forks of python-docx, python-pptx and openpyxl (measured from their licence files).

Not public: the weights, the 88 environment pipelines, the tasks and rubrics beyond the one worked example, the custom judge prompt, and the training logs. Without the tasks, the recipe cannot be reproduced, only re-implemented. "Team of 5" and "~$300k" appear only in the tweet. They are reported, unverifiable, and absent from the technical post.

What I take from it

The model is a careful but unremarkable result: a strong open MoE, RL-tuned toward one domain, gains about four points there and gives up one or two on code. The recipe is the contribution. It treats non-verifiable RL as data engineering: the reward is only as good as the boundary each criterion draws, so the effort goes into adversarial criteria, task filtering and walling the grader off from the policy, not into a stronger judge.

What the post does not yet show is evidence that the defences held: reward-versus-human-agreement curves over training, the hacking classifiers' catch rate, any held-out grader. The tweet's claims to check are mostly about the bill and the comparison. The ones I would want the tech report to answer are about the reward.

Other GLM-5.3-Flash derivatives this site has measured: the MLX port and the abliterated FP8 variant. For the verifiable side of the problem, see Scaling agentic RL and MiMo's graded environments; for the road not taken here, multi-harness RL.