MiMo-V2.6 RL environments: 7,780 tasks, five graders, and two judges you bring yourself

mdjsonmcpcopy↗ai

2026-09-26 · 24 min · reinforcement-learning · agents · open-source · llm · benchmarks · evaluation · explainer

Four days ago this site read the MiMo-V2.6 release and called its reinforcement-learning kit the part worth keeping: roughly 7,000 graded task environments with their verifiers, a distilled 9B model to start from, and a GRPO baseline with its metrics printed. It also said I could not find the kit. It is now in three places. The Hugging Face dataset XiaomiMiMo/MiMo-V2.6-RL-oss and the Docker Hub repository xiaomimimo/mimo-v2.6-rl-oss, which holds the task images, were both created on 2026-09-25. The training recipes, and two graders that are not in the dataset, are in a fork of verl at XiaomiMiMo/verl, last pushed on 2026-09-26. That fork was created on 2026-09-21, the day before my search, though I cannot tell what it held then. GitHub's repository search leaves forks out by default, which may be why I missed it.

The repost that spread it said "~7K of the RL environments … across domains like code, cyber, general, music and web dev." That is close. What I want to know is narrower: what exactly is a task here, what grades it, and what would I have to supply to run it. So I pulled every parquet file, all 925 rubric files, one complete knowledge-work environment and the Docker Hub tag list, and I read the five graders line by line. I ran none of Xiaomi's code and pulled no image. Every count below is Measured unless I say Reported.

What an environment is, here

Reinforcement learning for an agent needs three things per task. A prompt: the text the policy sees. An environment: the state it acts on, here a container with a repository, a binary, or a workspace of files. And a verifier: a program that runs after the episode, inspects what the agent left behind, and returns a number, the reward. The policy learns to raise that number, and only that number, so the verifier is the task specification as far as training is concerned.

Every arm in the release trains with GRPO (Group Relative Policy Optimization, from DeepSeekMath). The trainer samples $G$ rollouts of the same prompt, a group, grades each one, and scores each rollout against its siblings rather than against a learned value function:

where $r_i$ is rollout $i$'s reward, $\bar{r}$ the group mean, $\sigma_r$ the group's standard deviation and $A_i$ the advantage that scales rollout $i$'s gradient. The code, cyber and general recipes set norm_adv_by_std_in_grpo: false, which drops the $\sigma_r$ and leaves $A_i = r_i - \bar{r}$; the web-dev and music configs keep verl's default and divide. Two consequences matter for reading the graders. A constant added to every rollout's reward cancels. And a group whose rollouts all earn the same reward has zero advantage everywhere and teaches nothing, which is why the code and cyber arms drop such groups (filter_groups) before the update.

What is in the box

The dataset card is a table and two links. The files say more.

Rows are rows of each domain's parquet, and I checked that every row's task id is distinct. The report's four domains come to 6,780 against its "approximately 7k", and music to 1,000 against "approximately 1k", so the headline counts hold. Every docker_image a row names resolves to a tag through the dataset's image-mapping.jsonl, all 3,764 of them. The image sizes are Docker Hub's compressed per-tag figures; shared base layers count once per tag, so a pull of everything would move less than 7.5 TB; how much less, I did not measure.

"7K environments" needs one qualification. A web-dev row is a prompt, not an environment: all 2,093 of them run in the same image. A music row is a single turn of text with no environment at all. Counted as distinct places an agent can act, the release is 4,688 environments carrying 7,780 tasks.

The dataset itself is 41,292 files and 12.10 GB (the sum of the listed file sizes, at commit 639865fd), licensed Apache-2.0. Almost all of it, 41,284 files and 12.08 GB, is the knowledge-work environments. The other four domains are one parquet each: the code parquet is 13.3 MB because it carries the hidden tests; music is 135 KB.

Every row follows verl's RLHFDataset schema: prompt (one user message), data_source, ability, reward_model and extra_info. The reward_model.ground_truth field is empty in all 7,780 rows. What a verifier needs travels in extra_info.instance_json, or lives in the environment directory, the image or the grader's code.

Five graders

The widget below walks one rollout through each grader and then through a GRPO group. The rules are transcribed from the released code; the seven sibling rewards are illustrative values each grader can actually return. The prose after it carries the same content.

Code: the tests arrive after the agent leaves

Each of the 2,698 code rows is a GitHub-style issue (median 1,603 characters) and a container image with the repository checked out at the task's base commit. The grader is OpenSourceCodeEnvironment in XiaomiMiMo/MiMo-Agent, the harness library the verl fork pulls in as a submodule (a fork of mini-swe-agent v1.9.0, since largely rewritten). Its reward contract is three steps:

1. Reset every path the row's test_patch touches back to the captured base commit.
2. git apply the test patch.
3. Run test_command; the reward is 1.0 if and only if it exits 0.

The trick is where the tests live. They are not in the image. They ship inside test_patch, together with the script that runs them: mimo_test_command.sh appears in all 2,698 patches. In the code's own words, "The patch is applied only at reward time, so the hidden tests and the script that runs them never exist while the agent works." Setup also refuses an image whose git history reaches past the base commit (git rev-list --all --not <base>), because a reachable future commit is the reference fix sitting in git log. And a patch that fails to apply is reported as REWARD_TESTBED_CORRUPTED so the trainer can mask it instead of scoring a wrong answer. Prime Intellect's 365,000-task catalogue withholds grading material the same way; it is becoming the standard move.

Hidden from the agent is not hidden from the internet, though: the patches sit in plain text in code.parquet. Anyone evaluating on these repositories later should assume the tests are in someone's pretraining crawl.

By the file extensions in each test patch (my heuristic, not a field), the set is 1,179 Python tasks, 721 Go, 388 JavaScript, 166 TypeScript, 117 across fourteen other languages, and 127 whose test files carry no extension I mapped. Every row has an 1,800 s verifier limit. The images are the expensive part: median 3.1 GB, largest 13.72 GB, 6.68 TB for the arm.

Cyber: crash in the right function or score zero

The 1,000 cyber rows come from ARVO, a corpus of reproducible OSS-Fuzz vulnerabilities. The prompt is one line naming a sanitizer, a bug type, a function and a file, wrapped in a fixed template. The image carries the source and the sanitizer-instrumented binary; the agent writes a proof-of-concept input and submits it to a small HTTP server that runs as root and executes the binary as a separate unprivileged user with a 60 s limit.

The report explains why this grader exists: "The difficulty is not triggering a crash, since complex C/C++ projects expose dozens of reachable crash paths, but triggering the specific vulnerability described." Differential testing against a fixed binary, as CyberGym does, breaks when the patch is incomplete; an LLM judge is not repeatable. So the reward is a string match. The server walks the sanitizer's stack trace, skips frames outside /src/, fuzzer and sanitizer frames, takes the first project frame, and pays 1.0 only when that function name, the sanitizer and the bug type all equal the target's. Only the last submission counts. The file named in the prompt is parsed but not compared.

What the 1,000 contain, by the description line: 683 AddressSanitizer, 235 MemorySanitizer and 82 UndefinedBehaviorSanitizer targets; 388 heap-buffer-overflows and 225 uses of uninitialised memory; 176 projects by top-level source directory, led by nDPI (70), FFmpeg (55) and ImageMagick (52). One thing to know before you split it: there are only 693 distinct task statements. 501 tasks sit in 194 groups that share an identical line, the largest eleven deep, because the same function and bug type recur across commits. Split by statement, not by row, or the validation set leaks.

General: 925 simulated offices and a rubric

This is the richest part of the release and the one the report describes at length.

expand

Each of the 925 environment directories under general/envs/ holds a workspace/ of documents (median 13 files; PDF, CSV, XLSX, DOCX, PPTX), a system/ of SQLite databases with their schemas, a tools/ directory with one Python module per mock enterprise system, and the grading files. The directory names encode an industry and a language: 16 industries from accounting, audit and tax (157) and finance and insurance (135) down to agricultural co-ops (7), 504 environments in English and 421 in Chinese, and a t1 to t5 tag I read as difficulty (111, 307, 338, 137 and 32). Between them they mock 6,130 system instances under 4,343 distinct names, 3 to 18 per environment with a median of 6: workday_hcm, servicenow_irm and yonyou_nc_finance are the most common. An environment is 23 to 154 files, median 7.3 MB, the largest 450 MB.

manifest.json says how one runs. The pod has two containers. The agent's container gets the workspace at /work/workspace. A sidecar gets the databases and the tool modules, starts one MCP server per module on ports 39101 upward, and the agent reaches them over MCP. After the episode run_verify.py takes the agent's last message as answer.md, snapshots every database read-only, and calls the environment's verify.py.

The grader is a rubric in verifier_meta.json. Across the 925 files there are 5,125 items, 2 to 10 per environment (median 5). 4,437 of them (86.6%) are LLM-judged; 688 are Python rule checks against the databases or deliverable files. 695 environments have no rule item at all. 722 tasks are read-only; 178 require the agent to change a database, and those are where the rule checks earn their keep. The judge sees one item at a time with up to 20,000 characters of extracted evidence per file, and must answer 0 or 1: the prompt says a partially met item is unmet and missing evidence is a 0. The environment's score is $\sum_i w_i s_i / \sum_i w_i$ over the items, with $w_i$ the item's weight and $s_i \in \{0, 1\}$ its verdict.

Two details in that scorer are worth knowing. First, a missing weight counts as 1.0, and 834 of the 925 files set no weights at all. The environment I stepped through in the widget, a Los Angeles property-disposal review, weights its five judged items 0.2, 0.2, 0.25, 0.2 and 0.15 (their sum is 1.0). They ask whether the answer names the current installment deal and excludes the superseded ones, reports the county and city transfer-tax components ($9,141.00 and $37,395.00) with their approved total of $46,536.00, sorts the closing protections into approved and unresolved, states the approved payment split and the all-cash fallback, and concludes that the deal is not ready to close unconditionally. Then the file lists a sanity check that the databases still hold their anchor records, with no weight. It counts 1.0. So a rollout that leaves the databases alone and gets every judged item wrong scores 0.5. Under GRPO that half cancels in $r_i - \bar{r}$ whenever the whole group left the databases alone, so it acts as a heavy penalty on tampering rather than as free reward. 202 environments carry such a sanity item; its median share of the weight is 0.2, and in 9 it is half or more.

Second, the judge is yours. verify.py reads the endpoint, key and model from GA_JUDGE_URL, GA_JUDGE_KEY and GA_JUDGE_MODEL, and the shipped example fills in gpt-4o-mini. The report says that "During RL, a self-hosted MiMo-V2.6-SFT model serves as the grader to support stable scoring." That model is not on the Hub. Whatever judge you pick defines 86.6% of this arm's reward. If the judge fails, the runner writes no reward rather than a zero, "instead of training on a false 0", and the trainer masks the rollout.

The other 64 general rows are terminal tasks in the Terminal-Bench format, one image each. The first asks for a repair to a vendored Snakemake workflow engine; by category there are 18 software, 18 ML and 15 security tasks. Their tests travel base64-encoded in the row, are copied in only after the agent exits, and start with an anti-hack guard that zeroes the reward for a planted conftest.py or sitecustomize.py, or for any protected file whose hash changed. The limits are 1 CPU, 2,048 MB, no internet and a 900 s agent budget.

Web dev: rank the siblings, not the page

The 2,093 web-dev rows are website briefs, median 825 characters, 2,076 of them in English by my character count, all run in one shared image. The dataset carries no verifier for them. The grader is a service in the verl fork, recipes/design/grader_service, and it does not score a page on its own. It collects the screenshots of all eight siblings in a group and asks a vision model, eight times, which pages are clearly better than the rest and which clearly worse, as an ordinary person would see them side by side. The eight orders form a Williams Latin square, so every screenshot appears in every position once; a code comment reports that this flattened a first-position bias "from ±0.28 to ±0.03 votes".

A second judge call scores how much of the brief the page implements, from 0 to 1, and that becomes a deduction. The reward is

with the query-fit deduction $d$ at 0 for a score of 0.9 or more, 0.2 from 0.6, 0.4 from 0.4 and 0.8 from 0.2. Below 0.2 the reward is 0 outright, and so is a page whose scripts throw on load. A net of one vote either way is treated as noise and zeroed. The affine map puts $r$ between 0.0667 and 1; neither the shift nor the ÷3 changes which siblings GRPO pushes up.

The consequence is stated in the code: "A single rollout's reward is therefore meaningless in isolation." It is a training signal, not a metric, which is why there is a separate evaluation mode that scores one page absolutely with a single vision call. And the vision model is yours to supply; the service ships with no default model.

expand

Music: well-formed and human-sounding, not what was asked

The 1,000 music rows ask for a piece in ABC notation, a plain-text music format, under explicit constraints. The first one reads, in Chinese: an English country dance in E major, 129 BPM, 4/4, 56 bars, for violin, flute and cello in three voices. The briefs are 504 Chinese and 496 English; 746 are in 4/4, and they ask for one to six voices.

The scorer is recipes/design/music/scorer. It pulls the ABC out of the answer, renders it to MIDI with abc2midi, and rejects the piece (reward 0) on any Error line, 10 or more bar-length warnings, a blank line inside the score, or a MIDI channel that changes instrument. A surviving piece is scored on 18 features in 6 groups: rhythm (weight 0.22), texture (0.20), tonal (0.16), register (0.16), acoustic (0.14) and structure (0.12). Each feature is scored by whether it falls inside the 10th to 90th percentile band of a human reference corpus. That band score is 85% of the reward; the other 15% is Jensen-Shannon agreement with the corpus's pitch-class, interval and duration histograms.

Nothing in that path reads the brief. The key, tempo, meter, bar count and voice count are in each row's extra_info, and the parquet builder says what they are for: "they are also the material for grading instruction-following, which this scorer does not do." So the music reward pays for a well-formed score that sounds statistically like human music. A piece in D minor at 60 BPM for solo piano is scored by exactly the same rules as one that honours the E-major brief, with no penalty for ignoring it. The report's music result, 45.7 after SFT to 52.5 after RL on an internal benchmark, is measured by something I cannot see; I would not assume it measures adherence either.

The 9B starting point, and the baseline

MiMo-V2.6-Distill-Qwen-9B is Qwen3.5-9B after supervised fine-tuning on MiMo-generated trajectories: 77.4B tokens, of which 27.2B carry loss, across code, cyber, general and visual work. There is no music in its SFT mix. It is MIT-licensed and is the only V2.6 checkpoint on the Hub that is not an "-RL" model. The two that are, MiMo-V2.6-Pro-RL and MiMo-V2.6-Flash-RL, are the shipped post-trained models (the 1.02T and 309B MoEs), named for the run that produced them; unlike V2.5, no base or SFT checkpoint of either was released.

The report's Table 6 starts GRPO from the 9B SFT checkpoint, separately per domain, and reports every row going up. Reported, not measured:

Seven of the eleven rows are public benchmarks you can rerun. The four "mini" sets are internal, described as following "the same task distributions as their corresponding training sets", and none of them is in the release. Neither are the RL-trained 9B checkpoints, so the right-hand column is a target to hit, not a model to download.

What the fork does give you is the reference run, in each arm's launch script:

The code arm also carries the multi-harness experiment: four "mini" harness configurations (mini-mimocode, mini-bash, mini-claude-code, mini-codex) mixed per step by a seeded hash, matching the four training harnesses of the report's Table 7. That is the "composable mini-harness" the report promised. Table 7 is also a reminder of why harness choice matters: the same Qwen3.5-9B scores 36.4 on SWE-bench Verified in one harness and 60.6 in another.

Released, described, missing

Two of those gaps you can close yourself, at a cost: carve a validation split (by statement, for cyber), and pick a judge knowing it is not the one Xiaomi used. The internal eval sets and the RL checkpoints are what would make a baseline comparable, and those you cannot supply. If your 9B run lands at 64 on SWE-bench Verified instead of 66.2, nothing in the release tells you whether the difference is your cluster, your judge or your luck.

What one arm costs to run

Nothing here runs on a workstation. Every arm but music launches one Kubernetes pod per rollout (the code arm's default ceiling is 1,024 concurrent pods), and the defaults assume four to eight nodes of eight GPUs. The images are the other bill: the code arm alone is 6.68 TB of compressed layers across 2,698 tags, and a GRPO step with 64 prompts × 16 rollouts touches up to 64 of them 16 times over. The knowledge-work and web-dev arms add judge calls: one per judged item in general, up to 10 per rollout; in web dev a query-fit call per rollout and 8 pick rounds per group. Music is the cheap one, a CPU-bound scorer and one binary.

The take

This is a real release, and a more careful one than the dataset card suggests. The code grader keeps the tests out of the pod until the episode ends and refuses images with leaky history. The cyber grader replaces a flaky oracle with a deterministic one and says why. Masking a broken testbed instead of scoring it zero appears in three separate graders. Those are the details that decide whether RL learns the task or the grader, and they are all readable.

The limits are in the same code. Two of five arms are only as good as a judge model you choose, and 86.6% of the knowledge-work reward is that judge's opinion. The web-dev reward has no absolute meaning by design. The music reward ignores the brief. The cyber set repeats its statements. And the baseline cannot be checked end to end, because four of its eleven rows and every RL checkpoint stay in-house. Nearly 4,700 environments with readable graders is still, as far as I know, more than any other lab has released alongside a model. It is just not yet a benchmark.

Measured: row counts, field values and every tally above, from the five parquet files, image-mapping.jsonl, all 925 verifier_meta.json files and the full file listing of XiaomiMiMo/MiMo-V2.6-RL-oss at commit 639865fd; image counts and sizes from the Docker Hub API for xiaomimimo/mimo-v2.6-rl-oss; grader logic read from XiaomiMiMo/verl at a2ad9f6 and XiaomiMiMo/MiMo-Agent at 467f0a1, the fork's pinned submodule. Languages for the code set are my heuristic from test-patch file extensions. Reported: Tables 5, 6 and 7, the quotes and both figures, from the MiMo-V2.6 technical report, and the 9B's training mix from its model card. I ran no rollout, pulled no image and executed none of the released code. The trainer underneath is verl, the open version of HybridFlow. Previously: MiMo-V2.6.