# MiMo-V2.6: a trillion parameters, thirty RL steps, and a 20× that is a decoder > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.thesatyajit.com/articles/mimo-v2-6 > date: 2026-09-22 > tags: llm, mixture-of-experts, reinforcement-learning, agents, inference-optimization, explainer This site read [MiMo-V2-Flash](/articles/mimo-v2-flash) nine months ago as a piece of systems thinking: a 309B MoE that kept 256K context cheap by making five layers in six sliding-window with a 128-token window, and buying back reach with one global layer per block. **MiMo-V2.6** keeps that skeleton and changes everything around it. Two models, both natively omnimodal, both open-weighted under MIT, both produced by the same six-day reinforcement-learning run — plus a desktop app, a subscription tier, and a mode called **Pro UltraSpeed** claiming "up to 20× faster output." Three things in this release are worth an engineer's time, and only one of them is a benchmark. Let me take the parameter counts first, because the card's headline number is unusual enough that it is worth confirming it is real. ## 1.02T, confirmed, and shipped in four bits The card above reads **524.12B parameters**, and that is the first thing to explain, because it is wrong by almost exactly a factor of two and for an interesting reason. Hugging Face counts elements per dtype, and this checkpoint stores its expert weights as MXFP4 packed two values to a byte inside `U8` tensors. The Hub sees 500.1B bytes and reports 500.1B parameters; there are 1,000.2B weights in them. Add the FP8 attention projections and the BF16 spine and the real figure is 1.02T. The card says "Sparse MoE, 1.02T total / 42B activated." I summed the safetensors headers of all 130 shards. The weights are stored as **MXFP4** — `save_format: "mxfp4"` in the index metadata, 4-bit values packed two per byte into `U8` tensors with a separate block scale, alongside FP8 attention projections and BF16 for norms, embeddings and the output head. Unpacking that packing: | | Pro | Flash | |---|---:|---:| | stored parameters | **1,024.22B** | **310.76B** | | MoE FFN experts | 1,000.19B | 302.80B | | everything else in the backbone | 21.06B | 5.98B | | multi-token-prediction module | 1.93B | 0.99B | | vision encoder | 739M (679M in the 28 blocks) | 729M | | audio patch encoder + its projector | 268M (101M + 168M) | 261M | | **active per token** | **41.90B** | **15.45B** | | on disk | 566.0 GB | 172.9 GB | 1,024.22B and 41.90B against the card's "1.02T / 42B": exact. The technical report's Table 1 agrees, and Flash's 310B is the same size class as the V2-Flash this site covered — a 309B model grown by one point and given ears. The disk figure is the part worth dwelling on. A trillion BF16 parameters is about 2 TB. Shipping in MXFP4 puts it at **566 GB**, which is the difference between a model you can pull and one you cannot. Xiaomi did not release a BF16 checkpoint and then a quantization; the release *is* the four-bit checkpoint, with the attention projections left at FP8 and every layer's `o_proj` in the `ignored_layers` list, untouched. That is a choice about what open weights mean in 2026, and I think it is the right one.
Pro is 70 layers at hidden 6144, 60 sliding-window and 10 global, window 128, 128 query heads over 8 KV heads, 384 routed experts with top-8 and **no shared expert**, and a 1M-token position budget. It is the V2-Flash recipe at roughly 3× the width and 5× the expert count. ## "Omnimodal" is four encoders and a speech vocabulary The word gets used loosely, so here is what is actually in the checkpoint. Vision is a 28-layer ViT at hidden 1280, 24 sliding-window and 4 global layers, with sink-augmented SWA and alternating row-major/column-major token serialization so information crosses window boundaries — 679M parameters in the blocks. Audio is two pieces: a 6-layer **patch encoder** in the main weights that folds four frames into one token (25 Hz → 6.25 Hz) — 101M of transformer plus a 168M two-layer projector up to the 6144-wide backbone, where the report's 127M figure is the encoder alone — and a separate **308M audio tokenizer** with 24 encoder and 24 decoder layers and 20 RVQ codebooks, shipped as its own 1.87 GB `audio_tokenizer/model.safetensors` in the same repository. The piece people miss is `speech_embeddings.0 … .19` in the main weights: twenty tables of (1280 × 1024), one per RVQ codebook. Those are for *emitting* speech tokens, not just consuming them. With the tokenizer's decoder in the same repo, the release is genuinely round-trip audio, not audio-in-text-out. That is more than most "omni" cards mean. ## The RL run, with its unit costs visible The claim of the release is not the architecture, it is the training. Xiaomi streamed the production run live and published the ledger. Two things about this table are unusual. First, **30 steps**. A full reinforcement-learning production run on a trillion-parameter model, from the mid-trained checkpoint to the shipped one, is thirty optimizer steps — each one consuming 1,568 prompts × 16 rollouts and 2.7–3.7B tokens. Batch size is doing all the work; the gradient count is tiny. That is worth internalizing if your mental model of RL is PPO on a game. Second, **the router is frozen**. The report is explicit: "we freeze the MoE router" to suppress training drift at scale. So the RL run moves expert weights and attention but not the routing distribution, which means the specialization learned in pre-training is held fixed while the experts are re-tuned inside it. [Routing collapse is the classic MoE failure](/articles/mixture-of-experts-from-scratch), and freezing the router is the bluntest possible defence against it. It also means the RL phase cannot discover that a task wants a different expert — a tradeoff the report states and does not quantify. The grading is the interesting research contribution. Binary pass/fail cannot rank two solutions that both pass, so Xiaomi scales the *reward* compute instead: **Groupwise Reward Synthesis** builds per-task rubrics offline from contrasting rollouts, and **Groupwise Advantage Redistribution** ranks passing trajectories online and moves advantage toward the better ones. Judged against the policy's own samples, which is what makes it a self-improvement loop rather than a distillation from a fixed teacher. The reported effect is not just accuracy — it "steers the model towards shorter paths and fewer tokens per task," which is the metric agent operators actually pay for. ## What the released RL environments actually contain This is the part of the release I would keep if I could keep only one thing, because almost nobody ships it. The technical report's Table 5: | Domain | Task family | Tasks | Verifier | |---|---|---:|---| | Code | Software engineering | ~3k | Executable tests | | Cyber | Vulnerability reproduction | ~1k | Rule checks | | General | Knowledge work | ~1k | Rubric-based judging | | Visual | Web development | ~2k | Visual grading | About **7,000 graded task environments, each with its verifier**, plus roughly 1,000 music-generation tasks, plus `MiMo-V2.6-Distill-Qwen-9B` — Qwen3.5-9B supervised-fine-tuned on 77.4B tokens of MiMo-generated trajectories (27.2B of them loss-bearing) — as a common starting point, plus an end-to-end RL framework and a composable mini-harness. And then the thing that makes it a baseline rather than a press release: a reproducible GRPO run on that 9B model, from that SFT checkpoint, on those environments, **with denominators printed**. SWE-bench Verified 61.1 → 66.2 (avg@3). Terminal Bench 2.1 37.1 → 52.8 (avg@1). Toolathlon-Verified 35.2 → 38.0. OfficeQA Pro 19.5 → 24.8. Eleven evaluations, all improved, metric named on every row. A separate multi-harness experiment improves all 21 dataset-harness pairs. **Read what this is.** The ~7k released environments are the *reproduction kit*, not the environment set that trained Pro and Flash — the report describes five synthesis pathways and "thousands of environments" for the production run and does not publish a count for it. What you can run is a 9B-scale study with a real verifier per task and a published baseline to beat. That is still, as far as I can tell, the most complete agentic-RL environment release from a frontier lab. As of writing I cannot find the framework or the environments on GitHub or as a Hugging Face dataset — only the 9B checkpoint. The report says they are open-sourced; the artifacts are not yet locatable. ## The benchmark claim that does not survive MiMo-V2.6-Pro scores **46.32 on the Artificial Analysis Intelligence Index v4.3**, and the announcement's framing — "surpassing Kimi K3 and Qwen3.8 Max to become the strongest open-source model to date" — checks out on AA's board as of today: Qwen3.8 Max (0902) at 45.42, GLM-5.3 (max) at 44.78, Kimi K3 (max) at 43.59.
The index number survives. The sentence next to it does not. "On par with Claude Opus 5 and GPT-5.6 Sol across most agent benchmarks" is checkable against the model card's own evaluation table, which is where both comparison columns come from. Three rows ahead, two level, nine behind, out of fourteen where all three models have a score. The single widest gap is **Terminal-Bench 4.0 at 34.9 against Opus 5's 49.0** — 14.1 points, and that row is not disputable because all three figures are Artificial Analysis' own measurements under the same mini-swe-agent harness at pass@1 averaged over three repeats, copied into the card. The cybersecurity block is worse: ExploitBench 47.9 against 70.0 and 78.5. There is a fair version of the claim and the card almost writes it. Pro is **ahead of GPT-5.6 Sol** on 8 of those 14 rows, and it is ahead of *both* on AutomationBench and Terminal-Bench 2.1 — the two most workflow-shaped benchmarks in the set. "Competitive with the frontier on general-agent workflows, behind it on terminal work and security" is what the table says, and it is a perfectly good thing to be. Which is the other half of the story, and the card does not make it: MiMo-V2.6-Pro bills at \$0.435 in and \$0.87 out per million tokens. Being nine rows behind at that price is a straightforwardly attractive trade. I am not going to put a multiple on it, because the card prints no comparator price and the frontier price lists move — the ratio is for the reader to take from a current one. That is what the Pareto chart on the announcement is for, and it is a stronger argument than parity. ## UltraSpeed is a decoder, and its ceiling is 8× "MiMo-V2.6-Pro-UltraSpeed, delivering up to 20× faster output speed at the same quality." The announcement never says what it is. The repository does. There is a `dflash/` directory in `MiMo-V2.6-Pro-RL`, holding a config, 14 KB of modelling code, a mask embedding, and a 5.54 GB `dflash_draft_model.safetensors`. The config declares `DFlashDraftModel`: 5 layers, hidden 6144, sliding window 1024, `"is_causal": false`, `"block_size": 8`, `"mask_token_id": 151675`, and `target_layer_ids: [0, 15, 31, 47, 69]`. Read `spec_generate` and the mechanism is unambiguous. Each decode step: 1. Fill the next **8** positions of the output with the **mask token**. 2. Embed them with the *target's* embedding table, and run the 5-layer drafter **bidirectionally** over that block — one pass — conditioned on the target's hidden states concatenated from five of its 70 layers and projected down. 3. Push the drafter's output through the *target's* `lm_head` to get 7 drafted tokens. 4. Run the target once over the whole block, sample, and accept the longest matching prefix. That is a **masked-diffusion block draft used as a speculative drafter** — one denoising step, verified exactly. And because the target's attention is causal, position *i*'s logits depend only on the accepted prefix, so every emitted token is a sample from the target's own distribution. "At the same quality" is not marketing here; it is true by construction. What is also true by construction is the ceiling. Eight positions per block, one target forward per block, so **at most 8 tokens per target forward**, and the draft pass is not free. At any acceptance rate, with any draft cost, the shipped configuration cannot produce 20× fewer target forward passes. Slide acceptance to 100% and draft cost to zero and you get exactly 8×. So the 20× is either measured against something other than the same serving setup, or UltraSpeed runs a larger block than the released config — and the price list is the hint. UltraSpeed bills at **\$4.35 in / \$8.70 out per million tokens: exactly 10× standard Pro.** That is the honest read of speculative decoding in general and it is worth stating plainly: **it does not create throughput, it converts throughput into latency.** You burn far more compute per token — a target forward on a block of 8 instead of 1, plus the drafter, plus reserved capacity so the batch never fills — and you get the answer sooner. A 10× price for a claimed 20× speed is the exchange rate, printed. It is a real product for latency-sensitive work; it is not a faster model. I cannot test the 20×. I have not served this model and the API is not something I can benchmark honestly from here. What is measured is the `block_size: 8` in the shipped `dflash/config.json`, the single draft pass and single verification pass in `dflash.py`, and the card's own "predicts 7 subsequent tokens in a single forward pass." The arithmetic follows from those. If UltraSpeed serves a different block size, the ceiling moves and this section is about the released file rather than the product. ## The demos, and what they are The announcement is mostly a showcase — game development, Blender scenes, frontend design, slide decks, an orchestral piece converted to MIDI. Most of it is unfalsifiable from outside. One category is not, because it runs in published benchmark environments with published task specifications: