Inkling-Small: what on-policy distillation actually buys a reasoning model

Thinking Machines' Inkling shipped with an unusually candid pitch: "not the strongest overall model," a broad multimodal base meant for fine-tuning. Inkling-Small is the smaller sibling promised in that release, and its own pitch is just as specific: "an efficient open-weights model that achieves comparable performance to Inkling at a quarter of its size." Same architecture family, same 256-expert MoE backbone, same 1M-token context — but a materially different post-training story. Inkling-Small was post-trained from an earlier checkpoint using on-policy distillation with Inkling as the teacher, then pushed through two weeks of scaled agentic-coding RL. The result, per Thinking Machines: Inkling-Small now surpasses Inkling on reasoning and agentic coding benchmarks, while Inkling keeps the edge on knowledge and factuality.

That is the interesting part of this release — not the size, the recipe. On-policy distillation is a genuinely different training signal from ordinary distillation, and it is worth being precise about why. Along the way there is also a smaller, checkable finding: the parameter counts both Thinking Machines channels quote for Inkling-Small and Inkling do not match what the released weights actually contain.

Same backbone, one size down

Inkling-Small shares its architecture with Inkling almost feature-for-feature — the full mixture-of-experts and hybrid-attention design is covered in the Inkling piece, so here is just the shape, read from each model's config.json:

Same routing scheme (256 routed experts, 6 active, 2 always-on shared experts, sigmoid router with post-top-k norm), same relative position bias and short-conv mixing, same encoder-free image/audio path, same million-token context. Inkling-Small is a narrower, shallower cut of the identical design — fewer layers, a smaller residual stream, and a tighter expert width. What changes is everything downstream of pretraining.

Off-policy vs on-policy: who generates, who scores

Ordinary distillation — call it off-policy — has the teacher generate the training data. The teacher produces a sequence of tokens (an answer, a reasoning trace, a full trajectory), and the student is trained by cross-entropy to reproduce those exact tokens. It is imitation: match the teacher's output distribution on the teacher's own text.

That works fine for short, single-step outputs. It runs into a specific problem for anything autoregressive and long — like a chain of reasoning. At inference time the student has no teacher transcript to fall back on; it has to sample its own next token from its own distribution, condition on that, sample the next one, and so on. The moment a sampled token differs even slightly from what the teacher would have produced at that step, the student is in a state its training never covered — and every token after that is generated conditioned on an increasingly unfamiliar prefix. This is exposure bias: errors compound because the training signal only ever showed the model teacher-generated prefixes, never its own.

On-policy distillation removes the reference trajectory entirely. The student generates its own rollout, token by token, under its own policy — and the teacher's only job is to score the tokens the student actually produced (as a per-token reward or a log-probability target, depending on the recipe). There is no teacher transcript to drift away from, because training never showed the student one. Whatever state the student's own sampling puts it in, that is exactly the state it gets graded and corrected in. Drag through the two modes below and scrub the rollout step to see the difference concretely:

This is precisely why on-policy distillation matters more for a reasoning model than for a plain chat model: a reasoning trace is long, autoregressive, and self-referential — later steps depend directly on the model's own earlier steps. A student trained only to imitate a teacher's specific path is fragile exactly where it counts, the moment its own sampling wanders off that path. A student whose own rollouts are the only thing ever scored has no such cliff to fall off.

Thinking Machines describes the Inkling-Small recipe directly: "we post-trained an earlier checkpoint, Inkling-Small (preview), in part using on-policy distillation with Inkling as the teacher. Starting from that checkpoint, we continued scaling agentic coding RL for two weeks." Inkling — the larger, already-trained sibling — is the sole teacher; the smaller model generates, Inkling grades.

This site has covered two other takes on the same idea, and the contrast is worth naming. Kimi K3's post-training trains nine separate RL specialists (three domains times three effort levels) and then uses Multi-Teacher On-Policy Distillation to collapse all nine back into one shipped checkpoint — many teachers, all of them versions of the model itself. Agents-A1 does something similar with six domain specialists, routing each training trajectory to the one teacher that owns its domain. Inkling-Small's version is the simplest point in that space: one teacher, and it is not a specialist expert of the student — it is a wholly separate, larger, already-shipped model. Same underlying mechanism (student generates, teacher scores the student's own tokens), different teacher cardinality and a different relationship between student and teacher.

Two weeks of RL — read the disclosure level honestly

After the on-policy distillation stage, Thinking Machines says it "continued scaling agentic coding RL for two weeks." Read that number for what it actually specifies and what it does not.

It tells you the wall-clock duration of one training phase. It tells you nothing about cluster size, GPU count, rollout throughput, number of environments, or total compute — so "two weeks" from a 64-GPU pod and "two weeks" from a full GB300 NVL72 cluster are the same sentence describing wildly different amounts of work. Scaling agentic RL is mostly an environments-and-infrastructure problem — verified, reproducible task environments at scale is usually the actual bottleneck, not algorithm novelty — and none of that infrastructure detail is disclosed here either: no environment count, no rollout count, no reward model description beyond "agentic coding." Compare that to Kimi K3's post-training write-up, which at least names concrete infrastructure numbers (sandbox counts, checkpoint latencies) for its agentic RL stage. Thinking Machines' own blog names the training hardware for the base models (NVIDIA GB300 NVL72) but not specifically for this RL phase. Two weeks is a real number and a real signal that the recipe kept running rather than stopping early — it is just not, by itself, a compute disclosure.

The parameter count: stated vs measured

Both Thinking Machines channels — the announcement blog and the Hugging Face model card — quote the same rounded parameter counts for both models:

"Inkling-Small is a Mixture-of-Experts transformer with 276B total parameters, 12B active, trained on NVIDIA GB300 NVL72 systems." — Thinking Machines blog

Params (B) (activated/total): Inkling-Small "12/276", Inkling "41/975" — HF model card, evaluations table

Fetching each repository's safetensors metadata directly from the Hugging Face API (api/models/thinkingmachines/{Inkling-Small,Inkling}, checked 2026-08-03) gives a different number — the literal count of parameters in the released weight files:

No accusation implied here — both numbers come from official Thinking Machines channels, and this is simply what the weight files measure against what both channels quote. It is consistent across both models and both channels, so it reads as a rounding-and-carry-forward convention rather than a one-off typo. The active-parameter figures (12B / 41B) cannot be checked the same way — they describe how many parameters fire per token, which depends on live MoE routing at inference and cannot be read off static weight metadata. Take those as self-reported.

The "a quarter of its size" framing is worth checking on its own terms too. A literal quarter means Inkling should be 4x Inkling-Small. On the stated numbers, 975 ÷ 276 ≈ 3.53x; on the measured numbers, 952.38 ÷ 265.96 ≈ 3.58x. Either way, Inkling-Small is closer to 28% of Inkling's size than 25% — "a quarter" is a round-down of a real but smaller ratio, not a precise figure. One more data point that tracks the same rough ratio: Tinker's stated output pricing is $1.20 per million tokens for Inkling-Small against$4.05 for Inkling — Inkling-Small at about 30% of Inkling's price, in the same neighborhood as the ≈28% size ratio.

Benchmarks — where it wins, and where it doesn't

Thinking Machines' own evaluation suite backs the headline claim: Inkling-Small beats its own larger sibling on most reasoning, coding, and agentic benchmarks.

That pattern — Small ahead of its own larger sibling, and ahead of every similarly sized open peer Thinking Machines tested — holds cleanly on SciCode, GPQA Diamond, ARC-AGI-1/2, CritPt, and Toolathlon Verified. It is not universal: on SWE-Bench Pro Inkling-Small (55.9%) sits in a three-way near-tie, edged out slightly by MiMo V2.5 (56.1%) and Minimax M2.7 (56.2%) even as it still beats its own sibling Inkling (54.3%). And it does not hold at all on knowledge-recall tasks. Thinking Machines states that exception directly: "Inkling maintains an advantage on knowledge coverage and factuality." SimpleQA Verified is the sharpest case:

Inkling-Small loses to its larger sibling by more than 23 points here, and a similar gap shows up on AA Omniscience (Inkling-Small −9.0 vs. Inkling +2.1). Both are pure knowledge-recall benchmarks, not reasoning or agentic ones — exactly where the blog's caveat says to expect the loss, and the evaluation table backs it up cleanly. The next-largest gaps are Tau³ Banking (15.5% vs. 23.7% — an 8-point, roughly one-third relative deficit) and FORTRESS adversarial safety (71.6% vs. 78.0%); smaller, single-digit-point trails also show up on AIME 2026, Global-MMLU-Lite, and the multimodal audio/voice suite. None of these are reasoning or agentic benchmarks either — they cluster around knowledge, safety-adversarial robustness, and multimodal recall, consistent with "knowledge and factuality" being the one axis the bigger model still owns.

The take

Inkling-Small is a useful data point for a specific question: what does distillation from a bigger sibling actually buy you, mechanically? The answer here is not "compress the teacher's knowledge into a smaller container" — Inkling-Small is clearly worse at raw factual recall than Inkling, which is exactly what you would expect if the distillation target was never "know what the teacher knows." The target was "generate reasoning and agentic trajectories the teacher scores well" — and on-policy distillation is the mechanism that makes that the actual training signal, because it grades the student's own rollouts instead of teaching it to imitate someone else's. Layer two weeks of agentic-coding RL on top of that checkpoint and the result tracks: gains concentrate exactly in reasoning and agentic coding, and the one place the recipe doesn't touch — static factual knowledge — is the one place the bigger sibling keeps its lead.

The parameter-count gap is a smaller story, but it is the kind of thing worth checking rather than repeating: two official channels, one consistent 2.4-3.8% overstatement, verifiable in about two API calls. None of it changes what Inkling-Small actually is — an Apache-2.0, genuinely open-weights model that beats its own much larger sibling on most reasoning and coding benchmarks. It is just a reminder that "check the primary source" is worth doing even when the primary source is the model card itself.

Sources: the Inkling-Small announcement and the Hugging Face model card (architecture, training recipe, evaluations, pricing), cross-checked against the Inkling flagship card and this site's Inkling piece. Parameter counts were independently verified via the Hugging Face API's safetensors.total field for both repositories on 2026-08-03, not taken from either card. All benchmark numbers are Thinking Machines' own, on their own evaluation suite, with the harness caveats noted inline. Related reading: Kimi K3's Multi-Teacher On-Policy Distillation, Agents-A1's domain-routed on-policy distillation, mixture-of-experts from scratch, and scaling agentic RL. Neither Thinking Machines source publishes a static architecture or benchmark figure for this release — the blog's charts are rendered client-side from inline data, not static images — so the diagrams here are original illustrations of the mechanism, not reproductions of a paper figure.