# Inkling-Small: what on-policy distillation actually buys a reasoning model
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/inkling-small
> date: 2026-08-03
> tags: llm, mixture-of-experts, distillation, reinforcement-learning, explainer
Thinking Machines' [Inkling](/articles/inkling) shipped with an unusually candid pitch: *"not the
strongest overall model,"* a broad multimodal base meant for fine-tuning. **Inkling-Small** is the
smaller sibling promised in that release, and its own pitch is just as specific: *"an efficient
open-weights model that achieves comparable performance to Inkling at a quarter of its size."* Same
architecture family, same 256-expert MoE backbone, same 1M-token context — but a materially
different post-training story. Inkling-Small was **post-trained from an earlier checkpoint using
on-policy distillation with Inkling as the teacher**, then pushed through **two weeks of scaled
agentic-coding RL**. The result, per Thinking Machines: Inkling-Small now **surpasses Inkling** on
reasoning and agentic coding benchmarks, while Inkling **keeps the edge on knowledge and
factuality**.
That is the interesting part of this release — not the size, the recipe. On-policy distillation is
a genuinely different training signal from ordinary distillation, and it is worth being precise
about why. Along the way there is also a smaller, checkable finding: the parameter counts both
Thinking Machines channels quote for Inkling-Small and Inkling do not match what the released
weights actually contain.
## Same backbone, one size down
Inkling-Small shares its architecture with Inkling almost feature-for-feature — the
[full mixture-of-experts and hybrid-attention design is covered in the Inkling piece](/articles/inkling),
so here is just the shape, read from each model's `config.json`:
| | Inkling-Small | Inkling |
|---|---|---|
| Hidden size | 4096 | 6144 |
| Layers | 42 | 66 |
| Attention heads / KV heads | 32 / 8 | 64 / 8 |
| Sliding-window heads / KV heads | 32 / 8 | 64 / 16 |
| Sliding-window size | 512 | 512 |
| Routed experts | 256 | 256 |
| Active experts / shared experts | 6 / 2 | 6 / 2 |
| MoE expert intermediate size | 2048 | 3072 |
| Dense-layer intermediate size | 16384 | 24576 |
| Context length | 1,048,576 | 1,048,576 |
| Multi-token-prediction heads | 8 | 8 |
Same routing scheme (256 routed experts, 6 active, 2 always-on shared experts, sigmoid router with
post-top-k norm), same [relative position bias](/articles/how-llm-inference-works) and short-conv
mixing, same encoder-free image/audio path, same million-token context. Inkling-Small is a
narrower, shallower cut of the identical design — fewer layers, a smaller residual stream, and a
tighter expert width. What changes is everything downstream of pretraining.
## Off-policy vs on-policy: who generates, who scores
Ordinary distillation — call it off-policy — has the teacher generate the training data. The
teacher produces a sequence of tokens (an answer, a reasoning trace, a full trajectory), and the
student is trained by cross-entropy to reproduce those exact tokens. It is imitation: match the
teacher's output distribution on the teacher's own text.
That works fine for short, single-step outputs. It runs into a specific problem for anything
autoregressive and long — like a chain of reasoning. At inference time the student has no teacher
transcript to fall back on; it has to sample its own next token from its own distribution, condition
on that, sample the next one, and so on. The moment a sampled token differs even slightly from what
the teacher would have produced at that step, the student is in a state its training never covered —
and every token after that is generated conditioned on an increasingly unfamiliar prefix. This is
**exposure bias**: errors compound because the training signal only ever showed the model
teacher-generated prefixes, never its own.
**On-policy distillation** removes the reference trajectory entirely. The *student* generates its
own rollout, token by token, under its own policy — and the teacher's only job is to score the
tokens the student actually produced (as a per-token reward or a log-probability target, depending
on the recipe). There is no teacher transcript to drift away from, because training never showed the
student one. Whatever state the student's own sampling puts it in, that is exactly the state it gets
graded and corrected in. Drag through the two modes below and scrub the rollout step to see the
difference concretely:
This is precisely why on-policy distillation matters more for a reasoning model than for a plain
chat model: a reasoning trace is long, autoregressive, and self-referential — later steps depend
directly on the model's own earlier steps. A student trained only to imitate a teacher's specific
path is fragile exactly where it counts, the moment its own sampling wanders off that path. A
student whose own rollouts are the only thing ever scored has no such cliff to fall off.
Thinking Machines describes the Inkling-Small recipe directly: *"we post-trained an earlier
checkpoint, Inkling-Small (preview), in part using on-policy distillation with Inkling as the
teacher. Starting from that checkpoint, we continued scaling agentic coding RL for two weeks."*
Inkling — the larger, already-trained sibling — is the sole teacher; the smaller model generates,
Inkling grades.
This site has covered two other takes on the same idea, and the contrast is worth naming. [Kimi
K3's post-training](/articles/kimi-k3#post-training-nine-experts-then-one) trains **nine** separate
RL specialists (three domains times three effort levels) and then uses **Multi-Teacher On-Policy
Distillation** to collapse all nine back into one shipped checkpoint — many teachers, all of them
versions of the model itself. [Agents-A1](/articles/agents-a1) does something similar with six
domain specialists, routing each training trajectory to the one teacher that owns its domain.
Inkling-Small's version is the simplest point in that space: **one** teacher, and it is not a
specialist expert of the student — it is a wholly separate, larger, already-shipped model. Same
underlying mechanism (student generates, teacher scores the student's own tokens), different
teacher cardinality and a different relationship between student and teacher.
## Two weeks of RL — read the disclosure level honestly
After the on-policy distillation stage, Thinking Machines says it *"continued scaling agentic
coding RL for two weeks."* Read that number for what it actually specifies and what it does not.
It tells you the wall-clock duration of one training phase. It tells you nothing about cluster
size, GPU count, rollout throughput, number of environments, or total compute — so "two weeks" from
a 64-GPU pod and "two weeks" from a full GB300 NVL72 cluster are the same sentence describing
wildly different amounts of work. [Scaling agentic RL](/articles/scaling-agentic-rl) is mostly an
environments-and-infrastructure problem — verified, reproducible task environments at scale is
usually the actual bottleneck, not algorithm novelty — and none of that infrastructure detail is
disclosed here either: no environment count, no rollout count, no reward model description beyond
"agentic coding." Compare that to Kimi K3's post-training write-up, which at least names concrete
infrastructure numbers (sandbox counts, checkpoint latencies) for its agentic RL stage. Thinking
Machines' own blog names the training hardware for the base models (NVIDIA GB300 NVL72) but not
specifically for this RL phase. Two weeks is a real number and a real signal that the recipe kept
running rather than stopping early — it is just not, by itself, a compute disclosure.
## The parameter count: stated vs measured
Both Thinking Machines channels — the announcement blog and the Hugging Face model card — quote the
same rounded parameter counts for both models:
> "Inkling-Small is a Mixture-of-Experts transformer with 276B total parameters, 12B active,
> trained on NVIDIA GB300 NVL72 systems." — Thinking Machines blog
> Params (B) (activated/total): Inkling-Small "12/276", Inkling "41/975" — HF model card,
> evaluations table
Fetching each repository's `safetensors` metadata directly from the Hugging Face API
(`api/models/thinkingmachines/{Inkling-Small,Inkling}`, checked 2026-08-03) gives a different
number — the literal count of parameters in the released weight files:
| | Stated (blog + HF card) | Measured (HF `safetensors.total`) | Difference |
|---|---|---|---|
| Inkling-Small | 276B total | **265,956,439,090** (≈265.96B) | +10.04B, ≈3.8% above measured |
| Inkling | 975B total | **952,377,623,626** (≈952.38B) | +22.62B, ≈2.4% above measured |
No accusation implied here — both numbers come from official Thinking Machines channels, and this
is simply what the weight files measure against what both channels quote. It is consistent across
both models and both channels, so it reads as a rounding-and-carry-forward convention rather than a
one-off typo. The active-parameter figures (12B / 41B) cannot be checked the same way — they
describe how many parameters fire per token, which depends on live MoE routing at inference and
cannot be read off static weight metadata. Take those as self-reported.
The **"a quarter of its size"** framing is worth checking on its own terms too. A literal quarter
means Inkling should be 4x Inkling-Small. On the stated numbers, 975 ÷ 276 ≈ 3.53x; on the measured
numbers, 952.38 ÷ 265.96 ≈ 3.58x. Either way, Inkling-Small is closer to 28% of Inkling's size than
25% — "a quarter" is a round-down of a real but smaller ratio, not a precise figure. One more data
point that tracks the same rough ratio: Tinker's stated output pricing is $1.20 per million tokens
for Inkling-Small against $4.05 for Inkling — Inkling-Small at about 30% of Inkling's price, in the
same neighborhood as the ≈28% size ratio.
## Benchmarks — where it wins, and where it doesn't
Thinking Machines' own evaluation suite backs the headline claim: Inkling-Small beats its own
larger sibling on most reasoning, coding, and agentic benchmarks.
That pattern — Small ahead of its own larger sibling, and ahead of every similarly sized open peer
Thinking Machines tested — holds cleanly on SciCode, GPQA Diamond, ARC-AGI-1/2, CritPt, and
Toolathlon Verified. It is not universal: on **SWE-Bench Pro** Inkling-Small (55.9%) sits in a
three-way near-tie, edged out slightly by MiMo V2.5 (56.1%) and Minimax M2.7 (56.2%) even as it
still beats its own sibling Inkling (54.3%). And it does not hold at all on knowledge-recall tasks.
Thinking Machines states that exception directly: *"Inkling maintains an advantage on knowledge
coverage and factuality."* SimpleQA Verified is the sharpest case:
Inkling-Small loses to its larger sibling by more than 23 points here, and a similar gap shows up
on **AA Omniscience** (Inkling-Small −9.0 vs. Inkling +2.1). Both are pure knowledge-recall
benchmarks, not reasoning or agentic ones — exactly where the blog's caveat says to expect the
loss, and the evaluation table backs it up cleanly. The next-largest gaps are **Tau³ Banking**
(15.5% vs. 23.7% — an 8-point, roughly one-third relative deficit) and **FORTRESS adversarial**
safety (71.6% vs. 78.0%); smaller, single-digit-point trails also show up on **AIME 2026**,
**Global-MMLU-Lite**, and the multimodal audio/voice suite. None of these are reasoning or agentic
benchmarks either — they cluster around knowledge, safety-adversarial robustness, and multimodal
recall, consistent with "knowledge and factuality" being the one axis the bigger model still owns.
**Scope the evaluation methodology, not just the scores.** All of this is Thinking Machines
grading its own model against a provider-selected comparison set (Qwen3.5, MiMo V2.5, Minimax
M2.7, DeepSeek V4 Flash, Nemotron 3 Ultra, plus closed models Claude 4.5 Haiku, Gemini 3.5
Flash-Lite, and GPT 5.6 Luna) — a self-report, not a neutral third-party leaderboard. The card
itself discloses several protocol caveats worth carrying forward: SWE-Bench Verified and
Terminal-Bench 2.1 use an **internal, bash-only harness**, and external models' numbers on those
two are self-reported by their own vendors, not run in-house. Terminal-Bench 2.1 also zeroed out
"a small number of solutions... found to be contaminated from web search." HLE-with-tools numbers
for MiniMax M2.7, Claude 4.5 Haiku, Gemini 3.5 Flash-Lite, and GPT 5.6 Luna were run in-house by
Thinking Machines, not vendor-reported. None of this invalidates the results, but it means the
comparison set and the harness are both chosen by the same lab whose model is winning most of the
charts.
## The take
Inkling-Small is a useful data point for a specific question: what does distillation from a bigger
sibling actually buy you, mechanically? The answer here is not "compress the teacher's knowledge
into a smaller container" — Inkling-Small is clearly *worse* at raw factual recall than Inkling,
which is exactly what you would expect if the distillation target was never "know what the teacher
knows." The target was "generate reasoning and agentic trajectories the teacher scores well" — and
on-policy distillation is the mechanism that makes that the actual training signal, because it
grades the student's own rollouts instead of teaching it to imitate someone else's. Layer two weeks
of agentic-coding RL on top of that checkpoint and the result tracks: gains concentrate exactly in
reasoning and agentic coding, and the one place the recipe doesn't touch — static factual
knowledge — is the one place the bigger sibling keeps its lead.
The parameter-count gap is a smaller story, but it is the kind of thing worth checking rather than
repeating: two official channels, one consistent 2.4-3.8% overstatement, verifiable in about two
API calls. None of it changes what Inkling-Small actually is — an Apache-2.0, genuinely open-weights
model that beats its own much larger sibling on most reasoning and coding benchmarks. It is just a
reminder that "check the primary source" is worth doing even when the primary source is the model
card itself.
---
*Sources: the [Inkling-Small announcement](https://thinkingmachines.ai/news/inkling-small/) and the
[Hugging Face model card](https://huggingface.co/thinkingmachines/Inkling-Small) (architecture,
training recipe, evaluations, pricing), cross-checked against the
[Inkling flagship card](https://huggingface.co/thinkingmachines/Inkling) and this site's
[Inkling piece](/articles/inkling). Parameter counts were independently verified via the Hugging
Face API's `safetensors.total` field for both repositories on 2026-08-03, not taken from either
card. All benchmark numbers are Thinking Machines' own, on their own evaluation suite, with the
harness caveats noted inline. Related reading: [Kimi K3's Multi-Teacher On-Policy
Distillation](/articles/kimi-k3#post-training-nine-experts-then-one),
[Agents-A1's domain-routed on-policy distillation](/articles/agents-a1),
[mixture-of-experts from scratch](/articles/mixture-of-experts-from-scratch), and [scaling agentic
RL](/articles/scaling-agentic-rl). Neither Thinking Machines source publishes a static architecture
or benchmark figure for this release — the blog's charts are rendered client-side from inline data,
not static images — so the diagrams here are original illustrations of the mechanism, not
reproductions of a paper figure.*