2026-07-15 · 10 min · mixture-of-experts · multimodal · reinforcement-learning · attention
Why read this
Nichelast 15%Lays out Inkling's config (5:1 local/global, 256+2 experts, encoder-free inputs) and the effort-control claim; every number is vendor-reported.
- Widely used
- Open code or weights
LLM architectureNeeds datacenter GPUsPractitioner model
How this was scored
- Is it new?
- 1 of 3: An incremental tweak
- Can I trust it?
- 1 of 3: Spot-checks a few numbers
- Can I run it?
- 2 of 3: Open code or weights with real limits
- Will I understand it?
- 1 of 3: Partial mechanism
- Can I act on it?
- 1 of 3: General advice
- Will it last?
- 1 of 3: Relevant for months
- Does it affect many?
- 2 of 3: A widely used model, tool or lab release
- Only here?
- 0 of 3: Same as the press release
Score 38 of 100, ranked 542 of 577 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored
Most model launches lead with a leaderboard. Thinking Machines Lab's Inkling does the opposite: the announcement states plainly that it is "not the strongest overall model," and is instead "designed for broad adaptation through fine-tuning." That framing is the right lens for everything below. Inkling is an open-weights, multimodal mixture-of-experts foundation model — 975B total parameters, 41B active — with text, image and audio in one stack and up to a 1M-token context. A smaller companion, Inkling-Small (276B total / 12B active), ships in preview. The pitch is a customizable base, not a frontier trophy.
What makes it worth a close read is the mechanics: an attention design tuned for long context, an encoder-free multimodal path, a reinforcement-learning run whose reward scaled log-linearly while the model's reasoning got shorter on its own, and a knob that lets you dial how many tokens the model spends per query. Let's take them in turn — and keep the honest caveats in view throughout.
The backbone: sparse experts, hybrid attention
Inkling is a 66-layer decoder-only transformer. Two forms of sparsity run through it. In the feed-forward path, every layer is a mixture-of-experts: 256 routed experts plus 2 shared experts, with 6 routed experts active per token. A sigmoid-based router with an auxiliary-loss-free load-balancing bias decides which six fire — the same "drop the aux loss, use a bias term" trick that has become standard for keeping expert utilization even without a loss that fights the main objective. The 2 shared experts are always on, giving every token a common backbone of computation. Net effect: only 41B of the 975B parameters do work on any given token.
The attention path is a hybrid: of the 66 layers, 55 are sliding-window (512-token) local and 11 are global — an interleaved 5:1 ratio — with 64 query heads tied to 8 KV heads, over a 6144-dim residual stream. Five cheap local layers pass for every one exact global layer — the same local/global bargain that makes long-context serving affordable in MiniMax's sparse attention and MiMo-V2-Flash.
Then a cluster of small but telling choices — the kind you only catch by reading the config, not the launch post:
- Relative position bias, not RoPE (
d_rel=16,rel_extent=1024) — a learned bias on relative distance, chosen for cleaner extrapolation past the trained length. - Short depthwise convolutions (kernel size 4) in several places — after the key and value projections and on the residual branches — a cheap way to blend a little local context into each token before attention even runs. Convs-inside-a-transformer is a recurring "free lunch" for stability.
- An easy-to-miss one: a separate RMSNorm on the token embeddings, applied before the usual
per-block RMSNorms (
use_embed_norm=true). The residual stream is normalized at entry, not only inside each layer — extra insurance on embedding scale that most decoder-only stacks skip.
None of these are headline features; together they read as the fingerprint of a team tuning the backbone for stable long-context training rather than chasing a benchmark. Scrub the stack to see both sparsities at once — which layers are global, and which experts a token lights up:
Encoder-free multimodal
The multimodal design is deliberately minimal: no separate vision or audio encoder. Instead every modality is turned into tokens the transformer reads directly. Audio becomes discrete dMel spectrogram tokens; images are cut into 40×40-pixel patches and lifted by a small four-layer hMLP patch encoder; all modalities land in the shared hidden space and flow through the same experts and attention. There's no bolted-on CLIP-style tower whose representation you have to align — the model learns text, image and audio in one backbone. That is part of why it's pitched as an adaptation base: fine-tuning touches one stack, not a federation of encoders.
Controllable effort — the signature move
Inkling can vary how much it "thinks." The system message plus a per-token cost let you trade accuracy for token spend: turn effort down and it answers tersely; turn it up and it reasons at length, approaching its ceiling. The headline result is on Terminal-Bench-2.1, where the lab reports Inkling reaching Nemotron-3-Ultra-equivalent accuracy at roughly one-third the generated tokens. Drag the effort knob and read the tie line — the same score sits about 3× further right on the reference curve:
The efficiency framing matters more than any single point on the curve. A model that lets the caller choose the accuracy/latency trade-off, per request, is a different product from one with a fixed thinking budget — especially for the fine-tuning-and-deploy audience Inkling targets, who care about tokens-per-task cost at scale. (The curve shape above is illustrative; the ~63.8% Terminal-Bench-2.1 plateau and the ~1/3-token match are the real, vendor-reported anchors.)
Here is the sweep as Thinking Machines plot it, which is the measurement the interactive above is drawn from:
The RL story: log-linear reward, self-shortening reasoning
Post-training leaned on large-scale asynchronous reinforcement learning — over 30 million rollouts. Two findings stand out. First, the aggregate held-out eval reward rose log-linearly across those rollouts, climbing from 0.264 at the SFT-initialised checkpoint to 0.356 at release — a straight line on a log-rollouts axis, i.e. more RL compute kept paying off predictably rather than saturating. Second, and more surprising: with no brevity objective in the reward, the model's chain-of-thought became more concise on its own, "dropping grammatical overhead while remaining comprehensible." Reasoning compression emerged as a side effect of optimizing for correct answers. Drag the marker to watch reward climb as thought-length falls:
This connects back to controllable effort: a model whose reasoning is naturally terser is cheaper to run at any accuracy target, and the effort knob then lets you push that further.
Training and release
Pretraining ran on 45 trillion tokens of mixed text, image, audio and video, optimized with Muon for the large matrix weights and Adam for everything else (weight decay coupled to the squared learning rate), on NVIDIA GB300 NVL72 systems. Alongside the standard weights, Thinking Machines released NVFP4 weights for Blackwell — the same 4-bit format NVIDIA used to train Nemotron. The release is genuinely open: weights on Hugging Face (both standard and NVFP4), an API on Tinker plus Together, Fireworks, Modal, Databricks and Baseten, day-one vLLM / SGLang / llama.cpp integration, and a public Playground.
Results — read them as vendor-reported
Here are the headline numbers from Thinking Machines' own suite (at high effort). Reasoning first:
And the agentic / coding side, where the effort story is most relevant:
Multimodal and safety round it out: VoiceBench 91.4%, MMAU 77.2%, MMMU-Pro 73.5%, Global-MMLU-Lite 88.7%; on safety, FORTRESS Benign 95.9% / Adversarial 78.0% and StrongREJECT 98.6%. Inkling-Small tracks the big model closely on several evals (HLE text-only 31.6%, HLE with tools 47.8%).
The take
Inkling's most refreshing feature is its honesty about what it is. Thinking Machines did not build the model to top a leaderboard; they built a broad, open, multimodal base and tuned the ergonomics of adapting and running it — controllable effort so callers own the accuracy/cost trade-off, a 5:1 local/global attention hybrid and relative positions so 1M-token context stays affordable, an encoder-free multimodal path so fine-tuning touches one stack, and NVFP4 weights so it deploys cheaply on Blackwell. The two research results worth remembering are the log-linear RL reward (evidence that the post-training recipe kept scaling) and the emergent reasoning compression (shorter chains-of-thought with no brevity reward) — both their own measurements, both the kind of thing open weights will let others probe. Judge it not as "is this the best model" — the lab already answered no — but as a base you can take, fine-tune, and serve. On that axis, an open 975B/41B MoE with these ergonomics is a substantial thing to hand the community.
Built on Thinking Machines Lab's Inkling announcement and model card, plus the Hugging Face release. All benchmark and scaling figures are vendor-reported; the interactive diagrams are illustrations of the mechanism, with real endpoints noted inline. The three static figures are the announcement post's own charts, reproduced as published — the Hugging Face repository ships no figures of its own, and the model card page carries none either. Related reading: mixture-of-experts from scratch, NVFP4 training, MiniMax sparse attention, MiMo-V2-Flash, trillion-scale RL, the Muon optimizer, and how LLM inference works.