Neutrino-1: quantization is a training decision, not a deployment one

Fermion Research shipped three models on July 27, 2026: Neutrino-1 8B, Neutrino-1 0.6B, and a 0.6B-Chat variant, all on Hugging Face under Apache 2.0, all built on a ternary weight format — every projection weight is one of exactly three states, minus, zero, or plus. That part of the pitch is not new; I wrote about Ternary15M doing the same thing at 15M parameters a few days ago. What Neutrino-1 adds is scale (about 500x more parameters) and, more importantly, a controlled comparison that the smaller model never ran: what happens if you take the same ternary format and round a model into it after training, instead of training inside it from the start.

The answer is not "a few points worse." It's a cliff.

The cliff

Two 8B-class checkpoints, each about 3 GB, rounded into a ternary format after full-precision training, score 24.2 and 24.7 on 5-shot MMLU. Chance, on a four-choice test, is 25.0. Three models whose training ran inside the ternary constraint from the start — at the same roughly 2-3 GB artifact size — score 47.24, 65.75, and 72.1 (Neutrino-1 8B itself). Fermion's own line on this, and it's a good one: "rounding after training lands on the chance line; every model trained inside its own format clears it by twenty-two points or more."

The mechanism is worth sitting with, because it explains why this isn't a smooth tradeoff curve. Round a trained weight to the nearest of three values and the individual errors don't cancel — Fermion describes them as uncorrelated, so they "accumulate along the row as a random walk instead of cancelling. The feature that leaves the layer is not a noisier version of the right answer; it is a different number, and 36 layers compound the difference." Their framing: "the failure is not noise. It is amnesia, and its signature is the cliff: scores do not degrade toward chance, they arrive there." Train inside the constraint instead, and the optimizer never learns a solution the format can't store in the first place — there's no gap between the model that was trained and the model that ships.

This is the flip side of what Ternary15M already showed at tiny scale: quantization-aware training with a straight-through estimator gets hard-ternary inference to within +0.01 nats of the latent model, because the network never experienced anything else. Neutrino-1 is the same bet, replayed at 8B, with the missing control group finally run: skip the QAT and round instead, and the model doesn't degrade gracefully — it falls through the floor.

What's actually ternary

Not everything. Of the 8B's 8.19 billion parameters, 6.95 billion (all 252 projection matrices — seven per layer, attention and feed-forward, across 36 decoder layers) are ternary. The token embedding table and the output head stay int8; the RMSNorm gains stay fp32. The reasoning Fermion gives is about where rounding error can hide: "a linear layer's output feature sums hundreds of three-state contributions, so individual state errors cancel inside the sum; that summation is what makes the format survivable." An embedding lookup returns one row verbatim — there's no sum to average the error away — and the output head "decides tokens by small logit margins," where a rounding error can flip the argmax. Both stay out of the ternary lane for the same reason: no averaging effect to hide behind.

The scale mechanism is also more granular than Ternary15M's. Ternary15M ternarizes each output channel around one number: absmean(W), the mean absolute weight for that whole row. Neutrino-1 groups weights into fixed-size blocks along the input dimension and gives each block its own higher-precision scale — Fermion's phrase is "state times scale." Smaller blocks track the underlying weights more closely at the cost of more stored scales; one scale for an entire row (Ternary15M's approach) is the coarsest, cheapest case. The toy below runs the actual arithmetic — round clamp(w / scale, -1, 1) per weight, scale is each block's mean absolute value — so you can see the tradeoff move:

Where the bytes go

expand

This is the honest caveat behind "4.2 times fewer bytes than fp16 at bf16 on the same memory system," and it's worth being precise about it: that ratio is a whole-artifact number, not the per-weight ternary compression ratio. Log2 of 3 states is about 1.58 bits, which against a 16-bit float is closer to a 10x reduction — but the embedding and output tables (int8, one byte per value, no ternary discount) and the norm gains (fp32) drag the average down. At 8B, those non-ternary tensors are only 15.2% of the parameters but 32.8% of the bytes. At 0.6B the effect is worse: the same un-ternarized vocabulary is 47.5% of the file, because a smaller transformer has fewer projection weights to amortize a fixed-size vocabulary table against. It's the identical finding Ternary15M made at 15M parameters, where the FP32 embedding table was over 60% of that model's total parameters and dominated its 43 MB footprint — the direction of the effect is the same at both ends of a 500x scale range: the smaller the model, the more its un-quantized vocabulary — not its ternary matmuls — decides the file size.

On disk, Neutrino-1 8B is 3.88 GB; it downloads at 2.56 GB because the ternary lane compresses further in transit (Fermion reports 0.516-0.569 of raw bytes, layer-dependent). Neutrino-1 0.6B downloads at 328 MB.

Sparsity is learned, not imposed

Across the 8B's 6.95 billion ternary weights, the split is 62.63% zero, 18.68% plus, 18.69% minus — remarkably close to balanced between the two nonzero states, and remarkably far from an even three-way split. Fermion's framing is the one worth keeping: "most of the mass on zero: the format sets how much of each tensor falls silent, and the learned weights decide which connections go." A float layer can only make a connection small; a ternary layer, trained natively, can delete it outright and the training decides which ones.

expand

That spike is the interesting part, because nothing about the format explains it — the format sets how much falls silent on average, not where it clusters by depth. The four attention projections sit in a tight 61.84-63.51% band at every one of the 36 layers, almost boring in its consistency. The feed-forward down and gate projections are the exception: down reaches 72.47% zero at layer 3, gate reaches 70.48% at layer 4 — roughly ten points denser than the rest of the network — and both settle back to the ~62% baseline by layer 5. The single densest tensor in the whole model is the layer 1 down projection at 60.59% (its local minimum, immediately before the spike). Scrub through the real per-layer numbers below:

The state statistics are also stable across scale in a way that argues they're a property of the format and the training recipe, not of size: at 0.6B, fourteen times fewer parameters, the split is 62.26% zero / 18.87% plus / 18.86% minus — within half a point of the 8B on every axis.

How much of Qwen3-8B does it keep

Neutrino-1 8B is measured, on Fermion's own harnesses, against Qwen3-8B at bf16 — described in the post as "the full-precision base it was built from," which is itself worth flagging: this isn't an independently trained architecture being compared to an unrelated baseline, it's a model built from Qwen3-8B's own weights and then retrained natively in ternary. At 4.2x fewer bytes, Neutrino-1 8B holds:

Report the weakest number, not the flattering one: tool calling retention is 79%, and it's worse than that headline suggests once you look at the breakdown by category (BFCL v3, macro-averaged to 68.9 overall). Held-out, textbook function signatures score well — 82.3% simple, 83.5% multiple — but signatures drawn from real-world APIs in the wild score much lower: 61.6% live-simple, 52.0% live-multiple. Non-Python languages are worse still: 54.0% JavaScript, 43.0% Java. "Tool calling: 79%" is an average that buries a 40-point spread between the easy and hard slices of that same axis.

For what it's worth, the one place Neutrino-1 8B is reported to exceed the reference is answer-format discipline — Fermion's explanation is that discipline is a trained behavior, not a bulk statistical property of the weights the way knowledge is, so the format doesn't cap it the way it caps knowledge retention. At the small end, Neutrino-1 0.6B is compared directly to Qwen3-0.6B on ARC-easy: 53.45 vs 60.82, 87.9% retention, at one-eighth the precision and a 238 MB vs 1.50 GB download.

Serving it: the Neutrino Engine

The inference side ships as its own artifact — a pip install fermion package, a CUDA-enabled llama.cpp fork, and an MLX pack — with one stated design constraint: output has to be token-identical to a full-precision reference on every backend. Fermion gates every release on that: the speculative-decoding path (Neutrino-1 0.6B drafting for the 8B) was checked token-by-token against the undrafted path across 27,648 consecutive tokens before any drafted throughput number was published, and they report zero divergences.

Measured numbers: 33.7 tokens/second on a MacBook M5 (GPU path; 24.9 tok/s CPU-only), 30.7 tokens/second on an NVIDIA L4 at 4k context inside 4.68 GiB of VRAM (fits an 8 GB card), and 396 tokens/second undrafted on an H100 80GB — rising to 763 tokens/second with the 0.6B draft model, gated as above. Draft acceptance is prompt-dependent: near 100% on counting/enumeration, 96.5% on factual recall, roughly 80% on prose, and roughly 50% on code — code is where the smaller model diverges from the 8B's choices most often, so the speedup shrinks accordingly.

The core argument for why a smaller artifact is faster at batch size 1 is straightforward memory-bandwidth accounting: single-stream decode reads every weight once per token, so bytes-per-token divided by memory bandwidth sets a hard floor on latency that no kernel can negotiate around. Neutrino-1 8B's 3.88 GB artifact against roughly 16 GB for the same weights at fp16 puts that floor about four times lower before a single kernel runs.

One more honest number: KV cache growth is indifferent to weight format and scales with context regardless — 0.60 GB at 4,096 tokens, up to 6.04 GB at the model's full 40,960-token window. Past roughly 26,000 tokens the cache alone outweighs the 3.88 GB model artifact, so the memory story stops being about weights and starts being about context length.

What you can actually download

All three models are live on huggingface.co/FermionResearch as of today, Apache 2.0, no waitlist:

They're new enough that download counts were in the single digits at the time I fetched the org page — this is a same-day release, not an established artifact with a track record.

The take

Neutrino-1 is Ternary15M's bet — train inside the constraint instead of rounding into it, keep a shared scale next to the signs, let signed accumulation replace multiplies — replayed at roughly 500x the parameters, with a block-wise scale instead of one absmean per channel, a real inference engine with a correctness gate, and agentic/tool-use evals that a 15M TinyStories model has no business running. The MMLU cliff (24.2-24.7 at chance versus 72.1 trained in format) is the cleanest piece of evidence I've seen that quantization format is something you commit to before training starts, not a knob you turn afterward — and it's consistent with, not contradicted by, Ternary15M's own finding that training-aware ternary costs almost nothing when the network never knows another way to compute.

None of this is independently verified. Every number in this piece — the MMLU scores, the retention percentages, the sparsity statistics, the tokens-per-second figures — is self-reported by Fermion Research on their own harnesses, comparing their model to their own full-precision progenitor. The MMLU headline and the "96% general knowledge" figure use two differently-named metrics that are never reconciled in the source. The speed claims are anchored to an artifact-size ratio, not a measured full-precision baseline on the same silicon. And the model being celebrated for "holding" Qwen3-8B's knowledge was built starting from Qwen3-8B's own weights, not trained from scratch as an independent check on the method. The cliff is real and the mechanism is coherent; the specific numbers around it deserve the same scrutiny you'd give any single-lab benchmark table until someone else reproduces them.

Sources: Fermion Research, "Intelligence at one-eighth the bits," "Introducing the Neutrino-1 models," and "The Neutrino Engine" (all July 27, 2026); model weights at huggingface.co/FermionResearch. All figures and quotes are self-reported by Fermion Research with no third-party replication I could find. The two figures embedded above are reproduced from Fermion Research's own per-layer and per-tensor measurements published in "One-eighth the bits"; the three interactive components are mine. Related: Ternary15M, the from-scratch 15M-parameter version of the same bet.