# Neutrino-1: quantization is a training decision, not a deployment one

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/neutrino-1
> date: 2026-07-27
> tags: llm, quantization, ternary, bitnet, inference, efficiency
Fermion Research shipped three models on July 27, 2026: **Neutrino-1 8B**, **Neutrino-1 0.6B**, and a 0.6B-Chat
variant, all on Hugging Face under Apache 2.0, all built on a ternary weight format — every projection weight is one
of exactly three states, minus, zero, or plus. That part of the pitch is not new; I wrote about
[Ternary15M](/articles/ternary15m) doing the same thing at 15M parameters a few days ago. What Neutrino-1 adds is
scale (about 500x more parameters) and, more importantly, a controlled comparison that the smaller model never ran:
what happens if you take the *same* ternary format and round a model into it after training, instead of training
inside it from the start.

The answer is not "a few points worse." It's a cliff.

## The cliff

Two 8B-class checkpoints, each about 3 GB, rounded into a ternary format after full-precision training, score **24.2**
and **24.7** on 5-shot MMLU. Chance, on a four-choice test, is **25.0**. Three models whose training ran *inside* the
ternary constraint from the start — at the same roughly 2-3 GB artifact size — score 47.24, 65.75, and **72.1**
(Neutrino-1 8B itself). Fermion's own line on this, and it's a good one: "rounding after training lands on the chance
line; every model trained inside its own format clears it by twenty-two points or more."

<RoundingCliff />

The mechanism is worth sitting with, because it explains why this isn't a smooth tradeoff curve. Round a trained
weight to the nearest of three values and the individual errors don't cancel — Fermion describes them as
uncorrelated, so they "accumulate along the row as a random walk instead of cancelling. The feature that leaves the
layer is not a noisier version of the right answer; it is a different number, and 36 layers compound the difference."
Their framing: "the failure is not noise. It is amnesia, and its signature is the cliff: scores do not degrade toward
chance, they arrive there." Train inside the constraint instead, and the optimizer never learns a solution the format
can't store in the first place — there's no gap between the model that was trained and the model that ships.

This is the flip side of what Ternary15M already showed at tiny scale: quantization-aware training with a
straight-through estimator gets hard-ternary inference to within +0.01 nats of the latent model, because the network
never experienced anything else. Neutrino-1 is the same bet, replayed at 8B, with the missing control group finally
run: skip the QAT and round instead, and the model doesn't degrade gracefully — it falls through the floor.

## What's actually ternary

Not everything. Of the 8B's 8.19 billion parameters, 6.95 billion (all 252 projection matrices — seven per layer,
attention and feed-forward, across 36 decoder layers) are ternary. The token embedding table and the output head stay
int8; the RMSNorm gains stay fp32. The reasoning Fermion gives is about where rounding error can hide: "a linear
layer's output feature sums hundreds of three-state contributions, so individual state errors cancel inside the sum;
that summation is what makes the format survivable." An embedding lookup returns one row verbatim — there's no sum to
average the error away — and the output head "decides tokens by small logit margins," where a rounding error can flip
the argmax. Both stay out of the ternary lane for the same reason: no averaging effect to hide behind.

The scale mechanism is also more granular than Ternary15M's. Ternary15M ternarizes each output channel around one
number: `absmean(W)`, the mean absolute weight for that whole row. Neutrino-1 groups weights into fixed-size blocks
along the input dimension and gives *each block* its own higher-precision scale — Fermion's phrase is "state times
scale." Smaller blocks track the underlying weights more closely at the cost of more stored scales; one scale for an
entire row (Ternary15M's approach) is the coarsest, cheapest case. The toy below runs the actual arithmetic — round
`clamp(w / scale, -1, 1)` per weight, scale is each block's mean absolute value — so you can see the tradeoff move:

<BlockScale />

## Where the bytes go

<Figure
  src="/articles/neutrino-1/fig1.png"
  alt="Three donut charts. Left: Neutrino-1 8B by parameter count — 84.8% ternary projection weights (blue), 15.2% int8 embeddings (maroon), a sliver of fp32 norms. Middle: the same 8B by byte — 67.2% ternary, 32.1% int8 embeddings, 0.7% norms and metadata. Right: Neutrino-1 0.6B by byte — only 50.4% ternary, 47.5% int8 embeddings, 2.1% metadata."
  caption="Ternary weights are 84.8% of the 8B's parameters but only 67.2% of its bytes — and at 0.6B scale the un-ternarized vocabulary is nearly half the file (chart redrawn from Fermion Research's figures, 2026)."
/>

This is the honest caveat behind "4.2 times fewer bytes than fp16 at bf16 on the same memory system," and it's worth
being precise about it: that ratio is a **whole-artifact** number, not the per-weight ternary compression ratio. Log2
of 3 states is about 1.58 bits, which against a 16-bit float is closer to a 10x reduction — but the embedding and
output tables (int8, one byte per value, no ternary discount) and the norm gains (fp32) drag the average down. At 8B,
those non-ternary tensors are only 15.2% of the parameters but 32.8% of the bytes. At 0.6B the effect is worse: the
same un-ternarized vocabulary is 47.5% of the file, because a smaller transformer has fewer projection weights to
amortize a fixed-size vocabulary table against. It's the identical finding Ternary15M made at 15M parameters, where
the FP32 embedding table was over 60% of that model's total parameters and dominated its 43 MB footprint — the
direction of the effect is the same at both ends of a 500x scale range: the *smaller* the model, the more its
un-quantized vocabulary — not its ternary matmuls — decides the file size.

On disk, Neutrino-1 8B is 3.88 GB; it downloads at 2.56 GB because the ternary lane compresses further in transit
(Fermion reports 0.516-0.569 of raw bytes, layer-dependent). Neutrino-1 0.6B downloads at 328 MB.

## Sparsity is learned, not imposed

Across the 8B's 6.95 billion ternary weights, the split is **62.63% zero, 18.68% plus, 18.69% minus** — remarkably
close to balanced between the two nonzero states, and remarkably far from an even three-way split. Fermion's framing
is the one worth keeping: "most of the mass on zero: the format sets how much of each tensor falls silent, and the
learned weights decide which connections go." A float layer can only make a connection small; a ternary layer,
trained natively, can delete it outright and the training decides which ones.

<Figure
  src="/articles/neutrino-1/fig2.png"
  alt="Line chart of zero-state share against decoder depth for seven projection types across 36 layers of Neutrino-1 8B. The four attention projections (q, k, v, o) hold a flat band between 61.8% and 63.5% at every layer. The feed-forward down and gate projections spike sharply at layers 2-4, with down reaching 72.47% at layer 3 and gate reaching 70.48% at layer 4, then both settle back to the ~62% baseline by layer 5."
  caption="Attention holds a flat sparsity band across all 36 layers; the feed-forward down/gate projections spike roughly ten points above it at layers 2-4, then settle (chart redrawn from Fermion Research's figure, 2026)."
/>

That spike is the interesting part, because nothing about the format explains it — the format sets *how much* falls
silent on average, not *where* it clusters by depth. The four attention projections sit in a tight 61.84-63.51% band
at every one of the 36 layers, almost boring in its consistency. The feed-forward `down` and `gate` projections are
the exception: `down` reaches 72.47% zero at layer 3, `gate` reaches 70.48% at layer 4 — roughly ten points denser
than the rest of the network — and both settle back to the ~62% baseline by layer 5. The single densest tensor in the
whole model is the layer 1 `down` projection at 60.59% (its local minimum, immediately before the spike). Scrub
through the real per-layer numbers below:

<SparsityDepth />

The state statistics are also stable across scale in a way that argues they're a property of the format and the
training recipe, not of size: at 0.6B, fourteen times fewer parameters, the split is 62.26% zero / 18.87% plus / 18.86%
minus — within half a point of the 8B on every axis.

## How much of Qwen3-8B does it keep

Neutrino-1 8B is measured, on Fermion's own harnesses, against **Qwen3-8B at bf16** — described in the post as "the
full-precision base it was built from," which is itself worth flagging: this isn't an independently trained
architecture being compared to an unrelated baseline, it's a model built from Qwen3-8B's own weights and then
retrained natively in ternary. At 4.2x fewer bytes, Neutrino-1 8B holds:

<BenchBars
  title="Capability retained vs. Qwen3-8B at bf16, same public harnesses (self-reported)"
  unit="%"
  bars={[
    { label: "general knowledge", value: 96 },
    { label: "knowledge, re-annotated", value: 87 },
    { label: "strict instruction following", value: 87 },
    { label: "tool calling", value: 79, highlight: true },
  ]}
/>

Report the weakest number, not the flattering one: tool calling retention is 79%, and it's worse than that headline
suggests once you look at the breakdown by category (BFCL v3, macro-averaged to 68.9 overall). Held-out, textbook
function signatures score well — 82.3% simple, 83.5% multiple — but signatures drawn from real-world APIs in the wild
score much lower: 61.6% live-simple, 52.0% live-multiple. Non-Python languages are worse still: 54.0% JavaScript,
43.0% Java. "Tool calling: 79%" is an average that buries a 40-point spread between the easy and hard slices of that
same axis.

<Callout type="warn">
The MMLU headline (72.1) and the "96% general knowledge" retention figure are **not directly comparable** in
Fermion's own post. The retention percentages are computed against Qwen3-8B's score on an unnamed "general knowledge"
suite — Fermion never states Qwen3-8B's own 5-shot MMLU number anywhere in the piece, and never confirms that
"general knowledge" and "MMLU" are the same benchmark. The MMLU chart above only compares Neutrino-1 8B against
*other* ternary and rounded models at similar artifact sizes, not against its own full-precision progenitor. So
while the rounding-vs-native-training gap (24.2 vs 72.1) is well anchored, the honest answer to "what's the gap
between 72.1 and Qwen3-8B's own MMLU" is: **the source doesn't say, and you can't back it out from what's published.**
Every number in this section is self-reported by Fermion, on their own harness, with no third-party replication.
</Callout>

For what it's worth, the one place Neutrino-1 8B is reported to exceed the reference is answer-format discipline —
Fermion's explanation is that discipline is a trained *behavior*, not a bulk statistical property of the weights the
way knowledge is, so the format doesn't cap it the way it caps knowledge retention. At the small end, Neutrino-1 0.6B
is compared directly to Qwen3-0.6B on ARC-easy: 53.45 vs 60.82, 87.9% retention, at one-eighth the precision and a
238 MB vs 1.50 GB download.

## Serving it: the Neutrino Engine

The inference side ships as its own artifact — a `pip install fermion` package, a CUDA-enabled `llama.cpp` fork, and
an MLX pack — with one stated design constraint: output has to be **token-identical** to a full-precision reference on
every backend. Fermion gates every release on that: the speculative-decoding path (Neutrino-1 0.6B drafting for the
8B) was checked token-by-token against the undrafted path across 27,648 consecutive tokens before any drafted
throughput number was published, and they report zero divergences.

Measured numbers: **33.7 tokens/second** on a MacBook M5 (GPU path; 24.9 tok/s CPU-only), **30.7 tokens/second** on an
NVIDIA L4 at 4k context inside 4.68 GiB of VRAM (fits an 8 GB card), and **396 tokens/second** undrafted on an H100
80GB — rising to **763 tokens/second** with the 0.6B draft model, gated as above. Draft acceptance is prompt-dependent:
near 100% on counting/enumeration, 96.5% on factual recall, roughly 80% on prose, and roughly 50% on code — code is
where the smaller model diverges from the 8B's choices most often, so the speedup shrinks accordingly.

The core argument for why a smaller artifact is faster at batch size 1 is straightforward memory-bandwidth
accounting: single-stream decode reads every weight once per token, so bytes-per-token divided by memory bandwidth
sets a hard floor on latency that no kernel can negotiate around. Neutrino-1 8B's 3.88 GB artifact against roughly
16 GB for the same weights at fp16 puts that floor about four times lower before a single kernel runs.

<Callout type="note">
That fp16 comparison is a **size** argument (3.88 GB vs ~16 GB), not a measured one. Every throughput number Fermion
publishes for the Neutrino Engine is ternary-format-versus-ternary-format — against a "reference stack" running the
same public 27B ternary model (105.15 vs 97.80 tok/s), against `bitnet.cpp` running BitNet b1.58-2B (102.4 vs 89.0
tok/s on an M5), against a lookup-table CPU kernel (T-MAC, 2.12x), against an int4 GEMV kernel (+12-13%). I could not
find a measured fp16 Qwen3-8B throughput number on the same M5, L4, or H100 hardware anywhere in the post. The
tokens/second figures are real and gated for correctness, but the *speedup over full precision* claim is anchored to
an artifact-size ratio, not to a same-hardware fp16 benchmark run.
</Callout>

One more honest number: KV cache growth is indifferent to weight format and scales with context regardless — 0.60 GB
at 4,096 tokens, up to 6.04 GB at the model's full 40,960-token window. Past roughly 26,000 tokens the cache alone
outweighs the 3.88 GB model artifact, so the memory story stops being about weights and starts being about context
length.

## What you can actually download

All three models are live on [huggingface.co/FermionResearch](https://huggingface.co/FermionResearch) as of today,
Apache 2.0, no waitlist:

| Model | Size | Role |
|---|---|---|
| Neutrino-1 8B | 2.56 GB download / 3.88 GB on disk | The frontier model |
| Neutrino-1 0.6B | 328 MB download | Draft model for speculative decoding, and usable standalone |
| Neutrino-1 0.6B-Chat | — | Conversational small model |

They're new enough that download counts were in the single digits at the time I fetched the org page — this is a
same-day release, not an established artifact with a track record.

## The take

Neutrino-1 is Ternary15M's bet — train inside the constraint instead of rounding into it, keep a shared scale next to
the signs, let signed accumulation replace multiplies — replayed at roughly 500x the parameters, with a block-wise
scale instead of one absmean per channel, a real inference engine with a correctness gate, and agentic/tool-use evals
that a 15M TinyStories model has no business running. The MMLU cliff (24.2-24.7 at chance versus 72.1 trained in
format) is the cleanest piece of evidence I've seen that quantization format is something you commit to before
training starts, not a knob you turn afterward — and it's consistent with, not contradicted by, Ternary15M's own
finding that training-aware ternary costs almost nothing when the network never knows another way to compute.

None of this is independently verified. Every number in this piece — the MMLU scores, the retention percentages, the
sparsity statistics, the tokens-per-second figures — is self-reported by Fermion Research on their own harnesses,
comparing their model to their own full-precision progenitor. The MMLU headline and the "96% general knowledge"
figure use two differently-named metrics that are never reconciled in the source. The speed claims are anchored to an
artifact-size ratio, not a measured full-precision baseline on the same silicon. And the model being celebrated for
"holding" Qwen3-8B's knowledge was built starting from Qwen3-8B's own weights, not trained from scratch as an
independent check on the method. The cliff is real and the mechanism is coherent; the specific numbers around it
deserve the same scrutiny you'd give any single-lab benchmark table until someone else reproduces them.

---

*Sources: Fermion Research, "[Intelligence at one-eighth the bits](https://www.fermionresearch.com/research/one-eighth-the-bits/)," "[Introducing the Neutrino-1 models](https://www.fermionresearch.com/research/neutrino-8b/)," and "[The Neutrino Engine](https://www.fermionresearch.com/research/the-neutrino-engine/)" (all July 27, 2026); model weights at
[huggingface.co/FermionResearch](https://huggingface.co/FermionResearch). All figures and quotes are self-reported by
Fermion Research with no third-party replication I could find. The two figures embedded above are reproduced from
Fermion Research's own per-layer and per-tensor measurements published in "One-eighth the bits"; the three interactive
components are mine. Related: [Ternary15M](/articles/ternary15m), the from-scratch 15M-parameter version of the same
bet.*