# Tinfield 1: 1.85% of a model, and a 51-billion-parameter hash table
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/tinfield-1
> date: 2026-09-22
> tags: llm, open-weights, benchmarks, evaluation, mixture-of-experts, explainer
Bad Theory Labs has been in these pages twice. [BTL-3](/articles/btl-3) was a
934 MB LoRA adapter on Qwen3.6-27B, honestly labelled as one.
[BTL-4](/articles/btl-4) was a 35B checkpoint whose LiveCodeBench table did not
close arithmetically and whose `config.json` disclosed a merged Unsloth adapter
the prose never mentioned. Both times the useful move was the same: stop reading
the card and start reading the artifact.
**Tinfield 1** went up on 2026-09-21. It claims **33.0 on Terminal-Bench 4.0**
and **62.0 on DeepSWE v1.1**, "ahead of Claude Opus 4.8" on both, from
**6.6B active parameters out of 177B**, at 256K context, with two quantized
builds that fit a 64 GB machine. The card is short, specific, and — this is new
for this lab — names its harness and its k.
This one is more interesting than BTL-4, and for the opposite reason. BTL-4's
card said less than its artifact. Tinfield's card says roughly the right things.
What the artifact adds is *scale*: how little of a 360 GB upload is actually new,
and what the one phrase nobody has explained — "the engram table" — actually
is.
## What the numbers mean before anything else
Start with the parameter counts, because three different ones are in
circulation and all three are correct under different conventions.
The safetensors headers, summed across all 131 shards, give
**179,999,981,459 parameters** — 180.0B, 360.0 GB on disk in BF16. The card says
177B. Qwen's card for the base model says "125B with 6B activated, plus 51B
n-gram embedding and 4B MTP." Reconciling:
| Slice | Measured |
|---|---:|
| Whole checkpoint | 180.000B |
| − multi-token-prediction module | −2.607B |
| − vision tower | −0.449B |
| **= the language model, as the card counts it** | **176.944B ≈ 177B** |
| of which: the engram table | 51.200B |
| of which: everything else | 125.744B ≈ Qwen's "125B" |
So "177B" is the text model including the hash table but excluding the drafter
and the image encoder. Qwen's "125B + 51B + 4B" is the same weights, sliced
differently — Qwen's 4B for MTP counts the embedding and output head the drafter
needs, which are shared.
Active parameters work the same way. Counting the dense spine, top-10 of 512
experts per layer, the shared expert, the hyper-connection mixers and the output
head, I get **6.67B per token** — the card's 6.6B. Drop the output head as well
and it is **6.04B** — Qwen's "6B". Neither is wrong; they differ by one
248,320 × 2,560 matrix.
The engram lookup contributes **2,560 parameters** to that per-token figure.
Hold that number.
## Only 1.85% of it is new
Tinfield 1 and `Qwen/Qwen3.8-Flash-Next` publish **identical** safetensors
indices: same 1,658 tensor names, same shard assignment, same
359,999,963,128-byte total. That is not a coincidence, it is an invitation — if
the layout is identical, you can range-request the same byte window out of the
same shard in both repositories and compare.
I did that at four offsets per tensor (0%, 33%, 66%, 99%), on early, middle and
late layers, across every named tensor group.
The pattern is the cleanest I have seen on a released fine-tune. **Every linear
map in the attention and residual-routing path moved**, in every layer sampled:
the linear-attention `in_proj_{qkv,z,a,b}` and `out_proj`, the full-attention
`q/k/v/o_proj`, and the hyper-connection input mixers. **Everything else is byte
for byte the base model**: all 512 experts in all 48 layers, every router, the
shared expert, both 248,320 × 2,560 embedding matrices, every RMSNorm, the
linear-attention state dynamics (`A_log`, `dt_bias`, `conv1d`), the sparse-attention
indexer, the entire 27-layer vision tower, the entire MTP module, and the whole
51.2-billion-parameter engram table.
That is **3,322,347,520 changed parameters out of 179,999,981,459** — 1.85%.
**353.4 GB of the 360 GB you download is a file you can already get from Qwen.**
This is not an accusation. Freezing the experts and training only attention is a
real, defensible recipe — it is cheap, it preserves the base's knowledge exactly,
and on an agentic post-train where you want to change *how* the model uses what it
knows rather than *what* it knows, it is arguably the right target set. BTL-4's
LoRA target map was the same shape and I said the same thing there. What I would
want on the card is one sentence naming it, because the shape of the update is
the single best predictor of how a fine-tune generalizes, and a reader budgeting
360 GB of bandwidth has a right to know that 98% of it is a re-upload.
Sampling four windows per tensor cannot prove a tensor is untouched *everywhere*
— only that nothing moved where I looked. The positive results are stronger than
the negative ones: CHANGED at 0.1% of a tensor is proof of an edit; SAME at four
offsets is strong evidence of none. The engram shards are the case I checked
hardest, at four offsets on four different shards, because 51.2B parameters is a
lot of surface to claim is unchanged.
## What an engram table actually is
Both GGUF cards say, in a sentence that has been quoted more than it has been
explained:
> the engram table is 26.8 GiB and always memory-mapped on the CPU side, since a
> token reads only a handful of its rows.
Here is what that refers to. In `config.json` the base architecture carries a
block nobody else's does:
```json
"ngram_size": 3,
"ngram_vocab_size_base": 20000000,
"heads_per_ngram": 8,
"split_ngram_parts": 128,
"ple_embed_dim": 2560,
"ple_layer_ids": [2]
```
and in the tensor index, at exactly one layer, 128 tensors named
`ple.ple_embedding.ngram_embedding.shard_0 … shard_127`, each
**(2,500,012 × 160) in BF16**. That is **320,001,536 rows × 160 =
51,200,245,760 parameters**, 95.4 GiB, **28.4% of the entire checkpoint**, living
in a single layer.
Three small `I64` tensors sit next to them, and they are the whole story. I
range-requested them and read them directly:
```text
ngram_heads_vocab_sizes = [20000003, 20000023, 20000033, 20000047,
20000059, 20000063, 20000069, 20000077,
20000081, 20000093, 20000107, 20000147,
20000153, 20000159, 20000161, 20000171]
ngram_heads_offsets = [0, 20000003, 40000026, 60000059, ...]
layer_multipliers = [23703573157769, 20109073645365, 8052911324071]
```
Those sixteen vocabulary sizes are **sixteen distinct primes just above twenty
million**. They sum to 320,001,446 rows; the shards hold 320,001,536, so ninety
rows are padding. The offsets are their running sum. The multipliers are three
odd 43–45-bit constants, one per n-gram position.
So the mechanism is a **multi-probe hashed n-gram embedding**. Take the last two
or three token ids, hash them with a polynomial whose coefficients are those
multipliers, and reduce modulo each head's own prime. Sixteen heads, sixteen
different primes, sixteen unrelated collision patterns. Each head returns a
160-wide row; concatenate sixteen of them and you have exactly the 2,560-wide
vector that `ple_embed_dim` asks for, which is then gated into the residual
stream through a small conv, a key projection and a value projection at that one
layer.
The distinct primes are the entire trick. A single hash table of that size would
collide, and a collision would be a silent, permanent confusion of two unrelated
n-grams. Sixteen independent moduli mean two n-grams that collide under
`P₀` almost certainly do not collide under `P₁`, so the concatenation averages
the noise away. This is the classic hashing-trick construction — it just has not
often been built at 51 billion parameters inside a frontier language model.
And the reason it can be that big is the ratio the widget ends on. The table
stores **51,200,245,760** values. A token reads **2,560** of them — one 160-wide
row per head. That is a gather of about 5 KB out of 95 GiB, with no matrix
multiply attached. Qwen's own card is precise about why this matters:
> Embeddings provide a unique axis for parameter scaling that requires less
> computation and is more amenable to offloading than Mixture-of-Experts.
That is the whole design. MoE sparsity still costs you a matmul per active
expert and wants the expert weights near the accelerator. A hashed embedding
costs a pointer chase. Which is why llama.cpp can leave 26.8 GiB of it on the
CPU side and still keep the model on a 64 GB machine.
**The engram table is Qwen's, not Bad Theory Labs'.** It is declared in the base
model's card — "N-gram Embedding: 20,000,000 (bigrams/trigrams at layer 2)" —
and every one of its 128 shards is byte-identical between the two repositories.
Tinfield's card does not claim otherwise; it says "Built on Qwen3.8-Flash-Next"
in the third line. But the phrase "the engram table" appears only in BTL's GGUF
cards, with no pointer to whose table it is, and that is where the
misattribution has been coming from.
## The quantization arithmetic closes, and it confirms the table
The Compact and Mini cards are the best-written part of this release, and unlike
BTL-4's LiveCodeBench table their numbers reconcile exactly:
| | Compact | Mini |
|---|---:|---:|
| experts | 40.4 GiB | 30.0 GiB |
| spine | 4.8 GiB | 4.8 GiB |
| **GPU-resident** | **45.2 GiB** | **34.7 GiB** |
| engram table, memory-mapped | 26.8 GiB | 26.8 GiB |
| **file on disk** | **72.0 GiB** | **61.5 GiB** |
40.4 + 4.8 = 45.2, and 45.2 + 26.8 = 72.0. The Mini column closes the same way
to a tenth of a GiB. The "5.2× smaller on the GPU side" figure is computed the
right way too — against BF16 *minus* the engram, since the engram is never
GPU-resident in either build, which is 239.9 GiB, and 239.9 / 45.2 = 5.3.
And the engram row is an independent confirmation of my parameter count. The
recipe puts the table at `IQ4_NL`, which is 4.5 bits per weight.
51,200,245,760 × 4.5 / 8 = 28.8 GB = **26.8 GiB**. That is the number on the
card, to three significant figures, from a count I made out of tensor shapes the
card never mentions.
One of the quants was measured rather than asserted, though not by BTL. The
fourth GGUF repo in the org is a mirror — its own first line says *"Built by
[needmorevramog](https://huggingface.co/needmorevramog), mirrored here"* — and
the quantizer who built it published the check:
## The denominators
Now the part that actually decides whether "ahead of Claude Opus 4.8" survives.
The card states its protocol, which BTL-4's did not: *"Evaluated with
mini-swe-agent at k=5 on the full task sets: 66 tasks for Terminal-Bench 4.0,
113 for DeepSWE v1.1."* It does not say what k=5 means, and against a published
Opus score that is the whole comparison — best-of-5 and mean-of-5 are different
metrics and one is roughly twice the other on hard agentic sets.
You can settle it with arithmetic, because a benchmark with 66 tasks can only
produce certain scores.
Under **best-of-5** the score is *(tasks solved at least once) / 66*, so it must
be a multiple of 1.515 points: 21/66 = 31.82, 22/66 = 33.33. **33.0 is not on
that lattice.** Under **mean-of-5** the denominator is 66 × 5 = 330 and the step
is 0.303 points; 109/330 = 33.03, which prints as 33.0. So **k=5 means five
repeats, averaged** — the conservative reading — and the two leaderboards the
card cites agree: Artificial Analysis reports Terminal-Bench 4.0 as "pass@1
averaged over three repeats per task," and DeepSWE v1.1 runs the full suite four
times and reports the share of rollouts that pass.
That is a good result for the card. It also means the three runs being compared
use k=5, k=3 and k=4 respectively, which nobody states.
Then the comparison table, which mixes agents — and says so, in a column:
| Model | Agent | Score |
|---|---|---:|
| GLM-5.3 (max) | Claude Code | 41.8 |
| GPT-5.6 Sol (max) | Codex | 37.3 |
| **Tinfield 1** | **mini-swe-agent** | **33.0** |
| Qwen3.8-Flash-Next | *(blank)* | 29.0 |
| Claude Opus 4.8 (max) | Claude Code | 23.6 |
| Gemini 3.8 Flash (high) | mini-swe-agent | 19.1 |
Printing the Agent column is the honest thing to do and I want to credit it. But
a reader should know what it implies: **Tinfield 1 vs Claude Opus 4.8 on
Terminal-Bench is not a model comparison, it is a (model, scaffold) comparison**,
and the scaffolds differ. [The harness effect](/articles/harness-effect) is the
long version of why that matters. Here the direction is at least not flattering
to Tinfield — mini-swe-agent is the deliberately minimal scaffold and Claude Code
is a full agent harness — so if anything the comparison understates the model. It
still is not the comparison the sentence implies.
The DeepSWE row is the stronger of the two claims, because DeepSWE v1.1 runs
**every** model on mini-swe-agent by design. Tinfield's 62.0 against Opus 4.8's
59 is scaffold-matched; only k differs (5 vs 4). That one survives reading.
## The base-model row does not match its source
The card's most important number is not 33.0. It is the pair 29.0 → 33.0, which
is the only figure that isolates what Bad Theory Labs' training actually bought.
The card sources it: *"Base model Terminal-Bench figure from Artificial
Analysis."*
Artificial Analysis runs Terminal-Bench 4.0 on all 66 tasks with mini-swe-agent
at pass@1 averaged over three repeats — so this *is* the harness-matched
comparison, and that is genuinely better discipline than BTL-4 managed. But the
number on AA's board today for `Qwen3.8-Flash-Next` is **0.252525…**, which is
50/198, **25.25** — not 29.0. Every other entry on that board is an exact
multiple of 1/198, so the grid is real; 29.0 is not on it either
(57/198 = 28.79, 58/198 = 29.29).
I cannot tell you which way this cuts. AA re-runs models, so 29.0 may be what the
board showed when the card was written and 25.25 may be a later re-measurement,
or a different reasoning-effort row. What I can tell you is that the citation
does not resolve against its source as of 2026-09-22, and that the discrepancy
runs *against* Bad Theory Labs' interest: if the base is 25.25, the delta
Tinfield bought is +7.75 points, not +4.0. A card understating its own result is
an unusual failure mode, and the fix is the same either way — print the base-model
row you measured yourself, on your own harness, at your own k.
There is a smaller thing in the same neighbourhood. The card says Tinfield is
"the second open-weight model" on the Terminal-Bench board. On Artificial
Analysis' board for the same benchmark, `GLM-5.3 (max)` sits at 41.92,
[`MiMo-V2.6-Pro`](/articles/mimo-v2-6) at 34.85 and `GLM-5.3-Flash` at 32.83 —
three open-weight entries, two of them above 33.0, on the board that supplied the
base-model figure. Different boards, different scaffolds, so the claim is not
false on the leaderboard it refers to. It just does not hold on the one the card
cites two paragraphs earlier.
## The things nobody will check
Two smaller observations, both of the kind that only show up if you open the
files.
**The vision tower is present, untouched and unevaluated.** `pipeline_tag` is
`image-text-to-text`, there are 27 vision blocks, a merger, an `image_token_id`
and a `video_token_id`, and every one of those tensors is byte-identical to
Qwen's. The card describes a terminal and software-engineering model and reports
no multimodal number. That is fine — you inherit what you did not train and do
not claim it — but a reader should know that roughly 0.9 GB of the download is an
image encoder nobody in this release has measured, and that Tinfield's
multimodal behaviour is entirely Qwen's. [BTL-4 shipped exactly the same
situation](/articles/btl-4) six weeks ago.
**Bad Theory Labs' own website still lists Tinfield as not built.**
`badtheorylabs.com` carries a "The stack" panel with four systems; the first is
*"Tinfield — Model family · Proposed … status: not built."* The weights have
been on the Hub for a day. That is a stale roadmap page rather than a
contradiction, but it is the only place the lab describes what Tinfield is
*for*, and it describes it in the future tense.
## What is genuinely good here
I have spent most of this piece on denominators, so let me be direct about the
parts that are better than most releases:
- **The protocol is stated.** Harness named, k stated, full task sets, no
subsetting. BTL-4's SWE-bench row got three words; this gets a sentence with
numbers in it.
- **The quantization work is real and it reconciles.** A 4.5-bit engram, 2-bit
expert gate/up, a range-searched `Q2_0` for the down projections that takes
weighted error from 51.9% to 11.7%, mean KL against Q8 reported for both builds
(0.287 and 0.340), and an explicit statement that *"the benchmark scores are
for the BF16 weights; this build has not been evaluated."* That last sentence
is the one most quantizers omit.
- **The operational notes are earned.** `--jinja` or tool calling breaks;
`-c` is total context, not per slot; skip n-gram self-speculation because it
accepted 16–25% of drafted tokens on agent traffic; raise
`iogpu.wired_limit_mb` on macOS before 45.2 GiB of weights meets the default
wired limit. Nobody writes that paragraph without having run the thing in a
loop.
- **The community build is labelled as not theirs**, with its own author and an
explicit "not built or measured by us."
## The take
Tinfield 1 is a 3.3-billion-parameter attention-path fine-tune of
Qwen3.8-Flash-Next, shipped as a 360 GB re-upload, with a quantization recipe
that is the actual craft in the release and a benchmark table that is mostly
honest and cites one number that does not resolve against its source.
The reusable part is the method, and it is the same four steps
[BTL-4](/articles/btl-4) needed, now with a fifth:
1. **Does the table close?** Weighted means must be consistent with their parts.
2. **Does the declared base exist, and does the config match?** Here it matches
so exactly that the tensor indices are identical files.
3. **What does the config leak?** `ngram_vocab_size_base`, `split_ngram_parts`
and three int64 metadata tensors explain a phrase the prose leaves as jargon.
4. **Do the weights agree with the story?** 1.85%.
5. **Does the printed score sit on the lattice its denominator allows?** This is
the new one, it costs one line of arithmetic, and it settled a question the
card left open in the right direction.
What I would want before quoting 33.0 anywhere: the base-model row re-run by Bad
Theory Labs on their own mini-swe-agent at k=5, next to Tinfield's, with the
sampling parameters printed. Every other number on this card has a denominator.
That one borrows someone else's.
Download both checkpoints and diff them in full. I sampled four windows per tensor group on a subset of layers; a fine-tune that moved a narrow slice of one expert matrix outside every window I read would show as SAME here. If a full diff comes back with any expert, embedding or engram tensor changed, the headline number is wrong and so is the recipe I inferred from it.
Load the model and instrument `ple_embedding`. Print the row indices it gathers for a known token sequence and compare them against `(m₀t₀ + m₁t₁ + m₂t₂) mod Pᵢ`. The shapes, the primes and the multipliers are measured; the composition is my reading of them. If the indices come from a different hash — a learned index, a different polynomial, a different n-gram split than 8 bigram / 8 trigram heads — the mechanism section is wrong even though the parameter count stands.
The argument is that 33.0 is unreachable on a 66-task best-of-5 lattice. It fails if the runs excluded tasks, if some tasks were scored partially rather than binary, or if the printed 33.0 is itself rounded from a figure computed on a different denominator. Publishing the per-task results settles it in one file.
Read on 2026-09-22 from the embedded `terminalBench40` field on AA's model page, where it is 0.252525… = 50/198. If AA's board showed 29.0 at any point before 2026-09-21, the card cited it correctly and it has since been re-measured — in which case the card is right and this section should say "was re-measured" rather than "does not resolve."
Run Opus 4.8 on Terminal-Bench 4.0 under mini-swe-agent at k=5 and publish it. If it lands near 23.6 anyway, the scaffold was not doing the work and Tinfield's margin is a model margin after all.
---
*Nothing here was executed against a model. No weights were downloaded: every
measurement is arithmetic on public metadata plus HTTP range requests totalling a
few megabytes against the safetensors shards of
[`badtheorylabs/Tinfield-1`](https://huggingface.co/badtheorylabs/Tinfield-1) and
[`Qwen/Qwen3.8-Flash-Next`](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), read
on 2026-09-22. Parameter counts are summed from the full headers of all 131
shards; the engram metadata tensors were read as raw int64. Benchmark figures are
quoted from the
[Tinfield 1](https://huggingface.co/badtheorylabs/Tinfield-1),
[Compact](https://huggingface.co/badtheorylabs/Tinfield-1-Compact-GGUF) and
[Mini](https://huggingface.co/badtheorylabs/Tinfield-1-Mini-GGUF) cards, with
comparison values from
[Artificial Analysis](https://artificialanalysis.ai/evaluations/terminalbench-4-0)
and the [DeepSWE leaderboard](https://deepswe.datacurve.ai/). On figures: I read
all five `badtheorylabs` Tinfield cards on the Hub, the org profile, and
`badtheorylabs.com`'s papers and thesis pages. BTL itself publishes no image at
all. The only two in the org are the perplexity chart above and a KL-divergence
chart of the same pair, and both arrived with the
[Q4_K_XL GGUF](https://huggingface.co/badtheorylabs/Tinfield-1-Q4_K_XL-GGUF)
mirror from `needmorevramog`, who made that quant; terms in
[NOTICE.txt](/articles/tinfield-1/NOTICE.txt). There is no architecture
diagram, no plot of either benchmark, no video anywhere, and no chart on the
main card at all: its headline numbers exist only as five markdown tables. So
the score lattice and the other two interactives above are mine, because there
was nothing to mirror.
Earlier reads of this lab:
[BTL-3](/articles/btl-3) and [BTL-4](/articles/btl-4).*