Tinfield 1: 1.85% of a model, and a 51-billion-parameter hash table

Bad Theory Labs has been in these pages twice. BTL-3 was a 934 MB LoRA adapter on Qwen3.6-27B, honestly labelled as one. BTL-4 was a 35B checkpoint whose LiveCodeBench table did not close arithmetically and whose config.json disclosed a merged Unsloth adapter the prose never mentioned. Both times the useful move was the same: stop reading the card and start reading the artifact.

Tinfield 1 went up on 2026-09-21. It claims 33.0 on Terminal-Bench 4.0 and 62.0 on DeepSWE v1.1, "ahead of Claude Opus 4.8" on both, from 6.6B active parameters out of 177B, at 256K context, with two quantized builds that fit a 64 GB machine. The card is short, specific, and — this is new for this lab — names its harness and its k.

This one is more interesting than BTL-4, and for the opposite reason. BTL-4's card said less than its artifact. Tinfield's card says roughly the right things. What the artifact adds is scale: how little of a 360 GB upload is actually new, and what the one phrase nobody has explained — "the engram table" — actually is.

What the numbers mean before anything else

Start with the parameter counts, because three different ones are in circulation and all three are correct under different conventions.

The safetensors headers, summed across all 131 shards, give 179,999,981,459 parameters — 180.0B, 360.0 GB on disk in BF16. The card says 177B. Qwen's card for the base model says "125B with 6B activated, plus 51B n-gram embedding and 4B MTP." Reconciling:

So "177B" is the text model including the hash table but excluding the drafter and the image encoder. Qwen's "125B + 51B + 4B" is the same weights, sliced differently — Qwen's 4B for MTP counts the embedding and output head the drafter needs, which are shared.

Active parameters work the same way. Counting the dense spine, top-10 of 512 experts per layer, the shared expert, the hyper-connection mixers and the output head, I get 6.67B per token — the card's 6.6B. Drop the output head as well and it is 6.04B — Qwen's "6B". Neither is wrong; they differ by one 248,320 × 2,560 matrix.

The engram lookup contributes 2,560 parameters to that per-token figure. Hold that number.

Only 1.85% of it is new

Tinfield 1 and Qwen/Qwen3.8-Flash-Next publish identical safetensors indices: same 1,658 tensor names, same shard assignment, same 359,999,963,128-byte total. That is not a coincidence, it is an invitation — if the layout is identical, you can range-request the same byte window out of the same shard in both repositories and compare.

I did that at four offsets per tensor (0%, 33%, 66%, 99%), on early, middle and late layers, across every named tensor group.

The pattern is the cleanest I have seen on a released fine-tune. Every linear map in the attention and residual-routing path moved, in every layer sampled: the linear-attention in_proj_{qkv,z,a,b} and out_proj, the full-attention q/k/v/o_proj, and the hyper-connection input mixers. Everything else is byte for byte the base model: all 512 experts in all 48 layers, every router, the shared expert, both 248,320 × 2,560 embedding matrices, every RMSNorm, the linear-attention state dynamics (A_log, dt_bias, conv1d), the sparse-attention indexer, the entire 27-layer vision tower, the entire MTP module, and the whole 51.2-billion-parameter engram table.

That is 3,322,347,520 changed parameters out of 179,999,981,459 — 1.85%. 353.4 GB of the 360 GB you download is a file you can already get from Qwen.

This is not an accusation. Freezing the experts and training only attention is a real, defensible recipe — it is cheap, it preserves the base's knowledge exactly, and on an agentic post-train where you want to change how the model uses what it knows rather than what it knows, it is arguably the right target set. BTL-4's LoRA target map was the same shape and I said the same thing there. What I would want on the card is one sentence naming it, because the shape of the update is the single best predictor of how a fine-tune generalizes, and a reader budgeting 360 GB of bandwidth has a right to know that 98% of it is a re-upload.

What an engram table actually is

Both GGUF cards say, in a sentence that has been quoted more than it has been explained:

the engram table is 26.8 GiB and always memory-mapped on the CPU side, since a token reads only a handful of its rows.

Here is what that refers to. In config.json the base architecture carries a block nobody else's does:

"ngram_size": 3,
"ngram_vocab_size_base": 20000000,
"heads_per_ngram": 8,
"split_ngram_parts": 128,
"ple_embed_dim": 2560,
"ple_layer_ids": [2]

and in the tensor index, at exactly one layer, 128 tensors named ple.ple_embedding.ngram_embedding.shard_0 … shard_127, each (2,500,012 × 160) in BF16. That is 320,001,536 rows × 160 = 51,200,245,760 parameters, 95.4 GiB, 28.4% of the entire checkpoint, living in a single layer.

Three small I64 tensors sit next to them, and they are the whole story. I range-requested them and read them directly:

ngram_heads_vocab_sizes = [20000003, 20000023, 20000033, 20000047,
                           20000059, 20000063, 20000069, 20000077,
                           20000081, 20000093, 20000107, 20000147,
                           20000153, 20000159, 20000161, 20000171]
ngram_heads_offsets     = [0, 20000003, 40000026, 60000059, ...]
layer_multipliers       = [23703573157769, 20109073645365, 8052911324071]

Those sixteen vocabulary sizes are sixteen distinct primes just above twenty million. They sum to 320,001,446 rows; the shards hold 320,001,536, so ninety rows are padding. The offsets are their running sum. The multipliers are three odd 43–45-bit constants, one per n-gram position.

So the mechanism is a multi-probe hashed n-gram embedding. Take the last two or three token ids, hash them with a polynomial whose coefficients are those multipliers, and reduce modulo each head's own prime. Sixteen heads, sixteen different primes, sixteen unrelated collision patterns. Each head returns a 160-wide row; concatenate sixteen of them and you have exactly the 2,560-wide vector that ple_embed_dim asks for, which is then gated into the residual stream through a small conv, a key projection and a value projection at that one layer.

The distinct primes are the entire trick. A single hash table of that size would collide, and a collision would be a silent, permanent confusion of two unrelated n-grams. Sixteen independent moduli mean two n-grams that collide under P₀ almost certainly do not collide under P₁, so the concatenation averages the noise away. This is the classic hashing-trick construction — it just has not often been built at 51 billion parameters inside a frontier language model.

And the reason it can be that big is the ratio the widget ends on. The table stores 51,200,245,760 values. A token reads 2,560 of them — one 160-wide row per head. That is a gather of about 5 KB out of 95 GiB, with no matrix multiply attached. Qwen's own card is precise about why this matters:

Embeddings provide a unique axis for parameter scaling that requires less computation and is more amenable to offloading than Mixture-of-Experts.

That is the whole design. MoE sparsity still costs you a matmul per active expert and wants the expert weights near the accelerator. A hashed embedding costs a pointer chase. Which is why llama.cpp can leave 26.8 GiB of it on the CPU side and still keep the model on a 64 GB machine.

The quantization arithmetic closes, and it confirms the table

The Compact and Mini cards are the best-written part of this release, and unlike BTL-4's LiveCodeBench table their numbers reconcile exactly:

40.4 + 4.8 = 45.2, and 45.2 + 26.8 = 72.0. The Mini column closes the same way to a tenth of a GiB. The "5.2× smaller on the GPU side" figure is computed the right way too — against BF16 minus the engram, since the engram is never GPU-resident in either build, which is 239.9 GiB, and 239.9 / 45.2 = 5.3.

And the engram row is an independent confirmation of my parameter count. The recipe puts the table at IQ4_NL, which is 4.5 bits per weight. 51,200,245,760 × 4.5 / 8 = 28.8 GB = 26.8 GiB. That is the number on the card, to three significant figures, from a count I made out of tensor shapes the card never mentions.

One of the quants was measured rather than asserted, though not by BTL. The fourth GGUF repo in the org is a mirror — its own first line says "Built by needmorevramog, mirrored here" — and the quantizer who built it published the check:

expand

The denominators

Now the part that actually decides whether "ahead of Claude Opus 4.8" survives.

The card states its protocol, which BTL-4's did not: "Evaluated with mini-swe-agent at k=5 on the full task sets: 66 tasks for Terminal-Bench 4.0, 113 for DeepSWE v1.1." It does not say what k=5 means, and against a published Opus score that is the whole comparison — best-of-5 and mean-of-5 are different metrics and one is roughly twice the other on hard agentic sets.

You can settle it with arithmetic, because a benchmark with 66 tasks can only produce certain scores.

Under best-of-5 the score is (tasks solved at least once) / 66, so it must be a multiple of 1.515 points: 21/66 = 31.82, 22/66 = 33.33. 33.0 is not on that lattice. Under mean-of-5 the denominator is 66 × 5 = 330 and the step is 0.303 points; 109/330 = 33.03, which prints as 33.0. So k=5 means five repeats, averaged — the conservative reading — and the two leaderboards the card cites agree: Artificial Analysis reports Terminal-Bench 4.0 as "pass@1 averaged over three repeats per task," and DeepSWE v1.1 runs the full suite four times and reports the share of rollouts that pass.

That is a good result for the card. It also means the three runs being compared use k=5, k=3 and k=4 respectively, which nobody states.

Then the comparison table, which mixes agents — and says so, in a column:

Printing the Agent column is the honest thing to do and I want to credit it. But a reader should know what it implies: Tinfield 1 vs Claude Opus 4.8 on Terminal-Bench is not a model comparison, it is a (model, scaffold) comparison, and the scaffolds differ. The harness effect is the long version of why that matters. Here the direction is at least not flattering to Tinfield — mini-swe-agent is the deliberately minimal scaffold and Claude Code is a full agent harness — so if anything the comparison understates the model. It still is not the comparison the sentence implies.

The DeepSWE row is the stronger of the two claims, because DeepSWE v1.1 runs every model on mini-swe-agent by design. Tinfield's 62.0 against Opus 4.8's 59 is scaffold-matched; only k differs (5 vs 4). That one survives reading.

The base-model row does not match its source

The card's most important number is not 33.0. It is the pair 29.0 → 33.0, which is the only figure that isolates what Bad Theory Labs' training actually bought. The card sources it: "Base model Terminal-Bench figure from Artificial Analysis."

Artificial Analysis runs Terminal-Bench 4.0 on all 66 tasks with mini-swe-agent at pass@1 averaged over three repeats — so this is the harness-matched comparison, and that is genuinely better discipline than BTL-4 managed. But the number on AA's board today for Qwen3.8-Flash-Next is 0.252525…, which is 50/198, 25.25 — not 29.0. Every other entry on that board is an exact multiple of 1/198, so the grid is real; 29.0 is not on it either (57/198 = 28.79, 58/198 = 29.29).

I cannot tell you which way this cuts. AA re-runs models, so 29.0 may be what the board showed when the card was written and 25.25 may be a later re-measurement, or a different reasoning-effort row. What I can tell you is that the citation does not resolve against its source as of 2026-09-22, and that the discrepancy runs against Bad Theory Labs' interest: if the base is 25.25, the delta Tinfield bought is +7.75 points, not +4.0. A card understating its own result is an unusual failure mode, and the fix is the same either way — print the base-model row you measured yourself, on your own harness, at your own k.

There is a smaller thing in the same neighbourhood. The card says Tinfield is "the second open-weight model" on the Terminal-Bench board. On Artificial Analysis' board for the same benchmark, GLM-5.3 (max) sits at 41.92, MiMo-V2.6-Pro at 34.85 and GLM-5.3-Flash at 32.83 — three open-weight entries, two of them above 33.0, on the board that supplied the base-model figure. Different boards, different scaffolds, so the claim is not false on the leaderboard it refers to. It just does not hold on the one the card cites two paragraphs earlier.

The things nobody will check

Two smaller observations, both of the kind that only show up if you open the files.

The vision tower is present, untouched and unevaluated. pipeline_tag is image-text-to-text, there are 27 vision blocks, a merger, an image_token_id and a video_token_id, and every one of those tensors is byte-identical to Qwen's. The card describes a terminal and software-engineering model and reports no multimodal number. That is fine — you inherit what you did not train and do not claim it — but a reader should know that roughly 0.9 GB of the download is an image encoder nobody in this release has measured, and that Tinfield's multimodal behaviour is entirely Qwen's. BTL-4 shipped exactly the same situation six weeks ago.

Bad Theory Labs' own website still lists Tinfield as not built. badtheorylabs.com carries a "The stack" panel with four systems; the first is "Tinfield — Model family · Proposed … status: not built." The weights have been on the Hub for a day. That is a stale roadmap page rather than a contradiction, but it is the only place the lab describes what Tinfield is for, and it describes it in the future tense.

What is genuinely good here

I have spent most of this piece on denominators, so let me be direct about the parts that are better than most releases:

• The protocol is stated. Harness named, k stated, full task sets, no subsetting. BTL-4's SWE-bench row got three words; this gets a sentence with numbers in it.
• The quantization work is real and it reconciles. A 4.5-bit engram, 2-bit expert gate/up, a range-searched Q2_0 for the down projections that takes weighted error from 51.9% to 11.7%, mean KL against Q8 reported for both builds (0.287 and 0.340), and an explicit statement that "the benchmark scores are for the BF16 weights; this build has not been evaluated." That last sentence is the one most quantizers omit.
• The operational notes are earned. --jinja or tool calling breaks; -c is total context, not per slot; skip n-gram self-speculation because it accepted 16–25% of drafted tokens on agent traffic; raise iogpu.wired_limit_mb on macOS before 45.2 GiB of weights meets the default wired limit. Nobody writes that paragraph without having run the thing in a loop.
• The community build is labelled as not theirs, with its own author and an explicit "not built or measured by us."

The take

Tinfield 1 is a 3.3-billion-parameter attention-path fine-tune of Qwen3.8-Flash-Next, shipped as a 360 GB re-upload, with a quantization recipe that is the actual craft in the release and a benchmark table that is mostly honest and cites one number that does not resolve against its source.

The reusable part is the method, and it is the same four steps BTL-4 needed, now with a fifth:

1. Does the table close? Weighted means must be consistent with their parts.
2. Does the declared base exist, and does the config match? Here it matches so exactly that the tensor indices are identical files.
3. What does the config leak? ngram_vocab_size_base, split_ngram_parts and three int64 metadata tensors explain a phrase the prose leaves as jargon.
4. Do the weights agree with the story? 1.85%.
5. Does the printed score sit on the lattice its denominator allows? This is the new one, it costs one line of arithmetic, and it settled a question the card left open in the right direction.

What I would want before quoting 33.0 anywhere: the base-model row re-run by Bad Theory Labs on their own mini-swe-agent at k=5, next to Tinfield's, with the sampling parameters printed. Every other number on this card has a denominator. That one borrows someone else's.

Nothing here was executed against a model. No weights were downloaded: every measurement is arithmetic on public metadata plus HTTP range requests totalling a few megabytes against the safetensors shards of badtheorylabs/Tinfield-1 and Qwen/Qwen3.8-Flash-Next, read on 2026-09-22. Parameter counts are summed from the full headers of all 131 shards; the engram metadata tensors were read as raw int64. Benchmark figures are quoted from the Tinfield 1, Compact and Mini cards, with comparison values from Artificial Analysis and the DeepSWE leaderboard. On figures: I read all five badtheorylabs Tinfield cards on the Hub, the org profile, and badtheorylabs.com's papers and thesis pages. BTL itself publishes no image at all. The only two in the org are the perplexity chart above and a KL-divergence chart of the same pair, and both arrived with the Q4_K_XL GGUF mirror from needmorevramog, who made that quant; terms in NOTICE.txt. There is no architecture diagram, no plot of either benchmark, no video anywhere, and no chart on the main card at all: its headline numbers exist only as five markdown tables. So the score lattice and the other two interactives above are mine, because there was nothing to mirror. Earlier reads of this lab: BTL-3 and BTL-4.