# Tiel-Coder-35B-A3B: the 4-bit tier is 5.16 bits, and the only new weight is a prompt
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/tiel-coder-35b-a3b
> date: 2026-08-26
> tags: quantization, gguf, moe, local-inference, llama-cpp, explainer
A GGUF repo is a strange thing to review. There is no paper, no training recipe, no ablation. There are files. But GGUF files are self-describing — architecture, hyperparameters, per-tensor quantization type, the tokenizer, the chat template, even the absolute path of the importance matrix on the machine that made them, all sitting in a header at the front of the file. You can read every one of those without downloading 200 GB, because HTTP range requests exist and the header is at byte zero.
So I read them. [peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF](https://huggingface.co/peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF) ships nine quantization tiers of [Ornith-1.5-35B-A3B](/articles/ornith-1-5), plus an importance matrix and a vision projector. Three things came out of the headers that the model card does not put in the foreground: the tier called `Q4_K_XL` averages 5.161 bits per weight, not four; the routed experts are 93% of the parameters but only a third of the per-token compute; and the difference between "Tiel-Coder" and "a re-quantization of Ornith-1.5" is a Jinja template, not a tensor.
| | |
|---|---|
| Repo | [peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF](https://huggingface.co/peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF) · MIT · 9,831 downloads at the time of writing |
| Base | [ornith-ai/Ornith-1.5-35B-A3B](https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B), `Qwen3_5MoeForConditionalGeneration`, MIT |
| Shape | 40 blocks · **30 linear-attention + 10 full-attention** · 256 experts, 8 routed · 262,144 context |
| Parameters | **34,660,610,688** in the GGUF · 2,946,429,568 active per token (8.5%) |
| Tiers | 9, from 11.45 GiB (`Q2_K_XL`, 2.84 bpw) to 35.81 GiB (`Q8_K_XL`, 8.88 bpw) |
| Extras | `mmproj-BF16.gguf` (0.84 GiB, unquantized) · `Tiel-Coder-35B-A3B.imatrix.gguf` (510 entries, 3,000 chunks) |
| Not shipped | the vision tower inside the text GGUFs, and the MTP head — 733 tensors per tier, no `nextn` |
| Runs on | stock `llama.cpp` — `LLM_ARCH_QWEN35MOE` is upstream in `src/llama-arch.cpp` |
## Reading the files without downloading them
Every GGUF opens with the magic `GGUF`, a version, a tensor count, a KV count, then the key-value block, then one record per tensor giving its name, shape, ggml type and offset. Fetch the first few tens of megabytes with `Range: bytes=0-33554432` and you have all of it.
The nine tiers share a byte-identical header of **11,010,605 bytes** — which is mostly the tokenizer: 248,320 token strings, 247,587 BPE merges, and a 29,157-character chat template. After that the tensor table differs, and that is where the interesting part lives.
The first thing I did was check the arithmetic. For each tier, compute every tensor's size from its shape and ggml block layout (`Q4_K` is 144 bytes per 256 weights, `Q5_K` 176, `Q6_K` 210, `Q8_0` 34 per 32, `IQ2_XS` 74 per 256, `IQ3_XXS` 98, `IQ4_XS` 136), sum, add the header:
```
tier Σ tensors + header published size Δ
Q2_K_XL 12,290,649,133 12,290,649,152 19
IQ3_XXS 13,211,175,981 13,211,176,000 19
Q3_K_XL 16,845,532,205 16,845,532,224 19
IQ4_XS 17,730,530,349 17,730,530,368 19
Q4_K_S 20,893,035,565 20,893,035,584 19
Q4_K_XL 22,360,476,717 22,360,476,736 19
Q5_K_XL 26,592,529,453 26,592,529,472 19
Q6_K_XL 31,843,798,061 31,843,798,080 19
Q8_K_XL 38,451,203,117 38,451,203,136 19
```
Nineteen bytes of alignment padding, every time. The element count also comes out identical on all nine: **34,660,610,688** parameters. That is the number the Hub reports as `gguf.total` for this repo, and it is exactly the full base checkpoint (35,951,822,704 parameters, from the safetensors headers) minus the vision tower (446,571,248) minus the MTP block (844,640,768).
So the files are what they say they are, the two removed components are genuinely removed, and I can now trust every per-tensor number that follows.
The safetensors headers are readable the same way: the first 8 bytes of each shard are a little-endian `u64` giving the JSON header length, and that JSON lists every tensor's dtype and shape. Sixteen range requests gave me the full parameter census of a 72 GB checkpoint without pulling a single weight.
## Does A3B actually mean 3B?
`config.json` says `num_experts: 256`, `num_experts_per_tok: 8`, `num_hidden_layers: 40`, `hidden_size: 2048`, `moe_intermediate_size: 512`. The GGUF key-value block echoes all of it — `qwen35moe.expert_count 256`, `qwen35moe.expert_used_count 8`, `qwen35moe.block_count 40`. But a config file is a claim. The tensor shapes are the fact.
One expert is `gate_up_proj[256, 1024, 2048]` and `down_proj[256, 2048, 512]`, so 2,097,152 + 1,048,576 = **3,145,728 parameters per expert**. Forty blocks of 256 gives 32,212,254,720 — 93% of the whole file. Routing eight of them per token gives 1,006,632,960.
Add the parts that always run — 30 linear-attention blocks at 33,718,464 each, 10 full-attention blocks at 27,263,488, the per-block shared expert, the router, and a 508,559,360-parameter output head against a 248,320-token vocabulary — and the active budget is **2,946,429,568**. A3B checks out, to within a rounding of the label.
What I did not expect is the composition. The mixture is only 34% of the active budget. An equal share is linear attention, because `layer_types` puts 30 [gated-delta blocks](/articles/ltc-gated-delta) in the stack and dense means dense — every one of them runs on every token. Another 17% is the output projection. The sparsity everyone talks about applies to a third of the work.
That has a practical consequence I will come back to: the part of this model that is *not* sparse is also the part that stays at 8.5 bits in every tier.
## The ladder buys fit, not speed
Nine tiers spanning 11.45 to 35.81 GiB. The card's `fits` column is a judgement call, so I recomputed it from the byte counts, the real KV geometry, and a stated allowance for `llama.cpp`'s compute buffers.
KV first, because this architecture makes it cheap in a way the file size hides. Only 10 of 40 blocks do full attention, and those have `num_key_value_heads: 2` with `head_dim: 256`. So K and V cost `10 × 2 × 256 × 2 × 2` bytes = **20 KB per token** at fp16 — 5.0 GiB at the full 262,144 context. The other 30 blocks hold a constant `32 × 128 × 128` fp32 recurrent state each, about 64 MiB in total, which does not grow with context at all. The card credits "only 2 KV heads" for this; the larger factor is that three quarters of the blocks have no KV cache to grow.
The card's `fits` column survives the check almost exactly. `Q4_K_XL` at 20.82 GiB really is snug-but-fine on a 24 GiB card — about 116k tokens of context left over, which is more headroom than "snug" suggests. The one line I would soften is `Q6_K_XL`: the card says it "will not leave usable context on 32 GB", but the arithmetic gives about 73k tokens, which is usable. Load the vision projector as well and it drops to roughly 30k — at which point the warning is fair.
The more interesting number is the one the ladder does not advertise. Reading weight bytes per decoded token — the thing that actually bounds decode on a bandwidth-limited GPU — the ladder spans 1,651 MiB (`Q2_K_XL`) to 3,129 MiB (`Q8_K_XL`). That is 1.9x, against a 3.1x spread in file size. Going from `Q8_K_XL` to `Q4_K_XL` cuts the file 42% and cuts per-token weight traffic 16%.
You do not quantize this model to make it fast. You quantize it to make it fit. Once it fits, the tier barely matters for throughput, because the bytes you kept are the ones you read every token.
And if it does not fit, the escape hatch is real: everything that is not a routed expert comes to 1.6 to 2.5 GiB across the whole ladder. Push the routed experts into system RAM with `--n-cpu-moe` and every tier fits a 12 GiB card, at the cost of moving 314 to 1,065 MiB of expert weights per token across whichever bus you have. That is precisely the regime [FreeToken](/articles/freetoken) argues you should measure rather than assume, and the numbers here say why: at `Q4_K_XL` it is 586 MiB per token, which a PCIe 4.0 x16 link turns into a hard ceiling somewhere around 40 tokens/second before any compute happens.
## What "dynamic" actually changes
Every tier is quantized against an importance matrix the author generated rather than borrowed, and cut with per-tensor recipes on top. Both claims are checkable from the headers, and both hold.
The imatrix ships in the repo. Parsing it: 1,020 GGUF tensors, which is 510 entries × two arrays each (`counts` and `in_sum2`), `imatrix.chunk_count: 3000`, `imatrix.chunk_size: 512` — 1,536,000 calibration tokens, matching the card's "3,000 chunks of 512 tokens". Every shipped tier carries `quantize.imatrix.entries_count: 510` and `quantize.imatrix.chunks_count: 3000` in its own header, so the matrix in the repo is the matrix that cut the files.
Those 510 entries are exactly the quantizable 2-D tensors of the model: 100 attention projections, 320 FFN matrices, 90 SSM projections. Two are conspicuously absent — `token_embd` and `output`. The calibration pass never measured the output head, which is consistent with it being pinned at Q6_K or Q8_0 in every tier rather than quantized on evidence.
Now the recipe. The tier name describes three tensor classes out of nine. In `Q4_K_XL`, the gate and up projections of the routed experts are Q4_K, the down projection is Q5_K, and the attention, the gated-delta output, the shared expert, the embedding and the output head are all Q8_0. The router — the 256×2048 matrix per block that decides which experts run — is left at F32 in every tier, including the 2-bit one. Weighted over the real parameter counts, that is 4.53, 5.61 and 8.50 bits per weight respectively, and 5.161 for the file as a whole.
Which is the honest answer to "is this a 4-bit model": the experts are, and they are 93% of it, and the average is 5.16.
Two of the labels do not survive inspection at all. `Q2_K_XL` contains **zero** `Q2_K` tensors — its experts are `IQ2_XS` and `IQ3_XXS`, and its widest tensors are `Q5_K` and `Q6_K`. `Q8_K_XL` contains zero `Q8_K` tensors — it is `Q8_0` with eleven `BF16` promotions. And `Q4_K_S` and `Q4_K_XL` both declare `general.file_type: 15`, which is `Q4_K_M`, despite differing by 1.47 GB. The names are a size ordering, not a description.
The per-block strip in that widget is the part I find most convincing. Across nine independently-cut tiers, the same blocks get promoted: **block 1** gets a wider cut in six of the nine tiers, and **blocks 34, 38 and 39** have their down projection promoted in eight of nine. Not block 0, not a random scatter — the same four indices, tier after tier. That is what an importance matrix looks like when it is actually driving the decision rather than decorating the README. Against a flat recipe at the tier's own expert width, the promotions cost 2.57 GiB on `Q4_K_XL`: 14% more file for a handful of matrices.
One human detail survives in the metadata: every tier carries `quantize.imatrix.file: /Volumes/Vault/ai/models/local/tiel-1.5-35b-a3b.imatrix.gguf`. This was cut on somebody's Mac, off an external volume.
## The benchmarks, and what n = 25 can hold
The claim is "ties Opus 4.6". The point estimates do tie: 12 of 25 each. The question is what 25 problems, run once, can distinguish.
A Wilson 95% interval on 12/25 runs from **30.0% to 66.5%**. Against the base model's 8/25, Fisher's exact test gives p = 0.39. Against Nail's 9/25, p = 0.57. Against the dense Qwen3.8-27B's 16/25 — which the card presents as clearly ahead — p = 0.39 in the other direction. On this sample, the only defensible statement is that all seven local models land in a band the experiment cannot resolve.
To its credit, the card says "one run each" in the footnote and repeats "treat small differences as noise" in the limitations. I would go further: with n = 25 and one seed, essentially every difference on that chart is small.
The speed claim needs the same care, and here the card's own numbers contain the correction. "The lowest mean time per attempt of the 35B-A3B family" is true: 12.3 minutes against 14.2, 15.7 and 27.5. But the median tells a different story — 8.6 minutes, slower than stock Qwen3.6-35B-A3B (5.5), KAT-Coder (6.8) and Nail (7.2). The mean is lower because Tiel lacks the others' tail of runaway attempts, not because a typical attempt is quicker. "Faster" is a statement about the tail. The card draws both bars and labels them, which is more than most do; the headline picks the flattering one.
There is also a harness confound that the footnote discloses in one line and then moves past. The local models ran through "the Pi coding agent"; the cloud arms ran through Claude Code at medium reasoning effort. Nail ran on MLX, Tiel and Ornith on `llama.cpp`. So "ties Opus 4.6" compares two models through two different agent scaffolds, and "beats Nail" compares two quantizations through two different runtimes. [The harness effect](/articles/harness-effect) is large enough to swamp a four-problem difference on its own.
The multi-turn card is the one I would trust most, because it exposes its own formula and the arithmetic reproduces: 0.2 × 46.5 + 0.8 × 72.4 = 67.22, 0.2 × 51.5 + 0.8 × 68.7 = 65.26, 0.2 × 34.5 + 0.8 × 67.0 = 60.50. It also states the trade against its own base plainly — up 3.8 on answers, down 5.1 on clarifying questions — and notes that a reader who weights clarification differently gets a different winner. That is the right way to publish a composite.
## The part that is not a weight
Here is the finding that reframes the whole release.
`Ornith-1.5-35B-A3B` ships a `chat_template.jinja` of 7,536 bytes, 150 lines. The template baked into every Tiel tier is 29,157 characters, 453 lines, and identifies itself as `qwen3.8-froggeric-v22.4.0`. Pull it out of the GGUF header and read it, and lines 168 to 188 are doing the work:
```jinja
{%- set _terse_lead = 'You are Tiel-Coder, a variant of Ornith-1.5-35B-A3B.
Answer directly, after thinking. Lead with the answer, then only what it
needs to be correct and usable.' %}
{%- set _terse_core %}
Never: open with preamble or pleasantries; restate the question; add filler
transitions; hedge with niceties; or repeat a point you've already made.
Always: keep essential steps, caveats, uncertainties, and specifics ...
{%- endset %}
{%- set _terse_on = terse if terse is defined else true %}
```
That block is prepended to an empty system prompt, or appended to yours if you supply one. It is on by default. The rest of the 453 lines are a genuinely useful piece of engineering — two selectable tool-call encodings, a `reasoning_effort` control the base template does not have, truncation caps for oversized tool arguments and responses, an injected warning after a failed tool call — but the thing that changes how the model answers is a system prompt smuggled into the chat template.
The card is upfront about this: "We changed the chat template, not the weights." And it does the experiment that proves it.
That third bar is the right experiment, and it is rare to see it published. It isolates the quantization from the template, and it says the quantization costs nothing measurable while the terseness prompt costs 4.3 points of MMLU-Pro for 20% shorter answers.
Two caveats on it. First, the sample: 100 questions is 0.83% of MMLU-Pro's 12,032. The quoted `±2.3` and `±2.6` are the spread across three seeds; the uncertainty from *which* 100 questions is about ±4.4 points at that accuracy and is not in the bars at all. It cancels in a paired comparison on identical questions, which is what this is — but it does not cancel if you carry "73.7" over to any other MMLU-Pro number you have read. Second, on the seed spread alone the 4.3-point drop is about 2.15 standard errors. Directionally supported by the token counts, not established.
If the terse block conflicts with your agent's own system prompt, the template takes `{"terse": false}` via `chat_template_kwargs` and serves you the model with your prompt only. That option is documented in a Jinja comment inside the GGUF and nowhere else, which is a good argument for reading the template of any model you deploy.
## What a GGUF-only release can and cannot tell you
It can tell you a lot. In one afternoon of range requests I confirmed the architecture, the expert count, the routing width, the context length, the rope base, the KV geometry, the exact parameter census, the per-tensor quantization recipe of nine tiers, the calibration corpus size, and the entire chat template — and I reconciled every file to within 19 bytes.
I also confirmed three of the card's structural claims against the artefacts:
- **No MTP head.** 733 tensors per tier, no `nextn` block. The claim that it was stripped is true; whether it was untrained when they stripped it is not visible from these files, because the tensors are not in them.
- **The vision projector is the base model's.** `mmproj-BF16.gguf` sums to **446,571,248** elements, which is the base checkpoint's vision tower to the parameter, and every weight in it is BF16. Identical element count is not bit-identity, but it is consistent with a straight pass-through, and it means the vision path is genuinely unquantized on the 2-bit tier as well as the 8-bit one.
- **It loads on stock `llama.cpp`.** `LLM_ARCH_QWEN35MOE` is registered in `src/llama-arch.cpp` on master, along with every KV key these files use — `full_attention_interval`, `ssm.inner_size`, `ssm.group_count`, `ssm.time_step_rank`, `expert_shared_feed_forward_length`. No patched build required.
What it cannot tell you is everything about behaviour. There are no eval logs, no per-problem results, no harness in the repo, and the two benchmarks that carry the headline — "SWE-bench-Live through the Pi coding agent" and "Claw-Eval" — have no public artefact I could reach from the model card. Every number on those four PNGs is self-reported and unreproducible from what ships. That is not an accusation; it is the normal state of a quantization repo, and this one discloses more of its method than most. It is just the boundary of what reading files gets you.
I also could not verify that the tiers were cut from the BF16 source rather than from a Q8_0 intermediate. The card says the imatrix was measured on a Q8_0 and the tiers cut from BF16, which is the right way round, and nothing in the headers contradicts it — but nothing in the headers confirms it either.
One last piece of metadata worth not trusting: `general.size_label` reads `256x2.6B` on every tier. Two hundred fifty-six times 2.6 billion is 665.6 billion. The housekeeping fields — `general.name`, `general.version: '4000'`, `general.finetune: '35b'` — are derived from a directory name somewhere and are cosmetically wrong. The architecture keys underneath them are all correct. Read the `qwen35moe.*` block, ignore the `general.*` block.
## The ledger
**Genuinely good work.** The per-tensor recipe is real and legible: the same four blocks promoted across nine independently-cut tiers is an importance matrix doing its job, not a marketing word. Shipping the imatrix itself, with the chunk count in its own header, means anyone can cut a tier that is not in the table. And the MMLU-Pro control arm — the same quantization carrying the vendor's template — is the experiment most quant repos skip, published in the one place it hurts the product's story.
**Convergent, not novel.** Pinning attention, the shared expert, the output head and the router high while the routed experts absorb the bit reduction is the Unsloth Dynamic pattern, and the card credits it as such. The 8.5-bit frame with 4.5-bit experts is now roughly the house style for MoE GGUFs; what varies is which specific blocks get promoted, and this repo makes that visible.
**What I would check before believing.** Every benchmark on the four cards is n = 25 one-shot or n = 100 × 3 seeds. The Wilson interval on the headline result spans 30% to 66%. The cloud comparison runs a different harness from the local one. If any of those numbers matter to a decision you are making, run your own — the model is MIT and the tier that was benchmarked is a 22 GB download.
**What I would watch.** Whether the "model" in a release like this keeps drifting toward the prompt layer. Nothing here is dishonest — the card says the weights are unchanged, and proves it with a control arm — but the artefact being distributed as `Tiel-Coder-35B-A3B` is Ornith's weights plus 453 lines of Jinja, and the benchmarks that distinguish it from its base are measuring the Jinja. That is a legitimate thing to publish. It is also a thing worth naming, because a system prompt inside a GGUF is a system prompt you will not see in your logs.
**The number I will actually use.** 20 KB of KV per token, 2.0 GiB of always-resident weights, 586 MiB of expert traffic per token at `Q4_K_XL`. Those three, plus [the offloading arithmetic](/articles/freetoken), tell you exactly what this model will do on your hardware — which is more than any of the four benchmark cards will.