Penjing-27B: reading a quantisation claim out of the GGUF header

A GGUF release is one of the few artefacts in this field that cannot lie to you. The filename is marketing; the tensor-info block at the top of the file is the build. Every tensor's name, shape and ggml type id is sitting there in the first few megabytes, and a Range: bytes=0-8388607 request is enough to read the whole allocation without downloading 6 GB of weights.

So when PollardWeights/Penjing-27B-Pollard appeared today with a ladder of quants of Qwen/Qwen3.8-27B and the line "a ladder of measured-allocation quants (bits placed by per-layer sensitivity, not a uniform crush)", the useful thing to do was not to read the card. It was to read the headers.

This article is about the method, not the model. Qwen3.8-27B has been covered elsewhere; what is new here is Pollard Weights, the toolkit that produced these files, and whether "measured allocation" names something specific enough to check. It does, and it checks out partly.

What "Pollard" is

The repo never says, so this is a reading rather than a fact: pollarding is the horticultural practice of cutting a tree back to the same knuckles year after year so it keeps its shape at a size you choose, which fits a tool whose pitch is "tell it the target's budget and it fits that." Not a person, not Pollard's rho. The model here is called Penjing, the Chinese art the Japanese word bonsai descends from — and this site has already covered a ternary Bonsai built on the same base model, which turns out to be the most useful comparison available. More on that below.

The mechanism, per the toolkit's own README, is a two-signal pipeline:

1. pollard-sensitivity points at a model and measures which tensor groups matter, by crushing each group one at a time and watching the KL against f16, plus that model's own noise curve.
2. pollard-fit --sensitivity then allocates bits to minimise KL for a stated size budget — --ram/--vram of the target machine, minus a working reserve, rather than a bit-width chart.

The README's claimed wins for that procedure are +6–27% lower KL on a dense 1.5B (5 of 5 sizes) and +21–43% on a 40-expert MoE (4 of 5), against "uniform imatrix-IQ at matched size", measured on 48K held-out wikitext tokens. Here is its own chart:

expand

Two things about that figure matter for what follows. It is real evidence, published with the script that regenerates it from raw data — python experiments/plot_kl_win.py — which puts it ahead of most quantisation marketing. And it is not evidence about this release: different models, different sizes, a different architecture class. The README is careful about this in a comment aimed at readers like me:

DENSE models -> imatrix-guided K-quants (IQ3_S / IQ4_XS / Q6_K). The importance matrix is the dense win; the per-layer measured-KL knapsack does NOT pay off on dense (no expert redundancy to reallocate from), so we don't claim it there. MoE models -> pollard-automap measured expert-allocation. Rule of thumb: imatrix = dense, automap = MoE.

Qwen3.8-27B is dense. config.json has no num_experts field at all: 64 blocks, hidden 5120, intermediate 17408, 24 query heads over 4 KV heads, and a hybrid attention stack — three gated-delta linear-attention blocks for every one full-attention block (full_attention_interval: 4). By the toolkit's own rule of thumb, this release is on the imatrix path, and the per-layer knapsack is the leg the toolkit says does not pay off here.

So what does the per-layer structure in these files actually look like?

The allocation, read off the files

Four of the six advertised rungs were in the repo when I checked at 15:46 UTC on 2026-09-22. I pulled the header of each — Range requests against huggingface.co/.../resolve/main/<file>.gguf, parsing magic, version, tensor count, the metadata KV block and then the tensor-info array — and resolved the ggml type ids, including ik_llama.cpp's extended range (153 = IQ2_KT, 158 = IQ1_KT), against ik_llama.cpp's own ggml.h. Every file reports the same 866 tensors and the same 27,320,697,856 elements.

The three low rungs have exactly one piece of per-layer structure, and it is the same piece in all three: blocks 0, 1, 63 and 64 keep the rung's primary type; blocks 2 through 62 have ffn_gate and ffn_up demoted to IQ1_S. Not a gradient, not a knapsack boundary that moves with the budget — one binary split, at the same four indices, at 1.91 bpw and at 2.12 bpw and at 2.15 bpw. Block 64 is not a transformer block at all: qwen35.nextn_predict_layers is 1, so it is the multi-token-prediction head, and it is kept at Q6_K (or Q2_K in the smallest build).

A KL-minimising allocation re-solved under three different size budgets would not normally produce the identical keep-set three times. A first-and-last-blocks heuristic would, and first-and-last is the oldest protective rule in llama.cpp's own mix.

The IQ3_S rung has no per-layer structure whatsoever. All 64 blocks get IQ3_S on ffn_down, ffn_gate and ffn_up; the only departures from uniform are by role — attn_v and attn_qkv bumped to Q4_K, token_embd to Q4_K, output to Q6_K — and those are the shape llama.cpp produces by itself for MOSTLY_IQ3_S on a model whose n_gqa is at least 4. Qwen3.8-27B's is 6. If there is a Pollard-specific allocation in the IQ3_S file, it is not visible in the tensor types.

The one role-level choice that is genuinely its own

There is a departure from stock, and it runs the wrong way. In the three low rungs attn_k and attn_v are demoted to IQ1_S on 15 of the 16 full-attention blocks, while attn_q and attn_output keep the primary type. llama.cpp does the opposite: for every IQ2-class file type it bumps attn_v to Q4_K when n_gqa >= 4, on the standard argument that K and V are tiny and sensitive.

Tiny is the operative word. On this model attn_k and attn_v are 5120 × 1024 each — 4 KV heads at head_dim 256 — 5.24M parameters per copy, so 78.6M across the 15 demoted blocks for attn_k and the same again for attn_v. Dropping each group from 2.125 bpw to 1.5625 bpw saves 5.5 MB, 11 MB out of a 7.34 GB file: 0.15%. The same demotion applied to ffn_gate and ffn_up over 61 blocks moves 10.87B parameters and saves 765 MB, 10.4% of the file. One of those is the size lever. The other is a rounding error bought with the most sensitivity-critical tensors in the stack.

That is the kind of thing a per-tensor KL sweep is supposed to prevent, and it is the single clearest place where I would want to see the sweep's output rather than the claim.

Nothing weighs what its name says

Measured bits-per-weight, computed from the same headers — element count times the ggml block size, summed, divided by the parameter count:

The gap is almost entirely two tensors. output.weight — the LM head, 1.27B parameters over a 248,320-token vocabulary — is Q5_K in all three low rungs, 874 MB, 13.4% of the 6.53 GB file. token_embd.weight is Q2_K, another 417 MB. Together they are 19.8% of the smallest build, at an average of 4.0 bits per weight, inside a file whose name says 1.75.

Protecting the LM head at low bit is correct and I would do the same. The point is narrower: a reader picking a rung by its name is not picking the precision of a fifth of the file, and the card's "which file should I choose?" section — which is written in RAM budgets, sensibly — does not say so either.

Two smaller things fall out of the same headers. The imatrix metadata is preserved in every file: quantize.imatrix.entries_count 496, chunks_count 510, and quantize.imatrix.dataset reading C:\pollard\phome\gemma4_calib.txt. The card describes the calibration corpus as "a mixed-domain corpus so the matrix sees every register the model serves"; the artefact says it is a calibration file carried over from a Gemma-4 build. Those are compatible — calibration text is text, and reusing a good corpus across models is normal practice — but the filename is the more informative of the two descriptions, and it is the one that ships inside the file. The builds were also made on Windows, which is why the IQ1_KT build's header is 18 MB and the IQ3_S build's is 11 MB.

The part that deserves straightforward credit

Most GGUF repos publish a size column and nothing else. This one publishes perplexity against an f16 reference, mean KL divergence, and top-1 agreement — the share of tokens where the quantised model's argmax matches what f16 would have picked, which is the number that actually predicts whether a build feels broken. llama-perplexity prints it as Same top p and almost nobody reports it.

The numbers are also not flattering, which is the tell that they were measured rather than chosen:

Read the top-1 column rather than the perplexity one. At the recommended default rung, one token in five is not the token f16 would have produced. A mean KL of 0.42 nats is a large distributional change. The card says as much implicitly by calling IQ2_KT "recommended default" rather than "near-lossless", and its errata line — "Single machine; replication invited" — is the right register. But an 87%-smaller file that disagrees with the original on 21% of tokens is a different product from a compression, and the honest framing is that this ladder is for people who want a 27B-class model on a 10 GB machine at all, not for people who want the 27B model in less space.

What 2.12 bpw costs this model on actual tasks

Perplexity and top-1 agreement are cheap to measure and weakly related to whether a model can still do a job. For this particular base model there happens to be a better answer already on the site. PrismML's Bonsai 2 whitepaper evaluates Qwen3.8-27B at IQ2_XXS — the same nominal quant type as Penjing's second rung, on the same base model — across 20 benchmarks in thinking mode, and I walked through that table in September. The relevant column:

That is a different IQ2_XXS file — different builder, different imatrix, almost certainly a different mix — so it is a proxy and not a measurement of Penjing's build. But it is the same model at the same nominal type and near enough the same size, and it says something the perplexity table cannot: at this bit budget the damage is not spread evenly. GSM8K barely notices. GPQA Diamond loses a quarter of its score. A "+31% perplexity" summary and a 25-point GPQA drop are the same file.

This is also why Bonsai is the right foil. It lands at 1.72 bits per weight and 5.95 GB on the same base — smaller than Penjing's 1.91 bpw / 6.53 GB rung — and claims 83.9 on the same 20-benchmark aggregate against FP16's 85.4. The difference is not a better allocator. It is quantisation-aware training: Bonsai retrained the model to live in ternary weights, and Penjing is post-training quantisation of a checkpoint that was never asked. Post-training quantisation at under two bits and QAT at under two bits are different products that happen to produce files of the same size, and no allocator closes that gap.

Also worth stating plainly: IQ3_S, IQ4_XS and Q6_K have no published quality number, and as of 15:46 UTC on 2026-09-22 IQ4_XS and Q6_K had no published file — the card's "which file should I choose?" section recommends Q6_K for stock llama.cpp, and llama-server -hf PollardWeights/Penjing-27B-Pollard:Q6_K does not resolve. The repo was mid-upload: the four GGUFs landed between 05:44 and 13:54 UTC the same day. That is a timing observation, not a criticism; by the time you read this the ladder may be complete. It is in the article because the three rungs with numbers are the three smallest, and the shape of the quality curve above 2.15 bpw is exactly the region a buyer cares about and nobody has measured.

What I would run next

All of it is cheap and none of it needs a new idea.

1. The sensitivity sweep's output, as a file. pollard-sensitivity produces a per-tensor KL ranking. Ship it next to the GGUFs. It converts "bits placed by per-layer sensitivity" from a claim into an artefact, and it would immediately settle whether the 0/1/63/64 keep-set is what the sweep said or what the default said.
2. An ablation on attn_k/attn_v. Build one more IQ2_KT with K and V left at the primary type. It costs 11 MB. If top-1 agreement moves by more than a point, the demotion is a bad trade at any budget; if it does not, that is a genuinely interesting result about hybrid-attention models and worth a note of its own.
3. PPL and top-1 for the upper rungs, which is one llama-perplexity run per file.
4. The same three numbers for stock llama.cpp IQ2_XXS at matched size, produced with the same imatrix. That is the comparison the toolkit's own chart makes on other models and the one this repo does not make on this one.