Hy4 preview: frontier of five, not frontier of seven

mdjsonmcpcopy↗ai

2026-08-28 · 18 min · mixture-of-experts · sparse-attention · long-context · benchmarks · hunyuan · quantization

Why read this

Notable· top 60%

Sums all 131 shard headers to confirm 770B/49B, shows IndexCache deletes indexer weights, and finds the GGUF's accuracy table has no primary source.

• Checked against the source
• Widely used
• Concrete numbers to act on

·LLM architecture·Needs a workstation GPU·Apache-2.0·Practitioner model

How this was scored

Is it new?

1 of 3: An incremental tweak

Can I trust it?

3 of 3: Reproduces the headline result, or shows from primary files it is wrong

Can I run it?

2 of 3: Open code or weights with real limits

Will I understand it?

2 of 3: Mechanism from first principles with figures

Can I act on it?

2 of 3: A concrete recipe, numbers or comparison

Will it last?

1 of 3: Relevant for months

Does it affect many?

2 of 3: A widely used model, tool or lab release

Only here?

2 of 3: A teardown or measurement few others did

Score 67 of 100, ranked 185 of 577 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored

There's no arXiv id attached to Hy4 preview. Tencent's Hy team shipped a Hugging Face repo, a matching GitHub mirror, and a research-page splash screen that's a client-rendered shell with nothing in it once you strip the JavaScript — so the model card itself, config.json, and the safetensors headers of all 131 weight shards are the primary source here, the same way an arXiv PDF would be for a paper. That's not a knock: the model card is unusually rigorous as these things go, and most of what follows is confirming that rigor from the raw files rather than catching it out.

Hy4 preview is a 770B-parameter, 49B-active Mixture-of-Experts model — Tencent's own numbers, and they check out to three significant figures once you sum every tensor's actual shape. What's more interesting than the size is a specific sentence in the card: "enough to put Hy4 preview at the open-source frontier." That's a narrower claim than "the frontier," and the model card's own 41-row benchmark appendix — cross-referenced against its 12-panel headline chart — shows exactly why the qualifier is there and exactly how much weight it's carrying.

The architecture checks out, down to which layers own an index

config.json names three techniques the card describes in prose but never diagrams: Gated DSA, IndexCache, and iHC. All three are visible directly in the tensor shapes.

Gated DSA is DeepSeek Sparse Attention (use_dsa: true) with an added elementwise gate on the attention output — self_attn.linear_gate.weight, shape [16384, 6144], sized to match the o_proj input exactly (num_attention_heads 64 × v_head_dim 256 = 16,384). Multi-latent compression sits underneath it: q_a_proj down to a 2048-dim latent, kv_a_proj_with_mqa down to 576 (512 KV-latent + 64 RoPE), both later expanded back out — the same MLA shape DeepSeek popularized, with a gate layered on top.

IndexCache is the more interesting mechanic, because it isn't just a runtime shortcut. indexer_types in config.json marks each of the 78 layers full or shared: 21 layers compute their own sparse-attention index (self_attn.indexer.{wk,wq_b,weights_proj,k_norm}), and the other 57 are supposed to reuse a nearby full layer's index instead of computing one. Checked directly against the safetensors headers rather than trusted from the config flag: layers 0, 1, and 5 (all marked full) do own self_attn.indexer.wk.weight; layers 2, 3, 4, and 6 (all marked shared) do not have that tensor in the checkpoint at all. IndexCache doesn't skip a computation at inference time — it removes the weights that would do that computation from the model entirely. At ~9.37M parameters per full-layer indexer, that's 534.2M parameters that simply don't exist because 57 of 78 layers were never given their own.

iHC (identity Hyper-Connections) is the "Residual Streams: 4" line in the spec table, and it shows up as hc_fn tensors shaped [8, 24576] per layer — 24,576 is exactly hidden_size × 4, confirming four parallel residual streams per layer rather than the usual one.

Route this same 78-layer structure through the MoE side and the concentration is stark: 96.6% of the backbone's 769.907B parameters sit in the 256-expert routed banks, and only the 8-of-256 a token actually routes to (23.253B) ever fires. That's a slightly higher concentration than the two MoE models covered on this site previously — PhoneLLM's 93.0% and VoiceMem's base model at 92.9% — consistent with the general trend of scaling MoE capacity mostly through the expert count rather than the always-on backbone.

130 to 483 percent generational gains — and a frontier claim read at full precision

The card's own headline chart plots 12 benchmarks, Hy4 preview against Hy3 (its own predecessor, shown as a lighter segment on the same bar) and six named models: Qwen 3.8 Max, DeepSeek V4 Pro 0813, GLM 5.3, Kimi K3, GPT 5.6 Sol, and Claude Opus 5.

expand

Every one of the twelve Hy3-to-Hy4 jumps is real and large — DeepSWE goes from 28.0 to 64.3 (+130%), ProgramBench from 3.0 to 17.5 (+483%), HorizonMath from 3.5 to 8.8 (+151%). Nobody needs help finding the generational story here; it's printed on the chart in two-tone bars.

The more careful question is what "frontier" means once six competitors are on the same axis. The card's own text is precise about this — "open-source frontier," not "frontier" — and that qualifier turns out to be load-bearing. Cross-referencing the chart against the appendix table's typed values (which is what the bars are actually plotting) and ranking Hy4 preview against just the four other open-weight models gives a very different picture than ranking it against the full seven, closed models included.

Against the open-weight field specifically, Hy4 preview leads on 5 of these 12 benchmarks and never falls below 4th of 5 on the rest — a genuinely strong showing, and the claim as written holds up. Widen the comparison to include GPT-5.6-Sol and Claude Opus 5, and the lead count drops to 1 of 12. Both numbers are true about the same chart; the sentence in the card is careful enough that only the first one is actually being claimed.

The footnote that disciplines its own comparison

Eight of the benchmarks in the appendix table are grouped under "Reasoning," and a footnote at the bottom of that table says something most vendor benchmark tables never volunteer: "Among the eight reasoning benchmarks, five Claude Opus 5 results use high-setting runs... The high-setting runs have truncation rates of 2.32% on HLE, 15.17% on ArXivMath, 28.76% on HorizonMath, 5.86% on MathArena Apex 2025, and 17.17% on BrokenArXiv." A truncated run hit its token or turn budget before finishing, and on a benchmark that requires a long derivation, an unfinished answer usually just scores as wrong — so a high truncation rate plausibly understates what that model could do with more room.

This is Tencent disclosing a limitation of their own comparison, not something dug out of the fine print against their interest — and it's worth taking that disclosure seriously in both directions. On four of the five benchmarks with this caveat, Claude Opus 5 beats Hy4 preview anyway, truncation and all — the caveat doesn't erase the gap. The one benchmark where Hy4 preview comes out ahead, HorizonMath, is also the single highest truncation rate of the five. That's not proof the result would flip with a larger budget; it's a reason to hold that particular win more loosely than the four losses next to it, using exactly the caveat the card itself supplied.

163 experts, 203 tasks, and what an internal eval can tell you

Public benchmarks aside, the card reports a blind side-by-side: "163 internal experts rated model outputs on 203 engineering tasks." Hy4 preview scored 2.99 against GLM 5.3's 2.92 (46.8% wins / 12.8% ties / 40.4% losses) and 2.99 against Kimi K3's 2.94 (51.2% wins / 7.9% ties / 40.9% losses).

There's no released task set or transcript here, so this number can't be re-run the way a public benchmark can — "163 internal experts" has to be taken on trust in a way the rest of this page doesn't ask for. What's checkable is the arithmetic of what was disclosed, and both triples sum to exactly 100.0%. Read past the win-rate headline, the picture is a real but narrow edge — a 6.4-point and 10.3-point win-minus-loss margin against models that still win the individual comparison over 40% of the time — not the blowout a bare "51.2% wins" might suggest on its own.

What ships, and what Tencent says it doesn't do yet

The deployment story is unusually concrete for a preview release: dedicated container images (vllm/vllm-openai:hy4-preview, lmsysorg/sglang:hy4-preview) rather than a request to add support upstream, native MTP-based speculative decoding wired into both serving stacks out of the box, and a full finetuning pipeline shipped alongside the weights. transformers_version: 5.16.2 in config.json and no auto_map/trust_remote_code requirement in the README both point to HYV4ForCausalLM being a native architecture in a recent transformers release rather than custom modeling code bundled with the repo — a smaller thing than the headline numbers, but it's the difference between "clone this repo to run it" and "pip install and go." The license itself is a plain Apache 2.0, with no upstream-license wrapper to track — simpler than the layered NVIDIA-plus-BSD terms on PhoneLLM Alpha 1, since Hy4 preview isn't a fine-tune of someone else's checkpoint.

The card's own "Known Limitations" section is worth quoting rather than paraphrasing, because it's more candid than most: "we are shipping with known issues — among them, spending longer than necessary reasoning through complex tasks, and a tendency to over-verify its own work." That second failure mode — a model that keeps re-checking work it's already gotten right — is a specific, checkable-in-practice claim rather than boilerplate, and it lines up with the recommended default of reasoning_effort: "high" running deep chain-of-thought unless a caller explicitly opts into "no_think".

The ledger

What holds up: the parameter math, at four decimal places across both the backbone and the MTP layer. IndexCache's savings, confirmed as missing tensors rather than a skipped computation. Both blind-eval percentage triples, summing to exactly 100.0%. The "open-source frontier" phrase, read at the precision it was actually written — Hy4 preview leads 5 of 12 headline benchmarks against the models that phrase is about.

What needs the asterisk spelled out: "frontier" without the "open-source" qualifier doesn't hold — 1 of 12 against the full seven-model field, including two closed frontier models the chart plots but the claim isn't about. And the one reasoning benchmark where Hy4 preview beats Claude Opus 5 outright carries the highest disclosed truncation rate of the five benchmarks with that caveat — a result worth keeping, not worth over-crediting.

None of this is a case against the release. A 770B/49B-active MoE with a real architectural idea in IndexCache, verified against its own weight files rather than taken on the config flag, delivering 130-to-483% generational gains and a legitimate (if precisely-scoped) claim to the open-weight frontier, on a model card candid enough to footnote its own comparison's weak point — that's a stronger showing than most flagship releases put up, headline number included.

The GGUF ships from Tencent's own compression team, and the size claim mostly holds

A month after the release above, AngelSlim/Hy4-preview-GGUF appeared on Hugging Face. AngelSlim isn't a third-party quantizer picking up a popular open-weight release — it's Tencent's own model-compression toolkit team, the same group that ships FP8 and speculative-decoding tooling for the Hy line elsewhere on GitHub. The repo is small and specific: two GGUF files, a README, and two patch files for a llama.cpp fork, because neither file runs on stock llama.cpp — the hyv4 architecture isn't upstream, and the more aggressive of the two quants needs a CUDA kernel that isn't either. Needing a patched build to run a GGUF at all isn't new to this model family: Hy3's own community GGUF ladder required a hy_v3-capable llama.cpp build too, just without a from-scratch quantization format riding along with it.

The promotional framing for the release was a single sentence: "compressed Hy4-preview from 1.5TB to ~200GiB GGUF and it still works well!" That's two separate numbers to check against two repos' own file listings — tencent/Hy4-preview's 131 safetensors shards for the baseline, AngelSlim/Hy4-preview-GGUF's file sizes for the result — rather than against the sentence itself.

Both ends of the claim are directionally real: the compression is genuine and large, and the baseline figure is a fair rounding. Where it slips is the smaller number. The GGUF repo's own README states the STQ1_0 file's size as 213.66GiB in its own table — not "~200GiB" — and that stated figure matches the raw byte count from the API to the second decimal place, so the model card itself is precise here. The rounding to "~200" happened somewhere between the model card and the announcement, not inside the model card. It's a modest inaccuracy, not a fabricated one: 6.8% low read as GiB, 14.7% low if "200" gets read as decimal GB instead — exactly the unit ambiguity worth being careful about, and exactly why GLM-5.3's own GGUF coverage on this site flags GiB-vs-GB as a place vendor numbers quietly slip.

What sub-2-bit actually means, mechanically, and where AngelSlim spent the bits

"Some down to 1.31-bit STQ1_0, some up to 2.06-bit IQ2_XXS" is a real description of two real llama.cpp formats, and both bpw figures check out exactly: STQ1_0 is 1.3125 bits per weight, IQ2_XXS is 2.0625. Neither number is hand-wavy marketing — both come from a concrete block structure, and the README spells out STQ1_0's in enough detail to verify by arithmetic.

STQ1_0 (from llama.cpp PR #22836) stores ternary weights — each one is exactly -d, 0, or +d, the same three-valued alphabet this site covered training a model natively on rather than squeezing a trained one down into afterward — with a structural rule that exactly one of every four weights in a group is forced to zero (3:4 sparsity, grouped by a stride-16 pattern chosen for SIMD alignment on the decode side). That constraint isn't arbitrary: choosing which one of four lanes is zero is 4 possibilities, and the sign of each of the other three is 2 possibilities, so a 4-weight group has exactly C(4,3) × 2³ = 32 possible patterns — which is exactly the size of the codebook a 5-bit index (a 4-bit code plus a 1-bit table-select) can address. Per 256-weight block: 32 bytes of 4-bit codes, 8 bytes of table-select bits, and one 2-byte fp16 scale — 42 bytes total, and 42 × 8 ÷ 256 = 1.3125 bits per weight, exactly. IQ2_XXS is llama.cpp's older, non-ternary i-quant format at the next rung up, 2.0625 bpw via its own fixed non-uniform codebook — well-established elsewhere in llama.cpp, not new to this release.

What is new is the encoder AngelSlim wrote for STQ1_0. The upstream PR's quantizer targets QAT checkpoints already sitting on the ternary grid: it sets the scale to the single largest-magnitude weight in the block (d = amax) and zeros whichever lane has the smallest magnitude. The README describes that as weak for post-training quantization of a checkpoint that was never trained ternary, and replaces both decisions — a weighted least-squares scale solve in place of amax, and zero-placement that minimizes each lane's actual contribution to reconstruction error rather than just picking the smallest weight — alternating between the two for three rounds of coordinate descent. Measured on 1,200 real expert rows: the scale fix alone cuts weighted sum-of-squared-error by 89.7%; the zero-placement fix cuts a further 4.1% off what's left. That's a real, measured improvement — but it's an improvement in reconstruction error, a proxy for how close the quantized weights sit to the originals, not a measurement of any downstream task. Keep that distinction in mind for the next section.

"Calibration data picks each layer's bit-width" holds up at the level of individual layer indices, not just as a description of the average. The recipe file assigning formats to tensors is checked into the repo, and it explicitly forks from Unsloth's GLM-5.2 UD-IQ1_M recipe — reasonable, since GLM-5.2 shares HY4's glm-dsa-family architecture (MLA, 256 routed experts at top-8, a DSA indexer). Unsloth's day-zero Dynamic GGUF quants for that same architecture family, one generation later, are covered in more depth elsewhere on this site: GLM-5.3, Z.ai's release built on GLM-5.2's byte-identical base. Two llama.cpp auto-detection rules that work for GLM silently fail for HY4 and had to be patched by hand: attn_output only gets its usual precision bump when a model has exactly 8 experts, and HY4's 256 means it would otherwise fall straight to IQ2_XXS uncorrected; and HY4's split MLA tensor names (q_b/k_b/v_b/kv_a_mqa) don't match llama.cpp's exact-substring check for the fused names GLM uses, so they'd get no automatic bump at all without an explicit override. ffn_down_exps is deliberately quantized two formats higher than the gate/up projections next to it, because it writes straight back into the residual stream where its error isn't attenuated by a following gate — the recipe file's own comment attributes the same reasoning to GLM's baseline recipe, generalized here into a full per-layer sweep. Running the CUDA kernel matters as much as the format choice: the README's own numbers show STQ1_0 falling back to 20.80 tokens/sec on plain CPU dequantization versus 204.56 tokens/sec once the kernel is actually linked in — a 9.83× gap the recipe file warns readers to check for directly (nm -D libggml-cuda.so | grep -ci stq1_0) rather than assume.

One naming note: the promotional post calls the file "MIX-STQ1_0." No file by that name exists in the repo — the actual filename is Hy4-preview-STQ1_0.gguf, and "MIX" doesn't appear anywhere in the README, the patches, or either recipe file as a name AngelSlim itself uses. It's a reasonable shorthand for what the file does — mix STQ1_0 and IQ2_XXS at the tensor level — just not what the release calls itself.

An accuracy table with no primary source anywhere in the release

The promotional post's last claim is the one that matters most and checks out least: "Accuracy barely moves vs BF16: MCP Atlas 83.7→83.2, SWE-Bench multi 82.9→81.3, MRCR 81.3→81.1, IFBench 73.5→72.5." All four are real, independently documented benchmarks — MCP Atlas is a published tool-use-over-MCP-servers benchmark, SWE-Bench Multilingual and IFBench and MRCR are all established elsewhere. What none of them are is present anywhere this piece could find in the actual release: not in the GGUF repo's README (read in full, both its English and Chinese sections), not in either patch file, not in either .tensortypes recipe file, and not in Tencent/AngelSlim's GitHub repository or its arXiv technical report.

What the model card discusses about quality is a different kind of number entirely — the weighted-SSD reconstruction-error reductions in the section above, measured against the encoder's own reconstruction target on 1,200 expert rows. That's a legitimate thing to report, and it's reported honestly as exactly what it is. It is not a benchmark score, and a large drop in reconstruction error doesn't mechanically imply a small drop in end-task accuracy — the two can move together or apart depending on which weights the error concentrates in. Whether the four cited numbers are real internal results that simply weren't published alongside the weights, or a template quality line reused across a run of similar announcements, isn't something this site can settle from the outside. What can be said plainly: as of this release, there is no public, checkable primary source for the specific 83.7→83.2 / 82.9→81.3 / 81.3→81.1 / 73.5→72.5 figures, which puts this claim in a different category from the size and format claims above — not disproven, but unverifiable, and unverifiable is the honest thing to call it rather than rounding it up to confirmed.

Related architecture on this site: Mixture of Experts, from scratch for the routing mechanics IndexCache and the shared expert sit on top of; PhoneLLM Alpha 1 and VoiceMem for two other MoE models whose active-parameter claims were checked the same way, against the raw safetensors headers rather than the README; vLLM and SGLang for the two serving stacks Hy4 preview ships dedicated container images for. On the GGUF update specifically: Hunyuan Hy3 for this model family's earlier community quant ladder; Ternary15M for the three-valued weight alphabet STQ1_0 borrows for a different purpose; and GLM-5.3 for the Unsloth GGUF-quantization lineage this recipe forks from, and for another release where a promotional "2-bit" turned out to describe two different files.