{
  "claim": "Splash hard-codes the model. Qwen3.8-27B's entire geometry is a struct in runtime/model/Qwen3_8.hpp, and the pack format is nine bytes per sixteen weights, so the whole checkpoint can be reconstructed with arithmetic instead of downloaded. It comes to 26.935 billion parameters in 15.155 GB — which is how I know the byte count underneath every throughput claim in this piece is right.",
  "method": "Every shape below is read from `Qwen3_8Layout` and `DFlashDraftLayout` in the cloned repository at commit f58d36d (Splash 1.0). Bytes come from `q4PackedBytes(o, i) = o * i / 16 * 9` in runtime/model/WeightStore.cpp — a 32-byte block of nibbles plus a bf16 scale and a bf16 bias for each group of 64 inputs, which is exactly 4.5 bits per weight. Norms, GDN convolution taps and the selector codebooks are bf16; the GDN recurrent state is fp32. Nothing here is quoted from a model card.",
  "source": "https://github.com/incoai/splash",
  "captured": "2026-09-19",
  "note": "The reconstruction lands at 15.155 GB against the launch post's stated \"15 GiB of weights\" and 1.266 GB against its \"1.2 GiB draft\" — so the post's weight figure is GB labelled GiB, and its draft figure is GiB. Both are inside rounding of the real numbers. Target plus draft is 16.42 GB against a 17.4 GB Hugging Face package; the balance is the 27-block vision tower, which QwenVision.cpp reads as bf16 rather than Q4 and which comes to 0.93 GB. 15.155 + 1.266 + 0.93 = 17.35. Decode never touches it, so it is not in any byte count in this piece.",
  "columns": [
    { "key": "part", "label": "tensor" },
    { "key": "shape", "label": "shape" },
    { "key": "each", "label": "bytes each", "align": "right" },
    { "key": "n", "label": "×", "align": "right" },
    { "key": "sub", "label": "subtotal", "align": "right" }
  ],
  "rows": [
    {
      "part": "GDN in-projection",
      "shape": "16,640 × 5,120",
      "each": "47.92 MB",
      "n": "48",
      "sub": "2.30 GB"
    },
    {
      "part": "GDN out-projection",
      "shape": "5,120 × 6,144",
      "each": "17.69 MB",
      "n": "48",
      "sub": "0.85 GB"
    },
    {
      "part": "attention in-projection (q, k, v and the output gate)",
      "shape": "14,336 × 5,120",
      "each": "41.29 MB",
      "n": "16",
      "sub": "0.66 GB"
    },
    {
      "part": "attention out-projection",
      "shape": "5,120 × 6,144",
      "each": "17.69 MB",
      "n": "16",
      "sub": "0.28 GB"
    },
    {
      "part": "SwiGLU gate, up, down",
      "shape": "17,408 × 5,120 twice, 5,120 × 17,408 once",
      "each": "150.41 MB",
      "n": "64",
      "sub": "9.63 GB"
    },
    {
      "part": "norms, GDN 4-tap convolutions, decays, time biases",
      "shape": "bf16 and fp32 vectors",
      "each": "0.10 MB on a GDN layer, 0.02 MB on an attention one",
      "n": "64",
      "sub": "0.005 GB"
    },
    {
      "part": "LM head",
      "shape": "248,320 × 5,120",
      "each": "715.16 MB",
      "n": "1",
      "sub": "0.72 GB"
    },
    {
      "part": "token embedding, also 4-bit",
      "shape": "248,320 × 5,120",
      "each": "715.16 MB",
      "n": "1",
      "sub": "0.72 GB"
    },
    {
      "part": "TARGET TOTAL",
      "shape": "26,935,320,064 parameters at 4.5011 bits",
      "each": "",
      "n": "",
      "sub": "15.155 GB"
    },
    {
      "part": "DFlash 2 draft, 5 layers at hidden 5,120",
      "shape": "qkv 6,144 · dynamic 1,280 · attn 4,096 · SwiGLU 17,408",
      "each": "187.3 MB",
      "n": "5",
      "sub": "0.94 GB"
    },
    {
      "part": "draft context projection, reading 5 target layers at once",
      "shape": "5,120 × 25,600",
      "each": "73.73 MB",
      "n": "1",
      "sub": "0.07 GB"
    },
    {
      "part": "selector codebooks, bf16, one predecessor and one successor",
      "shape": "248,320 × 256, twice",
      "each": "127.14 MB",
      "n": "2",
      "sub": "0.25 GB"
    },
    {
      "part": "DRAFT TOTAL",
      "shape": "no head of its own — it runs the target's",
      "each": "",
      "n": "",
      "sub": "1.266 GB"
    },
    {
      "part": "KV cache, int8 with an fp32 scale per token per head",
      "shape": "16 attention layers × 4 heads × 256",
      "each": "32.50 KiB per token",
      "n": "32K ctx",
      "sub": "1.09 GB"
    },
    {
      "part": "Gated DeltaNet state, fp32, fixed size whatever the context length",
      "shape": "48 layers × 48 heads × 128 × 128",
      "each": "147 MiB per request",
      "n": "read + written every step",
      "sub": "0.31 GB"
    }
  ]
}