Limite 1B - Violetto: the throughput claim is in config.json, not the blog

Paradigma released Limite 1B - Violetto yesterday, and it comes with two different taglines. The GitHub README says "A model for high-frequency mathematical intelligence." The release blog says "high-throughput solutions of difficult mathematical problems" and, in its meta description, "high-throughput mathematical reasoning."

Neither page contains a tokens-per-second number, a latency figure, a batch size, a hardware target, or the word "milliseconds". I grepped both. The serving instructions are a vLLM plugin that requires tensor_parallel_size=1, and that is the whole of the deployment story.

So the interesting question is not whether the claim is true. It is whether the claim is attached to anything. It is — just not in the prose. config.json is unusually explicit for a model with no technical report yet, and one field in it does all the throughput work.

The stack, from the config

1,035,253,888 parameters, bf16, 2.07 GB on disk. 48 blocks, hidden 1280, intermediate 3328, 10 query heads over 2 KV heads at head_dim 128, Qwen's tokenizer (bos 151643, vocab padded to 151,680), tie_word_embeddings: true. Nothing surprising so far.

Then the rest of the file, which reads like a NanoGPT speedrun changelog rather than a model card:

{
  "global_every": 4,
  "global_layers": [3, 7, 11, 15, 19, 23, 27, 31, 35, 39, 43, 47],
  "global_nope": true,
  "sliding_window": 1024,
  "sliding_window_convention": "k >= q - sliding_window, inclusive of the query token",
  "ve_layers": [1, 4, 7, 10, 13, 16, 19, 22, 25, 28, 31, 34, 37, 40, 43, 46],
  "ve_dim": 128,
  "mudd": true, "mudd_layers": [24, 47], "mudd_taps": 3,
  "mudd_tap_idx": { "24": [0, 12, 24], "47": [0, 23, 47] },
  "attn_gate_applied": "per_head_before_o_proj",
  "qk_norm": "rms_pre_rope",
  "rms_norm_has_weight": false,
  "attention_softmax_scale": 0.1,
  "rope_base_global": 1024.0,
  "rope_frac": 0.5,
  "softcap_logits": { "kind": "sigmoid", "a": 23.0, "b": 5.0, "c": 7.5 }
}

Four of these are worth stopping on, because the blog's one sentence about architecture — "strongly inspired by recent advancements made by the community on pre-training speedrun competitions" — is the only explanation on offer, and the fields say more than the sentence.

Value embeddings. model.value_embeds.weight is a second embedding table, 151,680 × 256, 38.8M parameters, injected at 16 of the 48 blocks. ve_stored_heads: 2 and ve_head_slice: "first_num_key_value_heads" mean it feeds the V path of the two KV heads directly from the token id, before QK-norm. This is the speedrun's U-net value-embedding trick, and it is 3.8% of the model spent on a lookup table that skips the residual stream entirely.

MuDD. Multi-way dynamic dense connections, at exactly two sites: block 24 taps the outputs of blocks 0, 12 and 24, and block 47 taps 0, 23 and 47. The parameters are trivial — mudd.dense1 is [32, 1280], dense2 is [48, 3, 32] — so this is a routing structure, not a capacity addition. Two deliberate long-range shortcuts at the half-way point and the exit.

Norms with no weight. rms_norm_has_weight: false throughout. Every RMSNorm is a pure normalisation with no learnable gain. That is a speedrun-derived simplification and it removes a hundred-odd thousand parameters, which is not the point; the point is that it removes a family of scale degrees of freedom that usually have to be tuned.

A pinned softmax scale. attention_softmax_scale: 0.1, not 1/sqrt(128) = 0.0884. Combined with rope_base_global: 1024.0 — three to four orders of magnitude below the 10⁶–10⁷ that a 131K-context model normally uses — and rope_frac: 0.5, so only half the head dimension is rotated at all, with the 12 global blocks using no positional encoding whatsoever. The positional scheme here is: local blocks get short-range RoPE on half their channels, global blocks get nothing and rely on the causal mask. It is the NoPE-plus-local-RoPE arrangement, committed to harder than most.

None of these four costs anything at inference. The fifth field does.

The only derivable throughput claim

36 of 48 blocks attend over 1,025 keys regardless of how long the sequence is. Only 12 grow.

The arithmetic is four numbers from config.json and nothing else. KV bytes per token per block = 2 (K and V) × num_key_value_heads (2) × head_dim (128) × 2 bytes = 1024 B. A global block at 131,072 tokens holds 134 MB; a windowed block stops at 1.05 MB. So the full 131K cache is 12 × 134 MB + 36 × 1.05 MB = 1.65 GB, against 6.44 GB if all 48 blocks were global. A factor of 3.91.

That is the mechanism behind "high-throughput", and it is worth saying plainly what kind of claim it is. It is not a speed claim — a windowed block and a global block cost roughly the same per token at short context. It is a concurrency claim: on one 80 GB accelerator with the 2.07 GB checkpoint resident and 4 GB set aside for activations and runtime, the 73.9 GB left over holds 44 simultaneous 131K-token sequences instead of 11. For a workload that is thousands of independent competition problems run in parallel, which is exactly what a maths model is for, that is the whole game.

It is also the ceiling, not a measurement. Nobody has published a real one.

The maths, and the one column that behaves differently

The headline is 94.01% on AIME 2026 at an estimated 1.71 × 10²¹ training FLOPs.

expand

The 6ND estimate is checkable. 6 × 1.035e9 × 275e9 = 1.71e21, so the x position implies about 275 billion training tokens — consistent with the blog's "less than 300B highly curated tokens", and a genuine 1–3 orders of magnitude below the models on the same dashed line. Sample efficiency is the real result here.

But read the axis footnote: "RL excluded; counted training stages vary by model." The blog credits its competition-level maths to "a mix of synthetic data generation, curated SFT and RL post-training", and the compute axis excludes the last of those for everyone. The chart is therefore a pre-training-efficiency chart wearing a capability headline, and the gap it shows is real but measured on the stage that is not the one doing the final work.

expand

Rank Limite in its own table, column by column:

Six of the seven columns are competition-format short-answer maths — AIME, HMMT, BeyondAIME, an olympiad shortlist — and Limite is first, third, third, fourth, fifth and sixth on them, against models up to 730 times its size. That is a real result and the BeyondAIME number in particular (74.25 against MUSE-Glimmer-30B's 70.00, which the blog names) is not a rounding artefact.

The seventh column is ArXivMath May 2026, and Limite is 7th of 12 at 25.08 — behind VibeThinker-3B (29.77), Qwen3.5-9B (31.64), Gemma-4-31B-IT (35.78) and a long way behind GLM 5.2 (56.67). It is also behind Qwen3.5-4B (26.33), a model it beats by four points on AIME 2026.

That inversion is the most informative thing in the table. On every benchmark whose format is "produce a boxed integer", the 1B is top-six. On the one benchmark whose format is not, the ordering reverts to something much closer to parameter count. Paradigma does not hide this — the column is right there in their own table, and the blog is unusually frank about the model's limits elsewhere, printing four worked examples of Violetto reinterpreting "What is photosynthesis?" as a cell-division recurrence and "What causes the seasons?" as a calendar arithmetic problem. A model described as "as lightly instruction-tuned as possible, to challenge the assumption that models need to be embedded in an assistant persona" is going to do that, and saying so with receipts is the right call.

It does mean the honest summary is narrower than "mathematical intelligence." This is a competition-problem solver. MiniCPM5-2B appears in the same table two rows down, and the comparison holds up: 90.21 to Limite's 94.01 on AIME 2026, at twice the parameters — but 21.80 to Limite's 25.08 on ArXivMath, a much smaller gap.

What is actually released

Four repositories under paradigma-inc: limite-1b-violetto (the RL'd model), limite-1b-base, limite-1b-base-soup and limite-1b-value-model — the training-time value model, which is an unusual and useful thing to release. The serving code is Apache-2.0, as are the Violetto and value-model repos; the two base checkpoints carry no licence tag on the Hub at the time of writing, which is presumably an oversight and is worth fixing before anyone builds on them. The base and Violetto configs are byte-identical apart from the weights, so the post-training changed no architecture.

What is not released: the technical report ("to be released soon"), the training data, the RL recipe, the eval harness, and any performance measurement at all. For a model whose name is "high-frequency", the last omission is the one to fix first, and it is an afternoon's work: vllm bench serve against the plugin at a few concurrencies, on a named card, with the prompt distribution the model is for.