MiniCPM5-2B: how 2 KV heads pay for a 131K window

openbmb/MiniCPM5-2B is a dense 2B-class language model that, per its own README, reaches "2B-class open-source SOTA" across 34 benchmarks, natively handles a 131,072-token context, and ships with an open speculative-decoding draft, a Q4/Q8/GPTQ/MLX/GGUF release matrix, and same-day recipes for vLLM, SGLang, llama.cpp, and five more inference backends. All of that is worth taking seriously. None of it comes with a tech report.

Pull the model repo's own metadata and it lists exactly two arXiv citations: 2506.07900 — MiniCPM4: Ultra-Efficient LLMs on End Devices — and 2602.09003, a data-tiering paper referenced once, for the pretraining data pipeline. Scroll to the bottom of the README and the BibTeX block still cites MiniCPM4. There is no MiniCPM5 paper. Every architectural and training claim this article makes about MiniCPM5-2B specifically rests on the model card, config.json, and the GitHub cookbooks — not a peer-reviewed report. Where the lineage runs through MiniCPM4's actual paper, this article says so explicitly and cites it as MiniCPM4, not MiniCPM5.

What's actually in the config

Model cards can round; config.json cannot. Here are the fields that matter, fetched directly from the repo:

{
  "architectures": ["LlamaForCausalLM"],
  "model_type": "llama",
  "hidden_size": 2048,
  "intermediate_size": 6144,
  "num_hidden_layers": 42,
  "num_attention_heads": 16,
  "num_key_value_heads": 2,
  "head_dim": 128,
  "max_position_embeddings": 131072,
  "rope_theta": 5000000,
  "vocab_size": 130560,
  "tie_word_embeddings": false,
  "torch_dtype": "bfloat16"
}

Two things worth stopping on before anything else:

It's LlamaForCausalLM. Not a MiniCPM-specific architecture class, not InfLLM v2's trainable sparse attention from the MiniCPM4 paper — a stock Llama-style transformer with grouped-query attention. The README says this outright: "MiniCPM5-2B uses the standard LlamaForCausalLM architecture, so mainstream inference engines can load it directly: no custom kernels, no model-code fork." That is a real, useful engineering decision — every backend in this article works on day zero because of it — but it also means MiniCPM5 did not inherit MiniCPM4's headline sparse-attention mechanism. What it inherited is the training philosophy: the data pipeline, the tiered data management, the RL recipe. The architecture reverted to something engines already know how to run fast.

"2B" is 2.52B. The Hugging Face API reports 2,516,756,480 BF16 parameters — 2.52 billion, 1.98B of them outside the embedding and LM head (vocab_size=130560 at hidden_size=2048 makes the embedding table alone about 267M parameters, doubled since tie_word_embeddings: false means input and output embeddings are separate weights). usedStorage on the repo is 5,033,557,096 bytes, which is 2,516,756,480 × 2 to within rounding — consistent with a straight BF16 release. None of this makes "2B" dishonest; every model in its comparison set rounds the same way. It's just worth naming once, in numbers, rather than letting the name do the rounding silently.

The lineage, from a paper that isn't about this model

Since MiniCPM5 doesn't have its own report, the honest way to talk about where it came from is to go read the paper it does cite and be careful about what transfers. MiniCPM4's abstract targets exactly this problem — efficient end-device LLMs — across four axes: architecture (InfLLM v2 trainable sparse attention), training data (the UltraFineWeb / UltraClean filtering lineage this site has covered separately), training algorithms (a µP-style hyperparameter search called ModelTunnel v2, plus chunk-wise RL rollouts), and inference systems (speculative decoding via FR-Spec, a CPU/GPU inference framework called CPM.cu).

expand

That figure is here for contrast, not lineage. InfLLM v2 is what let MiniCPM4-8B post the speed numbers in its own headline chart:

expand

MiniCPM5-2B is a different model, a different size class, and — per the README's own words above — a different attention mechanism entirely. What plausibly does carry over from MiniCPM4 to MiniCPM5 is everything data- and training-shaped: the UltraFineWeb/UltraClean filtering approach, tiered data management (now formalized in the 2602.09003 paper cited on this repo), and the general end-device efficiency mandate. What does not carry over, on the model's own admission, is the sparse-attention architecture. If MiniCPM5-2B is fast and cheap to serve at long context, the mechanism is downstream of GQA and aggressive quantization support — covered next — not of InfLLM v2.

Why 2 KV heads is the whole story

Grouped-query attention doesn't change how many query heads compute attention outputs — it changes how many distinct key/value projections have to be cached. MiniCPM5-2B ships 16 query heads sharing only 2 KV heads, an 8:1 ratio that is aggressive even among GQA models (Llama-3.1-8B ships 8 KV heads for 32 query heads, a 4:1 ratio, for comparison). Every group of 8 query heads reads the same cached key and value at inference time. The compute cost of attention doesn't move; the memory cost of storing that cache for every generated token, in every layer, for the life of the request, drops in direct proportion to the KV-head count.

The KV-cache arithmetic

This is the number the rest of the article turns on, so it's worth deriving instead of quoting. Per-token KV cache size, across the whole model, is:

layers = 42
kv_heads = 2
head_dim = 128
dtype_bytes = 2       # f16 — 2 bytes per element
context = 131_072     # native max_position_embeddings
 
per_token = layers * kv_heads * head_dim * 2 * dtype_bytes  # the "2" is K and V
total = per_token * context
 
print(per_token)        # 43,008 bytes/token
print(total)             # 5,637,144,576 bytes
print(total / 1e9)       # 5.637 GB

At the model's native 131,072-token context and f16 KV cache, that's 5,637,144,576 bytes — 5.637 GB — of KV cache alone, before a single weight is loaded. Now check it against a real deployment report: someone running MiniCPM5-2B-GGUF on a single RTX 3060 with

llama-server -m MiniCPM5-2B-Q8_0.gguf -ngl 99 -c 131072 -fa on --jinja \
  -np 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 --temp 1.0 --top-p 0.95

measured Q4_K_M at ~163 tok/s decode, 7.2GB total VRAM, and Q8_0 at ~113 tok/s decode, 8.2GB total VRAM, both with prefill around 5,800 tok/s, and reported that "most of that VRAM is the 131K KV cache." The GGUF weight files are 1.56GB (Q4_K_M) and 2.68GB (Q8_0). Subtract:

• Q4_K_M: 7.2 − 1.56 = 5.64GB implied KV cache
• Q8_0: 8.2 − 2.68 = 5.52GB implied KV cache

Both land within rounding of the derived 5.637GB. The claim checks out — not approximately, but to the byte on the Q4_K_M side. And the mechanism is exactly the 2-KV-head geometry above: an MHA model with 16 KV heads instead of 2 would need 8× that cache — 45,097,156,608 bytes, ~45GB — before weights, which doesn't fit on any consumer GPU sold today regardless of how aggressively the weights themselves are quantized. The 131K window isn't cheap because the model is small. It's cheap because the cache is small, and the cache is small because of a single config field.

The corollary the RTX 3060 report drew, and the slider above reproduces: drop the context window and the budget opens up fast, because weights barely move while the cache scales linearly with tokens. A Q8_0 build at 32K tokens instead of 131K needs roughly 2.68 + 1.41 ≈ 4.1GB — comfortable on an 8GB card with headroom to spare, or on a shared/multi-tenant box.

What 34 benchmarks actually say

The README's evaluation table compares MiniCPM5-2B against eight other models: three in its own 2B class (LFM2.5-2.6B, Qwen3.5-2B, Gemma-4-E2B-it) and five larger ones listed for reference (Qwen3.5-4B, granite-4.2-3B, Nemotron-3-Nano-4B, Gemma-4-E4B-it, LFM2.5-8B-A1B). The card's claim is an average of 53.9 across 34 benchmarks spanning nine categories, ahead of every model in the comparison set including the 4B-class ones (highest there: 51.1). Averaging MiniCPM5-2B's own 34 column values by hand reproduces 53.88 — the card's number is not a rounding trick, it's the real mean of the rows below.

The full table, transcribed as published (blue-bold-equivalent: bold marks the best result across all 9 models; a dagger † marks scores sourced from the official Artificial Analysis release rather than reproduced internally):

A pattern worth naming rather than skating past: MiniCPM5-2B doesn't win every row (Multi-IF, AIME 2025/2026, MMLU-Pro/Redux, MATH-500, several long-context and agentic rows go to a 4B-class model instead), but it wins the average, and wins it against models with meaningfully more parameters. On τ²-Bench Telecom and GAIA it leads the entire field, 2B-class or not. On AIME 2025/2026 it's within a point of the field-leading 4B model. That's a genuinely different shape than "small model wins on paper by cherry-picking benchmarks" — it's mid-pack-to-strong on most rows and exceptional on a handful, which is what actually moves an average that far above the next-best 2B model (33.2).

The training recipe: SFT, then RL teachers, then one distillation pass

expand

Reading the diagram left to right: base pretraining runs stable training, a short 4K-context decay phase, then a long decay phase that extends context in stages up to 512K (a training-curriculum detail — the shipped model's documented, served context is 131,072, matching max_position_embeddings). Mid-training then runs two more context stages on top of that base — 600B tokens at 32K context, then 400B tokens at 128K context — before 400B tokens of "deep-thinking" SFT (released as UltraData-SFT-2605, with the agent-specific portion as UltraData-SFT-Agent-2609, ~500K samples).

From the SFT checkpoint, training branches into parallel RL teachers — reasoning, general-task, and agentic tracks, each producing multiple expert checkpoints (16 total, 5 of them agentic) — using UltraData-RL-2609 (80K+ samples spanning math, code, general knowledge, and long-context reasoning). The reasoning-RL reward design is explicitly credited to JustRL ("Scaling a 1.5B LLM with a Simple RL Recipe," arXiv:2512.16649) — a real, citable paper. A separate piece of the recipe, the critic-based algorithm used specifically for the RL+OPD stage, is credited to something called "JustRL II," linked not to arXiv but to a Notion writeup. That is the actual, verified link in the README — worth flagging because an arXiv id (2511.05963) has circulated attached to "JustRL II" in some launch commentary; that id resolves to an unrelated paper ("Next-Latent Prediction Transformers Learn Compact World Models"), not to this work. Cite the Notion page if you cite JustRL II at all, and treat it as an unreviewed writeup, not a paper.

Then On-Policy Distillation (OPD) merges those 16 expert models back into one release checkpoint: at each response position, it computes full-vocabulary reverse KL divergence between student and teacher logits as the advantage estimate — replacing the usual verification-based advantage — and reuses each teacher's own RL training prompts as distillation data, so no separate distillation corpus had to be built.

The README states the net effect in two numbers: RL + OPD improves reasoning/general benchmarks by an average of +10.96 points over the SFT-only checkpoint, and agentic benchmarks by +6.96 points. Here is the chart that number comes from:

expand

Reading those gains benchmark-by-benchmark off that chart, rather than trusting the two averages alone — this is the same benchmark explorer from earlier, opened straight to its "what RL+OPD bought" tab:

NoLiMa gains 8.9 points, GPQA-Diamond gains 21.6, SWE-bench Verified gains 17.4, and τ²-Bench Telecom — already near-saturated at 92.98 after SFT — gains a comparatively modest 4.1. The gains aren't uniform, which is what you'd expect from a distillation step reusing each teacher's own training distribution rather than a generic capability boost applied everywhere at once.

One name from the release announcements that this article is not going to assert as fact: a training framework sometimes called "Meshy," described elsewhere as a scalable RL training system behind this release. It does not appear anywhere in the model card, the GitHub repository, or public documentation as far as this research could find — searched by name, directly, with nothing returned. If it's real, it shipped without a citable reference this article could locate; the RL-stack claims above are limited to what the README itself names and links.

The open release

Alongside the weights, OpenBMB released the training data itself:

DSpark: speculative decoding, and a speedup that doesn't decay smoothly

MiniCPM5-2B ships an open speculative-decoding draft, MiniCPM5-2B-DSpark. Per the community GGUF conversion of that draft, its GGUF architecture string is literally dflash — DSpark is built as DFlash plus a Markov head, which puts it in the same family this site covered in DFlash 2: the drafter already knew the answer, it just picked the wrong one, and adjacent to the confidence-scheduled verifier approach in DeepSeek's DSpark (a different, unrelated "DSpark" — DeepSeek and OpenBMB happened to land on the same name for different mechanisms). SGLang is the recommended serving path, with a purpose-built launch flag:

python -m sglang.launch_server \
  --model-path openbmb/MiniCPM5-2B \
  --trust-remote-code \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path openbmb/MiniCPM5-2B-DSpark \
  --speculative-dspark-block-size 7 \
  --port 30000

SGLang's own day-0 cookbook for this model publishes concrete single-GPU throughput numbers, reproduced in full below — worth being precise about what they do and don't establish, since a claim of "over 250 tok/s/user on coding tasks with DSpark enabled on a 5090" circulated around this release and this article could not confirm that specific framing against the cookbook page: the page's tables don't label whether DSpark was active for the runs, and don't mention a coding-specific workload — only a synthetic random benchmark (isl=1024, osl=1024).

Whatever the DSpark status of those specific runs, the community-built GGUF of the draft (aj9o9/MiniCPM5-2B-DSpark-GGUF) publishes a cleaner, fully-labeled comparison — same box, same target checkpoint, llama-benchy, decode-only, with and without the draft:

llama-server \
  --model MiniCPM5-2B-F16.gguf \
  -md MiniCPM5-2B-DSpark-F16.gguf \
  --spec-type draft-dspark \
  --spec-draft-n-max 7 \
  --host 127.0.0.1 --port 8080 \
  -ngl 999 -ngld 999 -fa on -np 1 -t 12 --jinja \
  -ctk q8_0 -ctv q8_0

VRAM cost is predictable and small — baseline 8,317MB, +1.8GB with the F16 draft loaded, 10.1GB total. The speedup is the interesting part precisely because it isn't a clean monotonic decay: strongest at 8k, inside the draft's 12,288-token training range, then weakening through 16k and 32k as the draft extrapolates past what it saw during training — and then recovering somewhat at 64k in this specific run. The uploader's own framing, worth repeating exactly: wall-clock speedup and accept length "are related but not the same number" — the official DSpark card's accept-length figures (~5.5 at T=0, ~4.1 at T=1.0 on in-distribution prompts) are reported to collapse toward ~1.6 well past the training range, which is a more pessimistic signal than the throughput curve above shows even at 64k. Both are real measurements; they're just measuring slightly different things, and only one number ships in most marketing.

For deeper context on why draft quality decays with distance from training data at all, and how other teams have addressed it, see this site's coverage of DFlash 2 and DSpark's speculative-decoding cousin over at LFM2.5 — the "does the speedup survive long context" question turns out to be a running theme across this entire family of drafters, not a MiniCPM5-specific quirk.

Deploying it: nine backends, nine chips, zero forks

Because MiniCPM5-2B is a stock LlamaForCausalLM, the GitHub repo ships cookbooks and matching Claude-Code/Cursor-style Agent Skills for every mainstream backend, with no custom kernel work required:

The released GGUF sizes, for reference — these are the numbers the VRAM budget explorer above uses:

Beyond the mainstream engines, OpenBMB partnered with the FlagOS Open Source Community to adapt the model across nine unrelated AI chip architectures via FlagRelease — Nvidia, Hygon, Metax, Iluvatar, Zhenwu, Mthreads, Kunlunxin, Ascend, and ARM-v9 — each getting its own published ModelScope/Hugging Face weights:

That ARM-v9 row is the one honest gap in this section worth naming directly: launch commentary around this release cited specific edge-hardware performance multipliers — Intel Core Ultra with OpenVINO, Arm Armv9 with SME2 giving roughly 1.7× prefill and 1.2× decode on mobile, and Rockchip RK3588/RK1828 numbers. This research checked the model card, the full GitHub repository (including every deployment doc and Agent Skill), and public search, and could not find those specific multipliers attached to MiniCPM5-2B anywhere verifiable. OpenVINO support for the MiniCPM5 family is real and documented by Intel — but for MiniCPM5-1B specifically in the material this research could locate, not confirmed for the 2B model at the cited multipliers. The FlagOS ARM-v9 row above is the verified edge-ARM claim this article can stand behind; the SME2 percentages are not repeated here because they couldn't be traced to a primary source.

Quickstart

# vLLM
pip install "vllm>=0.21"
vllm serve openbmb/MiniCPM5-2B --port 8000

# SGLang
pip install "sglang[srt]>=0.5.16"
python -m sglang.launch_server --model-path openbmb/MiniCPM5-2B --port 30000

# Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
 
model_id = "openbmb/MiniCPM5-2B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
 
messages = [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    enable_thinking=True, return_dict=True, return_tensors="pt",
).to(model.device)
 
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Recommended sampling for the "Think" mode: temperature=1.0, top_p=0.95. Tool calling emits XML-style calls; SGLang's built-in minicpm5 parser converts them to OpenAI-compatible tool_calls natively — start the server with --tool-call-parser minicpm5 (or auto), then send a standard OpenAI-style tools=[...] request and it just works, no custom parsing required on the client side. For the engines this site has covered end to end, see SGLang: the tree, and the language nobody remembers and vLLM: what PagedAttention turned into — both apply directly here, since nothing about MiniCPM5-2B asks either engine to do anything model-specific.

Beyond the card: what Artificial Analysis independently measured

Everything above comes from the model card or a GitHub repo commit MiniCPM5-2B ships in. One more data point circulates around this release that deserves its own section precisely because it is not in the card: Artificial Analysis independently benchmarked the model and published its own Intelligence Index score, separate from anything OpenBMB self-reports.

Per Artificial Analysis's own published article on the release: MiniCPM5-2B scores 15 on their Intelligence Index — "the highest Intelligence Index of any open weights model under 4B total parameters," 4 points clear of Granite 4.2 3B (11), and 1 point ahead of Qwen3.5 4B (14, estimated) "with 44% fewer parameters." It sits "level with Qwen3.5 9B (Reasoning, 15, estimated) at roughly 4x its size" — level with, not above, which is a meaningfully more modest claim than "beats models 4x its size" and worth stating precisely rather than rounding up. On token efficiency: MiniCPM5-2B used 19k output tokens per Intelligence Index task (11k of them reasoning tokens), "joint-lowest in the comparison model set with Granite 4.2 3B (19k)" — tied for lowest, not uniquely lowest, and notably not the 21k-vs-19k framing that has circulated for this comparison; by Artificial Analysis's own published figure, MiniCPM5-2B and Granite 4.2 3B post the identical 19k token budget. On agentic evaluation specifically, GDPval-AA v2 Elo of 831 "leads <4B models," about 110 points ahead of Ling 3.0 Tiny (718) and about 180 ahead of Granite 4.2 8B (647). OpenBMB's own launch post on X separately states a score of 20 on an "Agentic Index" — that specific figure comes from OpenBMB's own announcement, not from Artificial Analysis's article, and this research could not retrieve the full post to check its surrounding context (it returned an access-restricted response rather than the page).

What this article can and can't stand behind

To close where it opened: there is no MiniCPM5 tech report, and everything specific to this model in this article traces back to a model card, a GitHub repo, or an independent third-party benchmark — never a peer-reviewed source. Reproduced and verified directly against a primary source: the config shape, the 2.52B parameter count, the KV-cache arithmetic (to the byte), the 34-benchmark average (recomputed, not just quoted), the RL+OPD gain figures, the DSpark GGUF throughput table, the SGLang single-GPU numbers, the FlagOS nine-chip list, and the UltraData/UltraX dataset sizes. Explicitly not verifiable from anything this research could locate: the "Meshy" training-framework name, the specific Arm SME2 / Intel OpenVINO / Rockchip RK3588+RK1828 performance multipliers, and the full text of OpenBMB's own Agentic Index claim. Both categories are listed here on purpose — a model card this detailed deserves to be read exactly as carefully as it was written.

@article{minicpm4,
  title={Minicpm4: Ultra-efficient llms on end devices},
  author={MiniCPM, Team},
  journal={arXiv preprint arXiv:2506.07900},
  year={2025}
}

That's the citation the MiniCPM5-2B README itself asks you to use — for MiniCPM4, still, in 2026.