MiniCPM5-2B: how 2 KV heads pay for a 131K window

openbmb/MiniCPM5-2B is a dense 2B-class language model that, per its own README, reaches "2B-class open-source SOTA" across 34 benchmarks, natively handles a 131,072-token context, and ships with an open speculative-decoding draft, a Q4/Q8/GPTQ/MLX/GGUF release matrix, and same-day recipes for vLLM, SGLang, llama.cpp, and five more inference backends. All of that is worth taking seriously. None of it comes with a tech report.

Pull the model repo's own metadata and it lists exactly two arXiv citations: 2506.07900 — MiniCPM4: Ultra-Efficient LLMs on End Devices — and 2602.09003, a data-tiering paper referenced once, for the pretraining data pipeline. Scroll to the bottom of the README and the BibTeX block still cites MiniCPM4. There is no MiniCPM5 paper. Every architectural and training claim this article makes about MiniCPM5-2B specifically rests on the model card, config.json, and the GitHub cookbooks — not a peer-reviewed report. Where the lineage runs through MiniCPM4's actual paper, this article says so explicitly and cites it as MiniCPM4, not MiniCPM5.

What's actually in the config

Model cards can round; config.json cannot. Here are the fields that matter, fetched directly from the repo:

{
  "architectures": ["LlamaForCausalLM"],
  "model_type": "llama",
  "hidden_size": 2048,
  "intermediate_size": 6144,
  "num_hidden_layers": 42,
  "num_attention_heads": 16,
  "num_key_value_heads": 2,
  "head_dim": 128,
  "max_position_embeddings": 131072,
  "rope_theta": 5000000,
  "vocab_size": 130560,
  "tie_word_embeddings": false,
  "torch_dtype": "bfloat16"
}

Two things worth stopping on before anything else:

It's LlamaForCausalLM. Not a MiniCPM-specific architecture class, not InfLLM v2's trainable sparse attention from the MiniCPM4 paper — a stock Llama-style transformer with grouped-query attention. The README says this outright: "MiniCPM5-2B uses the standard LlamaForCausalLM architecture, so mainstream inference engines can load it directly: no custom kernels, no model-code fork." That is a real, useful engineering decision — every backend in this article works on day zero because of it — but it also means MiniCPM5 did not inherit MiniCPM4's headline sparse-attention mechanism. What it inherited is the training philosophy: the data pipeline, the tiered data management, the RL recipe. The architecture reverted to something engines already know how to run fast.

"2B" is 2.52B. The Hugging Face API reports 2,516,756,480 BF16 parameters — 2.52 billion, 1.98B of them outside the embedding and LM head (vocab_size=130560 at hidden_size=2048 makes the embedding table alone about 267M parameters, doubled since tie_word_embeddings: false means input and output embeddings are separate weights). usedStorage on the repo is 5,033,557,096 bytes, which is 2,516,756,480 × 2 to within rounding — consistent with a straight BF16 release. None of this makes "2B" dishonest; every model in its comparison set rounds the same way. It's just worth naming once, in numbers, rather than letting the name do the rounding silently.

The lineage, from a paper that isn't about this model

Since MiniCPM5 doesn't have its own report, the honest way to talk about where it came from is to go read the paper it does cite and be careful about what transfers. MiniCPM4's abstract targets exactly this problem — efficient end-device LLMs — across four axes: architecture (InfLLM v2 trainable sparse attention), training data (the UltraFineWeb / UltraClean filtering lineage this site has covered separately), training algorithms (a µP-style hyperparameter search called ModelTunnel v2, plus chunk-wise RL rollouts), and inference systems (speculative decoding via FR-Spec, a CPU/GPU inference framework called CPM.cu).

expand

That figure is here for contrast, not lineage. InfLLM v2 is what let MiniCPM4-8B post the speed numbers in its own headline chart:

expand

MiniCPM5-2B is a different model, a different size class, and — per the README's own words above — a different attention mechanism entirely. What plausibly does carry over from MiniCPM4 to MiniCPM5 is everything data- and training-shaped: the UltraFineWeb/UltraClean filtering approach, tiered data management (now formalized in the 2602.09003 paper cited on this repo), and the general end-device efficiency mandate. What does not carry over, on the model's own admission, is the sparse-attention architecture. If MiniCPM5-2B is fast and cheap to serve at long context, the mechanism is downstream of GQA and aggressive quantization support — covered next — not of InfLLM v2.

Why 2 KV heads is the whole story

Grouped-query attention doesn't change how many query heads compute attention outputs — it changes how many distinct key/value projections have to be cached. MiniCPM5-2B ships 16 query heads sharing only 2 KV heads, an 8:1 ratio that is aggressive even among GQA models (Llama-3.1-8B ships 8 KV heads for 32 query heads, a 4:1 ratio, for comparison). Every group of 8 query heads reads the same cached key and value at inference time. The compute cost of attention doesn't move; the memory cost of storing that cache for every generated token, in every layer, for the life of the request, drops in direct proportion to the KV-head count.

The KV-cache arithmetic

This is the number the rest of the article turns on, so it's worth deriving instead of quoting. Per-token KV cache size, across the whole model, is:

layers = 42
kv_heads = 2
head_dim = 128
dtype_bytes = 2       # f16 — 2 bytes per element
context = 131_072     # native max_position_embeddings
 
per_token = layers * kv_heads * head_dim * 2 * dtype_bytes  # the "2" is K and V
total = per_token * context
 
print(per_token)        # 43,008 bytes/token
print(total)             # 5,637,144,576 bytes
print(total / 1e9)       # 5.637 GB

At the model's native 131,072-token context and f16 KV cache, that's 5,637,144,576 bytes — 5.637 GB — of KV cache alone, before a single weight is loaded. Now check it against a real deployment report: someone running MiniCPM5-2B-GGUF on a single RTX 3060 with

llama-server -m MiniCPM5-2B-Q8_0.gguf -ngl 99 -c 131072 -fa on --jinja \
  -np 1 -ctk f16 -ctv f16 -b 2048 -ub 1024 --temp 1.0 --top-p 0.95

measured Q4_K_M at ~163 tok/s decode, 7.2GB total VRAM, and Q8_0 at ~113 tok/s decode, 8.2GB total VRAM, both with prefill around 5,800 tok/s, and reported that "most of that VRAM is the 131K KV cache." The GGUF weight files are 1.56GB (Q4_K_M) and 2.68GB (Q8_0). Subtract:

• Q4_K_M: 7.2 − 1.56 = 5.64GB implied KV cache
• Q8_0: 8.2 − 2.68 = 5.52GB implied KV cache

Both land within rounding of the derived 5.637GB. The claim checks out — not approximately, but to the byte on the Q4_K_M side. And the mechanism is exactly the 2-KV-head geometry above: an MHA model with 16 KV heads instead of 2 would need 8× that cache — 45,097,156,608 bytes, ~45GB — before weights, which doesn't fit on any consumer GPU sold today regardless of how aggressively the weights themselves are quantized. The 131K window isn't cheap because the model is small. It's cheap because the cache is small, and the cache is small because of a single config field.

The corollary the RTX 3060 report drew, and the slider above reproduces: drop the context window and the budget opens up fast, because weights barely move while the cache scales linearly with tokens. A Q8_0 build at 32K tokens instead of 131K needs roughly 2.68 + 1.41 ≈ 4.1GB — comfortable on an 8GB card with headroom to spare, or on a shared/multi-tenant box.

What 34 benchmarks actually say

The README's evaluation table compares MiniCPM5-2B against eight other models: three in its own 2B class (LFM2.5-2.6B, Qwen3.5-2B, Gemma-4-E2B-it) and five larger ones listed for reference (Qwen3.5-4B, granite-4.2-3B, Nemotron-3-Nano-4B, Gemma-4-E4B-it, LFM2.5-8B-A1B). The card's claim is an average of 53.9 across 34 benchmarks spanning nine categories, ahead of every model in the comparison set including the 4B-class ones (highest there: 51.1). Averaging MiniCPM5-2B's own 34 column values by hand reproduces 53.88 — the card's number is not a rounding trick, it's the real mean of the rows below.

The full table, transcribed as published (blue-bold-equivalent: bold marks the best result across all 9 models; a dagger † marks scores sourced from the official Artificial Analysis release rather than reproduced internally):

A pattern worth naming rather than skating past: MiniCPM5-2B doesn't win every row (Multi-IF, AIME 2025/2026, MMLU-Pro/Redux, MATH-500, several long-context and agentic rows go to a 4B-class model instead), but it wins the average, and wins it against models with meaningfully more parameters. On τ²-Bench Telecom and GAIA it leads the entire field, 2B-class or not. On AIME 2025/2026 it's within a point of the field-leading 4B model. That's a genuinely different shape than "small model wins on paper by cherry-picking benchmarks" — it's mid-pack-to-strong on most rows and exceptional on a handful, which is what actually moves an average that far above the next-best 2B model (33.2).

The training recipe: SFT, then RL teachers, then one distillation pass

expand

Reading the diagram left to right: base pretraining runs stable training, a short 4K-context decay phase, then a long decay phase that extends context in stages up to 512K (a training-curriculum detail — the shipped model's documented, served context is 131,072, matching max_position_embeddings). Mid-training then runs two more context stages on top of that base — 600B tokens at 32K context, then 400B tokens at 128K context — before 400B tokens of "deep-thinking" SFT (released as UltraData-SFT-2605, with the agent-specific portion as UltraData-SFT-Agent-2609, ~500K samples).

From the SFT checkpoint, training branches into parallel RL teachers — reasoning, general-task, and agentic tracks, each producing multiple expert checkpoints (16 total, 5 of them agentic) — using UltraData-RL-2609 (80K+ samples spanning math, code, general knowledge, and long-context reasoning). The reasoning-RL reward design is explicitly credited to JustRL ("Scaling a 1.5B LLM with a Simple RL Recipe," arXiv:2512.16649) — a real, citable paper. A separate piece of the recipe, the critic-based algorithm used specifically for the RL+OPD stage, is credited to something called "JustRL II," linked not to arXiv but to a Notion writeup. That is the actual, verified link in the README — worth flagging because an arXiv id (2511.05963) has circulated attached to "JustRL II" in some launch commentary; that id resolves to an unrelated paper ("Next-Latent Prediction Transformers Learn Compact World Models"), not to this work. Cite the Notion page if you cite JustRL II at all, and treat it as an unreviewed writeup, not a paper.

Then On-Policy Distillation (OPD) merges those 16 expert models back into one release checkpoint: at each response position, it computes full-vocabulary reverse KL divergence between student and teacher logits as the advantage estimate — replacing the usual verification-based advantage — and reuses each teacher's own RL training prompts as distillation data, so no separate distillation corpus had to be built.

Added 18 September 2026: the GGUFs are carrying the job name

All of the above comes from prose on a model card. It turns out there is a second, accidental source, and it agrees. Every file in openbmb/MiniCPM5-2B-GGUF carries a general.finetune metadata key, and OpenBMB's converter filled it with the internal checkpoint path rather than blanking it. It is byte-identical across Q4_K_M, Q8_0 and F16:

general.architecture = llama
general.name         = MiniCPM5 2.6B
general.size_label   = 2.6B
general.file_type    = 15                 # Q4_K_M
general.finetune     = 0822_job_758798.iter_000144.opd.C26_mopd_v4_fv_pg_rkl_lr1e6.16tch.mg2hf

Two things there. First, OpenBMB's own tooling labels this model 2.6B, not 2B — which is the same 2,516,756,480 parameters rounded up instead of down, and a small vindication of spelling the number out earlier.

Second, that job string decomposes, and every fragment I can read maps onto something the README describes in words: opd and mopd for On-Policy Distillation; fv for the full-vocabulary objective; rkl for reverse KL; pg for policy gradient; mg2hf for a Megatron-to-Hugging-Face export. And 16tch — sixteen teachers, matching the sixteen RL expert checkpoints exactly, from a field the card never says aloud.

The two genuinely new numbers are iter_000144 and lr1e6: the released checkpoint is iteration 144 of the OPD run, at learning rate 1e-6. Neither appears anywhere on the card, in the GitHub repo, or in the JustRL II writeup. A 144-step merge is a short run — which is what you would expect from a distillation stage that reuses its teachers' own prompts rather than building a corpus, and it puts a concrete number on "one distillation pass." Flagging the obvious caveat: this is my reading of an internal job name, not documentation. C26, v4 and the job id are opaque to me and I am not going to guess at them.

The README states the net effect in two numbers: RL + OPD improves reasoning/general benchmarks by an average of +10.96 points over the SFT-only checkpoint, and agentic benchmarks by +6.96 points. Here is the chart that number comes from:

expand

Reading those gains benchmark-by-benchmark off that chart, rather than trusting the two averages alone — this is the same benchmark explorer from earlier, opened straight to its "what RL+OPD bought" tab:

NoLiMa gains 8.9 points, GPQA-Diamond gains 21.6, SWE-bench Verified gains 17.4, and τ²-Bench Telecom — already near-saturated at 92.98 after SFT — gains a comparatively modest 4.1. The gains aren't uniform, which is what you'd expect from a distillation step reusing each teacher's own training distribution rather than a generic capability boost applied everywhere at once.

One name from the release announcements that this article is not going to assert as fact: a training framework sometimes called "Meshy," described elsewhere as a scalable RL training system behind this release. It does not appear anywhere in the model card, the GitHub repository, or public documentation as far as this research could find — searched by name, directly, with nothing returned. If it's real, it shipped without a citable reference this article could locate; the RL-stack claims above are limited to what the README itself names and links.

The open release

Alongside the weights, OpenBMB released the training data itself:

DSpark: speculative decoding, and a speedup that doesn't decay smoothly

MiniCPM5-2B ships an open speculative-decoding draft, MiniCPM5-2B-DSpark. Per the community GGUF conversion of that draft, its GGUF architecture string is literally dflash — DSpark is built as DFlash plus a Markov head, which puts it in the same family this site covered in DFlash 2: the drafter already knew the answer, it just picked the wrong one, and adjacent to the confidence-scheduled verifier approach in DeepSeek's DSpark (a different, unrelated "DSpark" — DeepSeek and OpenBMB happened to land on the same name for different mechanisms). SGLang is the recommended serving path, with a purpose-built launch flag:

python -m sglang.launch_server \
  --model-path openbmb/MiniCPM5-2B \
  --trust-remote-code \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path openbmb/MiniCPM5-2B-DSpark \
  --speculative-dspark-block-size 7 \
  --port 30000

SGLang's own day-0 cookbook for this model publishes concrete single-GPU throughput numbers, reproduced in full below — worth being precise about what they do and don't establish, since a claim of "over 250 tok/s/user on coding tasks with DSpark enabled on a 5090" circulated around this release and this article could not confirm that specific framing against the cookbook page: the page's tables don't label whether DSpark was active for the runs, and don't mention a coding-specific workload — only a synthetic random benchmark (isl=1024, osl=1024).

Whatever the DSpark status of those specific runs, the community-built GGUF of the draft (aj9o9/MiniCPM5-2B-DSpark-GGUF) publishes a cleaner, fully-labeled comparison — same box, same target checkpoint, llama-benchy, decode-only, with and without the draft:

llama-server \
  --model MiniCPM5-2B-F16.gguf \
  -md MiniCPM5-2B-DSpark-F16.gguf \
  --spec-type draft-dspark \
  --spec-draft-n-max 7 \
  --host 127.0.0.1 --port 8080 \
  -ngl 999 -ngld 999 -fa on -np 1 -t 12 --jinja \
  -ctk q8_0 -ctv q8_0

VRAM cost is predictable and small — baseline 8,317MB, +1.8GB with the F16 draft loaded, 10.1GB total. The speedup is the interesting part precisely because it isn't a clean monotonic decay: strongest at 8k, inside the draft's 12,288-token training range, then weakening through 16k and 32k as the draft extrapolates past what it saw during training — and then recovering somewhat at 64k in this specific run. The uploader's own framing, worth repeating exactly: wall-clock speedup and accept length "are related but not the same number" — the official DSpark card's accept-length figures (~5.5 at T=0, ~4.1 at T=1.0 on in-distribution prompts) are reported to collapse toward ~1.6 well past the training range, which is a more pessimistic signal than the throughput curve above shows even at 64k. Both are real measurements; they're just measuring slightly different things, and only one number ships in most marketing.

For deeper context on why draft quality decays with distance from training data at all, and how other teams have addressed it, see this site's coverage of DFlash 2 and DSpark's speculative-decoding cousin over at LFM2.5 — the "does the speedup survive long context" question turns out to be a running theme across this entire family of drafters, not a MiniCPM5-specific quirk.

Deploying it: nine backends, nine chips, zero forks

Because MiniCPM5-2B is a stock LlamaForCausalLM, the GitHub repo ships cookbooks and matching Claude-Code/Cursor-style Agent Skills for every mainstream backend, with no custom kernel work required:

The released GGUF sizes, for reference — these are the numbers the VRAM budget explorer above uses:

Beyond the mainstream engines, OpenBMB partnered with the FlagOS Open Source Community to adapt the model across nine unrelated AI chip architectures via FlagRelease — Nvidia, Hygon, Metax, Iluvatar, Zhenwu, Mthreads, Kunlunxin, Ascend, and ARM-v9 — each getting its own published ModelScope/Hugging Face weights:

That ARM-v9 row is the one honest gap in this section worth naming directly: launch commentary around this release cited specific edge-hardware performance multipliers — Intel Core Ultra with OpenVINO, Arm Armv9 with SME2 giving roughly 1.7× prefill and 1.2× decode on mobile, and Rockchip RK3588/RK1828 numbers. This research checked the model card, the full GitHub repository (including every deployment doc and Agent Skill), and public search, and could not find those specific multipliers attached to MiniCPM5-2B anywhere verifiable. OpenVINO support for the MiniCPM5 family is real and documented by Intel — but for MiniCPM5-1B specifically in the material this research could locate, not confirmed for the 2B model at the cited multipliers. The FlagOS ARM-v9 row above is the verified edge-ARM claim this article can stand behind; the SME2 percentages are not repeated here because they couldn't be traced to a primary source.

Added 18 September 2026: the tenth backend is a browser tab

Ten days after this article went up, MiniCPM5-2B acquired a deployment target that was not on the nine-backend list: the browser. victor/MiniCPM5-2B-WebGPU-Pi is a static Space — no sdk: gradio, no server-side inference, no GPU allocated to it at all — created 11 September and last pushed 16 September. It runs the Pi agent loop, a just-bash shell, and MiniCPM5-2B itself in a web worker, over WebGPU, with a virtual filesystem in IndexedDB and weights cached in OPFS. Its own framing: "No inference API, API key, or shell server is involved."

That is a different claim from anything else in this article, and it is checkable, so here is the check.

What actually ships

The Space does not carry the weights. It pins a companion model repo, Mike0021/MiniCPM5-2B-ONNX, at revision 04a6c49, and downloads from there. That repo is an int4 ONNX export, not a GGUF and not a re-upload:

The provenance claim is the one worth verifying first, because it is the one nobody can check by looking. The conversion notes publish the SHA-256 they say was verified on the official BF16 checkpoint, opening 14fb8e7f and closing b64402c. Hugging Face returns an LFS file's SHA-256 in the x-linked-etag header, so:

curl -sSL -I https://huggingface.co/openbmb/MiniCPM5-2B/resolve/\
abe115e887989b14f05e64a3b260648329324c3f/model-00000-of-00001.safetensors \
  | grep -i x-linked-etag
# x-linked-etag: "14fb8e7f0a18d53d1f239773758bf581cee7e456a4523a54622c3a245b64402c"

Exact match, and the same hash is still on main, so the pinned revision is not a moved target. This is a converted derivative of the official weights, from the official weights — not a re-quant of somebody's GGUF, which is what "community ONNX build" usually turns out to mean.

Measuring the download instead of quoting it

The Space says "The first download is about 1.84 GB." Asking the CDN rather than the card, with curl -sSL -I and reading Content-Length after the redirect:

1,844,177,822 bytes. "About 1.84 GB" is exact to three decimal places, which is not the direction these numbers usually round. The eight onnx/ files sum to 1,834,167,351 — the repo's usedStorage to the byte — and the remainder is the tokenizer and five small JSON files.

The interesting part is where those bytes are. onnx_data_5 and onnx_data_6 are 267,386,880 bytes each, and 2 × 267,386,880 = 534,773,760 = 130,560 × 2,048 × 2. That is the entire input embedding table at FP16, 510.00 MiB, unquantized, living in four 127.5 MiB pieces — exactly the "lossless splitting of the 510 MiB embedding allocation into four 127.5 MiB tensors" the conversion notes describe, visible in the shard sizes without opening a file.

Why cut it up at all? Because WebGPU has per-binding limits that a desktop GPU does not. The Space's own testing notes say inference passed "with the GPU device capped at a 128 MiB storage binding limit and 256 MiB buffer limit" — and 127.5 MiB is the largest round fraction of 510 that clears 128. One tensor that a CUDA allocator would never notice is, in a browser, the single hardest object to place.

The cost moves from the cache to the vocabulary

This is where the browser build inverts the entire argument of this article. At 131,072 tokens the 2-KV-head geometry means the cache is 5.637 GB and the weights are a rounding error. At the browser's 8,192-token budget the same 43,008 bytes per token comes to 352,321,536 bytes — 336 MiB, and the largest object in the tab is not the cache at all. It is that FP16 lookup table, 510 MiB of it, 29.2% of everything downloaded.

Which also explains an oddity in the file sizes: a 4-bit browser build (1.834 GB) is 17.5% larger than the 4-bit GGUF of the same checkpoint (1.561 GB). Parsing the GGUF's own tensor table settles why. In MiniCPM5-2B-Q4_K_M.gguf, token_embd.weight is held at Q4_K and the separate output.weight at Q6_K — tie_word_embeddings: false means llama.cpp gets to treat them differently, and it spends 150,405,120 bytes on the input side. The ONNX export spends 534,773,760 on the same matrix. One tensor, 3.6× the bytes, and it is most of the gap between the two files.

The export does claw some of that back on the output side, and the mechanism is worth naming because it is specific to this deployment. The conversion selects "last-position logits ... before the output projection" — the lm_head runs on one position instead of all of them. At the 1,890-token prompt they benchmark, all-position logits would materialize 1,890 × 130,560 × 2 bytes = 470.7 MiB of logits; last-position only needs 255 KiB. They measure a 9.5% first-token-latency improvement against their own identical-weight control, which is the right comparison to run and the right size of claim to make about it.

What it does and does not establish

Speed, on the hardware they tested: "warm local benchmark medians were about 35–42 decode tokens/s" on an AMD Radeon 8060S under Chromium. Set next to the SGLang cookbook's 496 tok/s single-user on an RTX 5090 and the DSpark GGUF's 109 tok/s baseline on a desktop box, a browser tab is roughly a third of local llama.cpp and a twelfth of a 5090 — which is a perfectly reasonable price for "no install, no key, no server."

The quality accounting is unusually candid for a community conversion, and it does not flatter itself:

Note the denominator: 41 prediction positions from six prompts. That is a conversion smoke test, and they say so — "scoped conversion checks, not a broad quality evaluation." The row that matters is the int4 one: 36 of 41, a mean KL four orders of magnitude above the FP16 row, and a written admission that "four-bit quantization has measurable quality loss," with a specific reproduced failure ("ineffective HTML/CDN edits and long, inconclusive reasoning" on a workflow the BF16 model completes). Two of the four rows in that table only compare the conversion to itself, which is worth saying out loud: they establish that packaging and the browser runtime are faithful, not that the model is.

And the honest gap, in their words: "Physical Android/iOS devices have not been tested." The 128 MiB-binding test is a desktop GPU with limits artificially lowered — it proves the tensor splitting works under phone-like limits, not under phone memory pressure or thermals. A 1.84 GB download into OPFS is also not a mobile-shaped ask. "Runs in your browser" is established for desktop Chromium with WebGPU and shader-f16; "runs on your phone" is not established by anything in this release, and the Space does not claim it.

What it does establish is the thing this whole article has been circling. MiniCPM5-2B is a stock LlamaForCausalLM with 2 KV heads, and that decision keeps paying out in places OpenBMB did not have to plan for: an ONNX exporter already knew this graph, a WebGPU runtime already had kernels for it, and a 336 MiB cache at 8K context fits in a tab. The nine-backend table above was assembled by OpenBMB. The tenth one was assembled by somebody else, in a weekend, without asking.

Quickstart

# vLLM
pip install "vllm>=0.21"
vllm serve openbmb/MiniCPM5-2B --port 8000

# SGLang
pip install "sglang[srt]>=0.5.16"
python -m sglang.launch_server --model-path openbmb/MiniCPM5-2B --port 30000

# Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
 
model_id = "openbmb/MiniCPM5-2B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
 
messages = [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    enable_thinking=True, return_dict=True, return_tensors="pt",
).to(model.device)
 
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Recommended sampling for the "Think" mode: temperature=1.0, top_p=0.95. Tool calling emits XML-style calls; SGLang's built-in minicpm5 parser converts them to OpenAI-compatible tool_calls natively — start the server with --tool-call-parser minicpm5 (or auto), then send a standard OpenAI-style tools=[...] request and it just works, no custom parsing required on the client side. For the engines this site has covered end to end, see SGLang: the tree, and the language nobody remembers and vLLM: what PagedAttention turned into — both apply directly here, since nothing about MiniCPM5-2B asks either engine to do anything model-specific.

Beyond the card: what Artificial Analysis independently measured

Everything above comes from the model card or a GitHub repo commit MiniCPM5-2B ships in. One more data point circulates around this release that deserves its own section precisely because it is not in the card: Artificial Analysis independently benchmarked the model and published its own Intelligence Index score, separate from anything OpenBMB self-reports.

Per Artificial Analysis's own published article on the release: MiniCPM5-2B scores 15 on their Intelligence Index — "the highest Intelligence Index of any open weights model under 4B total parameters," 4 points clear of Granite 4.2 3B (11), and 1 point ahead of Qwen3.5 4B (14, estimated) "with 44% fewer parameters." It sits "level with Qwen3.5 9B (Reasoning, 15, estimated) at roughly 4x its size" — level with, not above, which is a meaningfully more modest claim than "beats models 4x its size" and worth stating precisely rather than rounding up. On token efficiency: MiniCPM5-2B used 19k output tokens per Intelligence Index task (11k of them reasoning tokens), "joint-lowest in the comparison model set with Granite 4.2 3B (19k)" — tied for lowest, not uniquely lowest, and notably not the 21k-vs-19k framing that has circulated for this comparison; by Artificial Analysis's own published figure, MiniCPM5-2B and Granite 4.2 3B post the identical 19k token budget. On agentic evaluation specifically, GDPval-AA v2 Elo of 831 "leads <4B models," about 110 points ahead of Ling 3.0 Tiny (718) and about 180 ahead of Granite 4.2 8B (647). OpenBMB's own launch post on X separately states a score of 20 on an "Agentic Index" — that specific figure comes from OpenBMB's own announcement, not from Artificial Analysis's article, and this research could not retrieve the full post to check its surrounding context (it returned an access-restricted response rather than the page).

What this article can and can't stand behind

To close where it opened: there is no MiniCPM5 tech report, and everything specific to this model in this article traces back to a model card, a GitHub repo, or an independent third-party benchmark — never a peer-reviewed source. Reproduced and verified directly against a primary source: the config shape, the 2.52B parameter count, the KV-cache arithmetic (to the byte), the 34-benchmark average (recomputed, not just quoted), the RL+OPD gain figures, the DSpark GGUF throughput table, the SGLang single-GPU numbers, the FlagOS nine-chip list, and the UltraData/UltraX dataset sizes. Added on 18 September and measured rather than quoted: every file size in the WebGPU build (1,844,177,822 bytes, against a stated "about 1.84 GB"), the SHA-256 chain from that build back to the official BF16 checkpoint, the FP16 embedding's 510 MiB and its four-way split, and — from the shipped GGUFs' own general.finetune string — sixteen OPD teachers, 144 iterations and a 1e-6 learning rate.

Explicitly not verifiable from anything this research could locate: the "Meshy" training-framework name, the specific Arm SME2 / Intel OpenVINO / Rockchip RK3588+RK1828 performance multipliers, and the full text of OpenBMB's own Agentic Index claim. Add to that list, as of the update: any evidence that the browser build runs on a physical phone, which its own authors are careful to say they have not tested. All three categories are listed here on purpose — a model card this detailed deserves to be read exactly as carefully as it was written.

@article{minicpm4,
  title={Minicpm4: Ultra-efficient llms on end devices},
  author={MiniCPM, Team},
  journal={arXiv preprint arXiv:2506.07900},
  year={2025}
}

That's the citation the MiniCPM5-2B README itself asks you to use — for MiniCPM4, still, in 2026.

Update — 2 October 2026

A community EXL3 build gives the model a non-GGUF path: ewin-reg/MiniCPM5-2B-EXL3-Quantized (created 15 September). Its quantization_config.json declares 4.0 bits per weight with a 6-bit head (verified), and the single shipped model.safetensors is 1,730,109,447 bytes — 1.61 GiB (measured), matching the "1.61 GB" the announcement quotes. This is the EXL3 format, loaded with ExLlamaV3 and served through TabbyAPI, rather than the GGUF/llama.cpp stack the body of this article measured. The throughput figure — ~68–70 tokens/s on an NVIDIA Tesla T4 (reported) — is the publisher's, not re-run here.

What 4-bit EXL3 buys is the weight footprint, not the window. The 131K-token context this article leads with is a property of the 8:1 GQA ratio and the KV-cache arithmetic, and it does not move when you change how the frozen weights are stored. A Tesla T4 holds 16 GB; with the weights down to 1.61 GiB, almost all of that card is left for exactly the long-context KV cache the piece prices out — which is the regime that 68–70 tok/s is measured in.