2026-08-14 · 7 min · vlm · open-weights · on-device · edge · benchmarks · explainer
Liquid AI released LFM2.5-VL-3B on 2026-08-12 — a 3.1B open-weight vision-language model aimed at edge deployment. It is a non-reasoning model by design: it answers directly, which keeps latency low and, as it turns out, is the single decision that most of the performance story comes back to.
The blog lists four improvements over LFM2-VL-3B: screen understanding, function calling, grounding, and multi-image input. Three of those are ordinary gains. One of them is a capability appearing from nothing, and it is not the one the post leads with.
The row that changed
The two InternVL models still lead this row, and the gap is not trivial — 84.1 against 80.7. What changed is that GUI grounding went from a capability this line did not have to one where it beats every Gemma and Qwen model in the comparison, including ones two and a half times its size. For a model whose stated purpose is running on the device the screen belongs to, that is the row that matters most.
LFM2-VL-3B scored 6.0, 7.6 and 2.5 on the three ScreenSpot-v2 splits. Those are not weak scores; they are the scores of a model that could not do the task. LFM2.5-VL-3B scores 78.7, 81.2 and 82.2 — an average of 80.7 against 5.4, a 15× move filed under "significant improvements in screen understanding."
Liquid AI's own framing compares outward rather than backward: 80.7 "far ahead of the much larger Gemma-4-E4B (51.2) and Qwen 3.5 4B (78.5) and close behind the larger InternVL-3.5-4B (84.1)." I recomputed all four averages from the published splits and they reproduce exactly. But the comparison that tells you what happened is the one with its own predecessor.
For a model whose stated purpose is running on the device that owns the screen, this is the row that decides whether it is useful.
Architecture, and where the parameters went
The vision path is a SigLIP2 400M NaFlex encoder — "NaFlex" meaning it handles native aspect ratios rather than forcing a square crop — feeding a token-compression stage built from PixelUnshuffle plus an MLP connector. The diagram makes the compression ratio visible: four encoder tokens become one model token.
That compression is why the time-to-first-token numbers work. A heavier encoder produces more tokens, and every one of them has to be processed before the first output token appears.
On the language side, LFM2.5-VL-3B builds on the same pre-trained base as the LFM2.5-2.6B text model, pre-trained on roughly 34T tokens. Two details worth pulling out:
- The vocabulary was doubled to 128K by extending the existing tokenizer in place, specifically to support non-Latin scripts. Extending in place rather than retraining a tokenizer keeps the existing embeddings valid, which is the cheap way to do this and not the usual one.
- Vision pretraining was scaled 4× in tokens, with a mixture of curated and synthetic image-caption, OCR, grounding and instruction-following data. The grounding gain — RefCOCO precision@1 from 57.1 to 87.9 — is attributed directly to scaling synthetic grounding data.
Post-training is SFT with knowledge distillation from a larger teacher, plus something Liquid AI calls Antidoom training, followed by multi-reward RL.
The size-class claim, checked
published average row: 69.4 · 57.2 · 52.0 · 59.7 · 64.6 · 69.4 · 63.7 · 70.1
The averages I recompute from the 28 rows match Liquid AI’s published Average row on every model, so the table is internally consistent. The claim it supports is a narrow one, stated carefully: LFM2.5-VL-3B averages 69.4, exactly level with InternVL 3.5 4B at 4.7B parameters and 0.7 behind Qwen3.5-4B, also 4.7B. It beats both Gemma models, which are 5.1B and 8B. So “competitive against models twice its size” is true, and “better than models twice its size” would not have been.
I transcribed all 28 benchmark rows and recomputed each model's average. Every one reproduces Liquid AI's published Average row to within rounding, so the table is internally consistent — worth doing, because the claim rests entirely on that average.
The claim is narrow and it is stated precisely: LFM2.5-VL-3B averages 69.4, which is exactly level with InternVL 3.5 4B (69.4, at 4.7B parameters) and 0.7 behind Qwen3.5-4B (70.1, also 4.7B). It beats both Gemma models, at 5.1B and 8B.
So "competitive vision performance against models twice its size" is supported. "Better than models twice its size" would not have been, and the post does not say it.
The head-to-head view makes this sharper than the averages do. Counted row by row across all 28 benchmarks, LFM2.5-VL-3B is 14W–14L against InternVL 3.5 4B and 14W–14L against Qwen3.5-4B — a dead tie against both 4.7B models, from two different labs. Against the rest it is comfortably ahead: 27–1 over Gemma-4-E2B, 23–5 over Gemma-4-E4B, 22–6 over both 2B-class models.
Where it loses is worth naming. Qwen3.5-4B takes the document-heavy rows — DocVQA 94.8 to 91.1, InfographicVQA 80.3 to 70.2, OCRBench v2 58.7 to 47.5 — and MMMU-Pro 36.0 to 30.5. InternVL 3.5 4B takes ChartQA, MMMU, and all three GUI splits. If your workload is dense document OCR, the larger models are still worth their size.
One row moves the wrong way and the post does not mention it: CountBenchQA drops from 92.2 to 87.3, the only benchmark where the new model is meaningfully behind its predecessor. POPE also slips slightly, 89.2 to 88.7.
Function calling, added to a VLM
New to the VL line: ToolSandbox goes 26.4 → 59.5 and BFCL v4 20.5 → 32.5. The blog positions this as "on par with Gemma-4-E2B and ahead of Qwen3.5-2B," which understates it — 59.5 beats Gemma-4-E2B's 56.5 and Qwen3.5-2B's 47.7, and only Gemma-4-E4B (61.6, at 8B) and Qwen3.5-4B (65.0) are ahead.
Both InternVL models are marked N/A because they do not support function calling at all. That is the more interesting fact in the row: on the benchmark where InternVL was beating LFM on GUI grounding, it cannot compete, and a model that can both locate a button and call a tool is a different product from one that can only do the first.
The text-only instruction-following numbers are less flattering. IFEval 82.3 sits behind both Gemma models (83.0 and 87.9); Multi-IF 59.4 is well behind their 69.4 and 77.4. This is a vision model with tool use bolted on competently, not a text model that also sees.
Speed, and what is actually specified
The GPU numbers are the well-specified ones — vLLM 0.26, BF16, a 512×512 image plus 1,024 input tokens, median of five runs on one H100 SXM5. The on-device figures come with no stated quantization, prompt or batch size, so they are worth reading as claims rather than measurements. Both sets point the same way: the design is spending its parameter budget on being answerable immediately rather than on being right after thinking, and time-to-first-token is where a non-reasoning model collects.
The GPU measurements are properly specified: vLLM 0.26, BF16, a 512×512 image plus 1,024 input tokens, up to 256 output tokens, median of five runs per concurrency level, single H100 SXM5. On a 5-frame clip LFM2.5-VL-3B returns its first token in about 34 ms where the Gemma models take around 200 ms. Sustained output throughput reaches roughly 11K tokens/s at high concurrency — about 2× the 4B-class models — which Liquid AI works out to nearly 1B output tokens per day from one GPU.
The on-device figures are the ones to be careful with: 228 tok/s on an Apple M5 Max, 116 on an AMD Ryzen AI Max+ 395, 20 on a Galaxy S26 Ultra, in about 3 GB. No quantization, prompt, or batch size is stated for any of them. Given that GGUF, MLX and ONNX builds all ship day one and would each give a different answer, these should be read as claims rather than measurements.
What it is for
The honest summary is that this is a screen-and-document model that fits on a phone. It ties two 4.7B models on a 28-benchmark average, loses the dense-OCR rows to both, wins the real-world and grounding rows, and gained GUI grounding and tool calling in one release.
The non-reasoning choice is the through-line. It costs accuracy on the STEM benchmarks where thinking helps — MMMU-Pro 30.5 is the weakest column in the table — and buys 34 ms to first token and 11K tokens/s sustained. For an agent that has to look at a screen, decide where to tap, and do it again, that is the correct trade. For a model asked to reason about a diagram, it is not.
Two things I could not check: Antidoom training is named but not described anywhere in the post, and the on-device numbers have no stated conditions. Everything else in the table reproduces.