LFM2.5-DSpark: the best draft model in the release is the slowest one

2026-08-23 · 10 min · speculative-decoding · edge-inference · moe · liquid-ai · explainer

Liquid AI released DSpark draft models for three LFM2.5 checkpoints on 20 August. The headline is the usual shape — up to 3.18× on an H100, up to 2.87× on a MacBook — and the headline is fine. It is not the interesting part.

The interesting part is that the release contains a clean, published counterexample to the metric everyone uses to compare speculative drafters. LFM2.5-8B-A1B's draft has the highest acceptance length of the three — 6.95 tokens of a possible 10, against 5.02 and 4.81 for the dense models — and the worst realized speedup on device, at 1.18×. On MT-Bench it accepts 8.52 tokens per pass and returns 1.04×, which is to say nothing at all.

Liquid publish that table, explain it in two sentences, and label it "a subject of subsequent work". Those two sentences are worth expanding, because what they describe is not a bug in llama.cpp's Metal backend. It is arithmetic that applies to every sparse model.


Who	Liquid AI · 20 Aug 2026 · first speculative decoding release for LFMs
Models	draft checkpoints for LFM2.5-1.2B-Instruct, LFM2.5-2.6B, LFM2.5-8B-A1B
Size	~300M parameters each · 5 layers · block size 9
Integration	upstream in llama.cpp and SGLang · Safetensors and GGUF
Quality	greedy output identical to baseline by construction — benchmark accuracy unchanged
Agentic	BFCL function-call latency on a MacBook: 3.5 s → 1.5 s, a 57% cut
Trained	exclusively on AMD hardware, in Liquid's own framework

LiquidAI/LFM2.5-1.2B-Instruct-DSparkhugging face · snapshot 2026-09-08

parameters: 295.7M
repo size: 591.5 MB
architecture: Lfm2DSparkDraftModel
license: other
downloads: 1.9K
likes: 32
files: 5

parameters by dtype

BF16 295.7M

What DSpark is

Speculative decoding exists because decode is memory-bound: most of the latency is streaming weights from DRAM into SRAM, not arithmetic. A small draft proposes k tokens, the target verifies all of them in one pass, and the weight load is amortized across however many survive. This is true on an H100 and true on an iPhone, which is the property that makes it interesting for a company whose whole thesis is running on the device.

DSpark combines three pieces:

A DFlash-style parallel backbone, conditioned on context features from the target model, that runs one forward pass over a block and produces hidden states and base logits for all k draft positions at once.
A lightweight sequential head, modelled as a Markov chain between neighbouring tokens, biasing each position's logits toward continuations consistent with the token sampled just before it. Parallel drafting has no dependency between positions; this adds it back, which raises acceptance at later positions.
A confidence-scheduled verifier: a separate head predicts each draft token's acceptance probability conditioned on all previous ones being accepted, and a hardware-aware scheduler prunes low-confidence suffixes when verifying them would cost more batch capacity than they are worth.

A three-step diagram. Step 1, target model prefill: tokens A, B, C enter the target model, which emits D. Step 2, draft model proposes block: D plus three mask tokens enter a Parallel Block producing four sets of logits, then a Sequential Block adds dependency between neighbouring positions to yield candidates E, F, G, H, each with a confidence score, which a Hardware-Aware Prefix Scheduler uses to keep E, F and G and drop H. Step 3, target model verifies: D, E, F, G go through the target model in one pass; E and F are accepted, G is rejected and replaced by the target's own token G-star. — The three pieces: a parallel block draft, a sequential head that makes neighbouring positions agree, and a scheduler that trims the tail before verification. (Liquid AI, LFM2.5-DSpark.)

That third piece has a quiet fate worth noting. In the concurrency experiments, Liquid turn it off: "DSpark's confidence head can dynamically trim how many tokens to verify per request, but in our experiments the tokens it drops cost more than the compute it saves, so we serve a fixed verify window instead." A component of the published architecture, shipped in the checkpoints, disabled in the results — reported plainly. I would rather read that than not.

The training result nobody asks for

Before the speed tables, there is a small methodological finding that I think is the most transferable thing in the post.

Three side-by-side dual-axis charts of mean acceptance length and validation loss against training tokens, for LFM2.5-1.2B-Instruct over 15 epochs of 12.1B tokens, LFM2.5-2.6B over 15 epochs of 61.2B tokens, and LFM2.5-8B-A1B over 13 epochs of 27.1B tokens. In every panel validation loss falls smoothly and monotonically. Acceptance rises with it for the 1.2B, rises then declines after about 300 billion tokens for the 2.6B, and oscillates without trend for the 8B-A1B. — Validation loss falls smoothly in all three panels. Acceptance tracks it in one of them. (Liquid AI, LFM2.5-DSpark, Figure 1.)

The 1.2B draft behaves the way you would hope: acceptance climbs as loss falls, and the two agree about when to stop. The 2.6B draft peaks around 300B tokens and then declines for the remaining 600B while its loss keeps improving. The 8B-A1B draft oscillates between 6.8 and 7.2 with no trend at all, its best checkpoint arriving at 50B tokens out of 350B.

So Liquid select the epoch with the highest acceptance rate rather than the lowest loss. That is obviously right and almost nobody does it, because loss is the number your training loop already has. The general form: a draft model's loss is not its objective. It is trained by distillation on next-token prediction; it is used as a proposal distribution whose value is how often the target agrees with a whole block of it. Those two are correlated until they are not, and this figure is a picture of the point where they stop being.

The tables

block 9 · batch size 1 · temperature 0 · greedy output identical to baselinemean 1.18× · accepts 6.95 of 10

MoE · reasoning · 327.7M draft

MATH500

8.27

1.20×93 → 112

HumanEval

7.02

1.11×91 → 101

MBPP

6.93

1.09×89 → 97

GSM8K

4.02

1.43×90 → 129

MT-Bench

8.52

1.03×87 → 90

mean

6.95

1.18×90 → 106

acceptance length, of a possible 10realized throughput speedup (tok/s)

Select the 8B-A1B and stay on the MacBook. Its draft is the best of the three by the metric that is supposed to matter — 6.95 accepted tokens of a possible 10, against 5.02 for the 1.2B — and it delivers 1.18×, where the 1.2B delivers 2.54×. On MT-Bench it accepts 8.52 tokens per pass and returns 1.04×: essentially nothing.

Switch to the H100 and the same checkpoint returns 2.54×. Nothing about the draft changed. What changed is that verifying nine tokens through a mixture-of-experts touches far more expert weight than decoding one does, and on a machine whose Metal MoE path is not built for it, that extra traffic eats the entire win. Acceptance length is a property of the draft model; speedup is a property of the draft model and the machine, and this table is the cleanest illustration of the difference I have seen published.

The dense models do what the technique promises. LFM2.5-1.2B-Instruct averages 2.54× on the MacBook, peaking at 2.87× on HumanEval — 136 to 389 tok/s, which is a different category of interactive. LFM2.5-2.6B averages 2.27× on device and 2.67× on the H100. Liquid note that on-device throughput for the 1.2B now "far exceeds the throughput offered by most proprietary cloud models".

There is real variance in there worth not glossing: for the 1.2B, speedup ranges from 1.66× on MT-Bench to 2.56× on MATH500 on the H100 — a 52% spread driven entirely by the distribution of the underlying text. Speculative decoding is a bet on predictability, and conversational text is less predictable than mathematics.

Then the 8B-A1B, where the draft is best and the outcome is worst.

Why the MoE loses

Liquid give the reason in one clause: verifying k tokens "activates more experts and thus more weight traffic than a single decode step". Here is what that means quantitatively, because the shape of it is the whole story.

expert weight touched by one verification pass of 9 tokens0.57× the weight traffic per emitted token

active fraction12.5 %

block size k9 tok

expert reuse35 %

accepted6.95 tok

expert weight touched

50%

one decode step touches 12.5%

traffic per pass

3.99×

relative to one decode step

per emitted token

0.57×

still cheaper than decoding one at a time

Drag active fraction to 100% — a dense model — and both dashed bounds collapse onto each other at 100%: verifying nine tokens streams exactly the same weights as decoding one, so the whole block is free and the speedup is just the acceptance length. That is the case speculative decoding was designed for, and it is why the two dense LFM2.5 drafts get 2.3–2.9× on the same laptop.

Drag it back to an eighth and the curve climbs steeply before flattening. That climb is the bill: a sparse model’s central advantage is that one token only reads a slice of it, and verifying a block is precisely the operation that gives that advantage back. How much depends entirely on expert reuse across the block — routing locality is what stands between the two dashed lines — and on a backend whose MoE kernels are not written for wide verification, whatever reuse there is goes unclaimed. Hence 1.18× on a draft that accepts 6.95 tokens.

A dense model reads all of its weights to decode one token. Verifying nine tokens reads the same weights, once. The extra work is free — that is the entire premise of speculative decoding, and it is why the dense LFM2.5 drafts do fine on the same laptop with the same backend.

A sparse model reads a slice. LFM2.5-8B-A1B activates about 1B of 8B parameters per token, and its advantage over a dense 8B is precisely that it does not have to touch the other seven. But nine tokens route to nine different slices, and the union is much larger than any one of them. Verifying a block is the operation that gives a sparse model's advantage back.

How much of it comes back depends on how much the routing overlaps across the block, which is exactly the temporal locality that MoE serving engines like FreeToken build their caches on. If overlap is high, the union stays small and speculation still wins. If the backend's kernels are not written to exploit that overlap during wide verification — which is what "the current MoE implementation in llama.cpp's Metal backend" amounts to — none of it is claimed and you pay the independent-routing price.

So this is half implementation and half structural, and the honest version is: on a backend built for it, sparse verification is a caching problem with a good solution. On a backend that is not, a better draft model buys you nothing. The 2.54× the same checkpoint gets on the H100 is the proof that the draft was never the problem.

The part that will actually matter

A horizontal bar chart of end-to-end function-call latency on BFCL v3 across six categories — simple, multiple, parallel, parallel_multiple, live_simple and live_multiple — comparing LFM2.5-2.6B with DSpark against no speculation. With DSpark every category falls between 1.1 and 1.9 seconds; without it, between 2.6 and 4.0 seconds. Each category is annotated with its acceptance length, ranging from 4.42 to 5.72. — Where a two-times decode speedup stops being a benchmark number. Six BFCL categories on an M4 Max, 50 requests each: a mean of 3.5 s falls to 1.5 s. (Liquid AI, LFM2.5-DSpark, Figure 2.)

Liquid's stated goal for LFM2.5-2.6B was to make it "the first viable on-device agentic model", and the BFCL chart is the argument. In an agentic loop the model reasons before every tool call and the user waits through all of it — so decode latency is not a throughput statistic, it is the entire perceived responsiveness of the product. Cutting 3.5 seconds to 1.5 across six function-calling categories is the difference between a local agent that feels like a tool and one that feels like a demo.

embedding and LM head tied to the target — not carried by the draft+4.1% weights

decoder stack — 5 layers

241.2Midentical across all three drafts

hidden-state projection

21.0Mtakes the target's context features

Markov head

65.5Mthe sequential bias between neighbouring draft tokens

norms + confidence head

27.5kpredicts each draft token's acceptance probability

the draft

327.7M

against a 8.0B target

memory overhead

+4.1%

weights only, no extra KV

what it buys

1.18× / 2.54×

M4 Max / H100, mean over five tasks

Click through the three targets and watch the two right-hand boxes move in opposite directions. The 1.2B pays the most for its draft — nearly a quarter of its own parameter count — and gets 2.54× on a laptop. The 8B-A1B pays 4.1% and gets 1.18×. The cheapest draft is the one that is not worth having, at least on that backend.

The reason the drafts are this small at all is the last line of the table’s caption: embedding and LM head are tied to the target rather than duplicated. A 300M draft that had to carry its own vocabulary projection would be most of a billion parameters, and the whole trade would look very different on an 8 GB device.

And the cost of that is worth stating precisely, because "minimal memory increase" is doing some work in the announcement. A ~300M draft against a 2.6B target is +12.6% of weights; against the 1.2B it is +24.6%. That is not nothing on a phone. It is, however, a very good trade at 2.3–2.9×, and it is only that small because the embedding and LM head are tied to the target instead of duplicated.

What I would want next

The confidence head needs a machine where it pays. It is trained, shipped, and switched off. Liquid say the tokens it drops cost more than the compute it saves — on an H100 at moderate concurrency, which is exactly the regime where verification capacity is cheap. The scheduler is described as hardware-aware; the hardware where trimming a low-confidence suffix should obviously win is the 8 GB laptop, which is the configuration it was not evaluated on.

The MoE result deserves a locality measurement, not a promise. "Subsequent work" is fair, but the diagnostic is cheap: replay the routing traces and report how many unique experts a nine-token verification actually touches against one decode step. That single number decides whether this is a kernel problem worth fixing or a ceiling.

Block size 9 is fixed everywhere. Chosen by ablation on a subset, then applied to all three models across five datasets and two backends. Given that acceptance varies from 3.90 to 8.52 across those cells, the optimal block almost certainly does not — and on the MoE, block size is the direct knob on the weight-traffic curve above.

Fifteen epochs on the same corpus. The 2.6B draft trains on 61.2B tokens fifteen times over and its acceptance declines through the last two-thirds of that. Whatever is happening there — memorization, distribution drift away from the target's own outputs — it looks like a data problem being solved with an early-stopping rule.

The line worth keeping

Every speculative decoding release leads with acceptance length, and it is the right metric for the draft model in isolation. This release accidentally documents its limit: the draft that agrees with its target most is the one that made the least money, on the machine it was built for, because the pass that checks the agreement costs differently on different architectures.

Acceptance is a property of two models. Speedup is a property of two models and a machine. Publishing a table where those disagree by a factor of two, and saying so, is more useful than another 3× headline.