Mercury 3: a diffusion LLM at 1,068 tokens/s, and where the 4-8x goes in routed traffic

On 20 October Deedy Das posted that "Inception just dropped Mercury 3, a model 4-8x faster than Luna / Haiku for the same price and benchmark performance", and that "In real world use, OpenRouter shows that it's effectively 2-4x faster and 2-4x cheaper effective price." His source was Inception's launch post the day before: Mercury 3 runs "at over 1,000 tokens per second", outperforms GPT-6 Luna and Claude Haiku 5.5 "across a range of benchmarks", is "More than 40x the size of Mercury 1", and has a 1M-token context.

Mercury 3 is not an autoregressive model. It is a diffusion language model: it drafts a block of tokens at once and refines the draft over a few passes. That is the reason the speed claim is plausible at all, so I start with the mechanism. Then I check every launch number against what Inception actually published. That means its blog and charts, its 2025 Mercury tech report, its API docs, and OpenRouter's public model API, all read on 23 October 2026.

What I could not do: there is no Mercury 3 paper, no weights and no system card, and I made no calls to any hosted model. Every speed and quality number below is reported (the publisher's figure, not re-run), measured (I counted or computed it from a published file or chart), or reasoned (my arithmetic on the other two).

Why one token per pass is slow

An autoregressive model writes token $n+1$ only after token $n$ exists. Each new token costs one full forward pass. At small batch that pass is memory-bound: the GPU streams every weight out of memory to do a matrix-vector product's worth of arithmetic. The arithmetic units mostly wait. A 500-token answer is 500 sequential trips through the weights, and no amount of compute shortens a chain of dependencies.

A diffusion LM attacks the chain, not the trip. If one pass can fix several tokens, the same number of weight reads produces more text. That is the whole bet, and it is a bet about memory bandwidth, not about arithmetic: a pass over a 32-token block does more arithmetic than a pass over one token, but on a memory-bound GPU reading the weights once costs about the same either way.

How a diffusion LM writes

Diffusion for text replaces pixel noise with corruption of tokens. The masked-diffusion explainer derives it in full; here is the part Mercury needs.

Forward process. Take a clean sequence $x$ and corrupt it step by step into latents $z_1, \dots, z_T$, ending at a known noise distribution. In the masked (absorbing) variant, corruption replaces tokens with [MASK], and $z_T$ is all masks.

Denoiser. Train a network $p_\theta(x \mid z_t)$ to predict the clean tokens from a corrupted sequence. Mercury's tech report writes the loss as

where $\gamma(t) \ge 0$ weights each noise level, and $q$ is the corruption process. The network is an ordinary Transformer. The report is explicit that "Diffusion implies specific training and generation algorithms, but does not pose constraints on the architecture". Attention is not causal: every position reads every other one.

Sampling. Start from all masks. On each pass, predict every masked slot at once, commit the most confident predictions, and leave the rest masked for the next pass. A slot that has been predicted but not committed holds a draft that can still change. That is what Inception's diffusing stream mode shows: the whole answer text, re-sent as it is refined.

The catch. Committing $k$ slots from one pass samples each from its own marginal, not from their joint. Two masked slots that could read "hot dog" or "ice cream" can come out "hot cream". Fewer passes means more tokens per pass and more of this risk, which is the speed-quality dial every diffusion sampler exposes. LLaDA, Dream and iLLaDA all live on it.

Length and caching. Pure masked diffusion fixes the answer length in advance, and because attention runs both ways, no keys or values are final until the end, so there is no exact KV cache. The fix most of the field has converged on is block diffusion. Cut the answer into blocks. Diffuse inside the current block. Let it see every finished block, and treat finished blocks like an autoregressive prefix. Their keys and values cache exactly, and generation runs until a block emits an end token. Set Diffusion generalises the block, TwoTower gives the prefix its own causal network, and Flash-dLLM approximates a cache for the fully bidirectional case.

The widget runs a toy of that recipe: a 24-token answer, written left to right one token per pass, and with block diffusion committing $k$ tokens per pass inside blocks of $B$. With blocks of 8 and two commits per pass, it finishes in 12 passes instead of 24. With one block of 24 and four commits per pass, it finishes in 6. The faint words are drafts, and some of them are wrong until their neighbours are fixed.

What Inception says about its own sampler

Very little, and I want to be precise about which parts are documented and which are my inference.

• Reported (tech report, 2025). Mercury models are Transformers, and a denoising diffusion loss replaces the autoregressive one in every stage, including fine-tuning and alignment with RLHF or DPO. The method "extend[s]" Lou, Meng and Ermon's score-entropy discrete diffusion. Ermon is an Inception co-founder. Generation is "coarse-to-fine", "modifying multiple tokens in parallel". Serving uses "a proprietary inference engine" with "dynamically batched sampling and paging" and custom kernels.
• Reported (API docs, October 2026). Streaming is "block-by-block", where "A 'block' is a chunk of refined text the model emits as it decodes". Prefix caching is automatic, and cached input is billed at 10% of the input price. Temperature is clamped to 0.5–1.0, and there is no seed.
• Reasoned. Block-by-block streaming plus a prefix cache billed like an autoregressive model's is what block diffusion would look like from the outside: finished blocks are emitted and cached, and the current block is refined in parallel. Inception has not said so. It has also never published a block size, a step count or a remasking rule for any Mercury model, so the widget's numbers are illustrative, not Mercury's.

The report also states the limit of the idea: earlier diffusion LMs cut the number of forward passes "but fail to show improvements in wall-clock efficiency". Fewer passes only becomes speed with an engine and kernels built for it, and that engine is the part nobody outside Inception can read.

What changed from Mercury 1 to Mercury 3

Inception's lineage chart plots five models by size and by a quality average.

expand

I located each dot's centre in the published PNG and read it against the chart's gridlines (measured, to about a pixel):

So "40x" and "3x more intelligent" both hold on Inception's own axes: 50.6 / 15.9 is 3.2x (reasoned). Two things the chart does not say. The base of the size axis, Mercury 1 Mini, has no published parameter count, so "40x" tells you the ratio and nothing about the size. And Mercury 2 to 2.5 is roughly the same size, at 17.0x and 17.4x, so that 4.3-point step came from something other than scale: data, post-training or the sampler. From 2.5 to 3 the model grew 2.3x for 14.6 points (reasoned, from the measured positions).

Context has grown faster than anything else. The 2025 report gives 32,768 tokens natively and 128K with extension; Mercury 2 is 128K, Mercury 2.5 260K, Mercury 3 1M (reported, Inception's models page). Mercury 2 is described on OpenRouter as "the first reasoning diffusion LLM", and Mercury 3 keeps the four effort levels.

The benchmark table

expand

I counted the bold cells. Mercury 3 leads 8 of the 12 benchmark rows: GDPval, τ³-bench Banking, MultiChallenge, IFBench, GPQA Diamond, SWE-Bench Atlas, BrowseComp+ and DeepSearchQA (measured). It trails in 4. Luna leads AA-Omniscience at 46.94 to 46.44 and AA-LCR at 78.30 to 77.67. Haiku leads MRCR v2 at 78.40 to 75.50 and DeepSWE at 51.62 to 46.31. The margins where it wins are often large: 51.08 against 33.33 and 32.26 on SWE-Bench Atlas, and 76.84 against 61.82 and 54.65 on IFBench. These are reported scores; no third party has posted a run yet.

Three caveats sit in the fine print.

1. The headline index is not computable from the table. The scatter's "Intelligence Index" is "an unweighted mean across the benchmarks", and the lineage chart's footnote lists eleven of them. Three of the eleven, SciCode, SO Quality and Terminal-Bench v2.1, are not in the table. GDPval is an Elo near 1,200 and cannot enter an unweighted mean of percentages unrescaled. The unweighted mean of the table's eleven percentage rows is 61.8 for Mercury 3, 53.8 for Luna and 53.6 for Haiku (measured). The chart's index says about 50.6, 48.2 and 50.5. Neither is wrong; one is just not reproducible from what was published.
2. The name collides. Artificial Analysis also publishes an "Intelligence Index". OpenRouter's model API shows Artificial Analysis's values as 38.1 for GPT-6 Luna and 43.4 for Claude Haiku 5.5, and none yet for Mercury 3 (reported, read 23 October). Inception's index is a different scale with a different benchmark set. Its own scatter puts Mercury 3 about 0.2 above Haiku (50.64 against 50.46, measured from the dot centres) on a y-axis that spans only 47.5 to 51.5, which makes a near-tie look like a gap.
3. Medium effort, both ways. The comparison holds effort fixed by name, but "medium" means whatever each vendor decided it means, and none of them publish how many reasoning tokens it spends.

The speed claim: whose 1,068?

expand

Reading the dot centres off this chart gives 1,068, 239 and 130 tokens/s (measured), matching the table. The ratios are 1,068 / 240 = 4.45x over Haiku and 1,068 / 130 = 8.2x over Luna (reasoned). That is the "4–8x".

What is missing is the method. The blog says "1,068 tokens per second on NVIDIA GPUs". It does not name the GPU, the batch size, the prompt and output lengths, the effort level of the speed run, or who measured it. The comparators are given as "~130" and "~240", with no source. Artificial Analysis had no Mercury 3 page on 23 October; its model URL returned 404 (measured).

That is a step back from Mercury 1. The 2025 report attributes its speeds to "independent evaluations conducted by Artificial Analysis": 1,109 tokens/s for Mercury Coder Mini and 737 for Small, on H100s, with coding prompts of about 1,000 input and 1,000 output tokens, against the median of each rival across providers.

expand

Mercury 1's 1,109 and Mercury 3's 1,068 sit in the same band for a model Inception says is about 40 times larger. If both were measured the same way, that is the most interesting number in the launch. They were not measured the same way, as far as anything published shows, so I cannot say it.

What tokens per second does not count

Mercury's API counts reasoning tokens inside completion_tokens, and the launch compares the models at medium effort. Throughput is tokens per second, not answers per second. If Mercury 3 writes twice as many tokens as Haiku to reach an answer, the 4.45x becomes 2.2x in time to answer (reasoned). If it writes half as many, the gap doubles. None of the three vendors publishes tokens per answer for these runs, so the slider in the widget below is the honest form of the claim: a ratio with one unknown left in it.

What routed traffic shows

Deedy's second chart is OpenRouter's own data: the median (P50) throughput and the weighted average price actually paid, over three days.

expand

In routed traffic Mercury 3 is 302 / 79 = 3.8x GPT-6 Luna, 302 / 118 = 2.6x Luna's Fast tier, and 302 / 187 = 1.6x Claude Haiku 5.5 (reasoned). So "2–4x faster" holds against Luna and overstates the gap to Haiku. The larger story is how much each model keeps between the launch chart and the router. Mercury 3 keeps 302 of 1,068, 28%. Luna keeps 79 of 130, 61%, and Haiku 187 of 240, 78% (reasoned). The comparators' "~" numbers were already close to routed reality; Mercury's was not.

I can only offer explanations for that, and I am labelling them reasoned, not shown. A diffusion step's advantage is that it fills a memory-bound pass with more useful tokens. A busy server batching many requests also fills the pass, so the advantage shrinks as load rises, and the 1,068 may be a lightly loaded number. Short outputs leave less room for parallel blocks. OpenRouter also measures through its own proxy, with a definition of throughput I could not read from its public API, because the per-endpoint throughput fields are empty without a key. Each of these would move the number; I cannot tell which one did.

Price: "the same" for now

The table's "$0.10 / $0.35" is the launch price. The docs give the regular price as $0.20 per million input and $0.70 per million output, and neither the docs nor the blog says when the discount ends. At list price, Mercury 3 costs 2x Luna and Haiku per input token and 1.4x per output token (reasoned, from list prices). "Same price" is true for the discount window.

The effective-price claim needs the same care. Of the "2–4x cheaper", most is input. Paid input is $0.022 against $0.0434 and $0.0856, 2.0x and 3.9x. Paid output is $0.3594 against $0.4534 and $0.5698, only 1.26x and 1.59x (reasoned; one reply under Deedy's post made the same point). Paid input below a $0.10 list price means most input is served from cache. If all Mercury 3 traffic paid either $0.10 or the $0.01 cache rate, an average of $0.022 implies about 87% of input tokens were cache hits (reasoned).

The comparators also pay for things Mercury 3 does not charge. OpenRouter's model API lists cache-write fees for Luna and Haiku, and long-context tiers that raise Haiku to $0.50 / $2.50 above 100,000 prompt tokens, and Luna to $0.20 / $0.75 above 272,000. Mercury 3 has one rate across its 1M context (reported, OpenRouter model API). That, more than per-token list price, is why Haiku's paid output is above its list. For an agent that re-sends a long, cached transcript every step, those tiers matter more than the headline price.

What the customers say

The launch post quotes four early-access partners, and none of these numbers can be checked from outside. TinyFish says Mercury 3 completed 50% of 50 live Mind2Web navigation tasks, "significantly more than GPT-6.1-Sol". Octen reports a median generation time of 0.9 seconds and a median end-to-end response of 2.2 seconds for search answers on smart glasses. RevisionDojo says hints and marked answers "come back in about a second". All three are reported anecdotes, with no comparator scores or task lists published. A reply under the post also claimed that Mercury 2 and 2.5 are closed to new users and that existing API customers lose access on 15 November. The docs still list both models with prices on 23 October, and I found nothing from Inception saying so, so I treat it as unverified.

What holds

The mechanism is real, and the routed numbers are good. Mercury 3 is the fastest of the three on OpenRouter by a clear margin, at quality Inception reports as level or better. The launch's own 4–8x is the number to discount, by about 3.5x on Mercury's side, until someone outside Inception measures it the way Artificial Analysis measured Mercury 1. For a different route to the same goal, Uno compares a speculative-decoding approach against Mercury 2, and ALoDLM spends extra compute on the hard tokens inside each denoising step.