LFM3-Omni-2B in a browser tab: what the WebGPU Space actually loads and what it costs per second of speech

mdjsonmcpcopy↗ai

2026-10-31 · 15 min · webgpu · speech · omni · on-device · liquid-ai

Why read this

High· top 30%

Byte-for-byte accounting of the Space's files and cache allocator, plus a CPU run of the real speech codec; the headline speed claims stay reported.

• Interactive explanations
• Original analysis
• Runs in a browser

·Speech & audio·Custom licence·Practitioner teardown

How this was scored

Is it new?

2 of 3: A real new idea, method or capability

Can I trust it?

2 of 3: Measures key facts from files, code or configs

Can I run it?

2 of 3: Open code or weights with real limits

Will I understand it?

3 of 3: Mechanism carried by interactives built from real code or data

Can I act on it?

2 of 3: A concrete recipe, numbers or comparison

Will it last?

1 of 3: Relevant for months

Does it affect many?

2 of 3: A widely used model, tool or lab release

Only here?

2 of 3: A teardown or measurement few others did

Score 71 of 100, ranked 119 of 666 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored

Two days ago Xenova (the Transformers.js maintainer at Hugging Face) posted that LiquidAI's LFM3-Omni-2B runs "100% locally in a browser", "fully end-to-end, with no separate ASR or TTS models needed". The numbers in the post: about 280 tok/s on an M4 Max, 90 ms to the first token, 140 ms to the first audio, "custom WebGPU kernels". There is a public Space, LiquidAI/LFM3-Omni-2B-WebGPU, and its whole source is two JavaScript files.

I have no WebGPU adapter in my sandbox, so I cannot reproduce the speed. What I can do is read what the page downloads and how it lays that out, and run the part of the stack that is plain PyTorch on a CPU. Every claim below is labelled measured (I ran it), read (I read it in a file) or reasoned (arithmetic from read numbers). The post's own figures are reported.

(A sibling article on the text-only LFM3-1.7B is in review on this site; I mention it in plain text because it is not merged yet. LFM3-1.7B is the backbone here.)

What "omni" covers

Read, from the model card and config.json: inputs are text, images, video (with or without its soundtrack) and 16 kHz audio. Outputs are text, 24 kHz speech, or interleaved text and speech. One checkpoint replaces a speech recognizer, a vision-language model, an LLM and a speech synthesizer. It is a non-reasoning model, with a 32K context in the card (the config says 128,000 positions; I quote the card, since it is the support statement). Speech languages cover 19 locales; text covers 16 languages.

The model card's own diagram shows the plan. One stream of positions runs through the LFM3-1.7B backbone. Audio, image and text each get an encoder and a connector on the way in. On the way out a language-model head emits text, and a depthformer plus a codec emits speech.

expand

There is no paper. The only primary sources are the model repos, the GGUF repo and the Space, so this is a teardown of files, not of a method section.

What the page downloads

The Space is index.html (about 80 KB, the UI and a tool-calling agent) plus Lfm3OmniForConditionalGeneration.js (1.6 MB, minified). The bundle's defaults, read:

• model LiquidAI/LFM3-Omni-2B-GGUF, file LFM3-Omni-2B-Q4_K_M.gguf;
• mmproj mmproj-LFM3-Omni-2B-Q8_0.gguf, and it throws if there is none: "LFM3-Omni needs its mmproj (the speech encoder, depth decoder and codec)";
• default max new tokens 4096; system prompt Respond with interleaved text and audio.; recording capped at 30 seconds, up to 4 tool rounds, and nine page tools (location, time, battery, a timer, clipboard, theme).

Measured from HTTP headers: the two files are 1,099,630,080 and 506,842,528 bytes, 1,606,472,608 bytes together. The page does not ship weights itself; the browser fetches them from Hugging Face. The page does load Google Fonts, so "100% local" means the compute, not that the tab makes no other request.

I parsed both GGUF tensor tables with range requests (no weights downloaded). The split of the 1.61 GB, in bytes, is in the table. "Params" are the element counts of the tensors in that group.

Three things stand out.

"Q4_K_M" is not 4 bits. The backbone averages 4.78 bits per weight because half of every ffn_down and part of attn_v are Q6_K, and the 128,000-row embedding is Q6_K at 6.56. That is standard Q4_K_M behaviour, but it matters for bandwidth: the table that sets the speed is not the number in the file name.

The small file is not the small cost. The mmproj is 31.5% of the download. It is Q8_0, and a Q4_K_M mmproj of 291,586,720 bytes exists next to it. Switching would save about 215 MB (reasoned; I did not test quality at that quantization, and the codec decoder inside the file may be less tolerant than the encoders).

The vision tower rides along. 176 MB of it is in the mmproj the page requires, whether or not you ever show it a picture. I could not tell from the bundle whether the runtime keeps it resident on the GPU after loading, so the memory widget below has a switch.

My parameter counts agree with the model card: the safetensors header of the main checkpoint sums to 2,166,928,384 (card: "2.17B total"), of which the language model is 1,728,801,536, vision 164,206,336, audio encoder 109,505,344, depth decoder 83,898,560, code embeddings 49,569,792 and the two connectors 25,174,016 and 5,772,800. The codec is a separate 73,559,090-parameter file in audio_tokenizer/ and is not in the 2.17B.

Speech is 12.5 frames of 12 codes a second

Both directions run at the same rate, and I checked each from code, not from the diagram.

Input. The feature extractor is a log-mel with a 400-sample window and hop 160 at 16 kHz: 100 frames a second, 128 mel bins. The encoder's three stride-2 convolutions fold 8 frames into one vector. That is 12.5 positions per second. I re-implemented the front end from feature_extraction_lfm3_omni.py and ran it on 5.45 s of speech: 544 mel frames, 68 encoder tokens (measured, 12.48 per second).

Output. The codec has frame_samples 1920 at 24 kHz: 80 ms, 12.5 frames a second, each frame a set of 12 codes (num_codebooks). Each code is a finite-scalar-quantizer index with levels [8, 7, 6, 6], so 2,016 values (the config's codebook_size of 2,017 adds an end-of-audio code). That is log2(2016) = 10.98 bits a code, 12 codes a frame, about 1,647 bits a second (reasoned; a clip rounds up to whole frames, so my 69-frame test clip came out at 1,668).

One position per frame. Generation counts "a speech frame" as one slot. A frame's embedding is the sum of its 12 codebook embeddings, so the backbone steps once per 80 ms frame, not once per code. The 12 codes come from a small depth decoder: 3 layers, hidden size 1,024, bidirectional, with maskgit_schedule of [4, 4, 4]. The sampler reveals the codebooks in a random order, 4 at a time, so a frame costs one backbone step plus three passes of the depth stack. That is the design choice that makes real-time speech affordable: 12 autoregressive code steps per frame would be four times the passes.

Interleaving. In interleaved mode the output is blocks of 6 text tokens then 12 speech frames (interleaved_n_text 6, interleaved_n_audio 12), until <|text_end|> closes the text. While text remains, one second of speech comes with 6.25 text tokens. The text runs ahead of the audio, which is how the model can plan words it has not yet spoken.

The codec, run on CPU

The codec is the one part of the stack I could execute: 73.6M parameters, plain PyTorch on transformers 5.20.2 with the repo's modelling files (the repo ships its modelling files; I ran them from a local copy, with torch 2.14 on 2 threads, fp32). The speech is 5.45 s from Kokoro-82M ("The Golden Gate Bridge opened in nineteen thirty seven..."), which is synthetic speech, not a recording of a person. All of the following is measured.

• It encodes to 69 frames x 12 codes. Encode took 0.10 s, decode 0.38 s for 5.45 s of audio.
• The round trip does not preserve the waveform: waveform SNR is -0.4 dB. It preserves the spectrum: STFT-magnitude SNR is 10.6 dB. A codec that predicts log-magnitude and phase and rebuilds with an inverse STFT does not owe you the original phase, and the metric to read is spectral.
• Log-spectral distance (80-bin mel, bins within 60 dB of the peak): 3.02 dB for the round trip, against 14.4 dB for white noise and 17.3 dB for the same speech shifted by one frame. Those two are my controls for "what a bad answer scores".
• Streaming decode works as the code says. In one-frame chunks the median decode was 30.5 ms per 80 ms frame (p90 37.1 ms, first chunk 39.7 ms) on 2 CPU threads. Chunked output matched the one-shot decode to a max absolute difference of 3.5e-4 on a signal peaking near 1.

That last number is a floor for the browser: a 36M-parameter decoder with a 30-frame window costs about 0.38 of real time on two CPU threads, so on a GPU it is not the bottleneck. The code also tells you the latency the codec adds: the overlap-add trims (1280 - 320) / 2 = 480 samples at the start and holds back 960 samples (40 ms) of tail until the next frame arrives.

I then broke the codec on purpose. For each k, k of the 12 codebooks are replaced by random codes (mean of three seeds), decoded, and compared with the original.

Two codes out of 12 cost 1.5 dB; six random codes already sit at 11.2 dB, close to noise. The codes are not a coarse-to-fine stack where late codebooks can be dropped: every codebook is carrying information. That is the reason the depth decoder has to produce all 12 before a frame can be decoded, and the reason you cannot trade a few codebooks for latency (reasoned from this one clip; I did not test other voices or noisy input).

What a second costs

Now the budget. The rates are read; the weight traffic per step is reasoned from the table above, and it is a ceiling set by memory bandwidth, not a prediction.

• A text token reads the backbone blocks (876.3 MB) plus the tied LM head (215.0 MB, the whole Q6_K table for the argmax): 1,091 MB.
• A speech frame reads the blocks, three passes of the depth stack, and the codec decoder. The reference code scores all 12 codebook heads on each pass, which puts the depth stack at about 292 MB. If a runtime scores only the 4 revealed heads per pass it would be about 187 MB. With the higher figure a frame reads about 1,207 MB.

The post's "about 280 tok/s" has a hardware implication. If "tok" are text tokens, 280 of them is 280 x 1,091 MB, about 306 GB/s of effective weight traffic (reasoned). Apple lists 410 and 546 GB/s for the two M4 Max configurations (from memory; check your SKU), so that is a plausible 56-75% of the pipe. The post does not say what a token is when speech is flowing, so I do not know whether 280 counts frames.

If frames ran at that bandwidth the ceiling on speech would be 306 GB/s / 15.1 GB per second of speech: about 20x real time for speech only, and about 14x with interleaved text. So real-time speech is not the hard part. The reported 140 ms to first audio is likelier to be a pipeline latency than a throughput one. In interleaved mode the model emits 6 text tokens first. At 280 per second that is 21 ms, followed by one frame step and the codec's 40 ms tail hold-back (reasoned; it fits the reported 50 ms gap between first token and first audio, but I did not measure it).

Prefill is the other side. The backbone costs about 2.93 GFLOP per position (2 x 1.4667B non-embedding parameters, reasoned). One second of your speech is 12.5 positions, about 37 GFLOP through the backbone, plus roughly 3 GFLOP in the 110M-parameter speech encoder. A 30-second recording (the page's cap) is 375 positions, about 1.1 TFLOP. A 90 ms time to first token for that would need more than 12 TFLOPS effective. So the 90 ms number is almost certainly for a short utterance. The post does not say how long the one in the video is. (The page records the whole turn with MediaRecorder and then runs it; this is turn-based, not full duplex.)

The cache: 12 KB a position

The backbone is not a 24-layer attention stack. config.json says 24 layers: 13 gated short convolutions (kernel cache 7), 6 full-attention layers and 5 sliding-window layers (window 256), with 4 KV heads of dimension 128. Only the full-attention layers grow with the conversation:

6 layers x 2 (K and V) x 4 heads x 128 x 2 bytes = 12,288 bytes per position at f16.

That is a quarter of the 49 KB a 24-layer full-attention stack of the same width would need. One second of your speech adds 12.5 positions, 154 KB; one second of interleaved reply adds 18.75 positions, 230 KB (reasoned). The conversation can be long before the cache matters. What bounds it is a constant in the runtime.

The Space's bundle sizes the cache itself. Reading its allocator (minified, but the arithmetic is legible):

• full-attention K and V: one buffer each of shape [6, 1, maxLength, 4, 128];
• sliding layers get a ring of 2^ceil(log2(window + 512)) rows: with a 256 window that is 1,024, so [5, 1, 1024, 4, 128];
• conv state is f32, 24 x roundUp(2048 x 7, 64) elements (it allocates all 24 layers, 13 of which are conv; I take that to be padding to a uniform layout, not a bug);
• the default maxLength is the minimum of 8,192 and a device-limit cap, where the cap is min(maxStorageBufferBindingSize / o, maxBufferSize / (6 x o)) with o the bytes per position per layer for one of K or V.

With f16 and the maximum 8,192 that is 100,663,296 bytes of full-attention cache, 10,485,760 of ring and 1,376,256 of conv state: about 112 MB, small next to the 1.6 GB of weights. 8,192 positions are 655 s of your speech or 437 s of interleaved reply. The runtime also requests the adapter's own limits, not the 128 MiB WebGPU default, which matters because the single 215 MB embedding table is larger than that default binding.

The kernels

The bundle is a general WebGPU ML runtime, with WGSL templates, not a one-off for this model: about 130 templates, including shared pieces for Q4, Q8 and the IQ grids, subgroup and subgroup-matrix paths, a packed 4x8 integer dot product, split-K decode attention, a short-conv-state update, a ring-buffer index and fused "normed projection" kernels (RMSNorm folded into the matvec). I found the op domain com.xenova on a fused LlamaDecodeLmHeadArgmax, and feature checks for shader-f16, subgroups and chromium-experimental-subgroup-matrix.

Two things I read that explain the speed on the K-quant files.

1. Activations are quantized to int8 in 32-value blocks. writeQ8 takes the block's absolute maximum, divides by 127, rounds half away from zero and clamps to +-127, then packs four values to a u32. The matvec kernels (q8dot) multiply those against unpacked weights. Decode is bandwidth-bound, so shrinking the traffic is the only knob that matters, and integer dot products make the arithmetic cheap.
2. Greedy decoding is fused with the LM head. The argmax over the 128,000-row table happens inside the kernel, so the 128,000 logits are never written back. It is also why a text token reads the whole 215 MB table.

I did not run any of these. I can say they are present and what they compute, not how fast. Whether the shaders agree with a PyTorch reference is the question I would ask next, and it needs a WebGPU device.

Licence

The licence is LFM Open License v1.0 (license: other, lfm1.0), not an OSI licence. Grants are royalty-free, but section 5 conditions commercial use on the user and its legal entity not exceeding the Threshold: annual revenue of 10 million US dollars or more. Above it, commercial use "is not licensed under this Agreement". A qualified non-profit using it for research is exempt. Read the actual text before you ship a product; I am summarizing, not advising.

What I ran, and what I did not

Measured: safetensors headers (parameter totals), GGUF tensor tables (bytes per component), HTTP sizes of the files, the log-mel front end (68 tokens for 5.45 s), the codec round trip and its streaming decode and ablation on CPU.

Read, not run: the Space's bundle (defaults, kernels, cache allocator), config.json, the feature extractor, the depth decoder's sampling loop, the codec's decode and trim logic, the licence.

Reasoned: bytes per step, bandwidth ceilings, the KV figures and the latency decomposition.

Reported only: 280 tok/s, 90 ms to first token, 140 ms to first audio, "first true omni model running 100% locally in a browser". I could not run the Space (no WebGPU adapter) and I did not run the 2.17B model, because its bf16 weights are 4.3 GB and I had about 2.5 GB to work with. Quality is also untested: nothing here says the 4-bit model is as good as the BF16 one at speech recognition or answers, and a Q4_K_M checkpoint is a tradeoff the post does not discuss.

What to take from it

The interesting engineering is not the kernel count. It is that the model was shaped for this: a backbone that mostly avoids attention, 12.5 positions a second in and out, three passes of a small depth decoder instead of twelve, and a 36M-parameter codec decoder with a 30-frame window. Each of those lowers a per-second cost, and the budget above shows a second of speech comes out at roughly 15-22 GB of weight reads. That is within reach of a laptop GPU. The 1.6 GB download and the missing quality numbers are the open questions I would settle before building on it.