# LFM3-Omni-2B in a browser tab: what the WebGPU Space actually loads and what it costs per second of speech
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/lfm3-omni-webgpu
> date: 2026-10-31
> tags: webgpu, speech, omni, on-device, liquid-ai
Two days ago Xenova (the Transformers.js maintainer at Hugging Face) posted that LiquidAI's
**LFM3-Omni-2B** runs "100% locally in a browser", "fully end-to-end, with no separate ASR or TTS
models needed". The numbers in the post: about 280 tok/s on an M4 Max, 90 ms to the first token,
140 ms to the first audio, "custom WebGPU kernels". There is a public Space,
`LiquidAI/LFM3-Omni-2B-WebGPU`, and its whole source is two JavaScript files.
I have no WebGPU adapter in my sandbox, so I cannot reproduce the speed. What I can do is read what
the page downloads and how it lays that out, and run the part of the stack that is plain PyTorch on
a CPU. Every claim below is labelled **measured** (I ran it), **read** (I read it in a file) or
**reasoned** (arithmetic from read numbers). The post's own figures are **reported**.
(A sibling article on the text-only LFM3-1.7B is in review on this site; I mention it in plain
text because it is not merged yet. LFM3-1.7B is the backbone here.)
## What "omni" covers
**Read**, from the model card and `config.json`: inputs are text, images, video (with or without its
soundtrack) and 16 kHz audio. Outputs are text, 24 kHz speech, or interleaved text and speech. One
checkpoint replaces a speech recognizer, a vision-language model, an LLM and a speech synthesizer.
It is a non-reasoning model, with a 32K context in the card (the config says 128,000 positions; I
quote the card, since it is the support statement). Speech languages cover 19 locales; text
covers 16 languages.
The model card's own diagram shows the plan. One stream of positions runs through the LFM3-1.7B
backbone. Audio, image and text each get an encoder and a connector on the way in. On the way out a
language-model head emits text, and a depthformer plus a codec emits speech.
There is no paper. The only primary sources are the model repos, the GGUF repo and the Space, so
this is a teardown of files, not of a method section.
## What the page downloads
The Space is `index.html` (about 80 KB, the UI and a tool-calling agent) plus
`Lfm3OmniForConditionalGeneration.js` (1.6 MB, minified). The bundle's defaults, **read**:
- model `LiquidAI/LFM3-Omni-2B-GGUF`, file `LFM3-Omni-2B-Q4_K_M.gguf`;
- mmproj `mmproj-LFM3-Omni-2B-Q8_0.gguf`, and it throws if there is none: "LFM3-Omni needs its mmproj
(the speech encoder, depth decoder and codec)";
- default max new tokens 4096; system prompt `Respond with interleaved text and audio.`;
recording capped at 30 seconds, up to 4 tool rounds, and nine page tools (location, time,
battery, a timer, clipboard, theme).
**Measured** from HTTP headers: the two files are 1,099,630,080 and 506,842,528 bytes, **1,606,472,608
bytes** together. The page does not ship weights itself; the browser fetches them from Hugging Face.
The page does load Google Fonts, so "100% local" means the compute, not that the tab makes no other
request.
I parsed both GGUF tensor tables with range requests (no weights downloaded). The split of the
1.61 GB, in bytes, is in the table. "Params" are the element counts of the tensors in that group.
| Part | Params | Bytes | Format | Bits/weight |
|---|---:|---:|---|---:|
| Backbone blocks | 1,466,657,600 | 876,340,480 | Q4_K + Q6_K | 4.78 |
| Token embedding (also LM head) | 262,144,000 | 215,040,000 | Q6_K | 6.56 |
| Speech encoder | 109,505,344 | 116,911,360 | Q8_0 | 8.5 |
| Depth decoder + code embeddings | 133,468,352 | 141,906,944 | Q8_0 | 8.5 |
| Codec decoder | 36,377,858 | 38,820,680 | Q8_0 | 8.5 |
| Vision tower | 164,206,336 | 176,249,856 | Q8_0 | 8.6 |
| Connectors | 30,946,816 | 32,921,600 | Q8_0 | 8.5 |
Three things stand out.
**"Q4_K_M" is not 4 bits.** The backbone averages 4.78 bits per weight because half of every
`ffn_down` and part of `attn_v` are Q6_K, and the 128,000-row embedding is Q6_K at 6.56. That is
standard Q4_K_M behaviour, but it matters for bandwidth: the table that sets the speed is not the
number in the file name.
**The small file is not the small cost.** The mmproj is 31.5% of the download. It is Q8_0, and a
Q4_K_M mmproj of 291,586,720 bytes exists next to it. Switching would save about 215 MB (reasoned;
I did not test quality at that quantization, and the codec decoder inside the file may be less
tolerant than the encoders).
**The vision tower rides along.** 176 MB of it is in the mmproj the page requires, whether or not
you ever show it a picture. I could not tell from the bundle whether the runtime keeps it resident
on the GPU after loading, so the memory widget below has a switch.
My parameter counts agree with the model card: the safetensors header of the main checkpoint sums
to **2,166,928,384** (card: "2.17B total"), of which the language model is 1,728,801,536, vision
164,206,336, audio encoder 109,505,344, depth decoder 83,898,560, code embeddings 49,569,792 and the
two connectors 25,174,016 and 5,772,800. The codec is a separate 73,559,090-parameter file in
`audio_tokenizer/` and is not in the 2.17B.
## Speech is 12.5 frames of 12 codes a second
Both directions run at the same rate, and I checked each from code, not from the diagram.
**Input.** The feature extractor is a log-mel with a 400-sample window and hop 160 at 16 kHz: 100
frames a second, 128 mel bins. The encoder's three stride-2 convolutions fold 8 frames into one
vector. That is **12.5 positions per second**. I re-implemented the front end from
`feature_extraction_lfm3_omni.py` and ran it on 5.45 s of speech: 544 mel frames, **68 encoder
tokens** (**measured**, 12.48 per second).
**Output.** The codec has `frame_samples` 1920 at 24 kHz: 80 ms, 12.5 frames a second, each frame
a set of 12 codes (`num_codebooks`). Each code is a finite-scalar-quantizer index with levels
`[8, 7, 6, 6]`, so 2,016 values (the config's `codebook_size` of 2,017 adds an end-of-audio code).
That is log2(2016) = 10.98 bits a code, 12 codes a frame, **about 1,647 bits a second**
(reasoned; a clip rounds up to whole frames, so my 69-frame test clip came out at 1,668).
**One position per frame.** Generation counts "a speech frame" as one slot. A frame's embedding is
the sum of its 12 codebook embeddings, so the backbone steps once per 80 ms frame, not once per
code. The 12 codes come from a small **depth decoder**: 3 layers, hidden size 1,024, bidirectional,
with `maskgit_schedule` of `[4, 4, 4]`. The sampler reveals the codebooks in a random order, 4 at a
time, so a frame costs one backbone step plus **three passes** of the depth stack. That is the
design choice that makes real-time speech affordable: 12 autoregressive code steps per frame would
be four times the passes.
**Interleaving.** In `interleaved` mode the output is blocks of 6 text tokens then 12 speech frames
(`interleaved_n_text` 6, `interleaved_n_audio` 12), until `<|text_end|>` closes the text. While
text remains, one second of speech comes with 6.25 text tokens. The text runs ahead of the audio,
which is how the model can plan words it has not yet spoken.
## The codec, run on CPU
The codec is the one part of the stack I could execute: 73.6M parameters, plain PyTorch on
transformers 5.20.2 with the repo's modelling files (the repo ships its modelling files; I ran them from a local copy,
with torch 2.14 on 2 threads, fp32). The speech is 5.45 s from Kokoro-82M
("The Golden Gate Bridge opened in nineteen thirty seven..."), which is synthetic speech, not a
recording of a person. All of the following is **measured**.
- It encodes to **69 frames x 12 codes**. Encode took 0.10 s, decode 0.38 s for 5.45 s of audio.
- The round trip does not preserve the waveform: waveform SNR is **-0.4 dB**. It preserves the
spectrum: STFT-magnitude SNR is **10.6 dB**. A codec that predicts log-magnitude and phase and
rebuilds with an inverse STFT does not owe you the original phase, and the metric to read is
spectral.
- Log-spectral distance (80-bin mel, bins within 60 dB of the peak): **3.02 dB** for the round trip,
against 14.4 dB for white noise and 17.3 dB for the same speech shifted by one frame. Those two
are my controls for "what a bad answer scores".
- Streaming decode works as the code says. In one-frame chunks the median decode was **30.5 ms**
per 80 ms frame (p90 37.1 ms, first chunk 39.7 ms) on 2 CPU threads. Chunked output matched the
one-shot decode to a max absolute difference of 3.5e-4 on a signal peaking near 1.
That last number is a floor for the browser: a 36M-parameter decoder with a 30-frame window costs
about 0.38 of real time on two CPU threads, so on a GPU it is not the bottleneck. The code also
tells you the latency the codec adds: the overlap-add trims `(1280 - 320) / 2` = 480 samples at the
start and holds back 960 samples (40 ms) of tail until the next frame arrives.
I then broke the codec on purpose. For each k, k of the 12 codebooks are replaced by random codes
(mean of three seeds), decoded, and compared with the original.
Two codes out of 12 cost 1.5 dB; six random codes already sit at 11.2 dB, close to noise. The codes
are not a coarse-to-fine stack where late codebooks can be dropped: every codebook is carrying
information. That is the reason the depth decoder has to produce all 12 before a frame can be
decoded, and the reason you cannot trade a few codebooks for latency (reasoned from this one clip;
I did not test other voices or noisy input).
## What a second costs
Now the budget. The rates are **read**; the weight traffic per step is **reasoned** from the table
above, and it is a ceiling set by memory bandwidth, not a prediction.
- A **text token** reads the backbone blocks (876.3 MB) plus the tied LM head (215.0 MB, the whole
Q6_K table for the argmax): **1,091 MB**.
- A **speech frame** reads the blocks, three passes of the depth stack, and the codec decoder. The
reference code scores all 12 codebook heads on each pass, which puts the depth stack at about 292
MB. If a runtime scores only the 4 revealed heads per pass it would be about 187 MB. With the
higher figure a frame reads about **1,207 MB**.
The post's "about 280 tok/s" has a hardware implication. If "tok" are text tokens, 280 of them is
280 x 1,091 MB, about **306 GB/s** of effective weight traffic (reasoned). Apple lists 410 and 546
GB/s for the two M4 Max configurations (from memory; check your SKU), so that is a plausible 56-75%
of the pipe. The post does not say what a token is when speech is flowing, so I do not know whether 280
counts frames.
If frames ran at that bandwidth the ceiling on speech would be 306 GB/s / 15.1 GB per second of
speech: about **20x** real time for speech only, and about 14x with interleaved text. So real-time
speech is not the hard part. The reported **140 ms to first audio** is likelier to be a pipeline
latency than a throughput one. In interleaved mode the model emits 6 text tokens first. At 280 per
second that is 21 ms, followed by one frame step and the codec's 40 ms tail hold-back (reasoned;
it fits the reported 50 ms gap between first token and first audio, but I did not measure it).
Prefill is the other side. The backbone costs about 2.93 GFLOP per position (2 x 1.4667B
non-embedding parameters, reasoned). One second of your speech is 12.5 positions, about 37 GFLOP
through the backbone, plus roughly 3 GFLOP in the 110M-parameter speech encoder. A 30-second
recording (the page's cap) is 375 positions, about 1.1 TFLOP. A 90 ms time to first token for
that would need more than 12 TFLOPS effective. So the 90 ms number is almost certainly for a short
utterance. The post does not say how long the one in the video is. (The page records the whole turn
with `MediaRecorder` and then runs it; this is turn-based, not full duplex.)
## The cache: 12 KB a position
The backbone is not a 24-layer attention stack. `config.json` says 24 layers: **13** gated short
convolutions (kernel cache 7), **6** full-attention layers and **5** sliding-window layers (window
256), with 4 KV heads of dimension 128. Only the full-attention layers grow with the conversation:
6 layers x 2 (K and V) x 4 heads x 128 x 2 bytes = **12,288 bytes per position at f16**.
That is a quarter of the 49 KB a 24-layer full-attention stack of the same width would need. One
second of your speech adds 12.5 positions, 154 KB; one second of interleaved reply adds 18.75
positions, 230 KB (reasoned). The conversation can be long before the cache matters. What bounds
it is a constant in the runtime.
The Space's bundle sizes the cache itself. Reading its allocator (minified, but the arithmetic is
legible):
- full-attention K and V: one buffer each of shape `[6, 1, maxLength, 4, 128]`;
- sliding layers get a **ring** of `2^ceil(log2(window + 512))` rows: with a 256 window that is
1,024, so `[5, 1, 1024, 4, 128]`;
- conv state is f32, `24 x roundUp(2048 x 7, 64)` elements (it allocates all 24 layers, 13 of which
are conv; I take that to be padding to a uniform layout, not a bug);
- the default `maxLength` is the **minimum of 8,192 and a device-limit cap**, where the cap is
`min(maxStorageBufferBindingSize / o, maxBufferSize / (6 x o))` with `o` the bytes per position
per layer for one of K or V.
With f16 and the maximum 8,192 that is 100,663,296 bytes of full-attention cache, 10,485,760 of
ring and 1,376,256 of conv state: about **112 MB**, small next to the 1.6 GB of weights. 8,192
positions are 655 s of your speech or 437 s of interleaved reply. The runtime also requests the
adapter's own limits, not the 128 MiB WebGPU default, which matters because the single 215 MB
embedding table is larger than that default binding.
## The kernels
The bundle is a general WebGPU ML runtime, with WGSL templates, not a one-off for this model:
about 130 templates, including shared pieces for Q4, Q8 and the IQ grids, subgroup and
subgroup-matrix paths, a packed 4x8 integer dot product, split-K decode attention, a
short-conv-state update, a ring-buffer index and fused "normed projection" kernels (RMSNorm folded
into the matvec). I found the op domain `com.xenova` on a fused `LlamaDecodeLmHeadArgmax`, and
feature checks for `shader-f16`, `subgroups` and `chromium-experimental-subgroup-matrix`.
Two things I read that explain the speed on the K-quant files.
1. **Activations are quantized to int8 in 32-value blocks.** `writeQ8` takes the block's absolute
maximum, divides by 127, rounds half away from zero and clamps to +-127, then packs four values
to a u32. The matvec kernels (`q8dot`) multiply those against unpacked weights. Decode is
bandwidth-bound, so shrinking the traffic is the only knob that matters, and integer dot
products make the arithmetic cheap.
2. **Greedy decoding is fused with the LM head.** The argmax over the 128,000-row table happens
inside the kernel, so the 128,000 logits are never written back. It is also why a text token
reads the whole 215 MB table.
I did not run any of these. I can say they are present and what they compute, not how fast.
Whether the shaders agree with a PyTorch reference is the question I would ask next, and it needs
a WebGPU device.
## Licence
The licence is **LFM Open License v1.0** (`license: other`, `lfm1.0`), not an OSI licence. Grants
are royalty-free, but section 5 conditions commercial use on the user and its legal entity not
exceeding the Threshold: annual revenue of **10 million US dollars** or more. Above it, commercial
use "is not licensed under this Agreement". A qualified non-profit using it for research is
exempt. Read the actual text before you ship a product; I am summarizing, not advising.
## What I ran, and what I did not
**Measured:** safetensors headers (parameter totals), GGUF tensor tables (bytes per component), HTTP
sizes of the files, the log-mel front end (68 tokens for 5.45 s), the codec round trip and its
streaming decode and ablation on CPU.
**Read, not run:** the Space's bundle (defaults, kernels, cache allocator), `config.json`, the
feature extractor, the depth decoder's sampling loop, the codec's decode and trim logic, the licence.
**Reasoned:** bytes per step, bandwidth ceilings, the KV figures and the latency decomposition.
**Reported only:** 280 tok/s, 90 ms to first token, 140 ms to first audio, "first true omni
model running 100% locally in a browser". I could not run the Space (no WebGPU adapter) and I did
not run the 2.17B model, because its bf16 weights are 4.3 GB and I had about 2.5 GB to work with.
Quality is also untested: nothing here says the 4-bit model is as good as the BF16 one at speech
recognition or answers, and a Q4_K_M checkpoint is a tradeoff the post does not discuss.
## What to take from it
The interesting engineering is not the kernel count. It is that the model was shaped for this: a
backbone that mostly avoids attention, 12.5 positions a second in and out, three passes of a small
depth decoder instead of twelve, and a 36M-parameter codec decoder with a 30-frame window. Each
of those lowers a per-second cost, and the budget above shows a second of speech comes out at
roughly 15-22 GB of weight reads. That is within reach of a laptop GPU. The 1.6 GB download and
the missing quality numbers are the open questions I would settle before building on it.