# FastH3 Preview v1: four calls, one-tenth the attention, and what the 14x actually measures
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/fastvideo-fasth3
> date: 2026-08-29
> tags: video-generation, diffusion, sparse-attention, distillation, inference-optimization, explainer
[FastH3 Preview v1](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree) is
[FastVideo](https://github.com/hao-ai-lab/FastVideo)'s distilled version of MiniMax's H3 audio-video
diffusion transformer, and the [announcement post](https://haoailab.com/blogs/fasth3-preview/) leads with
two numbers: "up to 14x speedup on NVIDIA Blackwell GPU," and "generate 15s 768p video in less than 13s with
sub-realtime generation on 8xB200 GPUs." Both are real — they're read straight off FastVideo's own published
table, and this piece rebuilds that table from the numbers to check them. But both numbers are also
specific in ways the headline doesn't spell out: 14x is one clip length on one GPU count, and "sub-realtime"
turns out to describe one row of a three-row table, not the whole table.
That's the shape of this piece. FastH3's own components are unusually well-documented for an open-weight
release — the checkpoint ships a machine-readable `checkpoint_metadata.json` with its exact training config,
a `provenance.json` with commit hashes and a Weights & Biases run URL, and safetensors headers you can read
without downloading 148GB. So rather than repeat the announcement, this walks through what's actually
verifiable: the two mechanisms that make FastH3 fast, a parameter count reconstructed from the checkpoint's
own tensor shapes, and a decomposition of "14x" into the piece that comes from calling the transformer fewer
times and the piece that comes from making each call cheaper — because they turn out to be different sizes,
and the source lets you tell them apart.
That verification runs one level deeper than the checkpoint, too. FastVideo publishes the full
training-and-inference framework FastH3 comes from, and reading it directly — the hand-written CUDA kernels
behind "on Blackwell," the DMD2 training step, the model registry, the Apple Silicon runtime — either confirms
specific claims the announcement makes about hardware and the roadmap, or complicates them in ways the
checkpoint's own metadata can't show on its own.
## The base model: MiniMax H3, undistilled
FastH3 isn't a new architecture — it's the same weights MiniMax shipped, pushed through a training run that
changes how the checkpoint is *used*, not what it fundamentally *is*. [MiniMax H3](/articles/minimax-h3) is a
33B-parameter dense, single-stream Omni-Transformer (MiniMax's own description, on its own model card) that
takes text, image, video, and audio in and produces synchronized audio-video out, at up to 2K resolution
(768p by default), 4-15 second clips at 24 FPS, with 32kHz stereo audio. Text and vision conditioning come
from a separate encoder — [the minimax-h3 piece](/articles/minimax-h3) identifies it as a 32B-parameter
Qwen3-VL model — and attention across the whole packed sequence is, in MiniMax's own words, "full attention
in the initial release," with sparse attention only "planned for future publication." FastVideo built that
sparse attention first, and applied it as a distillation target rather than waiting for MiniMax to ship one.
Two things are worth separating cleanly before going further, because the announcement's headline collapses
them into one number: FastH3 lowers Base H3's cost in exactly two ways, and they're independent mechanisms
that happen to compound.
## Lever one: 49 calls become 4
A diffusion transformer's inference cost is, to a first approximation, `(cost per call) x (number of calls)`.
FastVideo's own accounting of Base H3's default schedule is 49 calls to its 33B transformer per generation —
stated directly in the announcement, not inferred. FastH3 replaces that with exactly four, and this number
isn't a rounded average: `checkpoint_metadata.json`, published alongside the weights, lists
`dmd_denoising_steps: [999, 749, 500, 250]` — four fixed timesteps, no more, no fewer, for every generation
this checkpoint produces.
Forty-nine to four is a fixed 12.25x reduction in transformer forward passes, and it would be tempting to
stop there and call that the whole story. It isn't, for the reason the chart above makes visible: end-to-end
wall clock also includes text/vision encoding, VAE decode, audio decode, muxing, and file output, and none
of that shrinks when the diffusion loop gets four times shorter. At four fixed calls, that fixed overhead is
a *larger* share of a shorter clip's total time than of a longer one's — which is why the observed speedup
sits below the 12.25x line at 5 and 10 seconds, and only climbs past it at 15 seconds, where there's enough
diffusion work left for the second lever to act on.
## What actually trains those four calls: Data-Free DMD2
The method that gets Base H3 down to four calls is [Distribution Matching Distillation
(DMD2)](https://arxiv.org/abs/2405.14867), and the "Data-Free" name in FastH3's checkpoint filename describes
something specific about *how* it's trained, not a claim that training was free of supervision. Reading the
recommended checkpoint's own `checkpoint_metadata.json` training config end to end:
Three networks, all initialized from the same MiniMax-H3 weights: a frozen **teacher** (dense attention,
never updated), a trainable **critic** — DMD2's own name for it is `fake_score` — that keeps dense attention
and is continuously updated to track the student's current output distribution, and a trainable **student**
that runs the sparse `VIDEO_SPARSE_ATTN_H3` backend and is the thing being distilled into a four-step
generator. `generator_update_interval: 5` in the real config means the student takes one gradient step for
every five the critic takes — the critic has to keep up with a moving target, so it trains harder.
What makes the recommended checkpoint "Data-Free" is `preprocessed_data_type: "text_only"` in that same
config — every one of its five listed training-data paths is reduced to prompts alone before training
starts. Combined with `rollout_mode: "simulate"` and `rollout_sample_type: "ode"` (DMD2's "backward
simulation"), the student generates its *own* four-step rollout starting from pure noise, and the teacher and
critic score that self-generated sample — never a video anyone recorded or rendered. FastVideo names the
alternative directly in the announcement: "the synthetic-video runs instead start from forward-noised
Base-H3 video-and-audio latents," a separate, non-default ablation covered below. Neither path ever touches
H3-Base's own training corpus; the difference is whether the rollout starts from noise or from a real
model's output.
## Inside the training step: what checkpoint_metadata.json doesn't show
The metadata gives the shape of the recipe; FastVideo's actual training-step code
(`fastvideo/train/methods/distribution_matching/dmd2.py`) shows the mechanism is more involved than "the
student takes four steps, the teacher and critic score it." Two things the config alone can't tell you:
`dmd_denoising_steps: [999, 749, 500, 250]` is the fixed schedule the student walks *forward* through to
build a simulated trajectory (`_student_rollout`, when `rollout_mode == "simulate"`) — but the teacher and
critic don't score the student's prediction at one of those four points. They score it at a separately,
uniformly sampled timestep (`_sample_score_timestep`, bounded by the config's own `min_timestep_ratio` and
`max_timestep_ratio`) that has nothing to do with the four-point schedule. That gap between "the schedule the
student denoises through" and "the noise level the loss is actually computed at" is DMD2's real
distribution-matching mechanic — matching the student's output distribution to the teacher's across a broad,
randomly sampled range of noise levels, not just at the four points that ship in the released checkpoint.
It also means one training iteration touches the student transformer far more than four times. In
`"simulate"` mode, `_student_rollout` always walks the *entire* fixed schedule under `torch.no_grad()` —
`for step_idx in range(max_target_idx)` runs all three transitions between the checkpoint's four denoising
steps unconditionally, regardless of which target index gets randomly sampled — and only then takes one more
call, graded or not, at the sampled target. That's exactly four student forward passes per `_student_rollout`
call, every time. `_critic_flow_matching_loss` calls `_student_rollout` again on *every* iteration (with
`with_grad=False`) purely to generate the sample the critic's loss trains against — four more student calls
that happen whether or not the generator itself updates that step. On the iterations where
`generator_update_interval` also fires the generator loss, `_student_rollout` runs a second time with
`with_grad=True` for another four. So a training iteration touches the student transformer four times just to
train the critic, and eight times on the one iteration in five where the generator updates too — "four calls"
describes what ships at generation time, not what training costs to compute.
The code also confirms, from the loss functions themselves rather than from metadata, which network runs
which attention: the student's rollout calls `predict_x0(..., attn_kind="vsa")`, while both the critic's
flow-matching loss and the teacher/critic scoring in `_dmd_loss` call `predict_x0`/`predict_noise` with
`attn_kind="dense"` throughout — exactly the split the earlier diagram describes, now verified against the
training loop rather than just the checkpoint's config.
One thing worth flagging plainly: the generic `DMD2Method` expects a student role model whose
`predict_x0(..., attn_kind="vsa")` actually executes sparse attention, but the one open MiniMax H3 training
wrapper in this codebase, `fastvideo/train/models/minimax_h3/minimax_h3.py`, raises `ValueError("MiniMaxH3Model
supports dense attention for training")` for any `attn_kind` other than `"dense"` — and its only wired-up
training config in the repository (`examples/train/configs/overfit_minimax_h3_t2va.yaml`) pairs it with plain
`FineTuneMethod`, not `DMD2Method`. No example or config anywhere in this checkout actually combines MiniMax
H3 with DMD2. The checkpoint's own metadata is unambiguous that VSA-sparse DMD2 distillation is what produced
FastH3, so that part isn't in question — but the exact glue code that made `attn_kind="vsa"` work for an H3
student isn't in FastVideo's public tree as cloned here. Either it lives outside this snapshot, or the public
`MiniMaxH3Model` class was written for plain finetuning and doesn't (yet) cover the distillation path the
released checkpoint actually went through. Source alone can't settle which, and it's the kind of gap that
lines up with the checkpoint's own self-reported `"known_recipe_deviation"` — training internals that have
moved on from what's merged upstream.
## Lever two: VSA-H3 makes each of the four calls cheaper
The second lever is [Video Sparse Attention (VSA)](https://arxiv.org/abs/2505.13389), adapted to H3's
particular attention pattern in `fastvideo/attention/backends/video_sparse_attn_h3.py`. H3 runs one joint
bidirectional attention over a packed sequence of `[text | condition keyframes | audio | generated video]`,
so VSA-H3 partitions only the video portion into 3D tiles over its own (t, h, w) latent grid. FastH3 is
trained on the 64-token tile shape — `(4, 4, 4)` — rather than the library's other supported shape, 256
tokens at `(4, 8, 8)`.
A coarse pooling stage scores every tile cheaply against the query first — nothing is skipped blind. Then
`compute_topk` (source: `max(1, min(ceil((1 - sparsity) x num_blocks), num_blocks))`) decides how many tiles
survive to expensive, exact fine-grained attention. FastH3's trained default is 90% sparsity, so roughly one
tile in ten does. Two details matter for what actually gets compressed: non-video *queries* (text, audio
tokens attending outward) are always dense, and non-video *keys* default to "exempt" — always included for
every video query, never competing for a top-k slot, with a separate "compete" mode existing in the code but
not used by FastH3. So a text or audio token never loses information to sparsity in either direction; only
video-to-video attention gets thinned.
The other detail is a learned gate the base checkpoint doesn't have. `to_gate_compress` blends a pooled
signal from the 90% of tiles that lose the top-k competition back into the output, so they aren't discarded
outright — just compressed. Reading the checkpoint's own safetensors headers directly confirms this is real,
trained weight, not a documentation claim: a `[7168, 5376]` `to_gate_compress.weight` tensor exists in all 50
transformer blocks (`num_attention_heads: 56` at `attention_head_dim: 128`, so `56 x 128 = 7168`, against
`hidden_size: 5376`). Base H3's loader zero-initializes this tensor, so untrained inference from the base
weights is exactly plain sparse attention with the compression branch contributing nothing; FastH3's
distillation run has actually trained it.
## The parameter count, reconstructed
FastVideo's own model card states 35B parameters for the released checkpoint. That number is checkable, and
it decomposes cleanly into the two pieces already described. MiniMax's own card puts the base H3 transformer
at 33B, and the trained `to_gate_compress` tensors are new weight the base checkpoint never had: `7168 x
5376 = 38,535,168` parameters per layer, and with a real value in all 50 transformer blocks —
`38,535,168 x 50 = 1,926,758,400`, about **1.93B** parameters. `33B + 1.93B ~ 34.93B`, which rounds to the
card's stated 35B almost exactly. The distillation run didn't just re-weight an existing network; it grew
the checkpoint by nearly two billion trained parameters to make the compression branch worth having.
The rest of what ships in the 148GB repository is worth sizing too, because "35B model" undersells the full
pipeline: the transformer shards total 70.1GB (that 35B figure, at bf16), but the `text_encoder` directory —
14 shards, a real tokenizer and chat template — comes to 66.7GB, which at bf16 is just over 33B parameters.
That's consistent with (if a couple billion larger than, presumably from a vision tower and projector on top
of the language backbone) the 32B Qwen3-VL encoder [the minimax-h3 piece](/articles/minimax-h3) identifies as
H3's text/vision conditioner. The video VAE adds another 10.4GB and the audio VAE 0.6GB. Run the full FastH3
pipeline and you're loading two frontier-scale transformers — a ~35B generator and a ~33B encoder — not one.
The checkpoint's `provenance.json` is also worth a mention on its own, separate from what it says: it ships a
base-model commit hash, a training-run ID, a Weights & Biases URL, and sha256 checksums for every file — more
supply-chain transparency than most open-weight drops include. It also self-reports a limitation nobody would
ask for: `"known_recipe_deviation": "Predates later continuous score-clock and FastGen-alignment
corrections."` This specific checkpoint (step 1300) is flagged, by its own metadata, as coming from before
FastVideo's training recipe was refined further — worth knowing if a later checkpoint in the same LoRA
collection supersedes it.
## What's actually on the GPU: three kernels behind one switch
"Up to 14x speedup on NVIDIA Blackwell GPU" implies something architecture-specific is happening, and it is —
but it's gated behind an opt-in switch most of the code path doesn't take by default, and it's a genuinely
separate, hand-written kernel from the one that runs on Hopper, not one kernel recompiled per architecture.
`fastvideo-kernel/csrc/attention/` ships `block_sparse_h100.cu` (ThunderKittens, Hopper `wgmma` and TMA,
compiled only under the `sm_90a` device pass) alongside `block_sparse_sm100a.cu` plus
`block_sparse_kernel_sm100a.cuh` (a warp-specialized kernel built on `tcgen05.mma`, TMA tensor maps, and
cluster-launch-control scheduling — Blackwell's fifth-generation tensor-core instructions, which don't exist
on Hopper at all). The kernel's own `CMakeLists.txt` is explicit about why a plain `sm_100` target won't do:
"`-arch=sm_100a` is NOT enough — it emits a plain `sm_100` target and `ptxas` rejects every `tcgen05` /
`setmaxnreg` instruction." Both are built only when the arch list says so: Hopper kernels compile only when
`TORCH_CUDA_ARCH_LIST` matches `9.0a`/`90a`/`sm_90a`, and the build prints its own fallback plan when it
doesn't — "ThunderKittens kernels: DISABLED (will use Triton fallbacks at runtime)." The Blackwell kernel
compiles only when the list matches `10.0a`/`100a`/`sm_100a` explicitly, for 64- and 128-token blocks.
Getting to that Blackwell kernel at runtime takes more than owning a Blackwell GPU, though.
`fastvideo/attention/backends/video_sparse_attn_h3.py` reaches it only through an explicit opt-in,
`FASTVIDEO_VSA_SM100A=1`, and only for the no-grad inference forward at FastH3's trained tile size (64
tokens) — "Grad-tracking forwards and every backward stay on Triton unchanged," in the module's own words,
because the Blackwell kernel returns no gradient at all. The generic dispatcher one layer down
(`fastvideo_kernel/block_sparse_attn.py`'s `block_sparse_attn_from_indices`) has no equivalent automatic
branch for Blackwell either: with no environment variables set, its rule is "use the compiled Hopper kernel if
the device reports `sm_90` capability, otherwise Triton" — so on Blackwell hardware, by default, even
inference falls back to Triton.
FastVideo's own benchmark reproduction script sets the switch itself:
`examples/inference/basic/basic_fasth3.py`'s `--vsa-kernel` flag defaults to `"sm100a"`, and its
`profile_environment()` sets `FASTVIDEO_VSA_SM100A="1"` accordingly as part of the `all` profile the repo's own
docs describe as "the fastest measured four-GPU Preview recipe." The switch that makes the announced Blackwell
numbers reachable is that script's default, not something a Blackwell GPU triggers on its own — and the same
docs note the honest fallback: "use `--vsa-kernel triton --no-fa4` if the Blackwell kernels are unavailable."
Two more details worth being precise about. First, the H3 backend's own module docstring states that at tile
64 "both forward and backward run the Triton block-sparse kernels directly" — read literally, that's the
correct description of a Blackwell-only build, where the Hopper kernel is compiled out entirely and Triton is
the dispatcher's only option; a build that also targets Hopper would prefer the Hopper kernel whenever the
device matches, comment notwithstanding. Second, FastVideo's own reproduction script and docs describe the
measured recipe as running on "four GB200 GPUs," while the announcement's table headers every column "B200" —
GB200 is Nvidia's Grace-Blackwell superchip (Blackwell GPU dies paired with a Grace CPU), not the standalone
B200 card the table's columns name. Both are Blackwell-generation silicon, so the architecture-level "on
Blackwell" claim holds either way, but the two documents don't agree on the exact SKU, and nothing in the
source resolves which one actually produced the published numbers.
It's also worth restating why this piece has used the 64-token tile throughout: it's FastH3's trained and
shipped default, not the library's own default. `VSA_H3_TILE_SIZE = (4, 8, 8)` — 256 tokens — is what
`video_sparse_attn_h3.py` falls back to when nothing overrides it, routed through an entirely different pair
of kernels (`block_sparse_attn_256.py`: a Triton "route A" expansion to 64-token tiles by default, or an
opt-in FA4 CuTe DSL fastpath via `FASTVIDEO_VSA_CUTEDSL=1` operating on 128-token physical blocks). Neither
hand-written CUDA kernel in this codebase ever sees a 256-token tile directly; FastH3 trains and ships at the
smaller, non-default geometry that's also the only one either hardware-specific kernel targets.
## What FastVideo actually measured
Every number below comes from one table in the announcement: warm end-to-end latency on B200 GPUs, median of
three timed requests after one full warmup (model loading and compilation excluded), and — in FastVideo's own
words — "end-to-end time includes encoding, denoising, decoding, audio, muxing, and file output." These are
full-pipeline numbers, not isolated diffusion-loop or attention-kernel benchmarks. Tests run at 1344x768 and
24 FPS with audio; the 5s/10s/15s clip lengths correspond to fixed shapes of 124/243/345 frames.
| Configuration | 5s (s) | 10s (s) | 15s (s) | Speedup vs Base H3 (1x / 4x) |
|---|---|---|---|---|
| Base H3 - Dense FA4 | 132.5 | 377.4 | 678.7 | 1.0x / 1.0x |
| Preview v1 Dense/DataFree - Dense FA4 | 18.3 | 50.2 | 91.3 | 7.24x–7.52x / 5.97x–7.54x |
| Preview v1 VSA/DataFree - 90% sparse | 16.2 | 31.1 | 47.2 | 8.16x–14.38x / 6.65x–12.48x |
*(1x/4x = 1 or 4 B200 GPUs; all timings and ratios FastVideo's own, quoted to the table's published
precision.)*
Two methodology notes the announcement states outright and that hold up: "no 8x speedup is claimed without a
matched Base H3 run" — Base H3 was never benchmarked at 8x GPUs, so the 8xB200 column exists only for the VSA
row, with no baseline to divide against. And the ratios are computed from FastVideo's internal unrounded
timings, not the two-decimal numbers printed in the table — which is why recomputing a ratio from the
published seconds sometimes lands a couple of hundredths off the printed one. That's expected rounding, not
an error in either direction.
## Decomposing 14x: two multipliers, not one
The middle row of that table — Dense/DataFree, the same four-step DMD2 distillation but *without* VSA, still
running dense FlashAttention-4 — is the isolation point that lets the two levers be pulled apart cleanly,
because it's the only configuration in the table that has one lever (distillation) but not the other
(sparsity). Dividing straight through:
| Duration | Distillation alone (Base / Dense, 1x) | VSA on top (Dense / VSA, 1x) | Combined |
|---|---|---|---|
| 5s | 132.5 / 18.3 = 7.24x | 18.3 / 16.2 = 1.13x | 8.18x (table: 8.16x) |
| 10s | 377.4 / 50.2 = 7.52x | 50.2 / 31.1 = 1.61x | 12.11x (table: 12.13x) |
| 15s | 678.7 / 91.3 = 7.43x | 91.3 / 47.2 = 1.93x | 14.34x (table: 14.38x) |
The distillation-alone multiplier is remarkably flat across clip length — about 7.2x to 7.5x, nowhere near
the naive 12.25x that "49 calls to 4" predicts on its own, for the fixed-overhead reason above. VSA's
multiplier *on top* of that, by contrast, is the one that actually grows with clip length: 1.13x at 5
seconds, rising to 1.93x at 15 seconds, because a longer clip has proportionally more diffusion compute for
90% sparsity to remove and proportionally less fixed encode/decode/mux overhead diluting the win. The 14.38x
headline is the product of both — a distillation multiplier that barely moves, times a sparsity multiplier
that does — and it's the 15-second row specifically where both are large enough, and fixed overhead small
enough, for the combined number to say "14x" instead of "8x."
One more thing worth stating because it's easy to assume otherwise: there's no hardware-generation multiplier
hiding in this number. Base H3 and both FastH3 variants are benchmarked on the same B200 GPUs — "14x speedup
on NVIDIA Blackwell GPU" means a Blackwell GPU compared against that same Blackwell GPU, not a new chip
compared against an old one. Whatever is driving 14.38x, it's the two algorithmic levers above and their
interaction with fixed pipeline overhead — not different silicon.
## Is it actually real-time?
"Sub-realtime generation on 8xB200 GPUs" is the other headline claim, and it means something specific:
generation wall-clock time faster than the clip's own duration — watching it would take longer than making
it. Checked against each of the three durations FastVideo tested, rather than just the one the announcement
leads with:
At 5 seconds, the fastest configuration shown (VSA/DataFree on 8xB200) takes 6.84 seconds — 37% *slower*
than real time. At 10 seconds it's 11.66 seconds, 17% slower. Only at 15 seconds does it cross the line, at
12.88 seconds, matching the announcement's "less than 13s" almost exactly. "Sub-realtime generation on
8xB200 GPUs" is accurate for precisely the duration FastVideo chose to lead with, and not for the other two
rows of their own table. Base H3 and the Dense/DataFree variant never approach real time at any duration or
GPU count shown — sub-realtime is a VSA-plus-8-GPU result specifically, not a general property of the
distilled model.
## The other checkpoints FastVideo shipped
The recommended VSA/DataFree checkpoint isn't the only thing in the release. FastVideo also publishes three
comparison LoRAs, all distilled to the same four DiT calls, grouped in one collection so the training-source
and attention axes can be tested independently of each other:
| Checkpoint | Training source | Attention | Step |
|---|---|---|---|
| VSA / Synthetic | forward-noised Base-H3-generated video | VSA, 90% sparse, tile 64 | 1300 |
| VSA / Synthetic (longer) | forward-noised Base-H3-generated video | VSA, 90% sparse, tile 64 | 1900 |
| Dense / Data-Free | prompts only | Dense FA4 | 1000 |
"Data-Free" and "Synthetic" describe the training-source axis (prompts-only vs. Base-H3-rendered video as a
starting point); "VSA" vs. "Dense" describes the attention axis. The recommended release sits at the
Data-Free x VSA corner; the ablations let you check what changes if you swap either axis independently — at
the cost of researching them yourself, since FastVideo doesn't publish latency numbers for the two synthetic
variants (they share VSA/DataFree's runtime characteristics, so the announcement doesn't repeat the table for
them).
One correction to that framing: the recommended VSA/Data-Free configuration isn't published only as the
~148GB full checkpoint this piece analyzes. `examples/inference/basic/README.md`'s "FastH3 Preview LoRAs"
section lists four launcher scripts, not three — `run_fasth3_lora_preview_vsa_datafree.sh` alongside the two
synthetic ablations and the dense one — all pulling adapters from the same
`FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA` collection. So the recommended configuration ships in two
forms: the full weights analyzed above, and a much smaller adapter over base MiniMax H3. Per
`examples/serving/README.md`, that adapter isn't a pure low-rank delta either: "FastH3 adapters are hybrid
startup patches: alongside low-rank factors they may contain dense deltas and a VSA compression-gate
replacement" — which makes sense once you know `to_gate_compress` doesn't exist as a real tensor in base H3 at
all (the loader zero-initializes it), so a rank-64 factor has nothing to correct there; the adapter has to
ship the whole `[7168, 5376]` gate as a dense replacement. And per the same README, strength 1 "approximates
the full student" rather than reproducing it exactly — the LoRA form is a compressed stand-in, not a
bit-identical alternative to the checkpoint this piece is built on.
## FastVideo is bigger than one checkpoint
Reading only the announcement, FastVideo could pass for a lab that ships one distilled checkpoint at a time.
The repository it's built from is a full training-and-inference framework: `fastvideo/models/dits/` alone
carries on the order of thirty diffusion-transformer architectures — Wan, Hunyuan, Cosmos, Flux, LTX2,
Kandinsky5, MatrixGame2/3, GLM-Image, and MiniMax H3 among them — and `fastvideo/attention/backends/` carries
eleven other attention implementations beside VSA-H3 and its own base-VSA sibling (dense FlashAttention, SDPA,
Sage Attention and its v3, Nabla, sliding-window attention, VMoba, and separate quantized-attention
training/inference paths). FastH3 is one entry in a
model registry (`fastvideo/registry.py`) that already lists on the order of ninety model identifiers.
The framework extends well past the DiT zoo, too: `apps/dreamverse/` is a realtime "vibe-directing" streaming
product with its own server and web UI, deployable on Modal, Docker, or a self-hosted server over SSH;
`comfyui/` ships a ComfyUI integration with its own examples and web assets; `docker/` and FastVideo's
OpenAI-compatible serving entrypoints back the exact REST surface `examples/serving/openai_fasth3.yaml` uses
to serve FastH3 itself; and `fastvideo/performance_dashboard/` is a small service (`api.py`, `metrics.py`,
`service.py`) for tracking benchmark results over time. This piece has already noted that FastVideo hasn't
published a VBench score or any other quality metric for FastH3 — worth adding that the gap isn't a tooling
one. `fastvideo/eval/` is a real, generic, multi-GPU evaluation harness (`Evaluator`/`EvalWorker`, with a
`"vbench"` metric group resolvable by name) built to score any model in the registry. The framework can
compute these numbers; nobody has published any for this checkpoint.
Apple Silicon is the clearest place where the announcement undersells its own codebase. `fastvideo/mlx_runtime/`
is not a stub: dedicated modules `minimax_h3.py` (1,286 lines), `minimax_h3_pipeline.py`,
`minimax_h3_video_vae.py`, `minimax_h3_audio_vae.py`, and `minimax_h3_conditioner.py` reimplement H3's DiT,
scheduler, VAEs, and text/vision conditioner natively in MLX, checked against the torch reference by a real
parity test suite (`fastvideo/tests/mlx/test_mlx_minimax_h3_parity.py`), with INT8/INT6/INT4 quantization of
the attention and FFN matrices and a working example entrypoint (`examples/inference/basic/mlx_fasth3.py`)
that already runs text-to-audio-video end to end on an M4 Max. FastVideo's own support matrix says as much,
plainly: "MLX FastH3 Preview T2VA... Apple M4 Max, 36 GB unified memory... Source runtime; T2VA only" — flagged
less mature than the packaged FastMetal-QAD releases sitting next to it in the same table, but real, tested
code, not aspiration. One thing doesn't carry over, though: H3's MLX attention is plain dense
`mx.fast.scaled_dot_product_attention` (`fastvideo/mlx_runtime/minimax_h3.py`); the sliding-window sparse
module the MLX runtime does have (`windowed_attention.py`) is wired only into the Wan/FastMetal path
(`fastwan.py`), not into H3, so none of VSA-H3's sparsity savings apply on Apple Silicon yet.
RTX and DGX Spark sit in a different place. The framework broadly documents both — the README's own feature
list states support for "H100, A100, 4090" across Linux, Windows, and macOS, and there's a dedicated DGX Spark
install guide (`docs/getting_started/installation/spark.md`) for Nvidia's GB10 ARM64 platform, complete with
its own from-source kernel build since no prebuilt ARM wheel exists. But neither document, nor anything else
found in this checkout, ties FastH3 specifically to either platform the way the MLX runtime does for Apple
Silicon. For RTX and DGX Spark, "explicit interest in testing" — the announcement's own phrase, addressed
again below — is a fair description of where things stand today. Apple Silicon is the one platform where that
interest has already turned into a shipped, if self-flagged-as-preliminary, runtime.
## What FastH3 openly says it isn't yet
Two limitations are worth naming because FastVideo states them directly rather than leaving them for someone
else to discover. The model card is explicit that this preview "supports text-to-audio-video generation"
only — FL2VA (first-frame-and-last-frame-to-video-audio) and Ref2VA (reference-conditioned) were not
distilled, despite Base H3 supporting both as a general omni-modal model. And on quality: "difficult motion,
fine detail, and some audio may remain below the base MiniMax H3 model" — a four-step, 90%-sparse student is
not claimed to match its dense, 49-call teacher on everything, and there is no VBench score, human-preference
study, or other quantitative quality metric published anywhere in the release to check that claim against.
Every number in this piece is a speed number; none of them are evidence about output quality, because
FastVideo hasn't published any.
The roadmap section of the announcement names what's next rather cautiously: an 8-step option (framed as
possibly *higher* quality than the 4-step model, not just slower), FL2VA and Ref2VA distillation, a
collaboration with NVIDIA's FastGen team on Parallel Decoding Distillation, and FP8/NVFP4 variants — plus
explicit interest in testing on RTX, DGX Spark, and Apple Silicon, since the checkpoint itself is hardware-
independent and B200 is only FastVideo's own controlled benchmark platform, not a requirement. That's an
accurate way to describe RTX and DGX Spark today, per the source dug into above — but it undersells Apple
Silicon specifically, where a real (if self-flagged "Source runtime") MLX port of H3 already exists and runs
end to end on an M4 Max, dense attention and all.
## Checked, in one table
| Claim | Status |
|---|---|
| Base H3 calls its transformer 49 times; FastH3 uses exactly 4 | Holds — confirmed independently via `checkpoint_metadata.json`'s `dmd_denoising_steps: [999, 749, 500, 250]` |
| "Data-Free" means no video is loaded, real or synthetic | Holds — `preprocessed_data_type: "text_only"` plus noise-start backward simulation, read from the checkpoint's own training config |
| Trained `to_gate_compress` gate adds ~1.93B parameters over base H3 | Holds — computed from the safetensors header shapes (`[7168, 5376] x 50 layers`), independent of the card's rounded 35B figure |
| "Up to 14x speedup on NVIDIA Blackwell GPU" | Holds for the specific cell it describes: 15s clip, 1xB200, VSA/DataFree vs. Base H3 Dense FA4. Not the speedup at 5s or 10s, and not a hardware-generation comparison |
| "Generate 15s 768p video in less than 13s...sub-realtime on 8xB200" | Holds only at 15s (12.88s). Worth qualifying at 5s (6.84s, 37% slower than real time) and 10s (11.66s, 17% slower) |
| Base model is a MiniMax H3 finetune | Holds — all three training networks (teacher, critic, student) initialize `from` the same MiniMax-H3 checkpoint per the training config, and the base model card confirms a 33B dense architecture |
| "On Blackwell" means a separate, hand-written CUDA kernel is running | Holds, but it's opt-in — `FASTVIDEO_VSA_SM100A=1` gates it (`video_sparse_attn_h3.py`); unset, even Blackwell hardware falls back to Triton by the kernel's own default dispatch, and FastVideo's benchmark script sets the switch itself |
| DMD2 training runs the exact code merged into FastVideo's public tree | Unresolved — `dmd2.py` implements the algorithm generically and matches the metadata's field names, but the only open MiniMax H3 training wrapper currently rejects `attn_kind="vsa"`, and no example config in this checkout pairs MiniMax H3 with `DMD2Method` |
| "Interest in testing on... Apple Silicon" | Complicates — a real, tested MLX runtime for FastH3 already ships in-tree (`fastvideo/mlx_runtime/minimax_h3.py`), flagged "Source runtime; T2VA only" in FastVideo's own support matrix; it runs dense attention, not VSA-H3's sparse mechanism |
## The take
FastH3 is a genuinely well-instrumented release: a checkpoint whose training recipe, tensor shapes, and
provenance are all readable without trusting the announcement's prose, which is rarer than it should be for
a "state of the art" claim. The two mechanisms are legitimately different kinds of savings — fewer transformer
calls from DMD2 distillation, cheaper attention within each call from a trained sparse-attention gate — and
having the Dense/DataFree ablation in the published table means you don't have to take FastVideo's word for
how much each one contributes; you can divide the table yourself and get 7.2x-7.5x from distillation and
1.1x-1.9x from sparsity on top, growing with clip length in exactly the direction the mechanism predicts. The
headline numbers are real numbers pulled from real cells in that table — they're just specific cells,
describing the longest clip length and the most GPUs FastVideo tested, and the announcement is honest enough
to publish the shorter, less flattering rows right next to them.
Reading the framework this checkpoint comes from adds the same texture, one level down. "On Blackwell" is a
real, hand-written, architecture-specific kernel — genuinely different tensor-core instructions from the
Hopper path, not a recompile — sitting behind an opt-in switch that FastVideo's own benchmark script happens
to flip by default, rather than something a Blackwell GPU does automatically. The training step that produced
the checkpoint runs more forward passes than the four the metadata advertises, and the specific glue code that
let an H3 student train with sparse attention doesn't appear to be in FastVideo's public tree as cloned here.
And "interest" in new hardware is further along for Apple Silicon than the announcement's cautious phrasing
suggests — a real MLX port already runs FastH3 end to end, dense attention and all. None of this contradicts
the checkpoint-level accounting above; it's the same honesty check applied one layer further down, into code
the announcement doesn't quote from at all.
For readers coming from the sparse-attention side, [VSA-H3's tile-and-gate design](/articles/minimax-sparse-attention)
sits alongside a growing family of ways to cut attention's quadratic cost in video and long-context models —
compare it with [Sol-Attn's training-free block routing](/articles/sol-attn) for video diffusion, or [a field
guide to the wider design space](/articles/attention-mechanisms). For the distillation side, [MrFlow's
training-free resolution reshuffle](/articles/mrflow-diffusion-acceleration) tackles the same
"fewer/cheaper diffusion steps" problem from a completely different angle. And for what a from-scratch
efficient video architecture looks like instead of a distilled one, [SANA-Video 2.0](/articles/sana-video2)
and [Chimera](/articles/chimera-diffusion) are the two most direct comparisons on this site.
---
*Sources: the [FastH3 Preview v1 announcement](https://haoailab.com/blogs/fasth3-preview/); the
[FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree model card and files](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree)
on Hugging Face, including its `checkpoint_metadata.json`, `provenance.json`, and safetensors headers, read
directly via the Hugging Face API; the [MiniMax H3 base model card](https://huggingface.co/MiniMaxAI/MiniMax-H3);
the [FastVideo repository](https://github.com/hao-ai-lab/FastVideo), specifically
`fastvideo/attention/backends/video_sparse_attn_h3.py`, `video_sparse_attn.py`, and
`video_sparse_attn_h3_probe.py`; the DMD2 training method at
`fastvideo/train/methods/distribution_matching/dmd2.py` and the MiniMax H3 training model at
`fastvideo/train/models/minimax_h3/minimax_h3.py`; the hand-written CUDA kernels and their build gating in
`fastvideo-kernel/csrc/attention/` and `fastvideo-kernel/CMakeLists.txt`, and the kernel dispatch logic in
`fastvideo-kernel/python/fastvideo_kernel/block_sparse_attn.py` and `block_sparse_attn_256.py`; the FastH3
benchmark reproduction script `examples/inference/basic/basic_fasth3.py` and its accompanying
`examples/inference/basic/README.md` and `examples/serving/README.md`; the Apple Silicon MLX runtime at
`fastvideo/mlx_runtime/minimax_h3.py` and `windowed_attention.py`, and FastVideo's own
`docs/inference/support_matrix.md`; the [DMD2
paper](https://arxiv.org/abs/2405.14867) (Yin et al.) and the [VSA
paper](https://arxiv.org/abs/2505.13389) (Zhang et al.). The validation-sample stills are FastVideo's,
shown for commentary. The tile-selection, pipeline, call-count, real-time, and kernel-routing diagrams are
original, built from the sources above.*