SANA-Video 2.0: keeping video attention linear without losing the picture

Video generation has a scaling problem that images mostly dodge. After a VAE compresses a clip, a single 1080p video still spans tens of thousands of latent tokens — and a standard video diffusion transformer runs full 3D softmax attention over all of them, at every layer, at a cost that grows with the square of the token count. Double the resolution or the duration and the attention bill doesn't double, it quadruples. That $O(N^2)$ wall is why most open video models cap out at a few seconds and lean on clusters.

SANA-Video 2.0, from NVIDIA, is an argument that you can walk around the wall instead of paying to climb it. It generates high-quality video up to 720p on a single GPU — the 5B model renders a 720p, 5-second clip in 13.06 seconds on one H100, which the paper clocks at roughly 120× faster than Wan 2.2-A14B — while claiming quality parity with much larger full-softmax systems. It does this by being disciplined about where quadratic attention is actually worth its price.

expand

The organizing idea is the same "spend cost where the signal is" instinct that the Mage-Flow explainer applied to image tokenizers — but pointed at attention and stretched across time. To see why it works, start with the family it comes from.

The SANA lineage: efficiency as a house style

SANA has always been the efficiency line. The original SANA image model made three bets that each cut a different cost: a deep-compression autoencoder (DC-AE) that squeezes an image by 32× per side instead of the usual 8×, so the transformer sees far fewer tokens; a Linear Diffusion Transformer that replaces quadratic self-attention with linear attention; and a decoder-only text encoder — a small Gemma LLM in place of the heavy T5 — for conditioning. Together they let a laptop-class GPU generate 1K and even 4K images. SANA-Video 1.0 carried the recipe to video with a 2B, pure-linear DiT and posted big speedups at competitive quality.

But pure linear attention pays for its $O(N)$ speed with expressiveness. Linear attention compresses all of the past into a single fixed-size state matrix $S \in \mathbb{R}^{d \times d}$; that state simply cannot encode every token-to-token interaction, and for video — where precise spatiotemporal correspondence and fine detail matter — the missing interactions show up as softness and drift. This is the exact tension the 2.0 paper sets out to resolve, and it borrows its fix from a place you might not expect: recent large language models. Qwen3-Next and Kimi-Linear keep a mostly-linear stack but insert a few softmax anchors — periodic full-attention layers that restore exact interactions at fixed depths — and route information across depth with attention residuals, a combination Kimi K3 runs at trillion-parameter scale. SANA-Video 2.0 asks whether the same trick unlocks long, high-resolution video. It answers yes.

Two levers, and why they compound

SANA's speed isn't one trick; it's two multiplicative ones. The first is upstream of the transformer entirely: a high-compression VAE (SANA-Video 2.0 uses LTX-VAE 2.3, with a stride of 8×32×32 — 32× on each spatial axis, 8× in time) turns a clip into a modest sequence of latent tokens before the DiT ever runs. A 5-second 720p clip that is millions of pixels collapses to on the order of ten thousand tokens. The second lever is the one this paper is about: given that token sequence, how does attention cost scale as the clip grows? Drag the length and flip the resolution:

The point isn't the exact numbers — it's the shape. Because attention is quadratic in the token count, a full-softmax DiT's cost curls sharply upward as clips lengthen and sharpen; the linear-dominated hybrid's stays low, so the gap widens precisely in the long, high-resolution regime where video is most expensive. The paper's compiled profiling puts the DiT forward pass at 1.55× faster at 5 seconds rising to 3.2× faster at 60 seconds versus a matched full-softmax baseline at 720p, and 2.01× faster even at 1080p/121 frames. Compression gives you few tokens; linear attention makes each token cheap; the two savings multiply.

The mechanism: 25% softmax, and residuals to carry it

So how much softmax do you actually need? SANA-Video 2.0's answer, established through reduced-resolution proxy studies, is one layer in four. Its backbone is a stack of Hybrid Linear–Softmax Attention layers at a 3:1 ratio: three gated-linear-attention layers — cheap $O(N)$ mixing — for every one gated-softmax anchor that restores the full-rank interactions the linear layers can't represent. Then a second mechanism, Block Attention Residuals (AttnRes), groups the layers into blocks and routes each completed block's summary forward into later linear layers, so the anchors' refreshed representations propagate across depth instead of decaying — worth about a 12% lift in deep-layer effective rank in the paper's probes. Flip the regime and toggle the residuals:

Two design choices are worth flagging as honest engineering, not magic. First, SANA-Video 2.0 is trained from scratch as a hybrid — it is not a pretrained softmax model that was later "linearized," a shortcut that usually leaves quality on the table. Second, the 25% figure is a measured trade-off point, not a round number: fewer anchors and quality slips; more and you're paying for softmax you didn't need. The paper's architecture diagram lays out both pieces — the hybrid layer, the 8-layer blocks, and the shared-query router that does the aggregation:

expand

The two models share this design at different sizes: the 5B is a 32-layer, width-2,560 backbone; the 14B is 40 layers at width-4,096 (14.25B parameters). Both operate on LTX-VAE 2.3 latents and draw text features from Gemma-2-2B-IT — the decoder-only text encoder carried straight from the SANA lineage — through cross-attention at every layer.

What "Video2" adds: making it a real generator

A cheap backbone is only half a video model; the other half is the training pipeline that teaches it motion and taste. SANA-Video 2.0 is trained with flow matching (the same few-step-friendly objective the FLUX 3 explainer walks through for video), then sharpened in stages: a Self-Flow distillation that compresses the sampler, Direct Preference Optimization, and an online Reward-Feedback-Learning RL loop. It generates 480p–720p at 81, 121, or 193 latent frames — multi-second clips, extendable to 8 seconds after fine-tuning — and, because the whole stack was built to be hardware-friendly, a final Sol-Engine pass (kernel fusion, caching, and sparse attention) squeezes out a further 3.58× end-to-end, which is what brings the 5B pipeline to that 13.06s figure. Here is a representative clip from the project page — a surreal "world in a bottle" bobbing on the ocean, the kind of shot whose value is in staying coherent across time:

The numbers — and what they are

The headline is latency, and it is dramatic. Reading straight off the paper's one-H100, 720p/5s profile and expressing each baseline as a multiple of SANA 5B's 13.06s, the field looks like this:

The efficiency claim only means something if quality holds, and here the evidence is a VBench score of 84.30 for the 5B at 40 sampling steps — essentially level with the 14B's 84.23 and with the Wan 2.2 quality point the paper marks on its chart — reached in a small fraction of the latency. The paper's own framing is the honest one: match full-softmax quality while keeping linear attention's long-sequence scaling.

Honest limitations

The ceiling is real: 720p is the top resolution and clips are seconds, not minutes — this is not yet a long-form or 1080p+ model, and the 720p/8s operating point comes from a small supervised fine-tuning stage ($\sim 10^4$ clips), so the highest-resolution, longest-duration quality is the least battle-tested part. The 25% ratio is validated at reduced-resolution proxy scale and then trusted at full scale. The VAE that does so much of the compression work is a licensed external component (LTX-VAE 2.3), not SANA's own DC-AE — worth noting because it means the headline contribution here is squarely the attention design, not the tokenizer. And as always with a fresh tech report, every number is the authors'. None of this undercuts the core result; it just sizes it.

The take

SANA-Video 2.0 is a clean, well-argued answer to the question that has quietly bounded open video generation: do you have to pay quadratic attention to get softmax-quality video? The answer is no — keep three layers in four linear, spend softmax only at periodic anchors, carry the anchors' work forward with residuals, and feed the whole thing from a high-compression VAE so the token count is modest to begin with. The savings compound exactly where video is most expensive, which is why the gap grows with length and resolution rather than shrinking. It pairs naturally with FLUX 3, which bets on scale and joint multimodality to reach video, and with Mage-Flow, which makes the same co-design argument for images: efficiency is an architecture problem, not only a compute one. Worth the usual caveats on vendor benchmarks and the 720p/seconds ceiling — but as a demonstration that linear attention can carry real video without visibly losing the picture, it's the most convincing one so far.

Source: SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation (Chen et al., NVIDIA, 2026), the project page, and the SANA repository. The teaser and architecture figures and the sample clip are the authors', shown for commentary; all benchmarks are paper-reported. The attention-scaling and hybrid-stack interactives are mine. Related: Mage-Flow on efficient tokenizers and FLUX 3 on flow-matching video.