# SANA-Video 2.0: keeping video attention linear without losing the picture > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.thesatyajit.com/articles/sana-video2 > date: 2026-07-24 > tags: diffusion, video-generation, linear-attention, efficient-inference, nvidia, explainer Video generation has a scaling problem that images mostly dodge. After a VAE compresses a clip, a single 1080p video still spans *tens of thousands* of latent tokens — and a standard video diffusion transformer runs full 3D softmax attention over all of them, at every layer, at a cost that grows with the *square* of the token count. Double the resolution or the duration and the attention bill doesn't double, it quadruples. That $O(N^2)$ wall is why most open video models cap out at a few seconds and lean on clusters. [SANA-Video 2.0](https://arxiv.org/abs/2607.21553), from NVIDIA, is an argument that you can walk around the wall instead of paying to climb it. It generates high-quality video up to 720p **on a single GPU** — the 5B model renders a 720p, 5-second clip in **13.06 seconds** on one H100, which the paper clocks at roughly **120× faster** than Wan 2.2-A14B — while claiming quality parity with much larger full-softmax systems. It does this by being disciplined about where quadratic attention is actually worth its price.
The organizing idea is the same "spend cost where the signal is" instinct that the [Mage-Flow explainer](/articles/mage-flow) applied to image tokenizers — but pointed at *attention* and stretched across *time*. To see why it works, start with the family it comes from. ## The SANA lineage: efficiency as a house style SANA has always been the efficiency line. The original **SANA** image model made three bets that each cut a different cost: a **deep-compression autoencoder** (DC-AE) that squeezes an image by 32× per side instead of the usual 8×, so the transformer sees far fewer tokens; a **Linear Diffusion Transformer** that replaces quadratic self-attention with linear attention; and a **decoder-only text encoder** — a small Gemma LLM in place of the heavy T5 — for conditioning. Together they let a laptop-class GPU generate 1K and even 4K images. **SANA-Video 1.0** carried the recipe to video with a **2B, pure-linear** DiT and posted big speedups at competitive quality. But pure linear attention pays for its $O(N)$ speed with **expressiveness**. Linear attention compresses all of the past into a single fixed-size state matrix $S \in \mathbb{R}^{d \times d}$; that state simply cannot encode every token-to-token interaction, and for video — where precise spatiotemporal correspondence and fine detail matter — the missing interactions show up as softness and drift. This is the exact tension the 2.0 paper sets out to resolve, and it borrows its fix from a place you might not expect: recent large language models. Qwen3-Next and Kimi-Linear keep a mostly-linear stack but insert a few **softmax anchors** — periodic full-attention layers that restore exact interactions at fixed depths — and route information across depth with **attention residuals**, a combination Kimi K3 runs at trillion-parameter scale. SANA-Video 2.0 asks whether the same trick unlocks long, high-resolution video. It answers yes. ## Two levers, and why they compound SANA's speed isn't one trick; it's two multiplicative ones. The first is upstream of the transformer entirely: a **high-compression VAE** (SANA-Video 2.0 uses LTX-VAE 2.3, with a stride of **8×32×32** — 32× on each spatial axis, 8× in time) turns a clip into a modest sequence of latent tokens before the DiT ever runs. A 5-second 720p clip that is millions of pixels collapses to on the order of ten thousand tokens. The second lever is the one this paper is about: given that token sequence, **how does attention cost scale as the clip grows?** Drag the length and flip the resolution: The point isn't the exact numbers — it's the *shape*. Because attention is quadratic in the token count, a full-softmax DiT's cost curls sharply upward as clips lengthen and sharpen; the linear-dominated hybrid's stays low, so the gap *widens* precisely in the long, high-resolution regime where video is most expensive. The paper's compiled profiling puts the DiT forward pass at **1.55× faster at 5 seconds rising to 3.2× faster at 60 seconds** versus a matched full-softmax baseline at 720p, and 2.01× faster even at 1080p/121 frames. Compression gives you few tokens; linear attention makes each token cheap; the two savings multiply. ## The mechanism: 25% softmax, and residuals to carry it So how much softmax do you actually need? SANA-Video 2.0's answer, established through reduced-resolution proxy studies, is **one layer in four**. Its backbone is a stack of **Hybrid Linear–Softmax Attention** layers at a **3:1 ratio**: three gated-linear-attention layers — cheap $O(N)$ mixing — for every one **gated-softmax anchor** that restores the full-rank interactions the linear layers can't represent. Then a second mechanism, **Block Attention Residuals (AttnRes)**, groups the layers into blocks and routes each *completed* block's summary forward into later linear layers, so the anchors' refreshed representations propagate across depth instead of decaying — worth about a **12% lift in deep-layer effective rank** in the paper's probes. Flip the regime and toggle the residuals: Two design choices are worth flagging as honest engineering, not magic. First, SANA-Video 2.0 is trained **from scratch** as a hybrid — it is not a pretrained softmax model that was later "linearized," a shortcut that usually leaves quality on the table. Second, the 25% figure is a *measured* trade-off point, not a round number: fewer anchors and quality slips; more and you're paying for softmax you didn't need. The paper's architecture diagram lays out both pieces — the hybrid layer, the 8-layer blocks, and the shared-query router that does the aggregation:
The two models share this design at different sizes: the **5B** is a 32-layer, width-2,560 backbone; the **14B** is 40 layers at width-4,096 (14.25B parameters). Both operate on LTX-VAE 2.3 latents and draw text features from **Gemma-2-2B-IT** — the decoder-only text encoder carried straight from the SANA lineage — through cross-attention at every layer. ## What "Video2" adds: making it a real generator A cheap backbone is only half a video model; the other half is the training pipeline that teaches it motion and taste. SANA-Video 2.0 is trained with **flow matching** (the same few-step-friendly objective the [FLUX 3 explainer](/articles/flux-3) walks through for video), then sharpened in stages: a **Self-Flow** distillation that compresses the sampler, **Direct Preference Optimization**, and an online **Reward-Feedback-Learning** RL loop. It generates 480p–720p at 81, 121, or 193 latent frames — multi-second clips, extendable to 8 seconds after fine-tuning — and, because the whole stack was built to be hardware-friendly, a final **Sol-Engine** pass (kernel fusion, caching, and sparse attention) squeezes out a further **3.58×** end-to-end, which is what brings the 5B pipeline to that 13.06s figure. Here is a representative clip from the project page — a surreal "world in a bottle" bobbing on the ocean, the kind of shot whose value is in staying *coherent* across time: