LongLive-Plug: distill the speedup once, plug it into 54 video models

A video diffusion model rarely ships once. A lab trains a large base, and then the ecosystem bends it into specialists: a world model you can steer with actions, a depth-conditioned ControlNet, a line-art colourizer, a talking-head generator, a robotics simulator. Each of those usually goes through a distillation stage before it is usable in production, to cut the sampling cost or to keep a long video from drifting. And that stage is paid again for every new specialist: new data, a teacher to supervise against, another optimization run.

LongLive-Plug (arXiv 2609.38154, 29 Sep 2026, NVIDIA Efficient-Large-Model with MIT's Song Han) asks whether that bill has to be re-paid each time. Its answer: distill the capability once, as a LoRA on a frozen base model, and then add that LoRA to compatible downstream models with no further training. The paper calls it a once-for-all framework and verifies the idea across 54 downstream models. I read the paper, the six released checkpoints' cards, and the training code to check which numbers hold.

What distillation buys a video diffusion model

Three different costs get distilled away, and all three are normally re-done per model.

Few-step sampling. A diffusion transformer denoises a latent over many steps — 30 to 50 is typical for the models here. Each step is a full forward pass of a multi-billion-parameter transformer, so the step count is the latency. Distillation trains a student to take the same trip in four steps. LongLive-Plug uses DMD2 (distribution-matching distillation), the same family as the few-step image students I covered in few-step Qwen-Image-2.1 and the FastH3 preview.

Classifier-free guidance, folded into one pass. Guidance sharpens how closely a sample follows the prompt. The standard recipe runs the model twice per step — once conditioned on the prompt, once unconditioned — and extrapolates between them. For noisy latent $z_t$ at step $t$ with condition $c$, write the two predictions as $v_c = F_{\theta_0}(z_t, t, c)$ and $v_\varnothing = F_{\theta_0}(z_t, t, \varnothing)$. The guided prediction at scale $w$ is

Two forward passes per step is twice the compute. CFG distillation trains the model to emit $v_{\mathrm{cfg}}^{(w)}$ from a single conditional pass, halving the per-step cost. (The base of few-step Qwen-Image-2.1 already ran CFG-free, so there the whole speedup was steps; for video models the CFG pass is a real second cost to remove.)

Long-context error correction. Autoregressive (AR) video models generate a long clip chunk by chunk, each conditioned on its own history. Errors accumulate: late frames smear. A long-context distillation step corrects that drift so the rollout holds up far past the training window.

Each capability is a different training recipe. The usual workflow runs the right one again on every specialized checkpoint. That is the tax LongLive-Plug is trying to stop paying.

The once-for-all idea: distill into a LoRA, then add it

The move is to treat a distilled capability as a reusable functional LoRA rather than a property baked into one checkpoint. Freeze a base model $F_{\theta_0}$ for a backbone family. Distill a capability once into LoRA parameters $\phi$. Then, for any compatible downstream model $F_{\theta_\tau}$ in that family, deploy it by adding the adapter:

Here $\oplus$ adds the LoRA update to the corresponding layers of the target while keeping the target's own task-specific weights. No downstream data, no re-distillation. The LoRA was fit against the base model's behaviour, and the claim is that behaviour is shared closely enough by the family's descendants that the same update still lands.

expand

What does "compatible" require? The paper is precise about the boundary. Reuse works for descendants of the base model — full fine-tunes, task LoRAs, and models that bolt on extra conditioning branches or widen their output channels. Those keep the base's layer structure and weight space, so an additive update to matching layers is still meaningful. It does not work across unrelated architectures, and the long-context adapter additionally needs the target to already run causal AR inference, because a LoRA changes weights, not attention masks. One capability is distilled once per backbone family; three families means three sets of adapters, not one universal one.

The paper verifies this on three families: Wan2.1-T2V-14B, Wan2.2-TI2V-5B, and MiniMax-H3 — the omni-modal 33B transformer I wrote up in MiniMax H3, whose few-step story I followed in the FastH3 preview. The verified set is 24 downstream models for each Wan backbone and 6 for H3, 54 in all, spanning eight task categories: world modeling, robotics, structure-conditioned generation, camera and trajectory control, editing and restoration, subject and avatar generation, audio and RGBA outputs, and domain or style adaptation (all reported, Table 5 and §4.2).

The clever part: a CFG LoRA you can still turn

There is a catch that makes naive reuse fail, and the fix is the paper's nicest idea.

Existing few-step methods such as CausVid and Self Forcing distill few-step sampling and CFG together. That bakes in one guidance scale — the one used during training. But downstream tasks want different guidance: a world model wants gentle guidance, a ControlNet wants strong. A single fixed scale cannot serve all of them, and you cannot just scale the whole coupled LoRA up to crank the guidance, because that also perturbs the few-step behaviour. The paper shows this failing directly: globally scaling the coupled LoRA on SCOPE darkens and distorts the scene, with severe collapse at weight 5 (reported, §4.3, Figure 5).

LongLive-Plug decouples the two. The few-step behaviour goes in one LoRA; CFG goes in a separate CFG-only LoRA $\phi_{\mathrm{cfg}}$. The CFG LoRA is trained at a single fixed teacher scale $w_{\mathrm{train}}$ to reproduce the guided prediction $v_{\mathrm{cfg}}^{(w_{\mathrm{train}})}$ from one conditional pass. The surprise is that its inference weight then behaves as a guidance dial. Scaling the adapter by $\lambda_{\mathrm{cfg}}$ approximately shifts the effective guidance to

a near-linear map, which matches prior observations that scaling a fine-tuning update scales its effect. So one CFG LoRA trained at a fixed scale gives continuous guidance control after the fact. Combine it with the few-step LoRA and the layer weight becomes

so you keep four-step sampling ($\lambda_{\mathrm{step}}$ fixed) while tuning guidance per target ($\lambda_{\mathrm{cfg}}$). The released Wan2.2-TI2V-5B card recommends a few-step-to-CFG weight ratio of 1 to 0.5 as a starting point, and notes those are adapter weights, not the inference CFG scale (reported, model card). On MiniMax-H3 the two LoRAs are, for now, recommended to be used separately rather than composed (reported, H3 cards).

The adapters are low-rank — the Wan branches all train at rank 128 (reported, Table 4) — which is what makes "distill once, carry forever" cheap to store and attach. For context on what a rank-128 LoRA on a 20B-class diffusion transformer looks like in practice, see Marigold V2.

Distill once, plug into many

The widget below is the whole argument made touchable. Pick a backbone family to see its frozen base, the reusable adapters, and how many downstream models the paper verified. The slider turns a downstream model's native step count into the four-step cost; the paper reports a 5x to 12.5x step cut across 20-to-50-step native schedules, plus one saved forward pass per step from folding in CFG. The cost panel is the economics: toggle the four Wan2.2-TI2V-5B specialisation tasks the paper measured and watch the re-distillation bill climb while the once-for-all path stays flat.

The numbers in that panel are the paper's. Every family reuses adapters distilled on its own base model, with no downstream training. For models with 20-to-50-step native schedules, attaching the base-distilled LoRA and running four CFG-free steps reduces denoising steps by 5x to 12.5x (reported, §4.2).

Does the transferred adapter actually generate well?

A speedup that wrecks quality is not a speedup. The headline comparison is on SCOPE, a Wan2.2-TI2V-5B world model. Native inference uses 30 steps. Naive four-step sampling — just truncating the schedule, no distillation — produces blurry video: FVD 805.5. The transferred LongLive-Plug adapter, also four steps, brings that to 478.7, within reach of 502.1 from a SCOPE-specific distilled model that was trained on SCOPE's own data (all reported, Table 1). The plug-in, with no target training, lands close to the bespoke run.

expand

On depth-conditioned control (Wan2.2-Fun-5B-Control, 600 cases, four steps with CFG disabled) the transferred adapter improves all six reported metrics over naive four-step sampling, including depth error si-RMSE from 2.135 to 1.641 and the DOVER technical-quality score from 8.90 to 10.11 (reported, Table 2). There are task-dependent trade-offs against per-target distillation on individual metrics — the paper is honest that transfer is not a free lunch on every axis — but the gap to a model trained on the target task is small.

Long context transfers too. A long-context LoRA distilled on an AR Wan base, moved onto ReWorld (trained on roughly 8-second windows), raises the mean of seven VBench quality dimensions at a 64-second rollout from 73.51 to 75.77, and beats the base at every tested length up to 8x the training duration (reported, §4.5, Table 3). On Matrix-Game 3.0, the same adapter lifts the total VBench score from 83.73 to 84.34 over a 62-second rollout, edging out even a model distilled specifically for it (84.30).

All figures reported from the paper; none re-run here.

The economics, spelled out

The reason to care is the bill. The paper tracks cumulative distillation cost as you add tasks to the Wan2.2-TI2V-5B family (reported, Figure 2). Both strategies share a one-time base distillation of about 80 H100 GPU-hours — 700 iterations on 32 GPUs, roughly 2.5 hours. From there they diverge.

Re-distilling per target adds 83.9, 150.0, 86.8, and 56.1 GPU-hours for depth-conditioned generation, world modeling, pose-conditioned generation, and robotics simulation respectively — 376.8 additional GPU-hours, about 456.8 in total for those four specialists. The once-for-all path adds zero: it reuses the base-distilled adapters and stays at about 80 GPU-hours no matter how many targets it serves. That is a 5.71x gap over four tasks (reasoned, from the paper's figures), and the gap widens with every new model. There is a data cost too: the depth-specific adapter needed 5,000 paired prompts with dynamic depth videos; the transfer needs none.

The break-even is immediate. Because every per-target run adds at least 56.1 GPU-hours on top of the shared base, reuse is cheaper from the first downstream model, not after some amortization point. The cost of a model you distill once and never again is the cost of distilling it once.

Where it breaks

The paper's limitations section is short and worth taking at face value.

• Compatible descendants only. Reuse requires models that share the base's weight space. A LoRA distilled on Wan does nothing for an unrelated architecture; each backbone family needs its own adapters.
• Long-context transfer needs causal AR inference. A LoRA updates weights, not attention masks, so it cannot grant autoregressive behaviour to a model that lacks it — only correct drift in one that already has it.
• Guidance control is approximate. The $\widetilde{w} = 1 + \lambda_{\mathrm{cfg}}(w_{\mathrm{train}} - 1)$ dial is near-linear, not calibrated; the paper says CFG LoRA weights "may require adjustment after transfer," and pushing the CFG-branch weight to 5 degrades quality.
• Task-dependent trade-offs. Against a model distilled on the target's own data, the transferred adapter wins on some metrics and gives a little on others. On MiniMax-H3 the camera-control transfer shows a visible quality-versus-control trade, and the few-step and CFG LoRAs are not yet recommended for combined use.

None of these undercut the core result. They draw its edges: this is reuse within a family, not a universal accelerator.

What shipped

Unusually for a plug-and-play claim, the pieces are all public. The code — training and inference, with four recipes covering CFG and few-step distillation — is in the LongLive-Plug/ directory of NVlabs/LongLive, released 28 Sep 2026. Six LoRA checkpoints are on Hugging Face: a few-step and a CFG adapter for each of MiniMax-H3, Wan2.1-T2V-14B, and Wan2.2-TI2V-5B. Each card lists its base model and the downstream models it has been verified against.

The framing I'd keep: distillation has been treated as a property you train into a checkpoint. LongLive-Plug treats it as a capability you train into an adapter — fit it against a frozen base, and it travels to the base's descendants for free. The underlying machinery is the usual DMD2 and CFG distillation; the move is deciding to keep the result reusable, and then showing the weight space of a model family is shared tightly enough for that to hold across 54 models. For the diffusion-transformer backbone all of this sits on, see the architecture explainer; for the flow-matching view of few-step generation, CSFM.