VRAM is a policy, not a property

Two generative-media releases landed this month with opposite claims about the same resource.

WanGP, DeepBeepMeep's super-app for Wan, MiniMax H3, LTX-2, Flux and roughly 230 other model definitions, leads its feature list with: "Low VRAM requirements: run select models with as little as 6 GB of VRAM." YuE2, M-A-P's music generator, opens its quick start with: "Linux · Python 3.12 · NVIDIA GPU with BF16 support and 24 GB VRAM."

Six gigabytes for a 33-billion-parameter video model. Twenty-four for a three-billion-parameter music model. Both numbers are correct. Neither is a fact about the model.

What a model needs in VRAM is set almost entirely by where you decide its weights live between the moments they are used, and both projects have made that decision in code you can read. So I read it, and I measured every checkpoint the claims rest on — not from a model card, but by pulling each safetensors header back over an HTTP range request and summing prod(shape) × sizeof(dtype) myself.

Where "6 GB" comes from

The README states the number without a subject. It traces to one entry in docs/CHANGELOG.md, version 9.21, 29 October 2025:

Wan 2.2 Ovi 10 GB for all the GPU Poors of the World: only 6 GB of VRAM to generate 121 frames at 720p. With 16 GB of VRAM, you may even be able to load all the model in VRAM with Memory Profile 3

The second, newer claim is in the README itself, for MiniMax H3 (v12.42, 5 August 2026):

But rejoice WanGP version is as usual Ultra Optimized: 5-6GB of VRAM only for 5s (124 frames) and 8-9GB of VRAM for 15s at 832x480.

Read those twice and note what is missing. No GPU is named. No generation time is given. No system-RAM figure appears anywhere near either sentence. The H3 claim gives a resolution for its 15-second case and not for its 5-second one, and it does not say which checkpoint it means — WanGP ships H3 as a full 33B and a pruned 20B, and those two files differ by 12.09 GiB on disk.

That omission is not how this project used to talk. Version 4.0, 13 April 2025:

New optimizations for old generation GPUs: Generate 5s (81 frames, 15 steps) of Vace 1.3B with only 5GB and in only 6 minutes on a RTX 2080Ti and 5s of t2v 14B in less than 10 minutes.

Frames, steps, VRAM, wall-clock, and the card. That is a complete claim about a low-VRAM configuration, and I grepped the whole changelog for a later one. There are exactly two other entries anywhere in the file that pair a VRAM figure with a time, and both are worth looking at:

The July 2025 line is for LTX Video at 22 GB — enough VRAM that almost nothing streams. The January 2026 line is the tell: "You should be able to run it with as low as 10 GB of VRAM. If you have at least 24 GB of VRAM you will be able to generate 20s at 720p in a single window in only 2 minutes." The VRAM figure is for the constrained config; the two minutes is for the unconstrained one. They are two different runs in one sentence, and the sentence never names a card.

So it isn't that the project stopped measuring time. It is that the timings that survive belong to the configurations where offloading barely happens, and the configurations the headline advertises have not had a published wall-clock in seventeen months.

The mechanism: one block computing, one block arriving

WanGP does not implement offloading itself. It depends on mmgp==3.8.0 (requirements.txt), the author's own "Memory Management for the GPU Poor", and the whole trick lives in src/mmgp/offload.py.

mmgp walks a model's named_modules() and looks for any nn.ModuleList with at least five entries. That becomes a tower; each entry is a floor — for a diffusion transformer, one transformer block. Everything that is not in a tower is the base. Then it replaces the forward of every submodule with a wrapper that checks, before running, whether the block that submodule belongs to is currently on the GPU:

# mmgp/src/mmgp/offload.py — hook_check_load_into_GPU_if_needed_default
def check_load_into_GPU_needed():
    self.ensure_model_loaded(model_id)
    if blocks_name == None:
        if self.ready_to_check_mem():
            self.empty_cache_if_needed()
    elif blocks_name != self.loaded_blocks[model_id] and blocks_name not in self.preloaded_blocks_per_model[model_id]:
        self.gpu_load_blocks(model_id, blocks_name)

gpu_load_blocks unloads whatever block was previously resident, copies the requested one in, and then — this is the part that makes the whole scheme viable — issues the next block's copy on a second CUDA stream so it overlaps with the current block's compute:

# mmgp/src/mmgp/offload.py — gpu_load_blocks
if self.async_transfers and blocks_name != None:
    prev = self.prev_blocks_names[entry_name]
    first = prev == None or prev != loaded_block
    next_blocks_entry = self.next_blocks_names[entry_name] if entry_name in self.next_blocks_names else None
    if first:
        cpu_to_gpu(torch.cuda.current_stream(), self.blocks_of_modules[entry_name])
 
    torch.cuda.synchronize()
 
    if next_blocks_entry != None:
        cpu_to_gpu(self.transfer_stream, self.blocks_of_modules[next_blocks_entry])

How many blocks get to stay resident is decided once, at load, by tune_preloading:

# mmgp/src/mmgp/offload.py — tune_preloading
base_size = self.blocks_of_modules_sizes[model_id]
current_budget -= base_size
current_budget = max(0, current_budget)
...
    current_budget -=  2 * max_floor_size
    current_budget = max(0, current_budget)
 
for floors, max_floor_size, tower_size in towers:
    tower_budget = tower_size / total_size * current_budget
    preload_blocks_count = int( tower_budget / max_floor_size)

The budget pays for the base first, then reserves two floors for the shuttle, and only whatever is left buys residency. Now the number that decides everything. mmgp's own profile 5 sets the transformer budget to 400 MB; profiles 2 and 4 leave it at 1200. WanGP overrides both in wgp.py:

# wgp.py — init_pipe
mmgp_profile = int(profile)
if mmgp_profile in (2, 4, 5):
    default_transformer_budget = default_transformer2_budget= kwargs.get("budgets", 100)
    if isinstance(default_transformer_budget, dict):
        default_transformer_budget = default_transformer_budget.get("transformer", 100)
        default_transformer2_budget = default_transformer2_budget.get("transformer2", 100)
 
    budgets = { "transformer" : default_transformer_budget if preload  == 0 else preload,
                "text_encoder" : 100 if preload  == 0 else preload,
                "*" : max(1000 if profile==5 else 3000 , preload) }

kwargs["budgets"] was set to an empty dict four lines earlier, so .get("transformer", 100) returns 100, in MiB (ONE_MB = 1048576). WanGP's default profile is 4. So on a default install, on every profile that offloads at all, the diffusion transformer's budget is 100 MiB — smaller than a single block of any model in this article. Feed that through the arithmetic above and preload_blocks_count is zero for all of them, every time — and mmgp has a name for the result, in the log line it prints at verboseLevel >= 1:

# mmgp/src/mmgp/offload.py — tune_preloading
if preload_total == 0:
    print(f"Async loading plan for model '{model_id}' : base size of "
          f"{(preload_total+base_size)/ONE_MB:0.2f} MB will be preloaded with a "
          f"{max_blocks_fetch/ONE_MB:0.2f} MB async"
          + (" circular" if len(towers) == 1 else "") + " shuttle")

That branch — the one where preload_total is zero and the whole stack becomes a shuttle — is the branch every model in this article takes. The block stack lives in system RAM and rotates through a two-block window. That is what "6 GB of VRAM" is describing.

The receipts

None of the numbers above came from a README. Each checkpoint's size is the Content-Length after the Hugging Face CDN redirect, and each block size is read out of the safetensors header itself.

Three things fall out of that table that no model card says.

"Wan2.2 Ovi v1.1 5s 10B" is 11.66B parameters. The video tower is 5,566,479,552 and the audio tower is 6,094,273,556, and you need both: models/wan/ovi/modules/fusion.py interleaves them block by block (vid_block = self.video_model.blocks[i]; audio_block = self.audio_model.blocks[i]), and ovi_handler.py declares them mutual co-tenants so mmgp never evicts one to make room for the other. Their blocks totals are identical to the parameter — 5,476,392,960 each — which is the twin-backbone design showing up in the file. The extra bulk on the audio side is a 528,915,456-parameter patch_embedding that lives in the base and never leaves the card.

Quantizing these models does not shrink the base. Ovi's video tower has a base of 171.8 MiB in BF16 and 171.8 MiB in int8; the audio tower is 1178.5 MiB either way. Quanto quantizes the linear layers inside the blocks and leaves embeddings, patch projections and heads alone. For Ovi that barely matters. For YuE2 it matters a lot, and I get to it below.

VRAM is nearly independent of model size here, and traffic is not. The full H3 int8 checkpoint is 31.70 GiB and the pruned one is 19.61 GiB. Under this policy the first holds 2876.2 MiB of weights and the second 2270.5 MiB — a difference of 0.59 GiB. The difference in bytes crossing PCIe on every forward pass is 11.98 GiB. The headline quotes the first figure.

The half that isn't quoted

Per forward pass through the block stack, at int8: MiniMax H3 moves 30.10 GiB, the pruned 20B moves 18.12 GiB, and Ovi moves 10.21 GiB across its two towers. Ovi's own default is 30 denoising steps (defaults/ovi_1_1.json), which is 306 GiB of host-to-device traffic for one five-second clip. H3's README says 15 to 20 steps is a minimum: 451 GiB.

What that costs in seconds depends on one thing I cannot measure — how long a single block takes to compute on your GPU. I do not have a 6 GB card, and the project publishes no timing for either model. So here it is a slider, and everything else is arithmetic on the block sizes above:

The shape is the honest part. While compute per block exceeds transfer per block, the prefetch on transfer_stream hides the copy completely and the VRAM saving is free. Past the crossover the GPU waits on PCIe and every extra byte of checkpoint is time on the clock. Two details make that crossover worse than the arithmetic suggests:

• The lowest-VRAM profile also turns off pinning. mmgp's profile 5 sets pinnedMemory=False. Copies out of pageable host memory have to be staged through a driver buffer, so the bandwidth term drops exactly in the configuration chosen by someone who had no VRAM to spare. Profile 4 pins "transformer" — but pinning a 31.70 GiB checkpoint needs 31.70 GiB of reservable RAM, and when it doesn't fit mmgp quietly downgrades: "Switching to partial pinning since full requirements for pinned models is X MB while estimated available reservable RAM is Y MB."
• Profile 4+ turns off prefetching entirely. if profile == 4.5: kwargs["asyncTransfers"] = False in wgp.py. The changelog sells it as saving "up to 1GB of VRAM with Flux 2", which it does — by removing the second in-flight block, and with it the overlap.

And then the number nobody quotes at all. mmgp's README is explicit about system memory:

Requirements:

• VRAM: minimum 6 GB, recommended 24 GB (RTX 3090/ RTX 4090)
• RAM: minimum 24 GB, recommended 48 GB

These RAM requirements are for Linux systems. Due to different memory management Windows will require an extra 16 GB of RAM to run the corresponding profile.

WanGP is built on mmgp and inherits every word of that, and its own hardware section — "Supported GPUs: RTX 40XX, RTX 30XX, RTX 20XX, GTX 16XX, GTX 10XX, Tesla V100, A100, H100" — gives no RAM figure whatsoever. Running H3 int8 on 6 GB of VRAM means holding 31.70 GiB of weights somewhere, and that somewhere is your DIMMs. On Windows, with pinning, a 64 GB machine is the entry ticket for the model whose advertised requirement is 6 GB.

To be fair to the claims: as far as I can check them from the artifacts, they hold. 2.00 GiB of resident weights for Ovi in a 6 GB budget leaves about 4 GB for latents and activations at 121 frames of 720p, which is plausible. 2.81 GiB for H3 int8 in a "5-6 GB" budget is tighter but not absurd. The problem is not that the numbers are wrong. It is that they are one column of a two-column table.

The same question from the other end

Now YuE2, which has the opposite problem. Its README asks for 24 GB and its checkpoint is 7,261,441,640 bytes. The Hub's own metadata agrees with my header parse to the parameter: 3,630,684,224. The model is called YuE2-3B; it is 3.63B.

expand

That architecture is visible in the checkpoint, tensor by tensor. Grouping the 628 tensors by name with the layer index stripped out:

Two equal-sized experts per layer, and only one of them runs at a time. pipeline.py exposes the stages separately — plan(), generate_semantic(), synthesize(), decode() — and decode() already knows this, because it moves the whole backbone to the CPU before loading the VAE (self._model.to("cpu")). The other three stages do not.

There is a switch for it. nar.py:

# src/yue2/nar.py
@contextmanager
def _offload_ar(model, enabled):
    """Temporarily move unused AR modules; this model cannot serve concurrently."""
    modules = [model.model.embed_tokens, model.lm_head]
    for layer in model.model.layers:
        modules.extend((layer.input_layernorm, layer.self_attn, layer.post_attention_layernorm, layer.mlp))
    moved = []
    try:
        if enabled:
            for module in modules:
                device = next(module.parameters()).device
                if device.type != "cpu":
                    module.to(device="cpu")
                    moved.append((module, device))

That list is exactly the AR expert plus the embedding pair — every tensor in the YuE2_AR file measured below, 4131.2 MiB in BF16 — idle for the entire flow-matching solve. offload_ar defaults to False in YuE2Pipeline.__init__, and --offload-ar exists in cli.py's argparse and appears in neither the README nor docs/generation.md.

To actually lower the floor, rather than the middle of it, you have to stop loading a tower you are not running. That is a loader change, not a flag — which brings me to the part I did not expect to find. WanGP already ships YuE2, and it made exactly that change.

WanGP's YuE2, measured against upstream's

models/TTS/yue2/yue2_handler.py hands mmgp three models:

# models/TTS/yue2/yue2_handler.py — load_model
if lm_decoder_engine in ("cg", "vllm"):
    pipeline.text_encoder._budget = 0
...
return pipeline, {"pipe": {"text_encoder": pipeline.text_encoder,
                           "transformer": pipeline.transformer,
                           "vae": pipeline.vae}}

The AR expert goes into mmgp's text_encoder slot, the acoustic expert into transformer, and the decoder into vae — three separate models to mmgp, each with its own residency and its own 100 MiB budget. To make that possible, DeepBeepMeep re-published the checkpoint as two files. They reconstruct the original exactly:

The 2,048 extra parameters are model.norm.weight, the one tensor both halves need, duplicated into both files — 4,096 bytes, plus 96 bytes of second safetensors header. A lossless split, checkable from the headers alone.

The decoder gets the same treatment. Upstream's m-a-p/YuE2-Vae is 530,512,720 bytes holding 132,616,130 FP32 parameters, of which 66,241,664 are an encoder that inference never touches — from_pretrained(..., decoder_only=True) skips loading it, and modeling_vae.py refuses to run the decoder in anything but FP32:

# src/yue2/modeling_vae.py — from_pretrained
requested_dtype = dtype if dtype is not None else torch_dtype
if requested_dtype not in (None, "auto", "float32", torch.float32):
    raise ValueError(...)
...
model.to(device=device, dtype=torch.float32).eval().requires_grad_(False)

WanGP's yue2/YuE2_VAE_bf16.safetensors is 132,731,320 bytes: the decoder only, in BF16. Four times smaller on disk, and half the resident bytes: 126.6 MiB against 253.2. Whether BF16 decoding changes the audio is a listening question I have not run, and neither project claims an answer.

And here is the tradeoff stated in code rather than prose, back in that handler: pipeline.text_encoder._budget = 0 when the decode engine is CUDA graphs or vLLM. A budget of zero means no block splitting — load the whole AR tower and keep it. CUDA graphs capture fixed device addresses, so block swapping underneath one is not an option. Fast decode costs you 1539.2 MiB of window turning into 2789.4 MiB of resident tower. Two lines, one honest knob.

What the int8 doesn't buy

Now the detail I would have gotten wrong from the model card. config.json says "tie_word_embeddings": false, and at vocab_size 184,704 by hidden_size 2048 that makes embed_tokens and lm_head two separate 378,273,792-parameter tensors — 1443.0 MiB in BF16, 20.8% of the whole model.

Quantizers like quanto and int8_convrot convert Linear layers. Embeddings and output heads are not Linear in the sense they operate on, and they stay BF16. Measured from the headers:

The layers halve exactly. The base does not move at all. So int8 on the acoustic tower saves 48.0% of its bytes and int8 on the AR tower saves 32.5% — because more than half of that file is an embedding pair the quantizer will not touch. If you budget for "int8 halves it", you will be 723.8 MiB short on the AR tower alone. This is the same effect as the Ovi bases above, just with much more at stake.

The 24 GB itself

I could not reproduce it, and I want to be careful about what that means.

From config.json and protocol.py: 28 layers, 8 KV heads, head_dim 128, BF16, so the KV cache costs 28 × 8 × 128 × 2 × 2 = 114,688 bytes per token. The context is 24,576 and the semantic stage caps at 9,000 tokens, so a full song's cache is about 0.96 GiB per branch, doubled under CFG. The NAR stage is chunked on purpose — chunk_ranges splits at (context - prefix_tokens - 3) // 2 frames — and at 48000/1920 = 25 latent frames per second, the 9,000-token cap is a six-minute ceiling on song length that fits in a single chunk. The VAE decodes in 1024-frame tiles with a 16-frame halo, and drops to 512 when memory_budget_gib <= 12.

Add the 6.76 GiB of weights to a couple of gigabytes of cache and activations and you are nowhere near 24. My reading is that 24 GB is a safe floor for the worst case — a six-minute song at the token cap, with CFG, with CUDA graphs preallocated — quoted as if it were the requirement for any song. That is a defensible engineering choice and an uninformative number, and the code contains the evidence that the authors know it varies: memory_budget_gib is a constructor parameter with a documented 12 GB branch. In the shipped code it does exactly two things — pick vae_core_frames, and cap a vLLM memory fraction in fast.py. It does not bound anything else, despite the name.

There is also an FP8 path, src/yue2/quantization.py, whose own docstring is the most honest sentence in either repository:

"""Opt-in experimental FP8 AR linear layers; NAR always restores exact BF16.
 
No quantized quality or speed claim is implied by enabling this module.
FP8 kernels require NVIDIA compute capability 8.9 or newer. Original weights
remain in CPU memory, outside the registered module tree, for exact restore.
"""

Note the last sentence: FP8 here keeps a full BF16 copy of every quantized tensor in system RAM, so it trades VRAM for RAM rather than saving memory overall. quantization_status() returns "quality_validation": "unvalidated", "performance_validation": "unvalidated" — the module says, in its own return value, that nobody has measured it.

The benchmark, and what "highest" is doing

expand

The README's claim: "YuE2 (best-of-8) achieves 6.9632 SongBench Avg, the highest observed mean among all evaluated settings." I sorted their own docs/benchmark-results.csv and it is exactly true — rank 1 of 17, at 6.963229315476184, over Mureka 9's 6.9377.

The margin is 0.0255 on a seven-point scale, which their own docs/benchmarks.md calls "descriptive, not claims of statistical significance". More to the point:

Best-of-8 selects by SongBench Musicality — a dimension of the SongBench average being reported. The docs say so plainly ("This selection uses a SongBench dimension and is not equivalent to one unselected pipeline call"), which is more than most releases manage. But note that the unqualified "YuE2" row is also a selection, best-of-2 on PER: there is no single-call number anywhere in that table, and docs/generation.md confirms that one pipe(**request) gives you one candidate. The thing you run locally is the row that sits fourth, behind Suno v5 by 0.1405.

One more line worth catching. Both YuE2 rows were scored with YuE2-Vae-legacy, and the default listening decoder shipped to you is YuE2-Vae. Their docs are direct about it: "Benchmark claims refer to the stated evaluation model and decoder; a new local generation is not itself a reproduction of the published aggregate."

What the demos can and cannot show

A muted, looping video cannot carry a song, so there is no clip here. What I can do is measure the published audio, which turns out to be more interesting.

assets/audio/examples.json in the model repo ships a SHA-256 for every sample. I downloaded three and all three matched — 87fec41a… for passion.mp3, and so on. That same manifest also answers a question the demo page doesn't: two of the six published samples are labelled "model": "YuE2 (earlier checkpoint)". Only passion.mp3 names m-a-p/YuE2-3B, the checkpoint you can download. Disclosed, in their own metadata, and easy to miss.

So I took passion.mp3 — the one made by the released weights — and looked at its spectrum.

expand

Averaging the magnitude spectrum over the whole track: content is at −49.7 dB at 16 kHz, −73.4 dB at 18 kHz, and −123.4 dB at 19 kHz. A fifty-decibel cliff inside one kilohertz is not music; it is a filter. The other two samples wall off at 18.09 and 18.14 kHz — the same edge across a rock track, a Mandarin nu-disco track and a jazz-funk cover, which is what an encoder does and not what a model does. ffprobe names it: 192 kbps, encoder tag Lavc61.19.

Nothing here is wrong. YuE2's pipeline writes audio.flac at 48 kHz, the VAE config confirms "sample_rate": 48000 with a downsampling_ratio of 1920, and the README's "48 kHz stereo without quantization" is a claim about that file. It is just that every demo you can listen to has had its top octave removed by the thing that packaged it, so the published artifacts cannot demonstrate the published claim. If you want to hear what the decoder actually does above 18 kHz, you have to generate it yourself.

One citation that doesn't resolve

The m-a-p/YuE2-3B model card carries the tag arxiv:2503.08638, so the Hub renders a paper link on the page. I pulled that ID from the arXiv API: it is "YuE: Scaling Open Foundation Models for Long-Form Music Generation", submitted 11 March 2025, last revised 15 September 2025. That is the YuE v1 paper — a two-stage autoregressive codec model, not an AR–NAR Mixture-of-Transformers with a symbolic planner and a flow-matching acoustic head. The repository itself keeps v1 on a separate branch and calls YuE2 a different system.

I searched and could not find a YuE2 paper. The benchmark protocol is documented carefully in docs/benchmarks.md and that is genuinely more useful than most papers, so this is a citation-hygiene issue rather than a missing-evidence one. But a reader who clicks that tag gets a paper about a different model, and the aggregate score in the README is not in it.

What I'd actually tell someone

If you are choosing hardware from these two numbers, invert both of them.

For WanGP, the VRAM figure tells you the size of a window, not the size of the model. What you should be sizing is system RAM — enough to hold the whole checkpoint, plus a pinned copy if you want profile 4 to be fast — and what you should be asking for is a wall-clock on a named card, which this project stopped publishing in April 2025. Then check vram_safety_coefficient and perc_reserved_mem_max before you conclude anything, because both silently change the plan.

For YuE2, the 24 GB is a conservative single number covering a workload whose peak scales with song length and whose stages differ by 4 GiB. Everything you need to run it smaller is already in the repository — offload_ar, memory_budget_gib, split-stage loading — and the README mentions none of it. WanGP's port is the existence proof: 1539.2 MiB of resident weights at peak, from the same checkpoint, without touching the sampler.

And for both: the next time a generative-media release quotes a VRAM figure, ask what the residency policy is, what the system-RAM figure is, and how long it takes on which card. The first number without the other three is a statement about someone's offload.py, not about a model.