# MiniMax Music 3: a five-minute song is 9,000 steps of a 2.5 kbit/s code
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/minimax-music3
> date: 2026-08-14
> tags: audio, music-generation, open-weights, flow-matching, rvq, explainer
[MiniMax Music 3](https://huggingface.co/MiniMaxAI/MiniMax-Music3) generates complete songs up to five minutes long from lyrics plus a music description, at 32 kHz 16-bit stereo. The weights went up on 2026-08-07 and the card was still being edited on 2026-08-13.
What makes it worth reading closely is that the card is unusually specific — it names a parameter count for four separate components — and all four of those numbers can be checked against the published files. They hold up, which is rarer than it should be.
## The shape of the thing
The split is between **structure** and **texture**:
- The **Global LLM (8B)** predicts the first RVQ codebook, frame by frame. That codebook is the semantic one — 16,384 entries — and it carries the song's long-range progression: where the chorus is, whether the vocal identity holds, how the arrangement evolves.
- The **Local LLM (646M)** predicts the remaining seven acoustic codebooks *within* each frame, restoring fine-grained detail the semantic codebook throws away.
The part that is not standard is what happens next. Rather than decoding from the discrete RVQ tokens, the synthesis stage fuses the **final hidden states** of both models and flows from there. The tokens are what the models predict; the hidden states are what actually gets rendered. MiniMax's argument is that continuous representations preserve more than the quantized codes do — vocal articulation, instrumental texture, temporal continuity — and the card is explicit that at inference time "waveform synthesis uses the fused LLM hidden states and does not require the discrete tokenizer decoder."
The tokenizer, in other words, is a training-time device. It shapes what the LLMs learn to predict and is then bypassed.
## Every number on the card, checked
I pulled the safetensors headers by HTTP range request — the 8-byte length prefix, then the JSON header that gives every tensor's dtype and shape — rather than dividing file sizes and hoping. That mattered twice.
The **Flow Matching** module is 9.73 GB across two shards. At bf16 that would be 4.86B parameters and the card's "2.4B" would be wrong by a factor of two. Every tensor in it is **F32**, so it is 2,431.9M, and the card is right.
The **Global LLM** is the nicer result. The config is `Qwen3ForCausalLM` with Qwen3-8B's exact shape — 36 layers, hidden 4096, intermediate 12288, 32 query heads over 8 KV heads — but `vocab_size` is **200,000** rather than Qwen3-8B's 151,936. Qwen3-8B is 8.191B parameters. Widening the vocabulary adds `(200,000 − 151,936) × 4096 × 2 = 393.7M` for an untied embedding and output head. That predicts 8.584B, and the index reports 8.584B.
So the card's two claims about this model — "initialized from Qwen3-8B" and "its embedding and output layers are first adapted to semantic music tokens" — are both visible in a single number, and the extra 48,064 vocabulary slots are where 16,384 semantic music tokens went.
The **Flow-VAE decoder** matches exactly too: `dav.pth` is 491.8 MB, which at fp32 is 123.0M parameters against a stated 123M.
The one component the card never mentions is a 25.2M condition encoder that takes 24 kHz audio in and produces conditioning at 44.1 kHz — the piece that would let you condition on a reference track rather than only on text.
## Why the repository is 57 GB
The parameters add up to roughly 12B. The repository is 57.35 GB. The difference is that it ships the entire model twice, in two runtime layouts — the SGLang-Omni one the card recommends, and a diffusers modular pipeline.
The arithmetic that shows these are the same weights rather than two models is clean: `flowmatching_vae.pth` is 2,457.1M parameters at fp32, and the diffusers `transformer` plus `condition_encoder` are 2,431.9M + 25.2M. Same total, not an approximation.
The oddity is the folder named `qwen_7B`. It holds an **`AbabForCausalLM`** — Abab being MiniMax's own model family — at the identical 8.58B shape as the `Qwen3ForCausalLM` sitting beside it. A directory named after one model family, containing another, holding what appears to be the same model converted for a different runtime. It is 18.48 GB of the repository and nothing in `modular_model_index.json` refers to it.
## The frame budget
The card's Limitations section gives two ceilings and does not connect them: songs "up to five minutes," and "audio generation is limited to 9,000 acoustic frames." Those only agree at **30 frames per second**, which is a number the card never states.
It is worth deriving, because it fixes the scale of everything else. Eight codebooks per frame — one at 14 bits, seven at 10 — is 84 bits per frame, or **2.52 kbit/s**. That is the representation the Global LLM is autoregressing over, and a full-length song is 9,000 steps of it against a 32 kHz stereo output that would be 1,024 kbit/s as raw PCM. Roughly 406× compression, with the flow-matching stage responsible for putting back everything that ratio removed.
The text side is separate and much tighter: 5,000 tokens total for lyrics and description combined.
## Control, and what it does not promise
Input is two fields. **Lyrics** may carry explicit section tags — `[Intro]`, `[Verse]`, `[Pre-Chorus]`, `[Chorus]`, `[Post-Chorus]`, `[Bridge]`, `[Instrumental]`, `[Solo]`, `[Outro]`. **Music description** covers style, emotional progression, vocal performance, instrumentation, arrangement, and production.
MiniMax recommends a three-part Structured Caption — Global Metadata (genre, BPM, key, scale, emotional arc, production profile), Vocal Details (gender, timbre, performance style, harmony, backing vocals, effects), and Arrangement (primary and secondary instruments, section-level instrument evolution, groove, bass, percussion, textures, spatial effects). There is a `music-caption-rewriter` skill for expanding a short prompt into one, installable with `npx skills add`.
The card is honest about what that buys:
> Section tags and music descriptions provide generative control rather than strict symbolic guarantees. The generated tempo, key, instrumentation, lyrics, and song structure may not always match every requested detail exactly.
Which is the right way to describe a model that has no symbolic music representation anywhere in it. You are conditioning a sampler, not programming a sequencer.
## Running it
CUDA only, and non-streaming only — you wait for the whole song. Full precision fits under 24 GB of VRAM; with automatic CPU offloading it needs about 22 GB; and streaming the language model layer by layer with `apply_group_offloading` gets it onto an 8 GB card, slowly.
That last path is the interesting one for anyone without a datacenter, and it is a consequence of the hierarchy: the 8B Global LLM is the only piece that has to be resident for the long autoregressive run, so streaming its layers costs bandwidth rather than correctness.
## What I'd flag
The engineering claims check out, which is the main thing I set out to test. What the card contains no evidence for is **quality** — there are no listening-test results, no comparison against Suno or Udio or any other music model, and no objective audio metrics. There is a demo page and a single `assets/minimax_ttm.wav`. For a generative audio model, that is the entire evaluation.
The license file is present in the repository but the HF API reports no license field, so it is worth reading `LICENSE` directly before assuming anything about commercial use.
And 25 downloads against 440 likes, a week after release, is the signature of a model far more people want to hear about than can actually run.