# Qwen Audio 3.0 TTS: an instructable LM-plus-flow-matching speech stack

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/qwen-audio-3-tts
> date: 2026-07-20
> tags: audio, tts, speech, generative, explainer
Modern text-to-speech has mostly converged on a shape: a language model predicts discrete speech
tokens from text, and a generative decoder turns those tokens back into a waveform.
[Qwen-Audio-3.0-TTS](https://funaudiollm.github.io/qwen-audio-3.0-tts/) — Alibaba's latest, in the
CosyVoice lineage — runs that shape hard and adds the things that make a TTS model actually usable:
instruction control, inline non-verbal events, 16 languages and 20 Chinese dialect regions,
one-pass long-form synthesis, and 48 kHz output. It currently sits at **#1 on the Artificial
Analysis Text-to-Speech leaderboard**. Here's how it's built.

## The stack

Two models do the heavy lifting. An **autoregressive LM** predicts a sequence of discrete speech
tokens from the text (and, for zero-shot cloning, a reference clip). A **flow-matching decoder**
turns those tokens into a mel-spectrogram. A **vocoder** reconstructs and super-resolves the
waveform to 48 kHz. Fronting all of it is a **12.5 Hz supervised speech tokenizer** — the piece
that quietly sets the model's latency. Click through the stages:

<TtsStack />

Splitting content (the LM) from voice-and-prosody (the flow-matching decoder) is what makes the
model *instructable*: you can change *what* is said and *how* it's said through different parts of
the system. It's also why the model is robust to a bad reference — a noisy or reverberant prompt
still conditions the LM, and there's no explicit denoising step to break.

## Why 12.5 Hz

The tokenizer's frame rate is the single most consequential number in an autoregressive TTS system,
because the LM emits one token per step and the steps are serial. Fewer tokens per second of audio
means fewer decode steps means less latency. Most neural speech codecs sit at 25–75 Hz;
Qwen's supervised tokenizer runs at **12.5 Hz**. Drag it and watch the step count move:

<FrameRate />

The word doing the work is **supervised**. A raw reconstruction codec at 12.5 Hz would throw away
too much to sound good; a *supervised* tokenizer is trained to keep exactly the content and speaker
information the LM needs, so it can afford the low frame rate. Short token stream, fast decode,
intact voice — that's the trade the tokenizer is engineered to win.

## Say what, and how

Controllability in most TTS models means "pick a preset voice." Qwen splits it into three
independent knobs. A **free-style natural-language instruction** sets role, emotion, speaking style,
rate, timbre, and accent for the whole utterance. **86 fine-grained inline tags** drop non-verbal
events — laughter, breathing, coughing, sighing — at the word level, inside the text. And the text
itself is the content. Switch the instruction and the delivery re-colors without a word changing:

<InlineControl />

On top of that: **16 languages and 20 Chinese dialect regions** (seven languages new this version),
**one-pass long-form synthesis up to three minutes**, a reproducible speaker fine-tuning protocol,
and vocoder **super-resolution to 48 kHz**. It also handles hard text-normalization cases and
degraded reference speech without a separate cleanup stage.

## The receipts

The project page leads with two radar charts across the **CV3-Eval** multilingual set — one for
content consistency (word-error rate) and one for speaker similarity — against MiniMax-Speech,
ElevenLabs v3, VoxCPM2, DotsTTS, and the Qwen3-TTS base. The shape tells the story: Qwen-Audio-3.0-
TTS holds a large, even envelope across all ~20 language axes, where the lighter baselines collapse
on the long-tail languages.

<Figure
  src="/articles/qwen-audio-3-tts/fig1.png"
  alt="Radar chart of content-consistency word-error-rate across roughly twenty CV3-Eval languages, comparing MiniMax-Speech, ElevenLabs v3, DotsTTS, VoxCPM2, the Qwen3-TTS base, and Qwen-Audio-3.0-TTS, with Qwen-Audio-3.0-TTS forming a large even envelope."
  caption="Content consistency (WER, lower is better — outer ring is better) across CV3-Eval languages; Qwen-Audio-3.0-TTS stays strong on the long-tail languages where lighter models fall in (project page)."
/>

<Figure
  src="/articles/qwen-audio-3-tts/fig2.png"
  alt="Radar chart of speaker similarity across roughly twenty CV3-Eval languages for the same set of models, with Qwen-Audio-3.0-TTS maintaining high similarity across the board."
  caption="Speaker similarity across the same languages — how faithfully a zero-shot clone matches the reference voice (project page)."
/>

The paper reports state-of-the-art results across SEED-TTS-Eval, CV3-Eval, instruction-following,
long-form, and acoustic-robustness suites; the leaderboard #1 is the headline. (Exact WER/SIM
figures live in those radar charts rather than a table on the page — the shape is the claim.)

## The training, briefly

The two-model split has a matching two-track training recipe — **five progressive stages**: the LM
and flow-matching decoder are **pretrained independently**, then **jointly trained** with a
high-quality-data annealing phase, then the LM gets a **reinforcement-learning** pass, and the
decoder gets its own **robustness** stage and then its own **RL** stage. The robustness stage is
what lets the flow-matching decoder cope with degraded prompts; the separate RL passes are what
sharpen intelligibility and speaker fidelity without the two objectives fighting.

## The take

Qwen-Audio-3.0-TTS isn't a new paradigm — it's the LM-plus-flow-matching recipe executed with taste.
The 12.5 Hz supervised tokenizer keeps it fast, the content/voice split keeps it controllable, the
inline tags and instructions make it expressive, and the multilingual coverage is broad and even
rather than English-plus-a-long-tail. The interesting lesson is how much of "good TTS" is now about
the surfaces you expose — frame rate, instruction grammar, tag vocabulary — rather than the core
generative trick, which the field has largely settled.

---

*Source: the [Qwen-Audio-3.0-TTS project page](https://funaudiollm.github.io/qwen-audio-3.0-tts/)
(Alibaba / FunAudioLLM). The radar figures are theirs; the interactive diagrams are mine.*