FLASepformer and JAEC: the parts Alibaba shipped, and the parts it didn't

Point two microphones at a restaurant table and every speech model downstream — ASR, diarization, a call transcript — inherits the same problem: the mixture has more than one voice in it. Separating that mixture back into individual speakers is the "cocktail party problem," and it's the oldest hard problem in the field, dating to Cherry's 1953 experiments on selective hearing. This week Alibaba's Tongyi Lab put two new speech front ends on ModelScope under the iic namespace: FLASepformer, a linear-attention speech-separation model, and JAEC, a neural echo canceller. Both ship with real weights, real READMEs, and real licenses. Neither ships quite what the announcement I was handed implies.

The separation model comes with a paper — FLASepformer: Efficient Speech Separation with Gated Focused Linear Attention Transformer (Wang, Jiang, Qiao, Shi, Tian; Interspeech 2025) — and the paper's own tables are where this piece starts, because the ModelScope card volunteers something most model cards don't: it tells you, in its own words, that the checkpoint you can download is not the checkpoint its most quotable number describes.

The headline number, and the card that contradicts it

The line I was handed to check reads: "FLA-TFLocoformer-L reaches 24.9 dB SDRi on WSJ0-2Mix." First, is that number real? Table 1 of the paper, WSJ0-2Mix, FLA-TFLocoformer-L, trained with dynamic mixing (DM):

SI-SNRi 24.8 dB, SDRi 24.9 dB.

It's real — quoted correctly, to one decimal place, from a real table. Now open the separation model's card:

本模型卡发布的是 FLA-SepReformer-B：模型采用纯时域建模，在 Libri2Mix-100（train-100）上训练，用于 8 kHz 单声道双说话人语音分离。

"This model card releases FLA-SepReformer-B: the model uses pure time-domain modeling, trained on Libri2Mix-100 (train-100), for 8 kHz mono two-speaker speech separation."

That's a different backbone (SepReformer, not TFLocoformer), a different scale (B, 14.2M params, not L, 22.6M), and a different dataset (Libri2Mix-100, not WSJ0-2Mix). The card goes further than most and states its own released number directly, in its evaluation section: FLA-SepReformer-B reaches 20.3 dB SI-SNRi and 20.7 dB SDRi on Libri2Mix-100 — and adds, unprompted, that "other results in the paper illustrate FLASepformer's performance and efficiency across scales; they do not mean this release includes other datasets or scale checkpoints." That sentence is doing the announcement's job for it, and the announcement didn't quote it.

Here's the paper's own headline figure — the same comparison the announcement's number is drawn from, with every variant plotted against inference time and a checkmark on every point trained with DM:

expand

Every point on that plot is a different combination of backbone, scale, and training recipe. The 24.9 dB point and the 20.7 dB point aren't two measurements of the same thing — they're two different dots. The paper gives enough of those dots, though, to walk from one to the other and see exactly which choice buys how much:

Three-tenths of the total 4.2 dB gap comes from swapping backbones at matched size — TFLocoformer for SepReformer, 15.1M params for 14.2M — and it's actually negative: 23.5 dB against 23.7 dB. Scaling up 50% (M to L) buys 0.8 dB. Adding dynamic-mixing augmentation buys another 0.6 dB. The other 3.0 dB — most of the total gap — comes from a single substitution that has nothing to do with the model at all: evaluating on WSJ0-2Mix, a clean, curated read-speech benchmark, instead of Libri2Mix-100, the noisier LibriSpeech-derived mixture set the released checkpoint was actually trained and tested on. Publishing one checkpoint out of a paper that reports several is ordinary practice — every efficient-attention paper does it. What the announcement got wrong wasn't the number. It was pairing that number with a checkpoint that doesn't produce it.

That FLA-TFLocoformer-M row is worth a flag of its own: it's a second real checkpoint, iic/speech_flatflocoformer_separation_timefrequency_8k_middle_libri2mix360, referenced right in the separation README's own ONNX export code, and its numbers match the card's evaluation section exactly (22.2 / 22.4 dB, Libri2Mix-360). It's the same pattern one size down: middle, not large; Libri2Mix, not WSJ0-2Mix.

What FLASepformer actually changes

FLASepformer isn't a new speech-separation architecture. It's a new attention module dropped into two existing ones. SepReformer (Shin et al., NeurIPS 2024) and TF-Locoformer (Saijo et al., IWAENC 2024) are both real, published baselines; FLASepformer's whole contribution is replacing the global attention inside each of them with Gated Focused Linear Attention, leaving everything else — the encoder/decoder, the local transformer, the speaker-split stage, TF-Locoformer's frequency modeling and Conv-SwiGLU blocks — untouched:

expand

In SepReformer, Gated FLA replaces a downsampled multi-head self-attention that still runs at $O(N^2)$ over a reduced length $N/2^R$. In TF-Locoformer, it replaces the plain softmax self-attention inside "Temporal Modeling," leaving "Frequency Modeling" alone. Both swaps are local — one module, in an otherwise unmodified network — which is exactly why the released config file for FLA-SepReformer-B reads the way it does. This is configuration.json, trimmed to the relevant block:

"module_separator": {
  "relative_positional_encoding": {
    "in_channels": 128,
    "num_heads": 8,
    "maxlen": 2000
  },
  "enc_stage": {
    "addition_conf": {
      "ega_addition_conf": {
        "gated": true,
        "linear_attn_type": "FocusedLinearAttention",
        "kernel_size": 7
      }
    },
    "global_blocks": {
      "in_channels": 128,
      "num_mha_heads": 8
    }
  }
}

Every number here is one the paper's prose also states, independently: "gated": true and "linear_attn_type": "FocusedLinearAttention" are the Gated FLA module; "kernel_size": 7 is the depthwise convolution's kernel, matching the paper's ablation default ($k=7$); "maxlen": 2000 is the $N=2000$ per second of audio the paper quotes as SepReformer-B's global-attention sequence length; and 128 channels split across 8 heads gives $d = 128 / 8 = 16$ per head — exactly the $d=16$ the paper cites in the same sentence as $N=2000$. Paper and shipped config agree to the last integer.

Focused Linear Attention, from the matrix multiply up

Standard attention for one query is a weighted sum of values, weighted by similarity:

Computing this the obvious way means building $QK^\top$ — an $N \times N$ matrix — before you can touch $V$. That matrix is the entire cost problem: $O(N^2 d)$ multiply-adds to build it and run the softmax-weighted sum, and $O(N^2)$ floats resident in memory while you do it. Linear attention's move is one associativity swap. Replace the softmax kernel with a factorizable one, $\mathrm{Sim}(Q,K) = \phi(Q)\phi(K)^\top$, and matrix multiplication is associative:

$(QK)V$ becomes $Q(KV)$. The parenthesized term is a fixed $d\times d$ matrix — it never depends on $N$ — so the whole thing costs $O(Nd^2)$ and never materializes an $N \times N$ anything. That's the entire $O(N^2) \to O(N)$ story: not a smarter approximation of softmax, a different order of operations on the same three matrices.

It isn't free. Plain (vanilla) linear attention swaps exactness for a real accuracy cost, in two specific, measurable ways, and Focused Linear Attention — introduced for vision transformers by Han et al., FLatten Transformer, ICCV 2023 and imported into this paper's speech setting — exists entirely to buy most of that cost back:

Focus. A simple kernel like $\phi = \mathrm{ReLU}$ produces attention weights that are too smooth — every value contributes roughly its average share, rather than a few relevant ones dominating. FLA fixes this with a focused function,

— raise each feature to power $p$ elementwise, then rescale to preserve the original norm. Raising to a power exaggerates whichever components were already largest, pulling each vector toward its nearest coordinate axis; the norm-preserving rescale stops that from blowing up the magnitude. The net effect, demonstrated on a real DeiT attention row in the FLatten paper, is a distribution moving from $[0.37, 0.19, 0.26, 0.18]$ — nearly flat — to $[0.75, 0.11, 0.09, 0.05]$ after $\phi_3$: sharp, softmax-like, from the same inputs. This paper sets $p=3$ for speech and confirms it by ablation (Table 4): $p=2$ or $p=4$ each cost a few tenths of a dB, $p=8$ or $p=16$ cost more.

Rank. Separately from focus, linear attention's weight matrix has a real ceiling: $\mathrm{rank}(\phi(Q)\phi(K)^\top) \le \min(N, d)$. Softmax attention has no such ceiling — its nonlinearity lets the effective rank run up to $N$. When $d \ll N$, as it always is here ($d=16$, $N=2000$ per second), that ceiling bites: many rows of the linear-attention matrix end up nearly identical, and the model's outputs at different positions start resembling each other regardless of what's actually in the audio. FLA's fix is a parallel depthwise convolution (DWC) added directly to the output: $O = \phi(Q)\phi(K)^\top V + \mathrm{DWC}(V)$. Because DWC contributes its own, independent local structure, the equivalent attention matrix $M_{eq} = \phi(Q)\phi(K)^\top + M_{\mathrm{DWC}}$ can exceed the low-rank ceiling of the first term alone — a full-rank correction bolted onto a low-rank base. The original FLatten Transformer used a 2D convolution, sized for image patches; this paper's one domain-specific change is swapping it for a 1D convolution over time, the natural axis for speech, with kernel size $k=7$.

Both of those are the FLatten Transformer's contributions. This paper's own addition sits on top of both: a Gated MLP — a norm, a linear layer, and an activation — computed from the block's own input and multiplied elementwise into the combined FLA+DWC output, letting the model down-weight the global-attention branch per token rather than trusting it unconditionally everywhere. Table 4's ablation prices this precisely: removing the gate costs SI-SNRi 23.5 → 23.4 dB and SDRi 23.7 → 23.5 dB — the smallest of the ablations, but the direction is consistent, and it's the one piece of this module that has no precedent in the FLatten paper.

Here's what those exact dimensions — $d=16$, $N=2000$ per second — mean in practice, for one attention head processing real audio at real durations:

The speed and memory numbers, and why they move opposite ways

The efficiency claim in the announcement: FLA-SepReformer-T/B/L run 2.29×, 1.91×, and 1.49× faster than their SepReformer counterparts, using 15.8%, 20.9%, and 31.9% of the original GPU memory. The measurement conditions, stated directly in the paper (§3.2, §4.2, Figure 1's own caption): 30 seconds of mixed audio, on a GeForce RTX A800, single-threaded, inference only — not training — averaged over 100 iterations per audio length. This matters because linear attention's advantage is a function of sequence length; a 2.29× at 30 seconds says nothing directly about 5 seconds or 5 minutes.

expand

The paper reports one concrete failure point on that curve: SepReformer-L runs out of memory at 57 seconds on an 80GB GPU. FLA-SepReformer-L, tracing the same x-axis, does not.

The part worth explaining is the pattern within the FLA family: the speedup shrinks from T to B to L (2.29× → 1.91× → 1.49×) while the memory fraction grows in the same order (15.8% → 20.9% → 31.9%). Both are the same mechanism, seen from two sides. Gated FLA only replaces the global-attention module — the one piece of the network whose original cost was $O(N^2)$. Everything else in the block (the down/up convolutions, the local transformer, DWC, the Gated MLP's own linear layers) costs the same whether the attention inside it is quadratic or linear, and that unchanged cost scales with the model's channel width $d$, not with the audio's length $N$. At 30 seconds, $N=60{,}000$ is enormous and fixed by the audio, while $d$ only grows modestly from T to L. So at small $d$, the $O(N^2)$ term the attention module used to pay is the dominant cost in the whole block — replacing it buys almost all of the block's runtime and memory back. As $d$ grows, the untouched, $d$-scaling rest of the network becomes a bigger share of the total, and there's proportionally less left for the attention swap to save — which is exactly a shrinking speedup and a growing "fraction of the original" memory footprint, moving together for the same reason. (The paper doesn't publish per-scale channel widths for T and L, so this is the mechanism the pattern implies, not a re-derivation of the exact 2.29×/1.49× figures from first principles.)

Worth noting in passing: TF-Locoformer without Memory-Efficient Attention has quadratic memory growth by the paper's own account, same as plain softmax attention; only the FLA-TFLocoformer variants and the ME-attention TF-Locoformer variants avoid it, and Figure 3(c) plots both "with ME" and "without ME" lines to show the difference directly.

Checking the released weights against the paper's own parameter count

Two files ship on the model repo: onnx_model.onnx at 57,949,408 bytes and pytorch_model.pt at 59,450,500 bytes. The paper states FLA-SepReformer-B has 14.2M parameters. A quick check: at 4 bytes per fp32 parameter,

Both come out a few percent above 14.2M — 2.0% for the ONNX file, 4.7% for the PyTorch checkpoint — which is the right direction and the right size for the discrepancy: an ONNX file carries graph structure (node names, shapes, operator metadata) alongside the raw tensors, and a .pt file is a zip container wrapping pickled tensor storages, both of which inflate the byte count without adding parameters. The paper's stated size holds up.

JAEC: reading what "returns the LP output" actually means

JAEC's card is unusually precise about its own scope, in a sentence that rewards reading closely:

"This release contains the neural time-delay estimation (TDE) and adaptive linear processing (LP) runtime. It accepts microphone and far-end reference signals and returns the LP output."

A full acoustic echo canceller for real-time calls is normally three stages. Echo doesn't arrive instantly — it leaves through the far-end speaker, bounces around a room, and returns on a delay that drifts as people and objects move — so stage one, time-delay estimation, finds and tracks that lag against the reference signal. Stage two, an adaptive linear filter, builds a model of the room's echo path from the aligned reference and subtracts a predicted echo. Stage three, a neural residual-echo suppressor (often labeled NLP, for nonlinear processor), cleans up whatever the linear model of the room got wrong — nonlinearities from a clipping speaker, double-talk, anything the linear filter can't represent. JAEC's own limitations section confirms which of the three shipped: "The output is the JAEC TDE+LP frontend result; this package does not include the nonlinear processing (NLP) network." Stage three isn't in the box. What ships is a neural delay estimator feeding a classical adaptive filter — closer to a well-instrumented linear AEC than to a full modern neural echo canceller.

expand

The card's own diagram corroborates the text: two boxes, TDE and LP, each internally a generic "Neural Network" module, ending in a subtraction that produces e(n). The file manifest agrees a third time — the entire model is one 105,216-byte weights file, weights/tde_lp.bin, sized for a delay estimator and a linear filter, not for a separate residual-suppression network on top.

The runtime constants an integrator actually needs, straight from the card: the pipeline processes 160 samples (10 ms) per call, and its output carries a fixed 352-sample (22 ms) algorithmic delay relative to the microphone input, regardless of what delay TDE estimates internally. Real-time-factor numbers on three platforms, median of 10 runs, CPU only:

An RTF of 0.0039 means one second of audio costs under 4 milliseconds of CPU time — comfortably real-time on a server core with no GPU involved, consistent with a 10 ms-per-frame classical filter rather than a heavy neural stack.

from modelscope.outputs import OutputKeys
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
 
aec = pipeline(
    Tasks.acoustic_echo_cancellation,
    model="iic/speech_jaec_aec_16k",
    device="cpu",
    trust_native_code=True,
)
result = aec(
    {"nearend_mic": "nearend_mic.wav", "farend_speech": "farend_speech.wav"},
    output_path="output.wav",
)
pcm16_bytes = result[OutputKeys.OUTPUT_PCM]

That trust_native_code=True flag is not decoration. The repository ships prebuilt native libraries — lib/jaec_arm.so, lib/jaec_x86.so, lib/jaec_x86.dll — and the pipeline runs them directly. Apache-2.0 covers redistribution of those binaries, but there's no source for the TDE/LP runtime itself in the repo, only compiled libraries plus the 105 KB weights file. The license permits commercial use of a black box; it doesn't make the box auditable.

Licenses, and why they aren't a footnote here

The two releases carry different licenses, and for a change this actually changes what you can do with them. FLASepformer's card states it plainly: CC BY-NC 4.0, and the limitations section is unambiguous — "本模型仅限非商业学术研究用途，严禁商用" ("this model is limited to non-commercial academic research use; commercial use is strictly prohibited"). The same card's own list of potential applications reads: separating overlapping speech in meetings, interviews, podcasts, and video for cleaner editable tracks; preprocessing for ASR; analyzing customer-service and multi-party calls; front-end support for speaker diarization and captioning. Every one of those, run for money, is exactly what the license forbids. JAEC ships under Apache License 2.0 — no such restriction, commercial use included, modulo the native-binary caveat above.

None of this is unusual by itself — plenty of research checkpoints ship non-commercial while an accompanying production component ships permissively. What's worth flagging is that a reader skimming an announcement that bundles "two new speech front ends from Alibaba" together would reasonably assume similar terms for both. They aren't similar. One of these two releases can go in a shipped product tomorrow. The other, on its own license, cannot.

The ledger

What's confirmed. The 24.9 dB SDRi number is real, and it's the paper's, not a fabrication — but it names FLA-TFLocoformer-L, WSJ0-2Mix, trained with dynamic mixing, and none of that is the checkpoint ModelScope hosts. The released FLA-SepReformer-B genuinely gets 20.7 dB SDRi on Libri2Mix-100, and its file sizes are consistent with the paper's stated 14.2M parameters to within ONNX/pickle overhead. The $O(N^2) \to O(N)$ mechanism is exactly what the paper says it is, and the released configuration.json names the same focused-attention module, gate, kernel size, and sequence length the paper's prose does, integer for integer. The 2.29×/1.91×/1.49× speedups are real, measured at 30 seconds of audio on an RTX A800, at inference — and they shrink as the model scales for a mechanistic reason grounded in what FLA does and doesn't touch. JAEC genuinely ships only a time-delay estimator and a linear filter, confirmed by its own text, its own diagram, and a 105 KB weights file too small to hide a third network in.

What isn't. No GitHub reference implementation exists for FLASepformer as of this writing — the paper states "we plan to release the source code at a later time," and a year on from the Interspeech acceptance, that plan hasn't produced a public repo I could find. The paper doesn't publish per-scale channel widths for FLA-SepReformer-T and -L, so the shrinking-speedup mechanism above is the right shape, not an exact reproduction of the measured ratios. And the license gap is the sharpest open question of the two releases: a separation model whose own card lists commercial applications it's licensed not to be used for.