Dessa: timing a speech-only conversation model from its own demo tracks

On 23 October Jack Dorsey posted that Block had built "Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence." In the replies he added that it is "small enough to run on your macbook", answered "yes" to whether it runs locally, and, asked which language model powers the voice, wrote "speech to speech. no llm".

That last line is the interesting claim. Every open full-duplex model I have looked at on this site starts from a text model. Block says Dessa does not: it was trained "from scratch on nothing but human conversation. Sounds! No text, no pretrained models."

So I went looking for something to run. There is nothing. The Dessa page has a "Download" button that is a placeholder, a line that says "Launch links are coming soon", a waitlist, and a form for "a small group of AI safety researchers" to request early access. There is no paper, no model card, no parameter count and no licence. A Hugging Face search for dessa and for models under block and squareup returned nothing relevant, and a GitHub search for dessa in the block organisation returned zero repositories (measured, 24 October).

What the page does publish is 21 conversation recordings, each with the two speakers on separate tracks. That turns out to be enough to measure the part of "authentic" that is about timing. Before the numbers, the mechanism.

What full-duplex means

A voice assistant built as a cascade takes turns. A voice activity detector waits for you to stop, speech recognition writes down what you said, a text model answers, and a TTS engine reads the answer out. Hugging Face's speech-to-speech pipeline is a clean example, and its turn policy is literally a set of millisecond thresholds. Two things are lost. Everything the transcript cannot hold (a laugh, a sigh, sarcasm, how loud you were) is gone before the model sees it. And the model cannot talk while you talk, so there are no backchannels ("yeah", "mm"), no finishing your sentence, no getting interrupted gracefully.

A full-duplex model drops the turns. It runs on a clock. At every tick it reads one frame of your audio and writes one frame of its own, whether or not anyone is speaking. Kyutai's Moshi is the reference design, and I took it apart frame by frame in the PersonaPlex article: a codec turns 80 ms of audio into a column of tokens, the model's own stream and the user's stream sit side by side in one sequence, and a transformer predicts the model's next column from everything so far.

In that design there is no turn-taking component. "Listening" is the model predicting that its own next frame is silence. A backchannel is a short burst on its own stream while your stream is busy, which is a likely continuation because people do it constantly in recorded phone calls. Being interrupted is your stream lighting up while its own is also active, and the likely continuation is to trail off. All of it is next-token prediction over two streams. That is also why the timing is the thing to measure: it is not tuned by a threshold, it is whatever the training data taught.

Moshi still leans on text in two places. Its backbone is Helium, a 7B text language model, and before each frame of audio it predicts a time-aligned text token of what it is about to say, which Kyutai calls the Inner Monologue. The Moshi paper says the Inner Monologue "significantly improves the linguistic quality of generated speech" (reported). Its audio pretraining was 7 million hours, transcribed with Whisper, and the full-duplex behaviour came from fine-tuning on Fisher, 2,000 hours of two-channel telephone conversation (reported, Moshi paper section 4.2). Moshi's latency is "160ms, 200ms in practice" (reported, Moshi abstract).

The text-free precedent

Training dialogue on audio alone has been tried. Meta's dGSLM (2022) is "the first 'textless' model able to generate audio samples of naturalistic spoken dialogues", a dual-tower transformer with cross-attention trained on the same 2,000 hours of Fisher, "without any text or labels" (reported, dGSLM abstract). It produced laughter and natural turn-taking. It did not produce sense: the Moshi paper's own comparison calls it "not able to generate coherent speech" (reported). dGSLM also generated offline rather than live, and its units came from a separately trained self-supervised speech encoder, which "no pretrained models" would rule out for Dessa.

So the precedent says the timing part is learnable from audio alone, and the content part is hard. Block says as much: "For a machine with no text to lean on, it's brutal." What would make it work is mostly scale, and that is where Block is least specific.

What Block discloses

Everything below is from the Dessa page or Dorsey's posts, and all of it is reported, not checked.

• Input and output. "Audio in, audio out. Dessa is a true speech-to-speech model." Full-duplex: "it listens and talks at the same time".
• Training data. "Nothing but human conversation", "no text, no pretrained models". Language was learned "from hundreds of times fewer words" than LLMs need.
• Training compute. "We figured out how to train Dessa on only one NVIDIA GB200 NVL72." That is one rack of 72 Blackwell GPUs. No duration is given.
• Inference. "Inference runs faster than real time", and "it runs locally on a MacBook Pro M4 Max or better".
• Behaviour. It knows "when to cut in and when to backchannel", and hears and speaks "emotion, sarcasm, laughter".
• Team. A small lab at Block, "founded by former members of Dessa, the Toronto deep learning startup", which is where the name comes from. Six researchers and five current and former interns are credited.

What is missing is everything I would need to place it: parameter count, codec and frame rate, hours of audio and where they came from, how the two speakers were separated in training data, latency in milliseconds, and any evaluation. "Most authentic" has no metric behind it on the page. There is no human preference study, no baseline, no benchmark.

Two claims I can check with arithmetic

"A single spoken word takes thousands of times more bytes than the same word in text." At a conversational 150 words a minute a word lasts 0.4 s. As 16-bit mono at 16 kHz that is 12,800 bytes, against about 6 bytes of text, roughly 2,000 times (reasoned). So it holds for raw waveform. But no speech model trains on raw waveform bytes; it trains on codec tokens. At Mimi's 1.1 kbps, the codec Moshi uses, the same 0.4 s is 55 bytes, about 9 times the text (reasoned). The real gap is not bytes, it is that the codec tokens also carry voice, room and prosody that the model has to learn before it gets to words.

"Hundreds of times fewer words." Block does not name the LLM it compares against. Taking Llama 3's reported "over 15T tokens" of pretraining as a stand-in, at about 0.75 words per token that is roughly 11 trillion words. Two hundred to nine hundred times fewer is 12 to 56 billion words. At 9,000 words an hour of speech, that is roughly 1.4 to 6.3 million hours of conversation (reasoned, and only as good as the assumed comparison and speaking rate). That is the same order as Moshi's 7 million hours, and three orders of magnitude more than all of Fisher. If the bracket is right, the hard problem Block solved is not the architecture. It is getting millions of hours of two-sided conversation with each speaker isolated, which is what a model that learns turn-taking from audio needs. Moshi got there by diarizing single-channel audio into simulated streams and then fine-tuning on Fisher. Block says nothing about how it did.

The MacBook. If Dessa steps at Moshi's 12.5 frames a second and is memory-bandwidth bound like Moshi, a 7B model in BF16 needs about 209 GB/s (reasoned, the arithmetic is in the PersonaPlex piece). Apple quotes up to 546 GB/s for the M4 Max. So "M4 Max or better" is consistent with anything up to a Moshi-sized model and does not by itself mean small. Dorsey says it is "really really small". Without a parameter count I cannot say more.

Measuring the demos

Block's own framing invites this: "Every clip below is a live recording of a person talking to Dessa. Nobody steers it." The page's player draws the two speakers as separate waveforms, and the files behind it are separate: one FLAC per speaker, 44.1 kHz mono, fetched from the page's own asset paths.

expand

The tracks are clean. In every recording the quietest tenth of Dessa's track sits between -72 and -120 dBFS, often digital silence, so it is the model's output written straight to disk rather than a microphone. The person's track has a microphone's noise floor, and while only Dessa is talking it is never louder than in silence, so there is no echo of Dessa leaking into it (measured). That means a simple energy detector per track is enough.

The method, which is a script of my own run on the published files and nothing of Block's:

per track:  decode to 16 kHz mono -> RMS in 10 ms frames
voiced:     18 dB above the track's 10th-percentile floor, never below -50 dBFS
talkspurt:  bridge pauses under 180 ms, drop bursts under 90 ms
switch:     a talkspurt that starts after the OTHER speaker spoke last
offset:     its start minus the end of the other's last talkspurt
            (negative = it started while the other was still talking)
overlap:    a talkspurt that starts and ends inside the other's talkspurt

I used 19 of the 21 recordings: the five headline demos, where the person is a Dessa team member (the page's portraits are named), and 14 of the 15 under "More samples", where the person is labelled "User". I left out "Noisy GPT live", whose person track mixes a human and another AI, and treat the headline call with GPT-Live-1 separately. That is 67.4 minutes of conversation (measured).

What the timing shows

Across the 19 calls, Dessa takes the floor 501 times. Its median offset is +0.24 s, the middle half falls between -0.09 and +0.62 s, 29.5% of its turns start before the person has finished, and 10.2% wait more than a second. The people in the same calls take the floor 492 times with a median of +0.39 s, a middle half of -0.03 to +0.65 s, 26.6% in overlap and 7.7% over a second (all measured).

Three things in that comparison matter.

The distributions have the shape of conversation. Both peak a little after zero, both have a long tail of overlapping starts and a shorter tail of slow ones. The cross-language study by Stivers et al. found that every language it tested shows "a general avoidance of overlapping talk and a minimization of silence between conversational turns", with average gaps varying "within a range of 250 ms from the cross-language mean" (reported). Dessa's median sits inside the range of the humans it was talking to, not seconds outside it the way a cascade's would. The numbers are not directly comparable to Moshi's 200 ms or to Full-Duplex-Bench latencies, which time different events, but they are of the same kind.

Dessa is, if anything, quicker than its partners. Its median is 0.15 s earlier than the people's, and it starts in overlap a little more often. The difference survives changing the detector: with the threshold at 12 or 24 dB, or the bridging at 300 ms, Dessa's median stays between +0.21 and +0.26 s and the people's between +0.39 and +0.42 s (measured). It is also not just the team. In the five calls with team members Dessa's median is +0.32 s against their +0.39 s; in the 14 with other users it is +0.22 s against +0.41 s (measured).

It backchannels at a human rate. Dessa makes 170 short bursts that start and end inside the person's talkspurt, 2.52 a minute, with a median length of 0.285 s; the people make 142, or 2.11 a minute (measured). Of Dessa's bursts of a second or less, Block's own subtitles put a plain "yeah" on 16, and put no words at all on 72 of 168: those are laughs, breaths and murmurs that the subtitler skipped, which is itself a sign that they are not words. Both speakers together overlap for 6.4% of all speech time, and Dessa does 61.5% of the talking (measured).

Two of Block's annotations, checked

Block marks a "Laugh" at 11 s in the first demo and writes that Dessa laughs "mid-sentence, the way people do". The detector puts the start of that burst at 10.92 s, while the person's phrase ("Winter is coming for sure, but you know") runs to 11.18 s. The laugh begins 0.26 s before they finish (measured). Mid-sentence is generous; end of sentence, in overlap, is accurate.

Block also marks "Speaking louder" from 139.82 s: "Ask it to speak up and it gets louder." I measured the median level of Dessa's voiced frames. Before the marker it is -31.6 dBFS. From 139.82 to 155 s it is -24.4 dBFS, 7.2 dB louder. After 155 s it is back at -32.0 dBFS (measured). So it did get louder, briefly, and drifted back within about fifteen seconds. An earlier request at 125 s, "Can you come closer so I can hear you?", made no visible difference: -33.9 dBFS between 100 s and 139.8 s.

Dessa and GPT-Live-1

The headline demo where Dessa talks to "GPT-Live-1" (Block's name; the page says it talks to "ChatGPT Live") is one five-minute call, so it is an anecdote, not a result. In it Dessa takes the floor 32 times with a median of +0.12 s and starts in overlap 40.6% of the time; GPT-Live-1 also takes it 32 times, with a median of +0.31 s and 12.5% in overlap (measured). The other model waits for a gap. Dessa does not. Whether that reads as natural or as rude depends on the listener; it is the behaviour a model trained on human two-party calls would be expected to have.

What this does not show

These are Block's chosen recordings. They show that Dessa can hold a timing distribution like a person's for five minutes at a time. They say nothing about how often it does, because there is no test set, no failure rate and nothing on how the 21 were picked from however many were recorded.

Timing is not content. The recordings show Dessa talking in the first person about "my best friend" and the board games it plays "in our home", and in "Artist Dessa" it says "I am in fact Dessa". A model trained only on human conversation will claim a human life, because that is what the data contains. How it knows its own name, if no text and no prompt steer it, Block does not say; some identity-specific training or conditioning is the obvious guess, and it is only a guess (reasoned). Factual accuracy, the thing that a text backbone mostly supplies, is not measured anywhere. Block's own caption for the movie demo is that it "knows a surprising amount, considering every fact it has come from raw sound", which is a careful way to put it.

And "most authentic" is not something I can check. It is a comparative claim with no comparison published.

What I would want next

If Block publishes a report, the useful parts would be: the parameter count and frame rate, so the MacBook claim means something; the data recipe, especially how two-sided audio was obtained at scale; Full-Duplex-Bench numbers, which time turn-taking, backchannels and interruptions the same way for every model; and a floor-transfer-offset distribution on held-out human conversations, which is the measurement above done properly. The early-access programme for safety researchers is a reasonable start. The safety argument on the page, that a model which hears emotion could notice "serious pain", cuts both ways for a model that also sounds exactly like a person; one reply under Dorsey's post was "Scam calls about to get next level".

For now Dessa is a demo page with unusually honest files. Two tracks per call is more than most voice launches give. On those files, the turn-taking claim holds up. The rest is a waitlist.