# ssftalk: an ElevenLabs-shaped API over a formant synthesizer, benchmarked against typed-in numbers
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/ssftalk
> date: 2026-10-19
> tags: speech, tts, audio, rust, benchmarks, interpretability, voice-agents
On 18 October 2026 solidSF posted a link to [`solidsf-inc/ssftalk`](https://github.com/solidsf-inc/ssftalk) with "we will explain sometime soon… for now just go try stuff". The README calls it a "Pure-Rust ElevenLabs v1 Voice Engine, Full-Duplex Rooms & Audio-LLM Mechanistic Interpretability". Its badges say "265.5x geomean speedup (112x RT TTS)" against cloud TTS, ElevenLabs Flash v2.5, Demucs, WhisperX and a dense audio-LLM. It also sells a cloud WebSocket API, paid in "solidSF Credits" with a \$10 welcome grant.
That is a lot of claim for a repository with one commit and no paper. So I did what the post said, minus the trying: I cloned it and read it. This piece explains what the engine actually is, then holds each claim against the file it points at.
Up front: I read all 31 tracked files at commit `eb7722a` and ported three visible functions to Python, run on our own inputs on a shared 4-core cloud VM, CPU only, on 19 October 2026. I did **not** build or run the crate. Its build script decrypts four source modules from a binary blob and compiles them in, and I was not going to run code nobody can read on a shared machine. Every number below is labelled **measured** (from the files or from my ports), **reported** (solidSF's figure, not re-run) or **reasoned** (arithmetic on the other two).
| | |
|---|---|
| Repository | `solidsf-inc/ssftalk`, one commit, `eb7722a`, 18 Oct 2026, Apache-2.0 |
| Dependencies | `serde` and `serde_json`, nothing else (measured, `Cargo.toml`) |
| Readable Rust | 191,541 bytes in `src/` (measured) |
| Encrypted Rust | `assets/ssftalk_moat_vault.bin`, 78,428 bytes (measured), about 29% of the crate's source by size (reasoned) |
| Weights, model files, downloads | none (measured: no URL, file fetch or tensor format anywhere in the code) |
| Paid surface | `wss://api.solidsf.com/v1/ssftalk/ws`, 5 to 25 micro-dollars a call (reported, `cloud_ws.rs`) |
## What a voice is here
Start with the part that decides everything else. Modern TTS, the kind ElevenLabs sells and the kind covered in [speech.cpp](/articles/speech-cpp) or [Paradee](/articles/paradee-tiny-tts), is a learned model: text becomes tokens, a network predicts acoustic features or codec codes, a vocoder turns them into a waveform. The voice lives in millions of weights.
ssftalk has no weights. Its only dependencies are `serde` and `serde_json`, and nothing in the code opens a model file or a URL other than its own loopback test servers and the paid WebSocket. A voice is a `VocalTractProfile` in `src/voices.rs`: a fundamental frequency, four formant frequencies, a spectral tilt and a breathiness. Six numbers, plus a pace and a robot flag. Jesse, the house voice, is 192 Hz with formants at 540, 1680, 2680 and 3650 Hz (measured).
That is a **formant synthesizer**, the family behind 1980s speech chips. You make a buzzy source at the pitch $F_0$, then shape its spectrum with resonances at the formant frequencies $F_1 \dots F_4$, which is roughly what a throat and mouth do to the vocal folds' pulses. Different vowels are different formant positions. The text side is rules, not a model: `g2p_english` in `src/phonetics.rs` turns letters into ARPAbet phonemes with a dozen hard-coded words ("the", "hello", "solidsf") and digraph rules for the rest.
The approach is legitimate and old: tiny, deterministic and fast, which explains the speed claims better than any kernel does. It also sets the ceiling; rule-based formant speech sounds like a speech chip. I could not hear it, because the synthesizer is in the part I could not read.
### The ElevenLabs part is the API's shape
"ElevenLabs v1" here means the request and response structs and the URL paths: `TtsRequest` with `voice_settings`, `previous_text`, `pronunciation_dictionary_locators`; `/v1/text-to-speech/{voice_id}/with-timestamps`; the stream-input WebSocket messages with `try_trigger_generation` and `flush`. An ElevenLabs client pointed at the loopback server gets JSON of the right shape back.
It is not a reimplementation of any ElevenLabs model, and it is not a copy of an open model under that name. The built-in "Rachel", "Adam", "Antoni", "Bella" and "Josh" are formant presets that carry 20-character IDs the code labels as ElevenLabs's (Rachel is `21m00Tcm4TlvDq8ikWAM`) and descriptions such as "equivalent to ElevenLabs Rachel". Rachel is 198 Hz with formants at 550, 1710, 2710 and 3690 Hz (measured). The README never says ssftalk is unaffiliated with ElevenLabs; it uses the name 24 times (measured).
### Where the neural model is: the paid side
The code does name neural models, but only as cloud engines. `WsEngine::Articulate150mNeural` is documented in `cloud_ws.rs` as "Remote `ARTICULATE-150M` + `Maya1-3B` + `SNAC-24kHz` expressive neural speech synthesis". Maya1 is a real open model: [`maya-research/maya1`](https://huggingface.co/maya-research/maya1), Apache-2.0, 3,300,928,512 parameters by the Hub's own count (measured), a Llama-style transformer that predicts SNAC codec codes, decoded by [`hubertsiuzdak/snac_24khz`](https://huggingface.co/hubertsiuzdak/snac_24khz) (MIT). Its card advertises inline tags like ``, ``, `` and `` and a voice described in words, which is exactly the surface ssftalk's README sells. I found no model called Articulate-150M on the Hub.
So the reasoned reading is: locally you get a formant synthesizer behind an ElevenLabs-shaped API; the expressive neural voice, if it exists, is an open model served from solidSF's machines for credits. I cannot confirm what the server runs, since I did not call it.
## The encrypted vault
Four files in `src/`, namely `vocoder.rs`, `hamiltonian.rs`, `sts_isolate.rs` and `duplex.rs`, are about 1.2 KB each: 5,008 bytes together (measured). Each holds a doc comment, a `dispatch_over_ws` function for the cloud, and one line:
```rust
include!(concat!(env!("OUT_DIR"), "/vault_vocoder.rs"));
```
The included file does not exist in the repository. `build.rs` makes it. It reads `assets/ssftalk_moat_vault.bin`, XORs every byte with a keystream derived from the eight ASCII bytes `solidSF!`, checks for the magic `SSFVLT01`, and writes each module it finds into the build directory, where the compiler picks it up. The key is in the build script, so this is not protection; it is obscurity, and the crate says so itself. `src/hardware.rs` embeds a "REVERSE-ENGINEER BOUNTY" string inviting anyone who decrypts the vault to email the author, who "wants to fly you to SF".
The licence question this raises is simple. The README says Apache-2.0, and the encrypted bytes are in an Apache-2.0 repository, but the source of the synthesizer, prosody, speech-to-speech, isolation, sound effects, transcription, alignment, dubbing and duplex rooms is not published in readable form. `ElevenLabsEngine` in `src/elevenlabs.rs` imports `HarmonicIstftVocoder`, `ArticulateStreamer`, `ProsodyPredictor`, `build_acoustic_prior`, `transcribe_scribe`, `isolate_voice_audio`, `speech_to_speech` and `H2HSpeechJury`, every one from a vault module. Ten of the 14 benchmark stages run vault code (measured). What is readable is the API layer, the voice presets, the G2P, the codecs, the "mechanistic interpretability" module and the cloud client.
## The benchmark compares the crate with constants
`examples/full_suite_benchmark.rs` is the source of every badge. Each stage does the same thing: time a loop of ssftalk calls with `Instant::now()`, then push a row whose baseline is a literal.
```rust
rows.push(BenchRow {
stage: "07. Schur-Feshbach Audio Isolation",
ssftalk_us: iso_us,
baseline_us: 95_000.0,
baseline_name: "Demucs Spectral U-Net (1s)",
...
```
There are 14 of these, from 3,200.0 µs for "WebRTC SFU Turn State Machine" to 1,850,000.0 µs for "Cloud Voice Design API" (measured). No Demucs, WhisperX, WavLM, FFmpeg or ElevenLabs call exists anywhere in the repository; no file says where the constants came from. The speedup is `baseline_us / ssftalk_us`, and the headline is the geometric mean of the 14 ratios. From the README's own table that mean is 265.5 (reasoned; it reproduces exactly).
The comparisons are not like for like even as typed. Three examples, all measured from the example's source:
- **Stage 10, "Scribe ASR + forced align", 85,170.5x.** The call is `transcribe(audio, Some("Welcome to solidSF deterministic Hamiltonian speech."), true)`: the transcript is passed in. Both test files do the same and then assert that the output has as many words as the hint they passed (6, 12 and 7). The baseline, WhisperX plus PyAnnote, has to find the words. ssftalk is told them.
- **Stage 12, "Full-duplex room + barge-in", 10,668.1x.** The loop creates a struct, pushes two participants, encodes a 12-byte header on 960 bytes and sets one field after an "interruption". It takes 0.30 µs (reported). The baseline is a WebRTC SFU, a networked media server.
- **Stage 7, isolation, 1,285.9x and "+21.9 dB".** The "noisy" input is ssftalk's own synthetic speech plus one sine wave, `0.045 * sin(0.15 n)`. A single steady tone is the easiest noise there is to remove, and Demucs is a music source-separation network.
The ratios run from 1.49x (the codec, against a constant for libmp3lame) to about 85,052x (ASR), a spread of more than four orders of magnitude (reasoned). A geometric mean over that is mostly a statement about which constants were typed. Drop stages 10 and 12 and it falls to 120.7x; keep only the four stages whose code is readable and it is 149.1x; keep the three that produce or carry speech end to end (TTS, streaming, codec) and it is 3.61x (all reasoned). None of these is a speedup either, for the same reason the original is not.
The synthesis number is internally consistent: 40,833.61 µs for 4.56 s of audio is a real-time factor of 0.00895, or 112x real time (reported; the arithmetic holds). For a formant synthesizer that is plausible. It says nothing about ElevenLabs, whose 420,000 µs is a typed constant.
## "Audio-LLM mechanistic interpretability"
This is the most interesting claim to read, because all of its code is visible, in `src/mechinterp.rs` and `src/hardware.rs`. It has three parts. None involves a language model.
### The 88.0% KV-cache saving
The README says ssftalk can "cut GPT-4o Realtime and Gemini Live audio KV-cache token cost by **88.0%**". The function is `prune_audio_llm_frames(pcm16, frame_samples, amplitude_threshold, min_active)`. It does this:
1. Mark every sample whose absolute value is above the threshold (250 in the benchmark).
2. Pack the marks into 64-bit words, 64 samples per word.
3. For each frame, count the set bits in its words. A frame with at least 8 is "active"; every other frame is "cached".
4. Report `cached / total` as `token_savings_ratio`.
That is an amplitude gate, a voice-activity detector of the simplest kind. It never sees an audio LLM, a token or a KV cache; the claim is that a model could skip the frames it marks. The doc comment says it finds "silent/stationary" frames; nothing in it measures stationarity.
The 88.0% comes from the benchmark's input: one second at 24 kHz, all zeros except samples 2,400 to 4,800 set to the constant 9,000. Ten per cent of the signal is a flat block and 90% is digital silence. With 480-sample frames that is 50 frames, 5 of which contain the block, so the honest answer is 90.0%.
The code returns 88.0% because of step 3. It finds a frame's words with `words_per_frame = frame_samples / 64`, integer division, and 480 / 64 is 7.5, which rounds down to 7. Each frame therefore reads 448 samples, and frame $f$ reads samples $448f$ to $448f + 448$ rather than $480f$ to $480f + 480$. The windows slide 32 samples further behind per frame, so by frame 10 the window has slid onto the end of the block and a sixth frame is counted active. My Python port of the function reproduces it: 6 active of 50, 0.8800; counting each frame's own samples gives 5 of 50, 0.9000 (both measured). The same slip means the last 6.7% of any input is never read at all when frames are 480 samples (reasoned: 1 − 448/480).
On real audio the number is different. I ran the same port on three of the speech clips from the [speech.cpp](/articles/speech-cpp) article, resampled to 24 kHz: it marks 18.4%, 13.5% and 21.3% of frames as cacheable (measured). A conversation with long listening gaps would score higher. A quiet room does not help: one second of steady noise at an RMS of 300, about −40 dBFS, gives 0.0% (measured), because every frame has more than 8 samples over 250. Every voice pipeline has a gate like this. It is not a result about GPT-4o.
### The 1-bit SAE circuit matcher
A sparse autoencoder, in interpretability work, is trained on a network's activations to split them into a dictionary of sparse, readable features. Matching binarized codes with XOR and popcount is a real speed trick.
`SaeVoiceCircuitMatcher` does the matching half: for each 256-bit query it finds the dictionary entry with the smallest Hamming distance, four `u64` XORs and popcounts per pair, in plain Rust (the "Tensor-Core `b1.xor.popc`" name refers to PTX that the code says runs only on the paid cloud). There is no autoencoder. In the benchmark the dictionary is 256 entries whose bits are four hash constants multiplied by the entry's index plus a small offset, and the 16 queries are the dictionary's own first 16 entries (measured). Each query finds itself at distance 0. No activations from any model are ever produced to encode.
### The 64-D "glass-box" voice signature
`GlassBoxVoiceSignature64D::from_audio` computes a 64-element vector. Some coordinates are honest features: the fraction of samples with each amplitude bit set, the zero-crossing rate, RMS, peak, an autocorrelation pitch estimate, a 24-segment loudness envelope. Others are not features of the audio at all. Coordinates 24, 25 and 26 are the constants 0.875, 0.75 and 0.9375; coordinate 32 is 1.0; coordinate 33 is 15/24; coordinates 13 to 15 are bits of a running hash of the samples (all measured). The benchmark's verification is "self-cos=1.000000", the cosine similarity of the vector with itself, which is 1 for every non-zero vector.
The "counterfactual ablation" API is a config flag read by the vault's synthesizer: zero some harmonics, drop breath noise. Mechanistic interpretability asks what a trained network learned; here someone wrote the rules, so there is nothing to discover.
## Voice cloning without formants
The README's clone row says "64-D Glass-Box $F_0$ + 4-formant tract extraction" in 305 µs. `add_cloned_voice` in `src/voices.rs` measures $F_0$ by autocorrelation on one window of at most 2,048 samples (85 ms at 24 kHz), clamps it to 85 to 310 Hz, and then sets the formants to a fixed template scaled by pitch:
$$
F_k = T_k \cdot \min\!\big(\max\big((F_0/165)^{0.35},\ 0.82\big),\ 1.22\big), \qquad T = (500,\ 1550,\ 2550,\ 3500)\ \text{Hz}
$$
Formants are never measured from the reference audio (measured). Breathiness is one minus the autocorrelation peak. So two speakers with the same pitch clone to the same voice. "Voice design" is the same idea from text: `tract_from_description` picks 120 Hz if the description says "male" or "baritone" and 195 Hz otherwise, nudged by words like "deep" or "bright" (measured).
The 1,572x against ECAPA-TDNN compares this with a constant for a neural speaker encoder. The two do different jobs: ECAPA produces an embedding that tells speakers apart; this produces a pitch.
## Two output formats that are not what they say
**MP3.** `OutputFormat::Mp3_44100_128` is ElevenLabs's default output format, so a drop-in client will ask for it. ssftalk's encoder writes the four bytes `FF FB 90 64`, which is an MPEG-1 Layer III frame header, then the ASCII tag `SSFM`, a length, and a j128 payload (measured, `encode_audio_output` in `src/codec.rs`). There are no MP3 frames after the header. A browser, `ffmpeg` or any MP3 decoder will reject the file or play noise. ssftalk's own decoder reads it back, which is all its tests check. That alone breaks "100% API parity" for the most common request.
**j128.** This one holds. j128 is a block ADPCM codec, and a careful little one. Each block is 15 samples, 0.625 ms at 24 kHz: two bytes for the first sample verbatim, one header byte with a 2-bit predictor choice (zero, previous sample, linear extrapolation, mean of two) and a 6-bit index into a 64-entry scale table, then 14 four-bit residuals. The encoder tries all four predictors and three neighbouring scales and keeps the one with the least squared error. Fifteen samples in ten bytes is 1,600 blocks a second, exactly 16,000 B/s or 128 kbps, a third of 16-bit PCM (measured).
The code comment claims "> 28 dB speech SNR". My Python port of the encoder and decoder, on the first 4 s of three speech clips, gives 39.9, 30.9 and 38.0 dB (measured). That clears the claim. It is still a 128 kbps waveform coder for 24 kHz mono speech; Opus carries the same speech well at 32 kbps, which is what this site uses for audio. Its "1.5x faster than FFmpeg libmp3lame" is, again, a ratio against a constant.
## The cloud client
`SsfTalkWsClient::connect` takes `wss://api.solidsf.com/v1/ssftalk/ws`, strips the `wss://` prefix, and calls `TcpStream::connect("api.solidsf.com")` (measured). Two things follow, both reasoned from the code rather than from a connection I did not make. The address has no port, which Rust's socket parser rejects, so the default URL fails before any packet is sent. And `wss` means WebSocket over TLS, but a crate whose only dependencies are `serde` and `serde_json` has no TLS: given a host and port, it would send the HTTP upgrade, including `Authorization: Bearer `, in clear text.
Every cloud test in the repository talks to `SsfTalkLoopbackWsServer`, a local mock in the same file that answers with a hard-coded `server_exec_us: 28.4` and debits a balance it keeps in memory. The GPU kernels the README lists for GB300 and Apple Metal are three placeholder strings, for example `"/* [solidSF CLOUD_WS VAULT: rosenberg_harmonic_istft + symplectic_cfm dispatched via wss://api.solidsf.com/v1/ssftalk/ws] */"` (measured). In the same X thread solidSF wrote that the work is "coming to apple metal shortly", while the README's benchmark header names "Apple Silicon Unified Memory (aarch64 NEON + Metal 3.1)". The second announcement post quotes a different product, `solidcv`, "12.1x faster than opencv", which is not in this repository.
## The ledger
Every claim above, in one place. Filter by verdict.
## What holds, and what to take from it
Some things here are real. The ElevenLabs-shaped types are a reasonable layer to put in front of any local engine. The j128 codec meets its own SNR claim. Truncating an agent's transcript at the character where the human interrupted ("heard characters") is the right idea for voice agents. And a deterministic formant synthesizer is fine to ship if you call it that; [earshot](/articles/earshot) is classic DSP presented honestly.
The rest is vocabulary. "Mechanistic interpretability", "sparse autoencoder", "KV cache", "Tensor Core" and "twelve theorems" name, in the code, an amplitude gate, a Hamming search, a hash, a struct and comments. The speedups divide a timed loop by a number someone typed. The parts that would show whether the voice is any good, the synthesizer and the recognizer, are the parts you cannot read.
Testing "faster than WhisperX" means running both on the same audio and machine and reporting word error rate next to the time, as in [Whistle](/articles/whistle-stt) and [VoiceStudio](/articles/voicestudio). Until solidSF publishes the explanation the post promised, nothing in this repository shows the numbers on its badges.
## What I could not check
- **The vault.** I did not decrypt, read, build or run the four modules in `ssftalk_moat_vault.bin`. Everything about synthesis quality, the recognizer without a hint, isolation on real noise, and timing is unverified here.
- **The timings.** All ssftalk microsecond figures are solidSF's (reported). I did not reproduce them.
- **The cloud.** I did not call `api.solidsf.com` or create an account. What it runs, Maya1 or anything else, is unconfirmed.
- **My ports.** The pruning and j128 numbers come from my Python translations of `src/mechinterp.rs` and `src/codec.rs`, written line by line from the Rust (integer division truncating toward zero, as Rust does). The pruning port follows the non-`aarch64` path; the NEON path packs bits the same way. My SNR clips are themselves TTS output from the speech.cpp article, decoded from 32 kb/s Opus, not studio recordings.