# Flint: 125 tokens a second is what 2.5B active parameters buy at 273 GB/s
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/flint
> date: 2026-10-23
> tags: mixture-of-experts, agentic-coding, on-device, apple-silicon, quantization, kv-cache, inference, benchmarks
Richard Socher [introduced Flint](https://x.com/RichardSocher/status/2112944385415426249) as Recursive's first model: "a fast local coding model built with our automated research system, the Eureka Machine", with "125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark", "up to 2x faster than comparable models", and "built from scratch in under 4 months". Unsloth [followed](https://x.com/UnslothAI/status/2113046042908541083) with Dynamic GGUFs, calling it "the strongest MoE model under 50B parameters" that runs "on 24GB RAM".
Four claims, then: a speed, a ranking, a provenance and a method. The speed one is the one I can check most completely, because decode speed on a laptop is arithmetic. Here is what I found, with every number labelled **measured** (I computed it from a published file), **reported** (Recursive's or Unsloth's figure, not re-run) or **reasoned** (my arithmetic on the other two). I did not run the model: the BF16 checkpoint needs a B300 or H200, and the 4-bit builds need a Mac or an NVIDIA GPU this machine does not have.
| Claim | What the files say |
|---|---|
| 39.8B total, 2.5B active | Holds. **Measured** 39,785,287,456 parameters in the headers; 2,515,488,544 active per token including the output head. |
| 125+ tok/s on an M4 Pro | Plausible to the decimal. Recursive's published curve implies a steady ~200 GB/s, 73% of the chip's 273 GB/s, from 1K to 252K tokens of context (**reasoned**). |
| Up to 2x faster than comparable models | 2.00x against Qwen3.6 35B-A3B on the M4 Pro, with Flint on Recursive's own engine. Same engine on both sides (llama.cpp on RTX Spark), it is 1.33x (**reasoned** from **reported** numbers). |
| Best MoE under 50B on agentic coding | Against three named MoE peers, on Recursive's chart. Leads five of six columns; loses LiveCodeBench v6 to Qwen 3.6 by 0.1 and wins SWE-bench Verified by 0.8 (**reported**). |
| Built from scratch | Nothing in the files contradicts it: a new tokenizer, and no row of its attention weights matches the closest Qwen shape (**measured**). That is not proof. |
| The Eureka Machine did it | Three contributions are named. No logs, ablations or code for the system are published. |
## The shape, read from the headers
A safetensors file starts with an 8-byte length and a JSON header listing every tensor's name, dtype and shape, so fifteen range requests give the model's anatomy without the 80 GB of weights.
**Measured**: 36,734 tensors, all BF16, 39,785,287,456 parameters, which is the 39.8B on the card. The config matches the card's specification table: 49 layers, the first 2 dense and the other 47 MoE; hidden size 2,048; 32 query heads and 4 key-value heads of dimension 128; 256 routed experts of width 512 per MoE layer, 6 selected per token, plus a shared expert of width 1,024; a 150,144-entry vocabulary with untied input and output matrices.
The attention is a hybrid. 35 layers use a sliding window of 513 tokens (the current token and 512 before it) with RoPE; 14 layers attend to the whole context with no positional encoding at all. Every layer has per-head QK RMSNorm and learned per-head attention sinks, which show up in the headers as a 32-value `softmax_offset` tensor per layer. The router uses a sigmoid score, like DeepSeek-V3's, and there is no bias-correction tensor beside it.
Where the parameters sit (**measured**, from the headers):
| Group | Parameters | Read per token |
|---|---|---|
| Routed experts (47 layers x 256 x 3 matrices) | 37.85B | 6 of 256: 887M |
| Attention projections (49 layers) | 925M | all |
| Output head | 307.5M | all |
| Input embedding | 307.5M | one row |
| Shared experts, 2 dense MLPs, routers, norms | 396M | all |
Add the "read per token" column and you get 2,515,488,544 active parameters with the output head and 2,207,993,632 without it. The card says "approximately 2.5B, including the output head" and "approximately 2.2B" for the body. Both hold to the parameter.
Two things in that table drive the rest of this article. The experts are 95% of the model and 35% of what each token touches; that is the point of an MoE. And the output head is 12% of the active parameters on its own, because the body is small and the vocabulary is not. Recursive's quantization story starts there.
## Why active parameters set the speed
At batch size 1, one token is one forward pass of matrix-vector products: each weight is loaded, used for one multiply-add, and dropped. The chip can compute far faster than it can fetch, so it waits on memory, and the speed limit is
$$
\text{tokens/s} \le \frac{\text{memory bandwidth}}{\text{bytes of weights read per token} + \text{bytes of KV cache read per token}}
$$
Total parameters decide whether the model **fits**. Active parameters decide how fast it **runs**. A 40B MoE that reads 2.5B per token runs like a 2.5B dense model, as long as all 40B are resident.
The weight term depends on the format. Recursive's Apple build, [`RecursiveSI/flint-int4`](https://huggingface.co/RecursiveSI/flint-int4), is MLX group-64 affine INT4: each group of 64 weights stores 4-bit codes plus a BF16 scale and a BF16 offset, 4.5 bits per weight. Its headers (**measured**) show the output head at 4 bits too (`lm_head.weight` is 1,024 bytes per row of 2,048 values), the embeddings at 8 bits and the router left in BF16. Summing what one token touches, with 6 of 256 experts counted per layer and one embedding row:
| Group (INT4 build) | MB read per token |
|---|---|
| Attention projections | 520 |
| Routed experts | 499 |
| Shared experts, dense MLPs, routers, norms, one embedding row | 259 |
| Output head | 173 |
| **Total** | **1,451** |
That is the "against common practice" choice in Recursive's post: most quantization recipes leave the output head in 16 bits. Left in BF16 it would be 615 MB, and every token would read 1.89 GB instead of 1.45 GB (**reasoned**), about 30% more. Recursive says it recovered the accuracy with an offline distillation stage on the model's own pre-sampled responses, with the 16-bit model as teacher. Its summary: "The final NVFP4 model scores within 0.7 points of the 16-bit model across our benchmarks." The chart below has both checkpoints: the six differences average 0.60 points (**reasoned** from **reported**), the largest single drop 1.2. The NVFP4 GGUF for NVIDIA quantizes the output head as well; its header lists `output.weight` in the NVFP4 type (**measured**).
The KV term grows with context. Each full-attention layer keeps a key and a value per token: 4 heads x 128 dims x 2 bytes x 2 = 2,048 bytes. Fourteen such layers make 28,672 bytes per token of context (**reasoned** from the config). The 35 sliding-window layers never hold more than 513 positions, about 36.8 MB in total however long the session gets. At 1K tokens of context the KV read is 66 MB, noise next to 1,451 MB of weights. At 128K it is 3.8 GB, more than twice the weights.
The calculator does the division for a few chips. Pick a format, a context length and an engine efficiency (the share of the chip's peak bandwidth the kernels achieve). The filled dots are Recursive's own numbers for its `rec` engine on an M4 Pro, read off the chart in the [`rec` README](https://github.com/recursive-oss/rec); the hollow ones are llama.cpp on the same Mac.
## Checking 125 tokens a second
Apple specifies the M4 Pro at 273 GB/s. At 1K context the INT4 build reads 1.517 GB per token, so the ceiling is 180 tok/s. Recursive's speed chart reports 131.22 tok/s with its MLX engine at 1K input context, which is 73% of that ceiling (**reasoned**). That is a believable number for a well-tuned kernel; nothing about it needs a trick.
The stronger check is the whole curve. The `rec` README plots decode throughput from 1K to 252K tokens of context on the same M4 Pro, 4-bit weights, five runs per context, prefill excluded. I read the nine points out of its SVG (**reported**) and multiplied each by the bytes my model says that token reads:
| Context | rec tok/s | GB read per token | Implied GB/s | llama.cpp tok/s | Implied GB/s |
|---|---|---|---|---|---|
| 1K | 131.2 | 1.52 | 199 | 88.0 | 134 |
| 8K | 113.7 | 1.72 | 196 | 79.5 | 137 |
| 32K | 82.4 | 2.43 | 200 | 58.9 | 143 |
| 128K | 39.1 | 5.25 | 205 | 24.2 | 127 |
| 252K | 22.6 | 8.89 | 201 | 8.6 | 76 |
Across all nine points the implied bandwidth for `rec` stays between 194 and 208 GB/s (**reasoned**). One constant explains a curve that falls by a factor of almost six. That is what a memory-bound engine looks like: the throughput drops because the bytes per token grow, and for no other reason. It also says the KV cache is held in 16 bits, which matches the source: the cache in `rec`'s `cache.py` is allocated as `bfloat16`.
llama.cpp is a different story. It reaches about 134 GB/s at short context and decays to 76 at 252K, so its long-context attention path, not bandwidth, becomes the bottleneck. The README's own summary, "1.4–2.6×" faster than llama.cpp, is the ratio of these two columns.
`rec` is where the Metal megakernel lives: `engine/src/rec_mlx/kernels/megakernel.py`, 1,337 lines, the kernel the post describes as fusing "the entire decode forward pass" on M4 and M5 chips. It supports three chips, M4 Pro, M5 Pro and M5 Max, and needs 48 GB of unified memory.
For RTX Spark the reported figure is 155.82 tok/s at 1K context with the NVFP4 checkpoint. NVFP4 stores 4-bit values with an 8-bit scale per 16, also 4.5 bits per weight, so the bytes per token are about the same 1.5 GB. Recursive does not state the Spark's bandwidth. Reporting puts the N1X at about 300 GB/s (LPDDR5X-9400 on a 256-bit bus); on that figure the engine runs at roughly 79% of peak (**reasoned**, on a bandwidth I could not confirm from Recursive).
## What "up to 2x" compares
The 2x is 131.22 against Qwen3.6 35B-A3B's 65.55 on the M4 Pro, which is 2.00x (**reasoned**). But the Flint row is Recursive's own engine and the chart does not label the peers' engine. On the RTX Spark rows, where both sides are named, Flint on llama.cpp does 96.61 and Qwen3.6 on llama.cpp does 72.65: 1.33x. Flint on llama.cpp on the Mac, 87.98, is 1.34x Qwen3.6's 65.55.
How much of that should the shape alone give? Qwen3.6 activates about 3B parameters per token to Flint's 2.5B, which on bytes alone is about 1.2x (**reasoned**, from the "A3B" in its name rather than a header count). So roughly: the smaller active set buys 1.2-1.3x, and Recursive's engine buys the rest of the 2x. The engine is open source under Apache-2.0, so that part is real and usable. It is just not a property of the model, and it only exists for three Apple chips.
The "5x+ faster than dense models with similar coding scores" is conservative on Recursive's own numbers: dense Qwen3.8 27B reads 13.01 tok/s on the same chart, a tenth of Flint's short-context figure. That is the dense penalty in its purest form: 27B parameters read per token instead of 2.5B.
## The benchmark claim, and its class
"Best MoE under 50B" is a statement about a chart, so here is the chart.
Read it with the class filter on, then off:
- **The class is three models.** Qwen 3.5, Qwen 3.6 and Laguna XS 2.1, each 33-35B total with 3B active. Within that set, Flint BF16 leads Terminal-Bench 2.1 by ten points (54.9 vs 44.9), SWE-bench Pro v2 by sixteen (89.7 vs 73.8) and IFBench by nineteen (89.2 vs 70.2). It leads SWE-bench Verified by 0.8 (74.2 vs 73.4), and it does not lead LiveCodeBench v6: Qwen 3.6 has 80.4 to Flint's 80.3.
- **The mean leans on instruction following.** Across all six columns Flint averages 80.6 to Qwen 3.6's 71.7. Across the four coding columns alone it is 74.8 to 68.1 (**reasoned**). Still a lead, a smaller one; IFBench is where the gap is widest.
- **The dense column is not a tie.** Dense Qwen 3.8 averages 84.6 and leads Flint on all four coding benchmarks, by 18.1 points on Terminal-Bench 2.1. "Rivals" is fair for Muse Glimmer and Gemma 4. For Qwen 3.8 the honest framing is the one in the speed chart: about 90% of its coding mean at ten times its speed.
- **The SWE-bench Pro v2 column is all Recursive's.** The dagger on the panel title marks every score in it as measured internally. Laguna XS 2.1 also carries daggers on four other columns, and Qwen 3.8 and Muse Glimmer on some. The card says plainly that evaluation setups vary and that Flint's Terminal-Bench score counts web-retrieved solutions as failures, which developer-reported scores may not. That is a stricter rule for Flint, which helps its case, but it also means the columns are not one experiment.
The 89.7 on SWE-bench Pro v2 deserves its own line, because the card puts it in the second paragraph. I cannot check it: the harness is mini-swe-agent, the timeout is up to five hours per task, and the score is pass@1 averaged over at least five attempts, but the task set and the v2 changes are not described on the card or the post. Treat it as Recursive's measurement under Recursive's settings.
The two Flint dots show the engine effect: same model, same quality, 68.89 versus 47.58 tok/s depending on whose kernels run it.
## Built from scratch?
"From scratch" cannot be proven from released weights. It can be contradicted, so I looked for a contradiction.
**The tokenizer is new.** Flint's `tokenizer.json` is a byte-level BPE with 150,000 entries and 149,744 merges, plus 144 special tokens (`<|begin_of_text|>`, ``, `` and reserved slots up to 150,143). Against eight candidate tokenizers, seven of them distinct (**measured**), the share of Flint's vocabulary strings that also exist in the other one ranges from 55% (DeepSeek-V3) to 71% (MiniMax-M2), and the share with the same id is 0.2% or less everywhere. The longest identical run of merges at the start is two. Independent BPEs trained on similar web and code text share most common strings, so 55-71% is what unrelated tokenizers look like; a fork keeps ids and merge order. The pre-tokenizer is a hybrid too: DeepSeek-V3's digit and CJK splits in front of an o200k-style letter rule, matching none of them exactly.
**The attention weights do not match the closest public shape.** Flint's attention projections have exactly the shapes of Qwen3-30B-A3B's (hidden 2,048, 32 query heads, 4 KV heads, dimension 128), and its two dense MLP layers are 6,144 wide like that model's. Same shapes are where a copy would show. I range-read the key and value projections of layers 0, 2 and 10 from both checkpoints (**measured**): the flattened cosine similarity is between -0.002 and 0.001, and the best match between any of Flint's 512 rows and any of Qwen's 512 rows is about 0.10, which is what random 2,048-dimensional vectors give. A rotation of the whole hidden space would defeat this test, and I did not test the experts, which have different shapes from any Qwen model. Nothing points that way.
**The design space is familiar.** 256 experts of width 512 is Qwen3.6 35B-A3B's expert grid; the config keys (`mlp_only_layers`, `decoder_sparse_step`, `shared_expert_intermediate_size`) are the Hugging Face Qwen-MoE vocabulary; the learned sinks and sliding windows recall gpt-oss. Recursive's post says the architecture search started "from a baseline with 35 billion total parameters and 3 billion active parameters", which is the Qwen3.6 size class. A borrowed shape is not borrowed weights.
What I cannot check is the four months. The card lists pretraining, mid-training, supervised fine-tuning and reinforcement learning, and the quantized cards describe the data as web, code, licensed and synthetic. No token count, compute budget or data mix is published.
## What the Eureka Machine is credited with
The Eureka Machine is Recursive's automated research loop. Its [earlier post](https://www.recursive.com/updates/first-steps-toward-automated-ai-research) describes it as proposing an idea, implementing it, running the experiment, validating the result against reward hacks and variance, and choosing the next experiment, with results on NanoChat autoresearch, the NanoGPT speedrun and a GPU kernel benchmark. For Flint, the announcement names three contributions:
1. **The shape.** Starting from a 35B-total, 3B-active baseline, it searched expert counts and widths and picked 40B total and 2.5B active as the best projected trade between speed and coding quality.
2. **The quantization recipe.** The 4-bit output head, plus the offline self-distillation stage that "nearly halved the number of training steps compared with our quantization-aware training baseline". The thread says low-bit training "shrank Flint by 71%"; the BF16 safetensors total 79.58 GB and the NVFP4 GGUF is 22.86 GB, a 71.3% reduction (**measured**).
3. **The kernels.** Low-bit projections, MoE, attention and prompt processing for Apple Silicon and RTX Spark, and the Metal megakernel.
Of those, the files let me confirm the outcomes: the shape is what the post says, the output head is 4-bit in both quantized releases, and the engine's throughput sits at a steady three quarters of peak bandwidth. What they do not let me confirm is the attribution. There are no search logs, no list of what was tried and rejected, no ablation of agent-found versus human-found choices, and the Eureka Machine itself is not released. Flint is consistent with the claim and silent on it.
## Running it, and the 24 GB line
The Apache-2.0 licence is on every repo and in a `LICENSE` file beside the weights; the card notes that hosted API terms and data practices are separate from the weight licence. The `rec` engine is Apache-2.0 too. Transformers modeling code is not included yet, so the BF16 checkpoint runs through an SGLang kit on one B300 or H200.
For a laptop the choices are, by file size (**measured** from the Hub API):
| Build | Size | For |
|---|---|---|
| `RecursiveSI/flint-int4` (MLX) | 22.57 GB | `rec` on M4 Pro, M5 Pro, M5 Max; 48 GB required |
| `RecursiveSI/flint-int4-gguf` (Q4_1) | 25.99 GB | llama.cpp with Metal; 48 GB required |
| `RecursiveSI/flint-nvfp4-gguf` | 22.86 GB | llama.cpp with CUDA; 48 GB recommended |
| `unsloth/flint-GGUF` UD-Q4_K_XL | 25.05 GB | llama.cpp, Unsloth Desktop |
| `unsloth/flint-GGUF` UD-Q3_K_XL | 18.49 GB | the 24 GB option |
| `unsloth/flint-GGUF` UD-IQ1_M | 11.35 GB | the smallest |
So "runs on 24 GB" means the 3-bit file, which Unsloth's own guide says "fits 24GB RAM setups at short contexts". That caveat is doing work: the full-attention KV cache adds 0.94 GB per 32K tokens (**reasoned**), and macOS keeps part of unified memory away from the GPU by default, so 24 GB is a short-context, single-app setup. The 125 and 150 figures are measured on 48 GiB and 64 GB machines with 4-bit weights, not on the 3-bit file. On a 24 GB GPU the 3-bit file fits in VRAM with almost four times the M4 Pro's bandwidth; once experts spill to system RAM, the slower link sets the pace, as [Edge0](/articles/edge0-streaming-moe) and [FreeToken](/articles/freetoken) count.
## What I make of it
The speed claim is the best-supported thing about Flint, and it is supported by arithmetic rather than by trust. A 2.5B-active MoE with a 4-bit output head reads about 1.45 GB per token; a 273 GB/s chip moves that 180 times a second at best; Recursive's engine gets 131, and holds the same 73% of peak as the context grows to 252K. The ranking claim is true on the chart as drawn, against three peers, with one column lost by a tenth of a point and the strongest coding numbers in the room belonging to a slower dense model. The provenance claim survives the checks I could run. The method claim is a description.
The interesting design lesson is the one the bandwidth model makes obvious: once the active body is 2.2B, the output head and the KV cache are what is left to attack. Flint quantizes one and starves the other with 35 sliding-window layers. Those two decisions do more for its laptop speed than any benchmark column.
Related reading on this site: [Tiel-Coder-35B-A3B](/articles/tiel-coder-35b-a3b) counts what a "4-bit" GGUF of a model in the same size class really weighs; [GLM-5.3-Flash-MLX](/articles/glm-5-3-flash-mlx) checks another "built for MacBook Pro" claim against its own memory tables; [Qwen3.8-27B on a 12 GB card](/articles/qwen3-8-27b-on-12gb) is the dense model that tops this chart, squeezed down; [continuous batching on a DGX Spark](/articles/dgx-spark-batching) covers the box the RTX Spark is a sibling of; [Laguna S 2.1](/articles/laguna-s-2-1) is the larger sibling of one of the three peers; and [Dream-RSI](/articles/dream-rsi) and [RSI-top](/articles/rsi-top) look at other automated-research loops and what they actually changed.