# DeepSeek-V4.1-Flash-1017: every byte retrained, nothing else touched, and what two DGX Sparks can hold
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/deepseek-v4-1-flash-1017
> date: 2026-10-17
> tags: deepseek, mixture-of-experts, benchmarks, agentic-coding, on-device
On 17 October DeepSeek replaced the model behind its `deepseek-flash` API name with
**DeepSeek-V4.1-Flash-1017**, five weeks after it released V4.1-Flash on 10 September. The
announcement thread calls it "stronger benchmark performance, faster task completion, and shorter
outputs than the previous version—a Pareto improvement in performance and cost." The same morning a
widely shared post put the model card's table on X and said it is "roughly on par with Opus 5", beats
GLM-5.3 "across the board and even GPT-5.6 Sol", and that you should "buy 2 DGX spark and run this
baby locally."
That is three different claims. One is about what changed, one is about a table, and one is about bytes.
All three can be checked from files, so I checked them. Labels: **measured** means I computed it from a
file in the repo, **reported** means DeepSeek's (or someone else's) number that I did not re-run, and
**reasoned** means my arithmetic on the other two.
## What a "minor update" is, in files
The card says it plainly: 1017 "is a minor update to DeepSeek-V4.1-Flash." For an open-weights
model, "minor update" can mean anything from a changed chat template to a new architecture under the
same name. The repo answers it. I pulled the file listing with blob hashes for both
`deepseek-ai/DeepSeek-V4.1-Flash-1017` and `deepseek-ai/DeepSeek-V4.1-Flash` from the Hugging Face API
and compared them file by file (measured):
- **Byte-identical:** `config.json`, `inference/config.json`, `model.safetensors.index.json` (7.47 MB),
`tokenizer.json`, `tokenizer_config.json`, the `encoding/` reference prompt encoder and its tests,
every file of the `inference/` reference code, and the `LICENSE`.
- **Changed:** `README.md`, `.gitattributes`, and all 48 `model-000NN-of-00048.safetensors` shards.
Each shard has a new hash and exactly the same size in bytes as its predecessor.
- **Removed from the new repo:** the September tech report PDF, `chat_template.jinja`, the two card
figures and the `evaluation/` folder with the DeepSWE reproduction patch. They are still in the
V4.1-Flash repo, which the new card links for the report.
Same sizes is a strong hint, not a proof, so I read the headers. A safetensors file starts with an
8-byte length and a JSON header listing every tensor's name, dtype, shape and byte range. Two
HTTP range requests per shard get it without downloading any weights. Across all 48 shards of each
repo there are **96,085 tensors**, and every one has the same name, dtype, shape and offsets in both
(measured). The architecture and the on-disk layout did not move at all.
So did the numbers inside move, or did DeepSeek re-upload the same weights? I range-read the first 4 KB
of 22 tensors spread across the model in both repos and counted the bytes that differ (measured).
Every one changed: the embedding (38% of bytes differ) and LM head (48%), attention projections in
layers 0, 20 and 39 (66–75%), routed experts in layers 0, 20 and 39 (38–48%), a shared expert (79%),
the router weights (52%) and router bias (5% of bytes, out of 1,536), the hyper-connection mixers
(77%), both **Engram n-gram tables** (27% and 31%), the vision tower and its aligner (52–56%), and the
DSpark draft layers, whose experts differ in 98.8% of sampled bytes. A low-precision format makes a
small weight change flip a byte often, so these percentages say "this tensor was trained" rather than
"by how much". What they rule out is a partial update: this is not a new LoRA merged into the
attention, or a fine-tune of the language model with the vision tower frozen. Every component I looked
at was trained again, including the 196.6 billion lookup-table parameters.
That has a practical consequence the card does not mention. Anything derived from the September
weights — a re-quantized expert bank, Engram tables pre-packed into per-rank NVMe shards, an
abliterated variant — has to be rebuilt from 1017. One of the community Spark repos says it in so many
words: "Engram content follows the weights."
## The machine the weights live in
Since only the weights changed, the September card's architecture description holds for 1017, and the
config confirms the parts I need. V4.1-Flash is a 40-layer mixture-of-experts transformer organised as a
**causal encoder-decoder**: the first 20 layers are an encoder, and the decoder's global KV cache is
projected from the encoder's final hidden states rather than from each decoder layer. The point of that
split is prefill cost: a prompt only has to run through the encoder half, so DeepSeek reports **8B
activated parameters per token in prefill and 16B in decode**.
I can check those two numbers from the headers. Each MoE layer has 384 routed experts plus one shared
expert, and a token is routed to 6 of the 384. One routed expert is three 2304 × 5120 matrices,
35.4 million parameters. Counting, from the headers, every non-expert tensor in layers 0 to 39 plus 6/384
of the routed experts plus the LM head gives **16.13B parameters per decoded token**; doing the same for
layers 0 to 19 only gives **7.89B for prefill** (measured). Both match the card's 8B and 16B.
The rest of the 763.2 billion stored parameters is less obvious, and it matters for anyone sizing
hardware (measured from the headers):
| Part | Parameters | Bytes on disk | Format |
|---|---:|---:|---|
| Routed experts, 15,360 of them | 543.58B | 288.78 GB | FP4 E2M1 + one E8M0 scale per 32 |
| Engram n-gram tables, layers 1 and 14 | 196.61B | 202.76 GB | FP8 + scales |
| Attention, shared experts, routers, mHC, norms | 6.98B | 7.20 GB | mostly FP8 |
| DSpark draft layers (3 × 128 experts) | 14.23B | 7.93 GB | FP4 / FP8 |
| Embedding + LM head | 1.32B | 2.65 GB | BF16 |
| Vision tower + aligner | 0.49B | 0.97 GB | BF16 |
| **Total** | **763.21B** | **510.29 GB** | |
The **Engram tables** are the unusual line. They are two embedding tables of about 384 million rows
each, 256 FP8 values per row, and the model does not multiply by them; it looks rows up. The reference
`inference/engram.py` normalises each token id (lowercased, accents stripped), hashes the 2-, 3- and
4-gram ending at the current position with per-layer multipliers, and reduces each hash modulo a
distinct prime for each of 8 heads (measured from the code). That is 3 n-gram sizes × 8 heads = 24 rows
per Engram layer per token, or 48 rows of 256 values plus 8 scale bytes each: about **12.7 KB read per
token** from 203 GB of tables (reasoned). That access pattern is why people put the tables on NVMe.
The KV cache is the other unusual number. DeepSeek reports **890 bytes of global KV cache per token**,
which it puts at roughly a quarter of V4-Flash's. At the 1M-token context the card recommends, that is
0.93 GB (reasoned). For local use the KV cache is a rounding error; the weights are the whole problem.
## What the table says
The 1017 card has one table: 11 benchmarks, with columns for 1017, the September V4.1-Flash, K3,
GLM-5.3, GPT-5.6 Sol and Opus 5. The repost's image is a screenshot of exactly this table; it has no
other source. All numbers below are reported by DeepSeek and none were re-run by me.
**Against the September model, every row improves (reported).** The gains run from +2.0 points
(ZeroBench-main, 49.0 → 51.0) to +11.1 (Terminal-Bench 4.0, 31.2 → 42.3). ProgramBench gains 8.8
(20.3 → 29.1), Apex Bench 7.5, HLE without tools 5.8, Automation-Bench 5.7. The median gain is 4.1
points and the mean 5.2 (reasoned). No row regresses.
Three things in the table deserve a footnote before you quote it:
- **Nine rows disappeared.** The September card also reported GPQA Diamond (90.9), Codeforces (3471),
MathArena Apex (65.6), Terminal-Bench 2.1 (90.6) and 3.0 (30.0), CyberGym (88.1), SEC-Bench Pro (62.8),
ExploitGym (15.3) and HLE with tools (63.9). The 1017 card and the API changelog report none of them
for 1017. "No row regresses" is a statement about the rows DeepSeek chose to keep. Maybe all nine went
up too; nothing published says.
- **The previous model's NL2Repo score changed.** The September Hugging Face card says 64.0; the 1017
card's V4.1-Flash column, and DeepSeek's own 10 September changelog, say 65.4. One of DeepSeek's two
pages has a typo. If 65.4 is right, 1017's gain there is 2.8 points; if 64.0, 4.2.
- **Terminal-Bench 4.0 changed agent scaffold.** The September card says its code-agent benchmarks,
Terminal-Bench 4.0 included, ran in the Minimal mode of DeepSeek Harness. The 1017 card says "We use
the Claude Code harness for Terminal-Bench 4.0." The old column's 31.2 is the same number as in
September, so it was most likely not re-run in Claude Code. The largest gain in the table may partly be
a scaffold change. The September card's own scaffold table points the other way, for what it is worth:
on Terminal-Bench 2.1, V4.1-Flash scored 88.0 in Claude Code and 90.6 in DSH Minimal (reported).
The new row, Apex Bench v1.1, has a V4.1-Flash score of 64.6 in the 1017 card and no entry in the
September card, so it was run on the old model for this comparison.
### "Roughly on par with Opus 5"
Pick Opus 5 as the comparison in the widget. 1017 is ahead on 4 rows: DeepSWE v1.1 (+3.0),
Automation-Bench (+10.2), Agents' Last Exam (+6.8) and Apex Bench (+6.3). It is behind on 7: HLE by
13.7 points (42.6 vs 56.3), Terminal-Bench 4.0 by 9.5 (42.3 vs 51.8), ProgramBench by 7.9, NL2Repo by 7.1, and the three
vision rows by 1.0 to 1.6 (reasoned from the reported scores). The unweighted mean over the 11 rows is
59.4 for 1017 and 60.8 for Opus 5, which is the kind of average that only exists in tweets: it adds
percent-of-tasks on a coding benchmark to percent-correct on a knowledge exam.
The fair reading is narrower than the repost's. On the agentic rows DeepSeek built this update for
(SWE tasks, automation, Agents' Last Exam, Apex) 1017 matches or beats Opus 5. On the hardest reasoning
row and on the two "build a whole program" rows it is clearly behind. "On par" is true for some work
and false for other work, and the table tells you which.
### "Outperforms GLM-5.3 across the board and even GPT-5.6 Sol"
Against GLM-5.3 this holds on every row where GLM has a number: 8 of 8 (reasoned). GLM has no score on
the three vision rows, and its HLE is the text-only subset, so the like-for-like comparison is 1017's
44.0 against 42.0. Against GPT-5.6 Sol, 1017 wins 9 of 11 and loses two: HLE (42.6 vs 44.5) and
ZeroBench-main (51.0 vs 53.0). "Even GPT-5.6 Sol" is mostly right, not "across the board".
Two caveats apply to every rival column. These are DeepSeek's numbers for other companies' models, and
the card does not say whether DeepSeek ran them or copied them from each vendor; the September card
lists harnesses for its own runs only. And running Terminal-Bench 4.0 in Claude Code, Anthropic's own agent, probably
suits Opus 5 better than the other columns (reasoned, not measured).
## "Shorter outputs" and "faster task completion"
Neither has a number on the card or in the API changelog. There are no token counts and no wall-clock
times. The only evidence is the second image in DeepSeek's announcement post: DeepSWE v1.1 Pass@1
against average cost per task, at several reasoning efforts, in the mini-SWE v2 agent.
Reading the points by eye off the log axis (approximate), V4.1-Flash at max effort cost about \$0.49 a
task for 74.2%, and 1017 at max effort costs about \$0.20 for 77.0%. Even 1017 at low effort, about \$0.11
a task, lands near 72.6%, within two points of the old model's best for less than a quarter of its
cost. The API changelog says per-token pricing did not change with the update (reported): `deepseek-flash`
is \$0.30 per million uncached input tokens and \$1.20 per million output tokens at peak, half that
off-peak. At unchanged prices, a 59% drop in cost per task (reasoned, from the chart reading) can only
come from fewer tokens per task, input and output together. That supports "shorter outputs" on this
benchmark, in this agent. It does not tell you the output-to-input split, or whether the same holds in
Claude Code, CodeBuddy or DeepSeek Harness, which the thread says the model was tuned for.
"Faster task completion" follows from fewer tokens if decode speed is unchanged, and the architecture
is unchanged, so it is plausible (reasoned). No timing is published.
The thread's second post says 1017 "uses reinforcement learning with self-feedback to improve
end-to-end project development." There is no report for 1017 and no detail beyond that sentence and four
demo clips, so I cannot say what the self-feedback is. The ProgramBench (+8.8) and Apex Bench (+7.5)
gains are at least consistent with training aimed at building whole projects.
## "Buy 2 DGX Spark and run it locally"
A DGX Spark is a GB10 box with 128 GB of LPDDR5x unified memory at 273 GB/s. Two of them hold 256 GB
nominal. A community repo for a four-Spark setup reports 121.7 GiB usable per node, and I keep another
6 GiB per node for activations and CUDA graphs, which leaves about 248.5 GB across two boxes (reasoned).
At the shipped precision the checkpoint is 510.3 GB (measured): **twice what two Sparks hold**. Take the
Engram tables out of memory and the rest is still 307.8 GB with the draft layers, 299.8 GB without
(reasoned). So the repost's advice does not work with DeepSeek's files as published.
It does work with a re-encode, and someone has already shown it on the September weights. 0xSero's
`DeepSeek-V4.1-Flash-Spark` re-encodes every routed expert as EXL3 trellis at an average of **2.77 bits
per weight**, keeps everything else byte for byte, reads the Engram tables from disk, and serves with
tensor parallelism over the two Sparks' network link. Its card reports about 39–41 tok/s for one code
stream and about 29 tok/s for prose, with DSpark speculative decoding proposing 7 draft tokens, a 262,144-token served context, and 2,050 tok/s prefill at 8K (reported, for V4.1-Flash, not
1017). It also says fresh acceptance of its current published image is still pending. By my
arithmetic that layout, 2.77-bit experts with the tables on NVMe, needs about 207.2 GB across two DGX Sparks (reasoned), which fits with roughly 40 GB to
spare. Because 1017 kept every shape and offset, the same recipe should apply to it unchanged; the
experts and the Engram shards have to be rebuilt from the new weights.
The bandwidth side is where the arithmetic gets interesting. A decoded token reads the 6 active experts
in each of 40 layers, every non-expert backbone tensor, and the LM head: **13.04 GB at the shipped
precision** (measured from the headers). Of that, the routed experts are only 4.51 GB. The FP8 attention
and shared-expert weights are 7.20 GB, and the BF16 head 1.32 GB. That is why squeezing the experts from
4.25 to 2.77 bits barely moves decode: the per-token read only falls to 11.46 GB (reasoned).
At 273 GB/s, one box reading 11.46 GB per token caps out at 23.8 tok/s. Splitting the layers across the
two boxes does not raise that for one stream, because the token still passes through both halves in
turn. Tensor parallelism reads both halves at once, which doubles the ceiling to 47.6 tok/s before any
interconnect cost (reasoned). The reported 39–41 tok/s for code sits under that ceiling only because
DSpark accepts several drafted tokens per verify pass, and code is predictable enough for it to accept
many; prose, at about 29, accepts fewer. Without speculation, expect well under the 47.6 (reasoned).
The one-box version of the question has a short answer. One Spark holds about 124.2 GB after my reserve.
Even at 2.5 bits per expert with the tables on disk, the model needs 188.9 GB (reasoned). It does not fit.
## Licence and the API side
The weights and code are MIT; the `LICENSE` file is byte-identical to September's (measured). The API
now serves `deepseek-flash` from 1017, and the legacy names `deepseek-v4-flash` and
`deepseek-v4-flash-vision-exp` are routed to it as well (reported, API docs). The thread's third post
opens the Search and Fetch APIs at \$0.75 per 1,000 successful requests each. Calls made through
server-side tools in DeepSeek Harness, Claude Code and Codex are free until 31 October 2026 at 16:00 UTC,
which the pricing page gives as 00:00 Beijing time on 1 November (reported).
If you use [DeepSeek Harness](/articles/deepseek-harness), the thread asks for v0.2.1-rc.1 or later
"for the best experience with the new model". It does not say what changed in the scaffold, and I did
not check.
## What holds
- **Same model, new weights.** Config, tokenizer, code and all 96,085 tensor shapes are identical;
every shard and every sampled tensor changed, Engram tables and vision tower included (measured).
- **Every published row went up**, by 2.0 to 11.1 points, but nine of September's rows are not
reported for 1017, the old NL2Repo figure moved from 64.0 to 65.4, and Terminal-Bench 4.0 changed
scaffold between the two columns (reported).
- **"On par with Opus 5"** is 4 wins and 7 losses, with a 13.7-point gap on HLE; on the agentic rows
the update targeted, it is fair. **"Beats GLM-5.3 across the board"** holds on all 8 shared rows;
against GPT-5.6 Sol it is 9 of 11 (reasoned from reported scores).
- **"Shorter outputs"** has no token count behind it, only a cost chart in which max-effort DeepSWE
tasks cost roughly 59% less at unchanged prices (reasoned from a chart reading).
- **"Two DGX Sparks"** fails at the shipped 510.3 GB and works with experts near 2.8 bits and the
Engram tables on NVMe, which a community build already showed on the September weights (reported);
decode is bound by the 7.2 GB of FP8 dense weights more than by the experts (reasoned).
- **Unsloth's "1-bit, 128 GB"** is two quants: 88.2% agreement is UD-IQ1_M (139.8 GB without the
Engram table), the one that fits is UD-IQ1_S (122.4 GB, 85.1%), and "accuracy" is next-token
agreement, not a benchmark (measured sizes, reported agreement).
## 19 October: 1-bit at home
Two days after the release, Unsloth posted GGUF builds of 1017 for llama.cpp: "We made 1-bit retain
~88.2% accuracy while shrinking size from 508GB to 250GB (−52%). Run on a 128GB+ RAM device." I read
the GGUF header of every shard of all nine quants by range request, without downloading any weights.
The file sizes match the ones in Unsloth's guide to the tenth of a gigabyte (measured), and every quant
keeps the Engram table in a shard of its own:
| Quant | Total | Engram table | Everything else | Routed experts | Top-1, wiki1100 / long17k |
|---|---:|---:|---:|---:|---:|
| UD-IQ1_S | 233.0 GB | IQ4_NL, 110.6 GB | 122.4 GB | 1.70 bits | 82.8% / 85.1% |
| UD-IQ1_M | 250.4 GB | IQ4_NL, 110.6 GB | 139.8 GB | 1.95 bits | 87.1% / 88.2% |
| UD-IQ2_XXS | 300.5 GB | Q5_K, 135.2 GB | 165.3 GB | 2.31 bits | 90.5% / 91.2% |
| UD-Q2_K_XL | 334.8 GB | Q5_K, 135.2 GB | 199.6 GB | 2.81 bits | 91.5% / 93.3% |
| UD-IQ3_XXS | 394.7 GB | Q6_K, 161.3 GB | 233.5 GB | 3.30 bits | 94.6% / 95.1% |
| UD-IQ3_S | 423.6 GB | Q6_K, 161.3 GB | 262.3 GB | 3.73 bits | 96.0% / 96.8% |
| UD-Q3_K_XL | 485.1 GB | MXFP8, 202.8 GB | 282.3 GB | 4.00 bits | 96.5% / 97.4% |
| UD-Q4_K_XL | 500.7 GB | MXFP8, 202.8 GB | 297.9 GB | 4.25 bits | 98.2% / 98.5% |
| UD-Q8_K_XL | 508.4 GB | MXFP8, 202.8 GB | 305.6 GB | 4.25 bits | 98.5% / 98.7% |
Sizes and bits per expert weight are measured from the headers; the agreement columns are reported by
Unsloth. Vision (`mmproj`, 0.97 GB) and the DSpark drafter (7.98 GB at Q8_0) are separate files, so
none of these totals includes them (measured).
**What "1-bit" keeps.** In UD-IQ1_M, nothing outside the routed experts is below 5 bits. Attention is
Q6_K in all 40 layers, the shared experts Q8_0 or Q5_K, the LM head Q6_K, the token embedding Q8_0, the
Engram projections Q6_K, and the routers, norms and hyper-connection mixers stay F32 (measured). The
1-bit type, IQ1_S at 1.56 bits, covers 45 of the 120 expert matrices: gate and up in the encoder half
(layers 0 to 19) and down in layers 0 to 4. That is 37.5% of the expert weights; the rest are 2.06 to
2.56-bit types (measured). The Engram table is IQ4_NL, 4.5 bits. There is no IQ1_M tensor in the file:
the name is the recipe, not the format. UD-IQ1_S pushes the 1-bit type to 73% of the expert weights and
drops 13 of the 40 attention layers to Q5_K. At the top, Q4_K_XL and Q8_K_XL keep DeepSeek's MXFP4
experts (288.78 GB) and MXFP8 Engram table (202.76 GB) as shipped, matching the checkpoint table above
(measured).
**Memory against the claim.** The 88.2% belongs to UD-IQ1_M, whose non-Engram weights are 139.8 GB,
or 130.2 GiB. That does not fit a 128 GB machine. The quant that does is UD-IQ1_S, at 122.4 GB or 114.0
GiB, and the guide says so; its agreement is 85.1% on long17k and 82.8% on wiki1100. On one DGX Spark,
with the 121.7 GiB usable I used above, that leaves about 7.7 GiB for the KV cache, compute buffers and
the page cache holding hot Engram rows (reasoned). The tweet pairs the bigger quant's score with the
smaller quant's machine. Its "−52%" is 51% by my arithmetic, against either the 508.4 GB Q8 GGUF or the
510.3 GB checkpoint (reasoned).
**What "accuracy" means.** It is top-1 agreement: how often the quant's most likely next token equals
the reference's, on two text sets. No benchmark was run on any GGUF, and the guide says its benchmark
table is DeepSeek's, "not measurements of the GGUFs in this guide." Note the ceiling too: Q8_K_XL, whose
weights are the checkpoint's, reaches 98.5% and 98.7%, so 1.3 to 1.5 points of disagreement belong to the
runtime, not the quant (reasoned). Against that ceiling, UD-IQ1_M agrees 10.5 to 11.4 points less often
than the exact weights. What that does to a long agentic coding run is not published.
**Speed.** No tok/s is published, nor anything behind "up to 3x faster than other repos". Here is the
bandwidth arithmetic instead (reasoned from the measured sizes). A decoded token reads 6 of 384 experts per
layer plus every dense weight. For UD-IQ1_S that is 1.80 GB of experts and 6.60 GB of everything else,
8.40 GB in all; for UD-IQ1_M, 8.95 GB. On a Spark's 273 GB/s that caps one stream at 32.5 and 30.5 tok/s
before any overhead, against 20.9 for the shipped precision. Experts under 2 bits cut each token's reads by only about a
third, because the dense weights dominate, as in the Spark section. A reply estimating 77 tok/s on a Spark assumed about 3.5 GB per token, which
leaves most of the dense weights out. Engram adds 48 rows of 144 bytes per token, each a 4 KB page read
from SSD when it misses the page cache. Unsloth reports 1.3 to 1.5x from DSpark; I did not check it.
On a datacenter cluster the same table is not on disk at all: [DeepInfer](/articles/deepinfer) shards
it across GPU memory in an engram-parallel group. At home it is a memory-mapped file, and the quant you can fit is the
one whose non-Engram column fits your RAM.
## How I checked
For the 17 October release I diffed the two Hugging Face repos' file listings and blob hashes, read all
48 safetensors headers of each by range request, sampled 4 KB of 22 tensors in both, read
`inference/engram.py` and `config.json`, and took every benchmark score from the two model cards and
DeepSeek's API changelog without re-running any. For the 19 October update I read Unsloth's post with its
thread and replies through the fxtwitter mirror, the Unsloth guide, and the GGUF repo's README, `engram/`
and `dspark/` notes. I listed the repo through the Hugging Face API for file sizes and parsed the GGUF
header of every shard of all nine quants (64 files) with range requests, reading tensor names, shapes and
ggml types; per-tensor bytes come from offset differences, so the per-component totals add up to the
listed file sizes. I did not download a GGUF, run llama.cpp or Unsloth's build, or reproduce the KL and
top-1 figures. Not verified: Unsloth's "3x faster", the DSpark speed-up, and any tok/s on real hardware.
## Related
- [A 200 GB MoE on one RTX 3090](/articles/big-moe-one-3090) runs the September V4.1-Flash with its
experts in DDR4 and the Engram tables memory-mapped from NVMe; the byte counts there match these.
- [DeepSeek DSpark](/articles/deepseek-dspark) explains the draft layers whose experts changed most in
1017, and why accepted length sets the speedup.
- [DeepSeek Harness](/articles/deepseek-harness) is the scaffold DeepSeek says 1017 is tuned for.
- [One small box, 64 users at once](/articles/dgx-spark-batching) covers what a single DGX Spark does
with a model that fits.
- [Kimi K3](/articles/kimi-k3) and [GLM-5.3](/articles/glm-5-3) are two of the table's comparison
columns, and [the mixture-of-experts architecture page](/architectures/mixture-of-experts) explains
routing.