# Needle 3: one checkpoint, five models, and a file that weighs more than the page says
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/needle-3
> date: 2026-09-18
> tags: edge-inference, tool-calling, quantization, model-architecture, explainer
[Cactus Needle 3](https://cactuscompute.com/needle) is one 20-layer, 121M-parameter checkpoint trained so that any depth from 2 to 20 layers — sliced out of the *same weights* — is a deployable model in its own right. That is the real headline, and it is a genuinely different claim from the two prior Needle pieces on this site: [Needle 2](/articles/needle-finetune) shipped one 45M model, and [needle-environments](/articles/needle-environments) audited the schemas you fine-tune it on. Needle 3 ships nineteen models for the storage cost of one.
This piece assumes you've read [Needle 2: the interesting number is what happens after you fine-tune it](/articles/needle-finetune) (the Simple Attention Network block, the CQ2-bit engine, the fine-tuning argument) and [Needle Environments](/articles/needle-environments) (the confidence/refusal contract, in more detail than I repeat below). Neither article covers slicing — Needle 2 shipped one fixed depth — so that part is new.
| | |
|---|---|
| Model | [Cactus Needle 3](https://cactuscompute.com/needle) · 25–121M params · Apache 2.0 |
| Sizes | **5 documented depths**: 2L/25M, 4L/29M, 8L/52M, 16L/98M, 20L/121M |
| Precision | CQ2-bit (2.125 bits/weight default; embedding and mHC gates at 4-bit) |
| Architecture | Laddered Simple Attention Network · 20 layers × 768 wide · Monarch Hadamard FFN, engram tables, 4-lane mHC, byte-level grammar |
| Data | 360B tokens, proprietary structured corpus |
| Speed | 400–4k tok/s decode, 1–10k tok/s prefill on a Raspberry Pi 5 — aggregate across all depths, not broken out per slice |
| Fine-tuning | LoRA on the frozen 20-layer base, then sliced to any depth, 4-bit `.cact` |
## The sentence Cactus put on its own banner
The tweet-length version circulating is "Needle 3 can match DeepSeek V4 Flash." Read the live page and Cactus is more careful than that themselves: the banner at the top says *"Needle 3 fine-tuned on the Cactus Platform passes DeepSeek V4 Flash from 4 layers up,"* and the body text narrows it further — *"4L can match DeepSeek V4 Flash when tuned on downstream tasks for one epoch."* That is three qualifiers the shorter claim drops: fine-tuned, not base; from 4 layers, not every layer; and on the benchmark it was tuned for, not in general. Credit where due — Cactus states this plainly on its own page, in a TIP banner nobody could miss. The looser claim is something the announcement acquired on the way out the door, not something the source says.
There's a smaller, sillier inconsistency on the same page. The hero paragraph says the whole model is *"a single 8-29 MB binary."* Scroll to the intelligence-laddering section and the same page says *"Intelligence laddering produces 9 to 29 MB CQ2-bit binaries."* Eight or nine — it's a one-word difference, it doesn't change any conclusion, and it's the kind of thing that survives when a number gets typed twice by two different people. I flag it because the bigger size claim below is not that kind of typo.
## What "sliceable" means, and why you can't just cut a transformer in half
Training one network so that a prefix of it is *also* a valid, independently useful network is not a new idea — it's the shape shared by [LayerDrop](https://arxiv.org/abs/1909.11556) (randomly drop transformer layers during training so any subset survives at inference), early-exit networks like BranchyNet and DeeBERT (attach a classifier at intermediate depths and stop once confidence is high enough), and [Matryoshka Representation Learning](https://arxiv.org/abs/2205.13147) (train an embedding so its first *k* dimensions, for every *k*, are themselves a usable lower-dimensional embedding). Needle 3 applies the same instinct to whole transformer *blocks* rather than embedding dimensions, and its specific recipe is worth separating from the family.
**Why you can't just truncate a normal transformer.** Train a 20-layer model the ordinary way and every block's function is defined relative to what the blocks before and after it do — layer 14 can specialize on the assumption that layer 6 already resolved local disambiguation and layer 18 will clean up what it leaves rough. Cut the stack at layer 8 and you don't get a smaller version of the model; you get a model whose blocks were never trained to work without the ones you removed. Blocks become **load-bearing** for specific other blocks, and naive post-hoc truncation breaks the load path. Needle's own training write-up names this directly: the ladder objective is a regularizer specifically because *"no single block can become load-bearing, and the early blocks are pushed to do real work rather than defer it."*
**How the depths are chosen.** A 20-layer Needle is blocks $f_0, \dots, f_{19}$. A depth-$d$ subnetwork keeps a subset $S_d$ of $d$ blocks, run in their original order, with the rest passed through as the identity — literally skipped, not zeroed. Two properties matter: the subsets must **nest** ($S_2 \subset S_3 \subset \cdots \subset S_{20}$) so one training run serves every depth, and each depth must stay **spread through the stack** so a shallow subnetwork still sees early and late computation rather than, say, only the first eight layers. Both properties fall out of one rule: start from the two endpoints, then repeatedly split the *widest remaining gap* at its midpoint. Twenty blocks give one fixed sequence —
```text
0, 19, 9, 14, 4, 6, 11, 16, 2, 7, 12, 17, 1, 3, 5, 8, 10, 13, 15, 18
```
— and subnetwork $d$ is that sequence's first $d$ entries. $S_2 = \{0, 19\}$: **every depth keeps the first and last block**, which is why the architecture diagram below labels them explicitly. $S_8$ is $\{0, 4, 6, 9, 11, 14, 16, 19\}$, and so on up to the full stack.
**How one run trains all nineteen depths.** The naive approach — one loss term per depth, every step — is $19\times$ the forward passes. Needle instead samples one path per optimizer step. With probability 0.8 it's an ordinary full-depth language-modeling step. Otherwise, a depth $d$ is drawn uniformly from $\{2, \dots, 19\}$, only the blocks in $S_d$ run, and the loss is
$$
\mathcal{L}_{\text{step}} = \frac{d}{2L}\Big[\mathrm{CE}(p_d) + \lambda\,\mathrm{KL}\big(\mathrm{sg}[p_L] \,\|\, p_d\big)\Big], \qquad \lambda = 1
$$
— cross-entropy at that depth, plus a KL term pulling $p_d$ toward the *frozen* full-model prediction $p_L$ (stop-gradient, so no extra backward pass beyond the teacher's own). That's self-distillation, and it's doing real work: cross-entropy alone gives a 2-layer step a fraction of a percent of the gradient budget a 20-layer step gets, and the teacher signal is what keeps the shallow rungs from being trained on noise. Because the sampling is unbiased, the expected objective over many steps is exactly the mixture a joint 19-loss-term objective would compute — at the cost of one forward pass.
The training report's own control experiment is the cleanest evidence that this changes what the blocks *learn*, not just what you can extract afterward: a ladder-trained network and an identically-configured non-ladder control, trained for the same number of steps, reached full-depth validation losses of **1.958 versus 2.037** — the ladder-trained model is *better at being a 20-layer model*, not just better at also being smaller ones. And its **16-layer exit matched the control's full 18 layers**. Asking every prefix to be useful is a regularizer that redistributes work toward the early blocks; that redistribution is precisely what naive truncation of an ordinarily-trained model never gets, because nothing during its training ever asked block 8 to be able to stand on its own.
**Where it actually degrades.** Cactus's own base-model chart (confidence-gated, the config you'd actually ship) shows four of the five depths: 11.7 → 36.8 → 80.7 → 86.0 on Mobile Actions as depth goes 4 → 8 → 16 → 20 layers. That is not a smooth curve — the jump from 8 to 16 layers is +43.9 points, more than triple the +5.3 gained going from 16 to 20. DroidCall degrades more gently: 21.0 → 36.5 → 40.0 → 47.0. Different benchmarks, different knees; the ladder doesn't buy uniform graceful degradation, it buys *a* curve you get to choose a point on.
And the 2-layer slice is missing from that chart entirely, "for an honest reason" in Cactus's own words: at 25M parameters, its confidence head withholds almost every call, so its gated score on both benchmarks is effectively zero. Forced to answer regardless — no confidence gate — it manages **0.9% on Mobile Actions** and **20.5% on DroidCall**. That's the direct answer to whether the smallest slice is "genuinely useful or just technically loadable": out of the box, it's the latter. It loads, it runs, and it does almost nothing. Fine-tuned on one product's tools, the same 25M slice reaches 66.7 on Mobile Actions and 56.5 on DroidCall — usable, though still short of DeepSeek's 60.5 on DroidCall specifically. Two layers of a generalist is a curiosity; two layers tuned on your twelve tools is a product, and that gap is the whole argument for shipping the ladder rather than just the 121M model.
## Simple Attention Networks: deleting the feed-forward layer and measuring what's lost
The block diagram names a Monarch Hadamard FFN, which sounds like Needle still has a conventional feed-forward sublayer — it's cheaper, not absent. The architecture's real premise is a separate, checkable research result Cactus published alongside the ladder: [*A Controlled Study of Attention-Only Transformers*](https://cactuscompute.com/blog/simple-attention-networks). A **Simple Attention Network (SAN)** is a decoder transformer with the feed-forward sublayer deleted from every block — nothing else changes: pre-norm, GQA, RoPE, a scalar residual gate, tied embeddings. For a fixed attention pattern, a SAN layer is *linear* in its inputs, and each head's output at a position sits inside the convex hull of the value vectors it attends to. A SAN layer can only **select and transport content already present in context** — it cannot synthesize a representation the context doesn't support.
Cactus ran the "delete the FFN and measure what's lost" experiment three ways, because deleting the FFN perturbs parameters, compute, and nonlinear depth all at once, and each control isolates one:
| Control | Configuration | Params | GFLOPs/tok | FFN's lead |
|---|---|---|---|---|
| Delete in place | 20L SAN vs. 20L transformer | 24M vs. 87M | 0.40 vs. 0.72 | **0.470 nats** |
| Match compute | 20L SAN vs. 9L transformer | 24M vs. 43M | 0.40 vs. 0.39 | **0.263 nats** |
| Match parameters | 20L SAN vs. 4L transformer | 24M vs. 24M | 0.40 vs. 0.20 | **0.006 nats** |
Delete the FFN with nothing else changed and you lose 0.47 nats — you deleted 72% of the model's parameters, so of course it's worse. Hand that parameter budget back to attention as *depth* instead, and the gap collapses to **0.006 nats, 0.27% of loss**, reproducing to one part in ten thousand across seed pairs, on a 105B-token training run. The FFN's *parameters* matter enormously; its specific *functional form*, once attention gets an equal budget, mostly doesn't.
The 0.006 nats isn't spread evenly, either. Every training document has a query, a reasoning trace, and an answer region; decomposed that way, the attention-only model is *ahead* on traces and every answer region by 105B tokens, and behind only on **query tokens** — the positions with the least context to route from, where only knowledge baked into weights (rather than present in the prompt) can help. That's the one place a purely content-transporting layer structurally can't compete, and it's a small slice of total loss (8% of tokens, a per-token gap five times the aggregate) precisely because most real text is mostly routable.
This is exactly the deficit Needle's **engram** — hashed n-gram memory, read by gather rather than matmul — exists to buy back. Cactus's own framing: *"Needle 2 was a Simple Attention Network with an engram memory bolted on for the facts that are not in context; Needle 3 keeps the attention-only stack and adds a ladder."* And it's exactly why the architecture fits tool calling specifically rather than being a general free lunch: **the function name, the argument values, and the schema are all already in the prompt.** Tool calling and structured extraction are routing problems almost by definition — select the right span, transport it into a call — which is the one regime a context-grounded, FFN-free model was never expected to be worse at.
## CQ2-bit, and a file that doesn't weigh what the page says
**Cactus Quants (CQ)** is not Q2_K or a ternary scheme, though it lands in similar bit-width territory. The mechanism: split a tensor's input dimension into groups of 128 weights, and multiply each group by a normalized Walsh-Hadamard matrix — an orthogonal rotation. Whatever a group's original weight distribution looked like, after that rotation its coordinates resemble a sample from a Gaussian, because a Hadamard transform spreads any one weight's contribution across all 128 positions in the group. That's the trick: instead of calibrating a codebook per group to whatever distribution that group happens to have (which is what schemes like Q2_K's per-block scale-and-minimum do), CQ needs only **one fixed Lloyd-Max codebook**, computed once, that works for every rotated group in the model. Activations get the same rotation at inference, which cancels against the weight-side rotation because the Hadamard matrix is orthogonal — so the matmul result is unchanged; only the storage format is.
The storage cost per weight is $b + \tfrac{1}{8}$ bits: $b$ bits per index, plus a shared 16-bit norm amortized over the 128-weight group ($16/128 = 0.125$). For CQ2, $b=2$, so **2.125 bits/weight** — not the 1.58-bit-per-weight, ternary $\{-1, 0, 1\}$ representation that schemes like BitNet use; CQ trades a slightly larger nominal budget for the codebook-sharing trick, and — per the earlier Needle 2 piece — it's trained against these quantization levels from pretraining onward rather than applied post-hoc, so the deployed 2-bit model is the one that was actually optimized, not a lossy compression of a different one.
That last part is checkable without downloading the model. A safetensors file keeps a JSON header at its front, and Needle 3's carries a `__metadata__` block describing the run that produced it. Two range requests — eight bytes for the header length, then the header — are enough to read the entire quantization-aware schedule out of a 242 MB file.
**Receipts.** The quantization-aware training run is recorded inside the checkpoint itself. Needle 3's safetensors __metadata__ carries the full schedule — step count, seed, the bit anneal, the dataset revision and its mixture weights — so the QAT description on the page can be checked against the artifact rather than taken on trust. One thing it shows that config.json does not: 31.15% of the training mixture is speech data, while config.json lists modalities as text only.
| field | value recorded in the checkpoint |
| :--- | ---: |
| stage | qapt |
| training steps | 10,000 |
| seed | 3407 |
| bits: start → end | 4.0 → 2.0 |
| anneal fraction | 0.5 |
| phase B bit scheme | embedding=4,stack/mhc=4,default=2 |
| knowledge-distillation weight | 1 |
| base checkpoint | needle3_rl.pkl |
| tokenizer hash | 9697fdd2fbb126dd8de173771e49d99d |
| dataset repo | Cactus-Compute/dataset |
| dataset revision | fd2ea1dac4b0… |
| mixture · speech-pretrain | 10.00% |
| mixture · structured-speech | 21.15% |
| mixture · structured-targeted | 13.50% |
| mixture · structured-text | 55.35% |
Mixture weights sum to 100%. The two speech sources — speech-pretrain and structured-speech — account for 31.15% of it. config.json's extras.modalities lists ["text"].
> method: Range-request the first 8 bytes for the header length, then the header itself; parse the JSON and read __metadata__. No full download needed (8,256 bytes of a 242 MB file).
> source: https://huggingface.co/Cactus-Compute/needle3/blob/main/checkpoints/needle3.safetensors
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/needle-3/data/checkpoint-provenance.json (15 rows)
The anneal is exactly what the page describes: start at 4 bits, end at 2, with the transition spread across the first half of training and a final phase that pins the embedding and mHC tensors at 4 bits — the same `embedding=4,mhc=4,default=2` scheme `config.json` ships with. Publishing the seed and the dataset revision alongside it is more than most releases bother with. The one thing the metadata contradicts is small and probably an oversight: 31.15% of the training mixture is speech data, and `config.json` lists the model's modalities as text only.
Here's where the checkable claim breaks. `config.json` states the model has **121,021,910 parameters**. At a flat CQ2-bit, that's $121{,}021{,}910 \times 2.125 \div 8 = 32{,}146{,}445$ bytes — **32.15 MB**, already past the page's 29 MB ceiling using nothing but Cactus's own published bit-cost formula and parameter count. And that's the *generous* estimate: `config.json`'s own `quantization.scheme` field reads `"embedding=4,mhc=4,default=2"` — the tied embedding table and the multi-lane hyper-connection gates are stored at **4 bits**, not the 2-bit default, because they're more precision-sensitive. So I downloaded the actual file.
`Cactus-Compute/needle3`'s `needle3.cact` on Hugging Face resolves, via its real CDN redirect (not a git-lfs pointer — the HTTP response after the redirect carries the real bytes), to a **Content-Length of 35,335,380 bytes: 35.34 MB, or 33.70 MiB.** That's roughly 22% over the page's stated ceiling in decimal-MB terms, and it's consistent with the config's own mixed-precision scheme rather than contradicting it — the embedding and mHC tensors at 4 bits explain *why* the real file lands above even the honest all-2-bit estimate. As one more cross-check: the companion `checkpoints/needle3.safetensors` (the fp16 checkpoint you fine-tune from) is 242,047,978 bytes, and that number resolves exactly rather than approximately. Strip the 8-byte length prefix and the 8,248-byte JSON header and 242,039,722 bytes of tensor payload remain; sum the 64 tensors that header describes and you get 242,039,722 — the same figure to the byte, every one of them F16. Those tensors hold 121,019,861 parameters, which is 2,049 *fewer* than `config.json`'s `total_parameters`. An overcount of 0.0017% is far too small to move anything in this piece; I record it because I measured it. The 121M figure is solid even where the file-size claim isn't.
**Receipts.** Cactus's page gives Needle 3 a range of 8 MB at 4 layers to 29 MB at 20. The 20-layer file it actually publishes is 35,335,380 bytes — 35.34 MB, 33.7 MiB — about 22% over the top of that range. The mixed-precision scheme in config.json (embedding and mHC at 4 bits, everything else at 2) explains why; the page's range reads as an all-2-bit estimate. Only the 20-layer file is published, so the smaller slices are not checkable the same way and I make no claim about them.
| artifact | bytes | MB (10^6) | MiB (2^20) | note |
| :--- | ---: | ---: | ---: | :--- |
| needle3.cact (shipped, 20L) | 35,335,380 | 35.34 | 33.7 | the deployable container the size claim is about |
| page's stated ceiling (20L) | 29,000,000 | 29 | 27.66 | cactuscompute.com/needle, "8 MB at 4 layers to 29 MB at 20" |
| checkpoints/needle3.safetensors | 242,047,978 | 242.05 | 230.83 | fp16 checkpoint you fine-tune from; not the deployed artifact |
| config.json | 1,273 | 0 | 0 | declares scheme embedding=4,mhc=4,default=2 |
The fp16 checkpoint reconciles exactly: an 8-byte length prefix plus a 8,248-byte JSON header leaves 242,039,722 bytes of tensor payload, and summing the 64 tensors in that header gives 242,039,722 bytes — the same number to the byte. Those tensors hold 121,019,861 parameters, all F16. config.json says total_parameters is 121,021,910, which is 2,049 more than the checkpoint contains (0.0017%). Too small to move any claim in the article — recorded because it was measured, not because it matters.
> method: curl -sSL -I — Content-Length of the final response after the CDN redirect, not the git-lfs pointer stub.
> source: https://huggingface.co/Cactus-Compute/needle3
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/needle-3/data/file-sizes.json (4 rows)
None of this touches the smaller slices directly — Cactus doesn't ship pre-sliced files, you generate them yourself with `needle build --layers N`, so there's no 4-layer or 8-layer `.cact` sitting on the Hub to download and check the same way. The published "8 MB at 4 layers to 29 MB at 20" range may undercount those too, by a similar mechanism, but I didn't measure them and won't claim a number I didn't check.
## "Beats DeepSeek V4 Flash" — on what, exactly
Cactus's own benchmark suite names six evaluations, split into two families reported as means: **Tools** (Mobile Actions, 961 rows, phone commands to Android intents, exact call; DroidCall, 200 rows, some requiring two calls in order, exact calls in order; BFCL v4, 3,641 rows, AST match, no call permitted on irrelevant requests) and **Extractions** (DSTC8, 1,813 dialogue turns, field F1; SNIPS gold, 700 rows, schema given; SNIPS 7-way, 700 rows, pick the schema among seven first). Base checkpoints run through the shipped CQ2-bit binary with the confidence gate on; baselines run at f16 under vLLM; DeepSeek V4 Flash runs through its cloud API. That asymmetry favors the baselines, not Needle — post-hoc 2-bit quantization would collapse models never trained for it, so keeping baselines at full precision is, if anything, generous to them. It's the same asymmetry [Needle 2's benchmarks](/articles/needle-finetune) disclosed, and Needle 3 states it again.
On this specific chart, "beats models 10x its size on mobile tool calls" holds up exactly: Needle3-20L scores 86.0 against LFM2.5 1.2B's 82.4 — a real win, at roughly a tenth the parameters and two bits against sixteen. The fuller claim narrows to something more interesting and more defensible: matching a *frontier* model requires fine-tuning, and only shows up from four layers up.
That figure is genuinely the strongest result in the release, and it holds up exactly as stated: on DroidCall, fine-tuning lifts every depth 18 to 36 points, and 4 layers (62.5) through 20 layers (70.0) all clear DeepSeek's 60.5 — the "from 4 layers up" language is precise, not rounded.
It does not hold on Mobile Actions, which the same Figure 3 shows as its second panel. There, fine-tuning lifts the smaller slices enormously — the 2-layer model goes from 0.9 to 66.7, a 65.8-point swing — but **no tuned subnetwork reaches DeepSeek's 88.4**, including the full 20-layer model, which actually scores a hair *below* its own untuned base (84.5 tuned vs. 86.7 base, both forced-call). Fine-tuning on a benchmark can very slightly hurt a slice that was already close to that benchmark's ceiling; more training data doesn't help what's already saturated, and here it cost 2.2 points. The banner claim is real, sourced, and true — on DroidCall. The same figure, one click away on the same page, shows it isn't true on Mobile Actions, and nothing on the page says so.
The extraction side gets less attention and deserves a sentence: on DSTC8 specifically, the base Needle3-20L (40.7) is beaten outright by LFM2.5 230M (53.0) — a *smaller* model — while on SNIPS gold and SNIPS 7-way it does land in the "matches 2-3x bigger models" range Cactus claims. "Extractions" is reported as a mean of three suites; the mean can be true while one of the three components isn't, and DSTC8 is the one that isn't.
**Why any of this is plausible at all** is a point this site has now made three separate times, about three unrelated products. [Parallel Constrained Decoding](/articles/parallel-constrained-decoding) works because restricting a full vocabulary distribution to a handful of legal JSON values is arithmetic you already have, once the candidate set is small. The [tiny browser models](/articles/tiny-browser-models) — a 41,321-parameter syntax highlighter, a 38,745-parameter date parser — reach 83–99% on their own narrow tasks at four to five orders of magnitude fewer parameters than a general chat model, because the *output space* is a handful of classes or a short structured value, not open-ended prose. Needle's whole scoping argument, going back to Needle 2, is the same shape at a different point on the curve: a fixed, small tool vocabulary needs no world knowledge, so a 121M model — or a 29M one, fine-tuned — can compete with a model a hundred times larger precisely because most of that larger model's capacity is spent on a problem Needle was never asked to solve. Three unrelated codebases, three unrelated size classes, the same underlying reason.
## "Empty list, not a guess" — what's actually doing the refusing
This is a real design commitment worth taking seriously, and it turns out to be enforced by three different mechanisms doing three different jobs, not one learned "abstain" class.
**The grammar itself.** *"A request no declared tool can serve returns the empty call `[]`, which is the whole contract for refusal"* — Cactus's own confidence guide is explicit that off-topic detection needs no score at all. The byte-level grammar compiled from your tool schemas apparently permits an empty `function_calls` array as one of its valid completions, so "nothing matches" isn't a confidence judgment, it's a parse that was always available.
**The confidence floor.** Every response separately carries a `confidence` — the *minimum* of a calibrated head scored over the finished call, and the raw decode probability of the call's own tokens — and the engine applies a hard floor of 0.1 beneath which a call is redirected into `suppressed_calls` rather than executed. This is the number for *"is this specific call, which the model was willing to produce, trustworthy,"* not for "was there a tool here at all."
**Rule-based grounding gates**, sitting on top of both and independent of the score: a call is withheld outright if the request negates the tool's verb, quotes a command attributed to someone else, fills a required enum with a value it never named, fills a required slot with a control word, states no quantity for a required number with no default, or reverses a source/destination pair. This is close kin to what [needle-environments](/articles/needle-environments) found from the other direction — every one of that repo's nine *critical* test cases turned out to be a refusal category (missing, negation, invalid), and the degenerate always-refuse model passed all of them. The grounding gates are the shipped, general-purpose version of exactly the guard those hand-written test cases were checking for.
**One exception that swallows all three:** a tool declared with `triggers` — regex patterns matched against the raw request — always produces a call for a matching request, skipping the confidence floor and the guess gates. Only the contradiction-style grounding checks still apply. That's a deliberate escape hatch for intents that must always reach a tool, and it means the refusal contract is *not* uniform across your own tool surface — it depends on whether you opted a given tool into pattern-matched routing.
```python
from typing import Literal
@needle.tool(triggers=[r"\b(turn|switch|power|flip)\b.*\b(on|off)\b", r"\btoggle\b"])
def control_device(device: str, action: Literal["on", "off", "toggle"]):
"Switch or toggle any named smart-home device."
return {"device": device, "action": action}
agent = needle.Needle(tools=[control_device, get_weather])
agent.complete("toggle the garage door")
# function_calls [{"name": "control_device", "arguments": {"device": "garage door", "action": "toggle"}}]
```
**What isn't published: a false-positive rate.** The confidence guide's own worked illustration — a scatter of "right" and "wrong" calls against a 0.70 threshold, showing 19 acted on with 1 wrong and 18 sent to confirmation with 6 actually right — is captioned *"illustrative calls."* It is a diagram of what a calibrated score *should* look like, not a reliability curve measured on a held-out set. BFCL v4's "no call on irrelevant" scoring is the closest thing to a real measurement, and it's folded into one aggregate accuracy number (50.2 for the base 20-layer model, below several *smaller* baselines — the same enterprise-API blind spot [Needle 2's BFCL score](/articles/needle-finetune) had) rather than broken out as its own false-positive rate. The guide adds one more honest caveat in passing: *"correct Spanish calls have been measured at 0.0,"* which is exactly the kind of number that should make you distrust any fixed threshold on a deployment that isn't English-only. Without a published reliability diagram or an isolated false-positive rate on genuinely out-of-scope requests, "calibrated" is a design intent I can confirm the *mechanism* for and not a number I can confirm the *value* of.
There's a concrete reason to want that number before trusting the mechanism completely. [GitHub issue #117](https://github.com/cactus-compute/needle/issues/117) against Needle 2 (engine 2.0.4, `cactus-needle` 2.0.13, not Needle 3 — I want to be precise about which generation this is) reported a LoRA-tuned `.cact` where JAX greedy decoding on the exported weights reproduced the trained refusal (`[]`) on 8 held-out rows, while the native engine, given bit-identical weights and near-identical first-token logits (mean absolute difference 0.04 on the base model, 0.54 tuned), emitted an actual tool call with plausible-sounding reasoning on 6 of those 8 — *"what's the capital of France?"* triggered a real `search_site` call in the engine and a correct refusal in JAX. All 393 prefix tokens matched between the two paths. The issue is closed on GitHub with no visible comment thread, so I can't tell you whether it was root-caused or went stale — only that it exists, that it is exactly the failure mode "empty list, not a guess" is supposed to prevent, and that it showed up specifically after fine-tuning, which is the workflow this whole release is built to encourage.
## The ledger
**Real and checkable.** The ladder mechanism — bisected block order, stochastic-depth sampling, self-distillation, all traced to the actual training report rather than taken on faith. The Simple Attention Network result underneath it, an honest ablation with three separate controls and a result (0.006 nats at matched parameters) that's small enough to be believable and large enough on query tokens to explain exactly why engrams exist. The DroidCall fine-tuning result: every tuned depth from 4 layers up genuinely clears DeepSeek V4 Flash, verified against Cactus's own figure.
**Narrower than advertised.** "Passes DeepSeek V4 Flash" is true on DroidCall and false on Mobile Actions, in the very figure that's cited for the claim. "Beats models 10x its size" checks out cleanly on Mobile Actions and doesn't extend to BFCL v4, where several *smaller* baselines beat the base 121M model — the same enterprise-surface gap Needle 2 had. The 2-layer slice is technically loadable and functionally inert until you fine-tune it, which "any depth from 2 to 20" doesn't advertise on its own.
**Doesn't hold up.** The 20-layer `needle3.cact` on Hugging Face is 35.34 MB, not the 29 MB the page states twice (once as "8-29 MB," once as "9-29 MB" — they can't even agree with themselves on the floor). No per-slice Raspberry Pi throughput table exists anywhere I could find, so "up to 4k tokens/sec" is an unattributed endpoint of a five-slice range, not a number you can assign to a specific depth. And no reliability diagram or false-positive rate backs "calibrated confidence" — the one figure that gestures at it says "illustrative" in its own caption.
None of that makes the ladder fake. The mechanism is real, published in enough detail to check, and the DroidCall result is a genuine frontier-matching claim from a 29M-parameter model. It's the file size and the throughput number — the two claims a reader could check without reading a single equation — that don't survive being checked.
A `needle3.cact` published by Cactus that measures at or under 29,000,000 bytes with all
twenty layers and the confidence head intact. I measured one artifact, twice, on
2026-09-18 — a repack, or a separate all-2-bit build I didn't find, would settle it. The
[file-sizes dataset](/articles/needle-3/data/file-sizes.json) has the exact bytes and the
command; anyone can re-run it in a second.
A Mobile-Actions number for the 121M checkpoint at or above 88.4% under the same harness.
My reading comes off Cactus's own Figure 2, where the 20-layer point sits at 86.0% below a
dashed 88.4% line. If that figure is superseded, or if I've misread which line is which,
the claim goes.
Any published per-slice throughput table — depth, hardware, tokens/sec, batch size. I
searched the page, the repo, the model card and the training report and found none. One
table makes this a real number instead of a range endpoint, and I'd rather have the table.
A reliability diagram, an ECE or Brier figure, or a false-positive rate on the refusal
gate, measured on a named eval set. The confidence head is real and it is in the
checkpoint — `confidence_head/probes` is right there in the safetensors header. What's
missing is evidence that its scores are calibrated rather than merely ordered, and the one
figure that gestures at it labels itself "illustrative."
This one I expect to survive, and I'd want to know if it doesn't. It would fall if the
`__metadata__` I read turns out to describe a different run than the weights beside it —
if, say, the recorded `tokenizer_hash` doesn't match the shipped tokenizer. I checked that
the header and the payload reconcile to the byte, but I did not verify the hash itself
against the tokenizer file.