# AgentJev gets both halves

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/agent-jev-06b
> date: 2026-09-22
> tags: explainer, llm, architecture, calibration, agents
[`malevrigns/agent-jev`](https://github.com/malevrigns/agent-jev) is a 0.6B
open-weights decision model with a claim list I have seen most of before. Beats
Laya on Typed Decisions, 79.25% against 77.00%. Sixty-four candidates in one
forward pass with KV prefix reuse, 92.4% of the computation removed, about 2x
faster. Sixty-eight milliseconds per decision, zero decoded tokens, no JSON to
parse. Twice Laya's context.

One item on that list is not like the others, and it is the reason this article
exists:

<Callout type="note">
**"A small permutation-equivariant head then scores the set: the order of options does not smuggle in a ranking."**
</Callout>

Every other claim needs a benchmark, a denominator and an argument about whose
hardware it ran on. That one is a claim about a *type signature*, and this site
has spent four articles building the experiment that tests it. Reverse the option
list. A per-option scalar scorer must flip exactly none of them by construction; a
vocabulary readout cannot.
[openjev's letter readout](/articles/what-decision-models-cannot-do) flips **10 of
36** reversed questions and carries a **+1.71 logit** prior for slot A. The hosted
model was measured this week flipping [**12 of
100**](/articles/jev-is-not-deterministic) under permutation.

The weights are Apache-2.0 and 1.2 GB, which is small enough to run on the four
CPU cores I have. So I ran it.

<ReversalLedger />

**The answer changed on 0 of 50 reversed questions, and on 0 of 2,520 permutations
of those same option sets. The largest movement of any single logit anywhere is
9.5e-07** — a few units in the last place of a float32. Every candidate's backbone
vector comes back *bit-identical* between the two orders. This is not a training
result and it is not a number that got small: nothing in this model has anywhere to
put a position prior, so the only thing left to measure is the width of the float.

That is a real result. Other models in this category have been measured at zero
flips on a small set; this is the first one where the invariance has been measured
at the level of the arithmetic rather than at the level of the answer. What makes
it worth an article rather than a footnote is *how* it is arranged, because it is not the
arrangement that was supposed to buy this — and because the same model's score for
an option does depend on what the other options say: a median of 0.088 logits,
where a per-option scorer moves by exactly zero.

<RepoCard repo="malevrigns/agent-jev" />

## The third arrangement

[The fork](/articles/what-decision-models-cannot-do) this site has been describing
since the cua-s1 piece is a genuine one, and I have stated it as a law more than
once: the shape does not say how options reach the scorer, and the two answers
fail in opposite directions. Isolate the options and order becomes inexpressible,
but a question spanning two options has no answer to find —
[Jev scores 0 of 100](/articles/jev-scores-zero) on exactly that. Share the
context and relational questions become answerable, but position acquires a prior.
Nobody gets both halves.

(The hosted model's 12 of 100 does not contradict that. Order-invariance is a
property of the shape; what a hosted API does to it on the way through a batched
fleet is a different question, and [the determinism
piece](/articles/jev-is-not-deterministic) takes that apart. What follows is a
model you can run yourself, where there is no fleet in the middle.)

AgentJev is arranged a third way, and it gets both halves. Here is the whole of
it.

<ThreeArrangements />

Read the serving file first, because its docstring is the clearest statement of
the design anywhere in the repository:

```python
# agent-jev · jev_service/prefix.py @ b66a8ce — the module docstring, in full
"""Inference-only shared-prefix KV branches, preserving independent causal paths.

Each branch receives a private cache. Sibling candidates never attend one
another. The trained candidate-set head runs after all branch vectors exist.
"""
```

Four sentences, and between them the whole design. The prefix is encoded once and
cached — it is whatever the candidate paths agree on, which is the state, the
question, the `[CANDIDATE]` marker and any leading words the candidates happen to
share. Each candidate is then a private branch off that cache, so no candidate is
in any other candidate's context. And every branch resumes at the same position
index, because they all continue the same prefix, so candidate 7 and candidate 40
are encoded at *identical* rotary positions. There is no slot for a prior to attach
to. That last property holds in the fallback route too, where each candidate gets
its own full sequence: the candidate text still begins at token `len(prefix)` in
every row.

```python
# agent-jev · jev_service/prefix.py @ b66a8ce — the shared branch, reflowed from its one-line-per-statement original
prefix = torch.tensor([entries[0][0][:prefix_length]], device=device)
encoded = backbone(input_ids=prefix, attention_mask=torch.ones_like(prefix), use_cache=True)
root_cache = encoded.past_key_values
for start in range(0, len(entries), engine.path_batch):
    chunk = entries[start:start + engine.path_batch]
    suffixes = [seq[prefix_length:] for seq, _ in chunk]     # candidate text only
    cache = copy.deepcopy(root_cache)
    cache.batch_repeat_interleave(len(chunk))                # one private copy per candidate
    hidden = backbone(input_ids=ids, attention_mask=attention,
                      past_key_values=cache, use_cache=True).last_hidden_state
```

`batch_repeat_interleave` is the whole trick: the cache is copied across the batch
axis, and the batch axis carries no attention. This is the same observation the
cua-s1 piece made about `score_pair_batch` — stacking sequences into one tensor
*looks* like sharing and is not — used deliberately rather than incidentally.

It is worth being precise about what the prefix sharing does and does not buy,
because the obvious reading is wrong. Sharing the prefix is an **efficiency**
device. It changes no property of the answer: the fallback route, which gives every
candidate its own full sequence, computes the same thing more slowly, and the
repository measured the two against each other and got a maximum probability
difference of 0.000508. Order-invariance does not come from the sharing.

So far this is an efficient per-option scalar scorer. The options would meet at
the softmax, four scalars long, and a relational question would be unanswerable.
The departure is one module further on:

```python
# agent-jev · agentjev/model.py @ b66a8ce
class CandidateSetEncoder(nn.Module):
    """Transformer over the candidate set of one question.

    No positional embeddings, so the mapping is permutation-equivariant:
    permuting the candidates permutes the outputs identically.
    """
```

The candidates do meet, in a two-layer transformer over the set of them, and they
meet *before* the scalar is read rather than after. A transformer with no
positional embeddings is a set function: permuting its inputs permutes its outputs
and changes nothing else. That is the property the whole article turns on, and it
is bought in the only place where cross-candidate information does not cost you a
position.

The whole scoring path is seven lines:

```python
# agent-jev · agentjev/model.py @ b66a8ce — AgentJevModel._score, annotations mine
def _score(self, cand_vecs: torch.Tensor, cand_mask: torch.Tensor) -> torch.Tensor:
    x = self.proj_in(cand_vecs)                  # [Bq, C, 1024] -> [Bq, C, 256]
    x = x * cand_mask.unsqueeze(-1).to(x.dtype)
    enc = self.set_encoder(x, cand_mask)         # candidates see each other here, and only here
    rep = cand_vecs + self.proj_out(enc)         # residual back onto the isolated vector
    logits = self.scorer(rep)                    # [Bq, C] — one scalar each
    return logits.masked_fill(~cand_mask, -1e4)
```

Note what the residual does. `cand_vecs` is the isolated per-candidate reading;
`proj_out(enc)` is the correction that knowing the rest of the set buys you. Zero
the set encoder's output and what is left is the shape of `open-jev-deberta` — a
learned scalar head over an isolated encoding, which is the per-option family
exactly. The whole departure from it is one additive term.

### The channel is real, and I can put a number on it

A residual term is easy to write and easy to have contribute nothing. So the second
half of the experiment below asks the complementary question: hold one candidate's
text fixed, change *which other candidates are on the list*, and see whether that
candidate's raw logit moves. Not its probability — that moves for free through the
softmax when the set changes size — its logit, the number the softmax is taken over.

A per-option scalar scorer must give exactly the same logit. Its scalars exist
before anything knows the set. AgentJev's move by a median of 0.088 logits
and by as much as 0.71.

<SetVsOrder />

More than five decades separate the two perturbations — 5.3 comparing medians,
5.9 comparing worst cases. That is the article in one picture: the
score depends on *what else is on the list* and not at all on *where on the list it
is*.

One control, because dropping an option changes the size of the set as well as its
contents, and a length effect would look the same. Take a candidate and two *other*
candidates, and compare the two subsets that drop one of those others. Both subsets
have the same cardinality and both still contain the candidate you are watching, so
the only difference is which one of its neighbours is present. That is a pure
content perturbation with the size held fixed: median 0.081 logits, worst
0.83, the same order of magnitude as before. It is the contents, not the count.

And the proportionate qualifier, because a tenth of a logit is not a lot: on this
sample, removing a losing option changed the winner in zero of 183 drops. The channel
is measurable and it is not, on these questions, decisive. That is roughly what you
would expect from a 256-dimensional summary competing with a 1024-dimensional
reading of the candidate itself — and it is why the open question is whether the
channel can carry a reference, not whether it exists.

### What the third arrangement costs

Two things, and neither is order.

**The cross-candidate channel is narrow.** A candidate arrives at the set encoder
as one 256-dimensional vector, projected down from the 1024-dimensional hidden
state at its last token. Option B does not get to read option D's text; it gets to
read a 256-number summary of it. So the channel exists — the bar above is not zero
— but whether it is wide enough to resolve *"ship to the address in option D"*, the
[case spelled out in the jev-scores-zero piece](/articles/jev-scores-zero), is
untested by them and by me. It is the obvious next experiment and there is a
falsifier for it at the bottom.

**Equivariance is exact in the maths and approximate in the arithmetic.** A
permuted set changes the summation order inside the attention, and floating-point
addition is not associative. The training code knows this: there is a
`permutation_invariance_kl` term that re-runs the head on a shuffled order and
penalises the divergence, with a comment saying it exists to *"guard the property
under bf16 numerics"*. Its weight is zero in the shipped protocol, which I think is
the right call — this does not need training, it needs a wide enough float. What I
measured on CPU is float32, and 9.5e-07 is what that costs. The repository's own
check ran on GPU under bf16 autocast and reported exactly `0.0` — but that is a
probability, on one three-option question, read off a head running in bf16, whose
significand is eight bits against float32's twenty-four. A difference of the size I
measured could not have shown up there. It is a smaller number on a coarser
instrument.

## The measurement, and how I got the numbers

Their check is `jev_service/verify_http.py`. It is one question, three options,
reversed once:

```python
# agent-jev · jev_service/verify_http.py @ b66a8ce — the entire permutation check
permuted['questions'][0]['options'] = dict(reversed(list(choice['questions'][0]['options'].items())))
shuffled = evaluate(permuted)
perm_error = max(abs(a[k] - b[k]) for k in a); assert perm_error < .02, perm_error
```

`permutation_max_absolute_error: 0.0`, on n = 1. True, and thin. So I ran it at
scale, on the benchmark's own data, through the shipped code, on four CPU cores.

There are two runs, and the first one licenses the second.

**Run one is strict and end-to-end.** Take a question from the Typed Decisions test
split, build the caller's payload twice — once with the option list as published,
once reversed — and push each independently through
`jev_service.contract.prepare`, `encode_paths`, the Qwen3 backbone,
`AgentJevModel._score`, the per-primitive temperature and the softmax. Nothing is
reused between the two: they are re-tokenised, re-batched and re-encoded, so the
rows of the tensor that reaches the backbone are in a different order in the two
runs. Realign the reversed answer and compare. Boolean questions are excluded,
because `contract.prepare` hardwires their candidate order and leaves the caller
nothing to reverse:

```python
# agent-jev · jev_service/contract.py @ b66a8ce — a boolean has no order for the caller to reverse
keys = ['true', 'false']
candidates = [semantic(criteria[k], f'{k} criterion') if k in criteria else k.upper() for k in keys]
```

Two things came out, and the second is the one I did not expect to be able to
state so bluntly.

**The answer never changed. 0 of 50**, Choice and Score questions with four or
five candidates. On the same test the openjev readout changes its mind on 27.8% and
the hosted model on 12%.

**The backbone vectors are bit-identical.** `max|Δv| = 0.0` on every question, on
every candidate, to the last bit. Reversing the list reverses the order of the rows
in the batch that goes into the backbone and changes not one number that comes out.
This is the load-bearing fact, and it is what makes the second run exact rather than
an approximation: the vector for candidate *j* is a function of candidate *j* and
the shared prefix, and of nothing else in the request.

**Run two is the stronger perturbation, for the price of the first.** Because the
vectors do not depend on the arrangement, scoring a *permuted* set of the same
vectors through `_score` is precisely what the service computes for a permuted
request. So: one backbone pass per question, then every permutation of the candidate
set — all 24 for a four-option question, all 120 for five — scored at the head and
realigned. 57 questions, **2,520 permutations**, zero answer changes, and
the largest realigned movement of any single logit anywhere in the sweep is
**9.5e-07**.

Two to the minus twenty-three is 1.19e-07, so that worst case is a handful of units
in the last place of a float32 mantissa, arriving in the set encoder's attention sum
and nowhere else. To put it on the scale this corpus has been building: `kev`
reports probabilities *"within 4e-6"* when the same questions are asked together or
separately in fp32, and calls that four parts in a million.

The whole test is short enough to paste. This is the core of it, against the
repository's own modules:

```python
# reverse one question and compare, using the shipped serving code
from jev_service.contract import prepare, encode_paths
from agentjev.model import AgentJevModel

def logits(state, question):                      # question is the caller's dict
    paths, _, _ = encode_paths(prepare({"state": state, "questions": [question]}), tok, 2048)
    batch = pad_into_a_batch(paths)               # ids, attention_mask, cand_end_pos, …
    vecs = model.encode_candidates(batch)         # [1, C, 1024], one per candidate
    return vecs[0], model._score(vecs, batch["cand_mask"])[0].tolist()

forward = {**q, "options": dict(pairs)}
reversed_ = {**q, "options": dict(reversed(pairs))}
vf, zf = logits(state, forward)
vr, zr = logits(state, reversed_)

assert (vf - vr.flip(0)).abs().max() == 0          # the backbone never saw the order
print(max(abs(a - b) for a, b in zip(zf, reversed(zr))))
```

The `assert` is the line worth running on any model in this category. It passes
here.

**Receipts.** Reversing a question's option list does not change AgentJev's answer, and neither does any other permutation of it. The answer changed on 0 of 50 reversed questions and on 0 of 2,520 permutations. The candidate vectors that come out of the backbone are bit-identical between the two orders, so the residual movement — worst case 9.5e-07 logits — is a few units in the last place of a float32, arriving inside the head's attention sum. The same head moves a candidate's logit by a median of 0.088 when you change which OTHER options are on the list. Position carries nothing; composition carries a lot.

| perturbation | n | measured | a per-option scorer gives |
| :--- | ---: | ---: | :--- |
| reverse the list — answer changed | 50 | 0 | 0, by construction |
| reverse the list — worst \|Δ\| on a backbone vector | 50 | 0.0 | 0, exactly |
| reverse the list — worst \|Δlogit\| | 50 | 4.8e-07 | 0, exactly |
| reverse the list — worst \|Δprobability\| | 50 | 6.0e-08 | 0, exactly |
| every permutation — answer changed | 2,520 | 0 | 0, by construction |
| every permutation — worst \|Δlogit\| | 2,520 | 9.5e-07 | 0, exactly |
| drop an option — median \|Δlogit\| on a survivor | 780 | 0.088 | 0, exactly |
| drop an option — worst \|Δlogit\| on a survivor | 780 | 0.71 | 0, exactly |
| swap which neighbour is present, set size fixed — median \|Δlogit\| | 900 | 0.081 | 0, exactly |

The end-to-end run is 50 questions because a 0.6B model in float32 on four contended CPU cores is a minute or two per reversed pair; the permutation sweep is large because it reuses one backbone pass per question. Cardinality is 4 and 5 only, which is what this benchmark contains — the contract accepts 255 and I have not tested there. Nor have I tested bf16, which is what the GPU service runs and whose significand is eight bits against float32’s twenty-four. The questions are a seeded shuffle of the Choice and Score half of the split, so the draw is random but it is not stratified: top-1 agreement with the teacher on this subsample is 0.720 against 0.7925 on the full 2,000, which at this N is within sampling noise and is not a claim that the slice is representative.

> method: Weights from aimeigaoshou/agent-jev, code from malevrigns/agent-jev at b66a8ce, float32 on CPU — the dtype jev_service/engine.py uses when --device is not cuda. Per question: build the caller's payload twice, once as published and once with the option list reversed, and push each independently through jev_service.contract.prepare, encode_paths, the Qwen3 backbone, AgentJevModel._score, the per-primitive temperature from temperatures.json and the softmax. Compare the two candidate-vector tensors. Because they come back bit-identical, scoring a permuted set of the forward vectors through _score is exactly what the service computes for a permuted request, so every permutation — 24 at four candidates, 120 at five — was then scored at the head and realigned. Leave-one-out drops each candidate in turn and re-scores the survivors; the size-controlled variant compares two different leave-one-out subsets that both contain the candidate being watched. Boolean questions are excluded: contract.prepare hardwires their candidate order to ['true', 'false'], so a caller has nothing to reverse.
> source: https://huggingface.co/aimeigaoshou/agent-jev
> captured: 2026-09-22
> data: https://ai.thesatyajit.com/articles/agent-jev-06b/data/order-invariance.json (9 rows)

<Callout type="warning">
**Denominators.** One benchmark, one CPU, float32. The end-to-end run is small — 50 questions — because a 0.6B model in float32 on four cores shared with whatever else is on the box is a couple of minutes per reversed pair, and the permutation sweep is large only because it reuses one backbone pass each. What makes me willing to state the null flatly anyway is that it is not a statistical null. A transformer with no positional embeddings is a set function; the backbone deltas are exactly zero; the residual is at the float's last bit rather than at any scale a prior could live at. If a permutation produced a flip, the explanation would have to be a bug, not a bias. Two things I have *not* run. bf16, which is the dtype the GPU service uses and where the repo's own `permutation_invariance_kl` guard says it was worried. And any cardinality above five — which matters more than it sounds, because `path_batch` is 16, so a question with more than sixteen candidates gets split into microbatches in the order the caller sent them. At four and five candidates there is one microbatch and that channel is closed. At 64 it is open on every request.
</Callout>

## 598,418,689 parameters, and 2.4 million of them are the model

"0.6B" is a claim, so I pulled the safetensors header for
`aimeigaoshou/agent-jev` with range requests and summed the shapes. 343 tensors,
all BF16, **598,418,689 parameters**. The README's own latency table says 598M, so
the claim and the artifact agree. That is the same check
[DiffusionGemma's 25.2B needed](/articles/any-model-can-be-jev) and it costs the
same five minutes; it is worth doing every time, and it is worth saying when a
project passes it.

The split is the interesting part:

| block | parameters |
|---|---:|
| `path_encoder.backbone` — Qwen3-0.6B, no LM head | 596,049,920 |
| `set_encoder` — 2-layer transformer, d=256 | 1,579,520 |
| `scorer` — RMSNorm → 1024 → 256 → SiLU → 256 → 1 | 263,681 |
| `proj_out` — 256 → 1024 | 263,168 |
| `proj_in` — 1024 → 256 | 262,400 |
| **decision head, total** | **2,368,769** |

Everything that makes this a decision model rather than a language model is
2,368,769 parameters — 0.4% of the checkpoint. The cross-candidate channel accounts
for 1,579,520 of them, and the order-invariance accounts for none: it is a
positional embedding table that was never created. For scale, the
[whole of cua-s1-forms](/articles/cua-s1-forms) is 706,048 parameters, so this head
is about three and a third cua-s1s bolted to a Qwen3.

<ModelCard repo="aimeigaoshou/agent-jev" claimed="0.6B" note="Weights only: a flat safetensors state dict with no modelling code, which is why the README has you wrap it in a torch checkpoint before the server will load it. AutoModelForCausalLM will not open this file — the keys are prefixed path_encoder.backbone, and there is no LM head to find." />

One detail the README's *"language-model head removed"* glosses. Qwen3-0.6B sets
`tie_word_embeddings: true`, so the LM head is the embedding matrix transposed.
Removing it frees exactly **zero** parameters. The saving is that no logit vector
151,936 wide is ever materialised, which is real and is a saving of compute and
bandwidth, not of weights.

## The option ceiling that is not there

This corpus has now found an option count welded into an export twice:
[`logits[batch_size, 25]`](/articles/jev-in-the-browser) in the only open ONNX
export, and [`K = 32`](/articles/laya-mlx) frozen into a Core ML bundle along with
batch and sequence length. A third would make it a pattern, so I went looking.

It is not there — and part of the reason is that there is nothing here to freeze it
into. Both earlier ceilings were properties of an *export*: a graph compiled to a
fixed-shape target. AgentJev ships a PyTorch state dict and a Python server. No
ONNX, no Core ML, no traced graph, so no shape got baked.

**Receipts.** Nothing in AgentJev's weights knows how many options a question has. Every limit the model appears to have is a `raise ValueError` in one 90-line file, `jev_service/contract.py`, plus one argparse default. The 64-candidate load the README times is a workload, not a ceiling — the contract accepts 255, the same number TypeSafe's Choice accepts, and the scoring head has no parameter that depends on the count at all.

| limit | value | enforced by | in the weights? |
| :--- | ---: | :--- | :--- |
| choice candidates | 2–255 | contract.py — `if not 2 <= len(keys) <= 255` | no |
| score levels | 2–10 | contract.py — `if not 2 <= len(levels) <= 10` | no |
| boolean candidates | exactly 2 | contract.py — `keys = ['true', 'false']` | no |
| candidate paths per call | 1024 | contract.py — `total_paths > 1024` | no |
| questions per call | 128 | contract.py — `total_questions > 128` | no |
| states per call | 1–32 | contract.py — `not 1 <= len(requests) <= 32` | no |
| tokens per candidate path | 2048 | server.py — `--max-tokens` default | no — config says 32,768 |
| candidates per backbone microbatch | 16 | engine.py — `path_batch=16` | no |

Two open decision models have now been found with an option count welded into an export — N=25 in an ONNX graph, K=32 in a Core ML bundle. This is the third one I have looked for it in and the first where it is genuinely absent, so 'decision models cap out at N options' remains a claim about particular files rather than about the shape. The one bound that is not in this file is the training distribution: the typed-decisions fine-tune saw questions with 2, 4 and 5 candidates and nothing wider, so 255 is what the contract accepts, not what the checkpoint has been shown.

> method: Read every bound in jev_service/contract.py at commit HEAD of github.com/malevrigns/agent-jev, then checked each against the weights. Parameter shapes come from the safetensors header of aimeigaoshou/agent-jev, pulled with HTTP range requests and summed: the scorer ends in `scorer.fc2.weight [1, 256]` and the candidate-set transformer carries no positional embedding table, so neither has a dimension indexed by the option count. The 2,048-token path limit is `--max-tokens`, an argparse default on jev_service/server.py; the published config.json declares max_position_embeddings 32,768.
> source: https://github.com/malevrigns/agent-jev
> captured: 2026-09-22
> data: https://ai.thesatyajit.com/articles/agent-jev-06b/data/ceilings.json (8 rows)

The scorer ends in `scorer.fc2.weight [1, 256]` — output dimension one, evaluated
once per candidate, the same *"N is a loop bound, not a weight"* property the
cua-s1 piece drew. And the set encoder, which is the one module that does see the
whole set at once, carries **no positional embedding table at all**; that is the
same absence that buys the order-invariance. A transformer with no positions has
no maximum length.

So every limit AgentJev appears to have is a `raise ValueError` in
`jev_service/contract.py`, plus one argparse default. The 64 candidates in the
headline is a workload, not a ceiling; the contract takes 255, which is the number
TypeSafe's own Choice takes. Even the 2,048-token context is
`--max-tokens`, against a published `config.json` that declares
`max_position_embeddings: 32768`. Doubling Laya's 1,024 is a real operational
difference and it is a launch flag, not an architecture.

The one bound that is *not* in that file is the training distribution. The
typed-decisions fine-tune saw questions with 2, 4 and 5 candidates and nothing
wider. 255 is what the contract accepts, not what the checkpoint has been shown.

## 92.4% of the tokens, 51% of the clock

The efficiency claim is the one with the most committed evidence behind it and the
most room to be misread, so I rebuilt it.

<Figure
  src="/articles/agent-jev-06b/fig1.png"
  alt="The project's own comparison panel. On the left, standard inference with no prefix cache: 610 ms for 64 options, 33,547 backbone token operations, state computed 64 times, redundant computation 92.4 percent. On the right, AgentJev shared prefix with key-value reuse: 299 ms marked 2.04 times, 2,551 backbone token operations marked minus 92 percent, state computed once across 64 options, numerical parity with a maximum difference of 0.000508 and the same choices. A footer claims evaluation of up to 255 candidates without linear slowdown."
  caption="The repository's own figure for the efficiency claim, and every number on it except two is in its committed prefix_verification.json. The two that are not: the state is labelled 1,500 tokens and is 472, and no committed measurement backs the 255-candidate footer — the only 255-option run in the repo executed with prefix sharing switched off. (malevrigns/agent-jev, assets/agentjev_shared_prefix.gif.)"
/>

`jev_service/verify_prefix.py` builds one fixed request — 64 Choice options and
one Boolean over a repeated state — and times it under all three encoder modes,
median of three after warmup. I re-ran its tokenisation and got the repository's
numbers exactly: 66 candidate paths, **33,547** path tokens, a 492-token shared
prefix on the Choice question and 493 on the Boolean. Sixty-four candidates paying
492 tokens each for the same state is the whole of the redundancy, and removing it
is the whole of the 92.4%.

<TokenClock />

So: **"92.4% computation reduced" is measured in backbone input tokens**, and it
is exact. What it buys on the clock is 2.04x. The gap between those two ratios is
the interesting engineering, and the middle row of that table is the proof that it
is real rather than a rounding: forcing every question to share its prefix feeds
the backbone 19% *fewer* tokens than the automatic mode and takes 17% *longer*.

The reason is that the branched pass stops being compute-bound. `auto` sends 2,551
tokens through a 0.6B model — about 2.2 TFLOP, which is nothing — but it does so as
four sequential microbatches of sixteen rows and roughly sixteen tokens each, after
copying a 492-token KV cache sixteen times per microbatch. Twenty-eight layers of
kernel launches over a batch that is 256 tokens wide does not fill a GPU. The
`auto` heuristic is one line and it is the right line:

```python
# agent-jev · jev_service/prefix.py @ b66a8ce — share the prefix only where it pays
use_shared = mode == 'shared' or (mode == 'auto' and len(entries) >= 8 and prefix_length >= 128)
```

Fewer than eight candidates, or a prefix under 128 tokens, and it falls back to
independent paths.

While I am in that loop: *"64 candidates in one forward pass"* is the one phrase in
the claim list that is not literally true. `path_batch` is 16, so a 64-candidate
question is one prefill of the prefix plus four batched branch passes — five
backbone invocations, not one. What happens once is the *encoding of the state*,
which is the part that matters and the part the 92.4% measures. It is a small
thing, and it is also exactly the four sequential microbatches that keep the
speedup at 2.04x instead of something larger.

That eight-candidate threshold is why the repo's own 255-candidate smoke test
reports `shared_prefix_questions: 0`: its state was short, so nothing was cached,
and 10,090 tokens went through the backbone at 684 ms. The figure's *"up to 255
candidates without linear slowdown"* is the one line on it that no committed
measurement supports.

Worth saying what the sharing does *not* cost: the maximum probability difference
against the independent-path route is **0.000508** and the chosen option does not
change. That is a checked claim in a category where people usually assert parity.

## The 68 ms is in a GIF

The per-decision latency figure is the one number in the claim list with no file
behind it. It comes from `assets/agentjev_reflex_demo.gif`.

<Figure
  src="/articles/agent-jev-06b/fig2.png"
  alt="The project's own reflex demo panel. On the left, standard LLM autoregressive 27B plus: latency 3600 ms, generated tokens 485, showing a thinking trace and a JSON object with a reasoning field. On the right, AgentJev-0.6B parallel reflex: latency single forward 68 ms, generated tokens 0, showing a direct probability distribution over four actions with read_failed_test at 54.3 percent. A summary bar reads: AgentJev-0.6B delivers 50x faster latency, 68 ms versus 3.6 s, 0 output tokens, zero JSON syntax failures."
  caption="Where the 68 ms comes from. The baseline is an unnamed 27B-plus model emitting 485 tokens on unstated hardware, so the 50x is a token-count ratio of the kind this site took apart for a 63x claim — a fair comparison of two ways to get an answer, not a speedup of the same computation. (malevrigns/agent-jev, assets/agentjev_reflex_demo.gif.)"
/>

The README's own latency table is more careful than its GIF, and it is worth
noticing which way the rounding runs:

| load | Laya (421M) | AgentJev (598M) |
|---|---|---|
| P50, 5 questions over 1 state | **41.53 ms** | ~60–70 ms |
| P90, same | **47.14 ms** | ~85 ms |

Laya's figures are measured to two decimals. AgentJev's are a tilde and a range,
and they are the *slower* of the two. That is a project rounding against itself,
which is the opposite of the usual direction and part of why I trust the rest of
the table. Note also the unit: 60–70 ms is per **case**, five questions over one
state, so the per-decision figure is nearer 13 ms — while the GIF's 68 ms is for a
single four-option Choice. Same magnitude, different workloads, and neither of
them is in a committed artifact the way every other number in this repository is.

## The benchmark, and the line its own card draws

Typed Decisions is [`LocalLLaMA/typed-decisions`](https://huggingface.co/datasets/LocalLLaMA/typed-decisions):
Apache-2.0, public, 400 test cases and 2,000 questions across four workflows,
five typed questions over one shared state. It is independent — its card says so
outright, *"not affiliated with TypeSafe and it does not reproduce their Jev
model"* — and it is the best-documented benchmark in this category, because it
publishes its own ceilings.

Who made it, I cannot tell you beyond the handle. The dataset is owned by a Hugging
Face account called `LocalLLaMA`, uploaded on 2026-09-16, with no paper, no author
list and no repository behind it; it has 2,599 downloads and eleven likes. That is
worth stating rather than glossing, because two open releases are now being ranked
against each other on it. What it does have is a build procedure written down in
full, four reference rows, two ceilings, and a Jev measurement with a date, a
request count and a bill attached — which is more provenance than most benchmarks
with an author list.

**Receipts.** The Typed Decisions table in AgentJev's README is three different measurements stacked in one grid, and the README says so. Two rows were run in this repository, five were copied from the dataset card. The Brier column survives that mixing — recomputing it here with AgentJev's own code reproduces the card's uniform baseline to three decimals. The ECE column does not: the same code on the same uniform predictor returns 0.046 where the card's table says 0.169, so the 0.1687 that sits next to Jev's 0.144 is not the same statistic.

| table row | kind | top-1 | measured by |
| :--- | :--- | ---: | :--- |
| AgentJev-0.6B | specialist, fitted here | 79.25% | this repo — typed_decisions/experiment.py, 400 cases |
| Laya, published checkpoint | specialist, fitted by its authors | 77.00% | this repo — laya-typed-decisions @ f9ab0b2, 19.2 s for 2,000 questions |
| TypeSafe Jev 1.13.0 | generalist, zero-shot | 72.7% | the dataset card — TypeSafe API, 2026-09-18, all 2,000 decisions, $0.016 |
| ModernBERT-base 149M | specialist | 64.6% | the dataset card — Adaptive Classifier 0.2.0, frozen encoder |
| MiniLM-L6 22M | specialist | 58.7% | the dataset card — same harness |
| Prior (label frequency) | reference | 47.0% | the dataset card — reads nothing |
| Uniform | reference | 30.8% | the dataset card — 1/C on every option |
| Teacher self-agreement | ceiling, not in the README | 73.5% | the dataset card — a fresh teacher sample against gold built from the others |
| Perfect scenario understanding | ceiling, not in the README | 70.4% | the dataset card — the latent factors fitted to gold, cross-validated |

Uniform is the check that settles it, because 1/C on every option leaves nothing to a fitting choice. Its Brier comes out at 0.2382 against the card's 0.238. Its ECE comes out at 0.046 with a first-index tie-break and 0.015 with a random one; no bin count between 10 and 20 and no soft/hard accuracy variant I tried gets within a factor of three of the card's 0.169. My prior is a reconstruction rather than the card's own, so its 0.1978 against 0.189 is the looser of the two checks.

> method: Rebuilt the 400-case / 2,000-question test split from LocalLLaMA/typed-decisions at revision ea93064, using typed_decisions/prepare_data.py's own convert() and jev_service.contract.prepare(), and checked that every question id and gold target matches the committed test_calibrated_predictions.json. Then ran typed_decisions/experiment.py's metrics() over a uniform predictor (1/C on every option) and over a per-question label-frequency prior fitted on the train split, and compared against the dataset card's reference rows.
> source: https://huggingface.co/datasets/LocalLLaMA/typed-decisions
> captured: 2026-09-22
> data: https://ai.thesatyajit.com/articles/agent-jev-06b/data/provenance.json (9 rows)

Two rows of AgentJev's table were measured in AgentJev's repository. The Laya row
is a real re-measurement — `convaiinnovations/laya-typed-decisions` at a pinned
revision, all 2,000 questions in 19.2 seconds — not a quoted figure, which is
better practice than the three-for-three pattern the
[jev-scores-zero piece](/articles/jev-scores-zero) catalogued. The Jev row is
quoted, but from a card that ran the model itself: all 400 cases through the
TypeSafe API on 2026-09-18, p50 710 ms per case, \$0.016 total. So the 72.7% is a
measurement by a third party with no stake in either specialist.

Now the part the README leaves out.

<CeilingLadder />

The card publishes a saturation point as well as a floor. A fresh teacher sample,
scored against gold built from the other samples, agrees **73.5%** of the time. A
model that recovers the latent factors each case was generated from exactly scores
**70.4%**. And the card states the reading in its own words:

<Callout type="warning">
**"Read 0.52 as the floor. Around 0.70 is strong. Around 0.75 is saturation."** And, two paragraphs earlier: *"A score much above 0.75 means a model has learned the teacher's quirks rather than the task."*
</Callout>

AgentJev scores 79.25%. Laya scores 77.00%. Both are above the noise floor of the
labelling process, and the gap between them — 2.25 points, bootstrap interval
[+0.65, +3.90] — sits entirely in the region the benchmark's author says is
teacher idiosyncrasy rather than task. Neither project mentions this. AgentJev's
README is careful in every other respect about what its number is —
*"agreement with the public teacher argmax… not a measured coding-agent success
rate"*, *"beating a row here does not mean a pull request merged"* — and the one
sentence it does not carry is the one the card wrote for it.

The README's own by-workflow table sharpens it further, and I do not think anyone
has done this arithmetic. The four workflows are 500 questions each, so the overall
delta is their mean:

| workflow | AgentJev | Laya | delta |
|---|---:|---:|---:|
| Invoice processing | 86.20% | 81.20% | **+5.0** |
| Customer service | 82.20% | 76.40% | **+5.8** |
| Security incidents | 76.80% | 77.60% | −0.8 |
| Agent-trace observability | 71.80% | 72.80% | −1.0 |
| overall | 79.25% | 77.00% | +2.25 |

The whole margin is two workflows. On the other two, the 421M encoder wins. A
result that is +5 here and −1 there is not a model that is 2.25 points better; it
is two models that are better at different things, averaged.

This is not a reason to disbelieve the result. A specialist fitted on 960 cases
from four workflows, evaluated on 400 cases from the same four workflows, *should*
learn the teacher. The hygiene on both sides is good — better than the
usual. The benchmark generates its two splits in separate runs at different seeds
and refuses to package if a case id or a state hash appears in both. AgentJev splits
by case id, reserves 120 development and 120 calibration cases out of the 1,200,
picks the checkpoint on development soft cross-entropy before the test split is
opened, and pins the seed and the dataset revision. It is a reason to read +2.25 points over Laya as what it is, which
is two specialists agreeing with the same teacher to within its own sampling noise.

The training run, incidentally, is 600 steps and **841 seconds**. Fourteen minutes
on one GPU takes the phase-4 checkpoint from 38.70% to 79.25%, which says rather
more about how much of this task is in the four workflows than about the
architecture.

## Calibration: published, unusual, and not what the number looks like

The [cua-s1 article](/articles/cua-s1-forms) had to be corrected for saying nobody
published a calibration number when two projects had, so I went looking carefully
here before saying anything.

It publishes plenty. Expected calibration error, sum-of-candidates Brier, soft
cross-entropy and score MAE, per primitive and per workflow, before and after
temperature scaling, for its own checkpoint *and* for the Laya checkpoint it
re-measured, with all 2,000 per-question probability vectors committed. The
temperature is one positive scalar per primitive, fitted on 180 boolean, 180 choice
and 240 score questions from cases reserved for exactly that and used for nothing
else. Only Bespoke's thirteen-dataset public suite goes further, and nobody in this
category has better calibration *hygiene* than a split that reserves 120 cases for
temperature fitting before the test set is opened. That is the protocol
[the RLCD piece](/articles/rlcd-calibrated-decisions) asked for, done without being
asked.

### The part nobody else in this category has

Before the numbers, the thing I did not expect to find in the training code.
`agentjev/synth.py` is a generator of decision questions whose gold distribution is
**exactly computable**: biased coins with a Beta prior, loaded dice, urn draws,
card draws, small Markov chains, Bayesian updates. Not a teacher's opinion of the
probability — the probability.

```python
# agent-jev · agentjev/synth.py @ b66a8ce — gen_coin, with the state string and four
# of its six questions elided
a0, b0 = rng.randint(1, 4), rng.randint(1, 4)
p = rng.betavariate(a0, b0)          # the world's actual bias, drawn from the prior
n = rng.randint(0, 20)
k = sum(1 for _ in range(n) if rng.random() < p)   # heads observed
K = rng.randint(1, k + 6)            # how many heads the goal needs in total
R = rng.randint(0, 12)               # flips left
post_a, post_b = a0 + k, b0 + (n - k)
if k >= K:
    p_goal = 1.0
else:
    # future flips share one unknown p, so this is the Beta-binomial posterior predictive
    p_goal = beta_binom_tail_at_least(K - k, R, post_a, post_b)

qs.append(_q("stop", "boolean",
             f"If we keep flipping for the remaining {R} flip(s), will the goal be reached?",
             BOOL, {"distribution": _round_dist([p_goal, 1 - p_goal])},
             "known_distribution", 1.0))
```

And `agentjev/losses.py` gives that label a supervision *type*, with a weight
attached to how much the label is worth believing: `known_distribution` and
`deterministic` at 1.0, `binomial_counts` and `multiclass_counts` scored as a
count NLL with the weight scaling in the number of trials, and
`empirical`/`heuristic`/`teacher` at 0.1–0.3. A teacher's distribution is
explicitly the *cheapest* kind of supervision in this loss.

That is the most serious answer to this category's standing complaint that I have
read. Every project here returns a probability; the criticism, restated at least
four times on this site and once by `kev` in its own model card, is that the number
is a statistic about a distribution rather than a verified frequency. Training on a
closed-form posterior is the obvious fix, and this is the first repository I have
seen do it.

Two honest limits. The phase checkpoints and the training configs are gitignored,
so I can read the generator and the loss but I cannot verify what mixture was
actually used or at what weight. And the generator's world is arithmetic — coins,
urns, Markov chains — so whether calibration learned on *"will this reach five heads
in three flips"* survives the trip to *"should this invoice be paid"* is precisely
the open question. The answer visible in this repository is: partly. The phase-4
checkpoint that came out of that pipeline is the one over-confident row in the whole
comparison, and it is 38.70% accurate.

Two things about the numbers themselves.

**The Brier column is comparable and the ECE column is not.** The README says
plainly that the Jev, ModernBERT and MiniLM rows are copied from the dataset card
and *"use the card's definitions"*. So I checked whether the definitions agree, by
running AgentJev's own `metrics()` over a uniform predictor — 1/C on every option,
nothing fitted, nothing to disagree about — on the same 2,000 questions. Brier
comes out at **0.2382** against the card's 0.238: the same statistic, so AgentJev's
0.0448 really does sit on the same scale as Jev's 0.148. ECE comes out at
**0.046** against the card's **0.169**. No bin count between 10 and 20, and no
hard/soft accuracy variant I tried, gets within a factor of three. The 0.1687 next
to Jev's 0.144 is two different statistics in one column.

**And the model is under-confident, not over-confident.** Both releases committed
their per-question probabilities and neither drew the curve, so here it is.

<Reliability />

Every populated bin of both models sits *above* the diagonal. AgentJev's mean
stated confidence is 0.624 and it is right 0.793 of the time; Laya's is 0.553
against 0.770. Neither is over-confident anywhere. That inverts the warning this
category usually needs — `kev`'s model card putting *"a `confidence: 0.92` from
this model is a statistic about its own distribution"* on the wall — because a
threshold at 0.5 here escalates decisions the model gets right 78% of the time.

There is a clean reason, and it is in the benchmark rather than the model. The
training target is a teacher *distribution*, the mean of three samples at
temperature 0.7, and the mean top mass of that gold across the test split is
**0.659**. A model trained on soft cross-entropy against those targets reports
0.624. It is reproducing the teacher's spread almost exactly. The 0.169 is
measuring the gap between the teacher's spread and the teacher's argmax, which is a
property of the labels, not a miscalibration of the model.

Two consequences follow, and both are visible in the repository's own tables.

Temperature scaling made the ECE **worse**: 0.1573 uncalibrated, 0.1687 after
fitting. The fitted temperatures are all above 1, so they soften a model that was
already too soft. This is not an error — the temperature was fitted to minimise
soft cross-entropy, which it does, from 0.85064 to 0.84943. It is the correct move
if you consume the distribution and the wrong move if you threshold the top-1, and
the two objectives genuinely disagree here.

And the best ECE in AgentJev's whole table belongs to the checkpoint it replaced.
Phase 4, before the fine-tune, scores **0.1050** — against the calibrated model's
0.1687 — while being 38.70% accurate instead of 79.25%, and it is the only row that
is over-confident. The dataset card made the same point about its own Prior
baseline: *"Prior also has the best ECE on the table, at 0.088, while knowing
nothing… That is the clearest argument for reading KL and Brier here instead of
ECE."* AgentJev's own table contains the demonstration, one row below the headline.

One more caveat on the Brier, since it is the column that survives. The loss is
soft cross-entropy **plus 0.1 times sum-of-candidates Brier**, so the model is
optimised directly on the metric it is reported against, on a distribution drawn
from the same generator. That is not a hidden fact — `protocol.json` states it —
but 0.0448 against Jev's 0.148 is an in-domain model trained on the scoring rule
against a cold model that was not.

## What the repository does that I wish more of them did

A note on evidence, because this is the most thoroughly self-audited release in
this category and the contrast with its own GIFs is instructive.

`verification.json` records the HTTP smoke tests with their failures spelled out:
question isolation moves an answer by up to **0.00282** under bf16, and the file
says so rather than claiming bit-exactness. `serving_verification.json` checks the
HTTP path against the offline evaluation path on 4 cases and 20 questions and gets
a maximum probability difference of **1.07e-07**. `selection.json` records that the
checkpoint was chosen before the test split was opened. `transfer_verified.json` is
ten SHA-256 comparisons. The Chinese serving notes go further than any English
README I have read this month — my translations, so read them as such: Phase 5 was
abandoned because *"policy value did not improve and the stall judgment
collapsed"*, the weights loaded are *"not a random head or a 27B pretending to be
Jev"*, and the unimplemented `TreeEncoder` is explicitly *"not falsely claimed as a
completed training speedup"* — which is true, and which I only noticed because they
said it.

One thing those files do not say loudly enough. Both `verification.json` and
`prefix_verification.json` were produced against `runs/phase4/final.pt`, not the
typed-decisions checkpoint on Hugging Face — you can see it in the `"model":
"phase4"` field of every response they record. The architecture and the tensor
shapes are identical so the timings carry over, but the 2.04x and the 92.4% were
measured on a different set of weights from the ones you download.
`serving_verification.json` is the one that used `agentjev_v1`.

That last one is worth pulling out, because it is a real limitation stated
plainly. The shared-prefix runtime exists in *serving* only. In training, every
`(state, question, candidate)` triple is still an independent sequence that
recomputes the prefix, and `TreeEncoder.forward` raises `NotImplementedError`. The
92.4% is an inference property.

The shipped integration is a Claude Code `PreToolUse` hook: `agentjev_hook.py`
posts the tool payload to the local server for `Bash`, `Write` and `Edit`, asks one
boolean and one four-level score, and blocks only when the score is level 3 *and*
the boolean says unsafe. It fails open on any error, which is the right posture for
a gate in front of your own tools and also means the gate is advisory. It is a
demo, and it has the tell — the audit log path is hardcoded to somebody's
`C:\Users\…` directory, inside a `try`/`except pass`.

## So what is actually new here

Four things, in order of how much I think they matter.

**A decision model can have both halves, and the place to buy them is the head.**
The fork I described as a law is a law about the *backbone*. Isolate the candidates
there — which the shared prefix does more cheaply than anyone else in this category
— and then let them meet in a function that has no positions. Position carries
nothing, composition carries 0.088 logits at the median, and both properties
come out of the same 2.4M parameters. What I still do not know is whether that
channel is wide enough to carry a *reference* rather than a comparison, and that is
now the most interesting open question in this category, because it is the one the
0/100 was supposed to have settled architecturally.

**Order-invariance is now measured somewhere, not just claimed.** Zero flips across
2,520 permutations, with bit-identical backbone vectors and residuals at
float32 epsilon. The hosted model the category is named after was measured this week
at 12 of 100. An open 0.6B that anyone can download has the property the closed one
is described as having.

**Somebody trained on probabilities that are actually true.** `synth.py` builds
decision questions out of coins, urns and Markov chains whose gold distribution is
a closed-form posterior, and `losses.py` weights that supervision at 1.0 while
weighting a teacher's distribution at 0.1–0.3. Every project in this category
returns a probability and every one of them has been asked what it means. This is
the first answer to that question that is not a temperature fitted after the fact.
Whether it transfers out of the arithmetic is unsettled, and the phase-4 row says
not completely.

**Everything else on the claim list is a real number with a denominator you have
to read.** 92.4% is tokens and buys 2.04x. 79.25% is above the benchmark's own
saturation line. 0.0448 Brier is optimised for and in-domain. 68 ms is a GIF,
against an unnamed 27B. None of that is dishonest — almost all of it is disclosed
somewhere in the repository, and this repository discloses more than most — but the
claim list and the evidence live in different files, and the claim list is the one
that travels.

<ChangeMyMind>

<Falsifier claim="AgentJev's answer is exactly invariant to the order of the option list.">
Zero flips across 2,520 permutations of 57 questions, worst movement 9.5e-07 logits, in float32 at cardinality 4 and 5. Two things would break it. Cardinality: run the sweep at 16, 64 and 255 options, where the set encoder's attention has far more terms to sum in a different order. Precision: run it under `torch.autocast(bfloat16)`, which is what the GPU service does and whose significand is eight bits against float32's twenty-four — the repository's own `permutation_invariance_kl` term exists because someone was worried about exactly that. I can name the place a real order dependence would live, and my test never reached it. `path_batch` is 16, so a question with more than sixteen candidates is split into microbatches *in the order the caller sent them*, and each microbatch is padded to its own longest suffix. Reverse a 64-option list and candidate 3 moves from a chunk with one set of neighbours to a chunk with another, which changes the padding it sits next to and therefore the reduction order in its attention. Below seventeen candidates there is one chunk and this cannot happen; above it, it happens on every request. The sweep at 64 and 255 options is the experiment, and it is the one that would actually find something.
</Falsifier>

<Falsifier claim="The cross-candidate channel is wide enough to answer a relational question.">
I have shown the channel exists — dropping an option moves a surviving option's raw logit by a median of 0.088 logits, where a per-option scorer would move it by exactly zero — and I have not shown it carries a *reference*. Those are different claims and the gap between them is the most interesting open question in this category, because the 0/100 on relational choice was supposed to have settled it architecturally. Build the "ship to the address in option D" case from the jev-scores-zero piece at n = 100, in AgentJev's own request schema, and score it. A per-option scorer gets 0 by construction. If AgentJev also gets 0, the channel is real and too narrow to carry a pointer — a 256-number summary of an option is not its text — which would be the most informative negative result available here. If it scores materially above chance, the fork this site has been describing for four articles is not a fork at all.
</Falsifier>

<Falsifier claim="79.25% on Typed Decisions is above the benchmark's own saturation point, so the margin over Laya is teacher idiosyncrasy.">
The dataset card puts teacher self-agreement at 0.735 and says a score much above 0.75 means the teacher's quirks have been learned. Both specialists are above it. The experiment that settles what the margin means is a transfer run: score AgentJev and Laya on typed decisions from a workflow neither was fitted on — a fifth generator, or Bespoke's contrastive holdout, or `kev`'s 764-record out-of-domain set. If the 2.25-point gap survives out of domain, it is a model difference and I am being too cautious. If it collapses or inverts, it was the teacher.
</Falsifier>

<Falsifier claim="AgentJev's ECE of 0.1687 is not the same statistic as the 0.144 printed next to it.">
Run the dataset card's own scorer over a uniform predictor on the test split and publish the number. AgentJev's `metrics()` gives 0.046 for that predictor; the card's table says 0.169. One of those is wrong about the other, and a uniform predictor has no free parameters to argue about. If the card's 0.169 reproduces under some binning I did not try, the ECE column is comparable after all and this objection dissolves.
</Falsifier>

<Falsifier claim="Training on closed-form posteriors is what makes this model's probabilities mean something.">
I found the generator and the supervision weights; I did not find an ablation, and the repository does not publish the phase checkpoints or the configs, so I cannot tell how much of the mixture those questions were. The experiment is cheap and nobody in this category has run it: train the same head twice from the same base, once on the synthetic exact-posterior families and once on a teacher-labelled set of the same size, and report reliability on a third, held-out domain with real outcome labels. If the exact-posterior run is no better calibrated out of domain, then closed-form supervision buys arithmetic rather than honesty and the argument above is decoration. I would genuinely like to be wrong about this one, because the alternative is that the fix is this easy and nobody had bothered.
</Falsifier>

<Falsifier claim="Removing 92.4% of the backbone tokens buys 2.04x because the branched pass stops being compute-bound.">
That is my reading of somebody else's GPU, which I have not touched. Profile it: report the time split between the prefix prefill, the `copy.deepcopy` plus `batch_repeat_interleave` of the KV cache, and the branch forward, at 8, 64 and 255 candidates. If the cache copy is a small fraction and the branch passes dominate, my explanation is wrong and the ceiling is elsewhere. Either way the fix is the same and the repository already knows it: `path_batch` is 16, so a 64-candidate question is four sequential microbatches when it could be one.
</Falsifier>

</ChangeMyMind>