# RLCD is not constrained decoding
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/rlcd-calibrated-decisions
> date: 2026-09-18
> tags: explainer, llm, training, calibration, architecture
TypeSafe's launch post makes exactly three technical claims, in one sentence, and it is worth reading the sentence slowly because the community has since collapsed all three into one: **"We built a new stack entirely focused on automation: with a new model architecture, parallel sampler for maximum efficiency, and training method we call Reinforcement Learning for Calibrated Decisions (RLCD)."**
Three things. A new architecture. A parallel sampler. A training method. The sampler is the part everyone reproduced — it is arithmetic on logits any decoder already computes, it needs no training to exist, and it is covered end to end, with runnable code, in [Parallel constrained decoding](/articles/parallel-constrained-decoding). The product claims around it are audited in [Jev's receipts, itemized](/articles/jev-system-one-models), and the downstream repos in [What the Jev ecosystem actually built](/articles/jev-ecosystem). None of those three is this article. **This one is about the architecture and the training** — what a System One model is as an artifact, what RLCD is disclosed to be, what the objective would have to be to produce a calibrated decision model, and how you would actually train one.
Start with the name, because the shorthand is wrong. RLCD is widely read as "reinforcement learning for constrained decoding," which would make it a name for the sampler. It is not. TypeSafe's own AI primer expands it in a card, in bold: **"Reinforcement learning for calibrated decisions."** Not decoding — *decisions*. The launch post's comparison table says the same thing in the "Optimizes for" row: **"Calibrated decisions: answers with epistemically honest probabilities on System One tasks."** The acronym names the *objective*, not the mechanism, and once you see that, the interesting question stops being "how does the fast thing work" and becomes "what were the probabilities trained against."
## What is actually published about RLCD
All of it. This is the complete specification, from the [AI primer](https://docs.typesafe.ai/introduction/machine-learning-primer):
**"RLCD optimizes for a different output contract:** the model does not generate text; it returns decisions and probabilities; higher probability should correspond to a greater chance that the answer is correct." Plus the operational gloss: outcomes assigned `0.2` should occur about 20% of the time, `0.8` about 80%, `1.0` all the time — and "these rates describe groups of predictions, not a guarantee about any single answer."
That is a description of what the output must satisfy. It is not a method. There is no loss function, no reward signal, no dataset, no optimizer, no hyperparameter, no ablation, and no measurement. I did not want to assert that from a skim, so I searched the whole corpus: `docs.typesafe.ai/llms-full.txt` is 835,504 bytes and contains the string `RLCD` seven times, all seven in the primer page or the feature card that links to it. Case-insensitive counts across that file: `loss function` 0, `Brier` 0, `expected calibration` 0, `reliability diagram` 0, `proper scoring` 0, `cross-entropy` 0, `fine-tun` 0, `checkpoint` 0. `ECE` as a whole word: 0 — all 48 case-insensitive matches are inside ordinary words like "piece". The same search across the launch post, `typesafe.ai/manifesto`, the `antibenchmaxxing` post and `evals.typesafe.ai` — TypeSafe's dedicated evaluation site, the one place you would put a calibration curve — returns the same thing. The single uppercase `ECE` on the eval site is inside the string `NEW RECEIPT`.
### The acronym collides with a published method
Worth naming before going further, because it makes searching for the method actively misleading. **RLCD was already taken.** [Yang et al., 2023](https://arxiv.org/abs/2307.12950) — Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng and Yuandong Tian, at Berkeley and Meta — published *RLCD: Reinforcement Learning from Contrastive Distillation for Language Model Alignment*, with [a reference implementation from Meta](https://github.com/facebookresearch/RLCD). It builds preference pairs by prompting a model twice, once with a prompt encouraging a principle and once encouraging its violation, trains a preference model on the resulting pairs, and RL-tunes against it. It is an RLHF-family method with a paper, code and ablations.
So the string has three referents now, and only one of them is documented:
| RLCD | what it is | published? |
|---|---|---|
| Reinforcement Learning from Contrastive Distillation | Yang et al. 2023, alignment from contrastive prompt pairs | paper, code, ablations |
| Reinforcement learning for calibrated decisions | TypeSafe's, the subject of this article | three sentences of output contract |
| "constrained decoding" | the community's guess, from watching the sampler | not a thing anyone claimed |
The middle one is the only one that is TypeSafe's, and it is the only one of the three with no method attached. If you go looking for "how is RLCD trained" you will find Yang et al. — a real, reproducible method for something else entirely — and it is easy to come away believing you have found the answer.
The one place the docs go near the training data, TypeSafe declines on the record. The launch post's FAQ has an entry titled "Where does our training data come from?", collapsed into an accordion so it does not show in a casual read. Its answer, in full: **"TypeSafe is primarily a data research lab, which is how the biggest results in AI get made. We make all the data ourselves. We wouldn't train on your data even if you asked us to (no offense). We do some pretty sophisticated stuff, but if you want to find out more, we'd have to hire you."**
That is a clear, honest refusal, and I would rather report it as a refusal than paper over it. An article about an undisclosed training method must not invent the method. So: everything below is marked. **Reported** means TypeSafe or a named repository says it. **Measured** means I ran it or read it out of a committed artifact. **Reasoned** means I am arguing from first principles and could be wrong, and there is a falsifier for it at the end.
## The architecture, from the only evidence there is
The FAQ has one more collapsed entry, "Is Jev just a smaller LLM?", and its answer is the entire published description of Jev's architecture: **"Jev is neither small nor an LLM, hence being off the intelligence Pareto curve."** Fourteen words. No parameter count, no layer count, no family, no context window on the models page, no architecture diagram anywhere on the site. That sentence is worth holding onto for a different reason too — it is TypeSafe asserting that Jev is a *large* model, which is the claim openjev's untrained 4B result presses hardest against.
What the docs *do* disclose, scattered across the primitive pages and the jaggedness page, turns out to be a lot more architecturally informative than the architecture answer:
- A **Choice** takes up to **255 options**; a **Score** takes **2 to 10** ordered levels; a **Noul** is one probability.
- **Score levels are scored independently.** In TypeSafe's words: *"Every level is evaluated separately. The model doesn't see a level's number or its neighbours, so 'worse than the previous level' means nothing to it, and numbers in the descriptions or the instructions don't help."* The ordering is imposed by your array index, client-side, not learned.
- **High-cardinality Choices run two stages.** From the launch post's Wikiracing note: *"Jev supports a cardinality up to 255. For the higher cardinality choices, we do a 2 stage-system of scoring independently then making an explicit choice, hence the occasional slowdown."*
- **Two context limits, not one:** *"64k tokens together for all `state` and `questions`"* and *"32k tokens for the `state` + the longest `question`."* The docs flag this as unusual themselves — *"the way to think about its context length is a bit different from other models."*
- **`score` can land between levels** (`1.30`, `1.035`, `1.06`), and `confidence` is *"a statistic computed from the probability distribution"* whose formula TypeSafe explicitly declines to give: *"The pros and cons of different computations is a specialized topic that we'll keep to a separate cookbook rather than this page."*
Take the two context limits first, because they are the clearest architectural tell on the whole site. A normal decoder has one limit: the sequence length. Two limits only make sense if the state and a question are processed as *one* sequence per question — so the longest single sequence the model ever sees is state plus the longest question, which is the 32k number and therefore the real context window — while the 64k is a *work* budget over the whole request, which is only a separate quantity if the state is encoded once and shared across question lanes rather than re-read per question. **Reasoned:** Jev's actual context window is 32k; the 64k is a billing-and-scheduling budget that exists because the state is amortised. That is the same shared-prefix structure the PCD article describes, visible from the outside through nothing but two numbers on a failure-modes page.
Now the per-option scoring. "Every level is evaluated separately" and "scoring independently then making an explicit choice" are not how a vocabulary softmax works. In the reproductions, a candidate's probability comes from its token's row in one 151,936-wide logit vector; the candidates compete inside a single normalisation, automatically, because they are entries in the same row. You could not score one of them "separately" if you wanted to. A model that scores each option on its own and then normalises is a different object: a scalar scorer applied once per (state, question, option) triple. **Reasoned:** Jev is in that second family, not the vocabulary-readout family every open reproduction of the *sampler* is in. That is also the cleanest reading of why a 255-option Choice needs a second stage and a 3-option one does not — 255 independent scorings is a real amount of work, and picking among the survivors is a second pass.
### What is in a decision model's checkpoint
TypeSafe ships no checkpoint, so I went and read four that do exist. Every shape below comes from the actual safetensors header, pulled with an HTTP range request — eight bytes of little-endian length prefix, then the JSON header — so nothing here is a model-card claim.
**Receipts.** Four open reproductions of the Jev shape, four structurally different checkpoints. Every parameter count and tensor shape below is read from the actual safetensors header over an HTTP range request, not from the model card. The last row is Jev itself, for which the corresponding cells are the finding.
| checkpoint | what the checkpoint adds | params | output space | loss | published ECE |
| :--- | :--- | ---: | :--- | :--- | :--- |
| com-kotobalabs/open-jev-deberta-v3-large | head.safetensors: Linear[1024,3072] + Linear[1,1024] | 3,147,777 | 1 scalar per option, softmax within the question | CE + Brier | 0.022 in-domain (n=3,508) / 0.035 OOD (n=4,012) |
| heman10x/rlcd-modernbert-151m | text + class projectors (2×768→768→768) + logit_scale | 151,378,177 total, no option-shaped tensor | dot product per option description; nothing option-shaped | CE + 1.0 × Brier | 0.0352 → 0.0335 after temperature (n=1,000) |
| DavidHatley/system-one-mini | 5 heads: Linear[768,768] → Linear[k,768], k = 2,2,5,5,2 | 69,328,144 total | five fixed questions, fixed option sets | not stated; checkpoint selected by validation mean NLL | 0.1273 raw → 0.2673 after per-head temperature (final renderer) |
| Foodoo1/Qwen3-14B-RLCD-Decision-LoRA | LoRA r=16 α=32 on 7 modules; modules_to_save: null | 0 new output params | one vocabulary row, sliced to candidate tokens | CE on the single decision token only | none reported |
| TypeSafe Jev 1.13 | not disclosed | not disclosed | ≤255 Choice options / 2–10 Score levels / 1 Noul, per request | not disclosed | none published anywhere |
The parameter sums cross-check two of the cards: heman10x's train_manifest.json claims 151,378,177 parameters and the header sums to exactly that; DavidHatley's card says 69.3M and the header sums to 69,328,144. Nobody publishes a calibration number for Jev, so its ECE cell is empty in the sense of not existing, not in the sense of my not having found it — see the article for the search. Each row's backbone and training run stay in the committed JSON and in the prose; the table carries the six columns the argument turns on.
> method: For each repo: GET bytes 0-7 of the safetensors file, read the 8-byte little-endian header length, GET the header JSON, sum the products of every tensor's shape. No weights were downloaded. Training and metric cells are quoted from each repo's committed README / open_jev_config.json / train_manifest.json / adapter_config.json, with its own stated n where it states one.
> source: https://huggingface.co/com-kotobalabs/open-jev-deberta-v3-large
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/rlcd-calibrated-decisions/data/checkpoints.json (5 rows)
Four repositories, four genuinely different answers to "what does this model's probability spread over":
**Nothing new at all.** `Foodoo1/Qwen3-14B-RLCD-Decision-LoRA` is a QLoRA adapter on Qwen3-14B with `"modules_to_save": null` — no head is saved because there is no head. The output space is the tokenizer's vocabulary at one position, sliced to the candidates' first tokens. This is the family the PCD article covers.
**A scalar scorer.** `com-kotobalabs/open-jev-deberta-v3-large` keeps the DeBERTa-v3-large backbone in one file and puts its decision head in a second, `head.safetensors`, which contains exactly four tensors: `0.weight [1024, 3072]`, `0.bias [1024]`, `2.weight [1, 1024]`, `2.bias [1]`. That is 3,147,777 parameters and it is a 3072 → 1024 → **1** MLP. One output. Per option. The 3072 is three concatenated 1024-wide vectors: the mean of the question's text tokens, the mean of the option's text tokens, and their elementwise product. Softmax happens afterwards, across the options belonging to one question. Its own `schema.py` states the design principle better than I can:
```python
# typed_decisions/schema.py — the whole Jev shape in three sentences
"""Every kind is the same thing to a model: a softmax over an option list.
`score` differs only in how the distribution is read out; `noul` is a 2-option
choice whose read-out is p(options[1]). That is the whole point of the Jev shape
and why one head serves all three."""
```
**A bi-encoder.** `heman10x/rlcd-modernbert-151m` is GLiClass over ModernBERT: the header has `model.text_projector` and `model.classes_projector`, each a 768→768→768 MLP, plus a single scalar `model.logit_scale`, and nothing whose shape depends on the option count. Its card advertises "25 candidate slots"; that number lives in `bundle_manifest.json` as `"max_capacity_logits": 25`, an ONNX export shape, not a weight. The header sums to 151,378,177 parameters, which is exactly the number its `train_manifest.json` claims — a small cross-check that the card was written from the artifact.
**The schema, welded in.** `DavidHatley/system-one-mini` has five heads whose final layers are `[2, 768]`, `[2, 768]`, `[5, 768]`, `[5, 768]`, `[2, 768]`. The answer space is frozen at training time; the card says so plainly ("The model learns a fixed schema and exact option sets, not arbitrary typed questions"). This is the design that is *not* Jev-shaped, and including it makes the contrast legible: everything that is interesting about a System One model comes from the answer set being chosen by the caller at request time, and that requirement is what forces the head to be one-scalar-per-option rather than K-wide.
### Two formulas TypeSafe does not publish, recovered from its own examples
The response contract has two derived numbers — `score` and `confidence` — and the docs define neither. `score` is described as "the probability-weighted answer across the levels"; `confidence` as "derived from probabilities", with the explicit note that they are keeping the formula for a cookbook they have not written yet.
Both are recoverable, because TypeSafe publishes worked request/response pairs on the primitive reference pages. I parsed every JSON answer object in the docs dump carrying both a distribution and a derived value, and checked two candidate formulas:
$$
\text{score} = \sum_{i=0}^{N-1} i \cdot p_i
\qquad\qquad
\text{confidence} = \frac{p_{\max} - 1/N}{1 - 1/N}
$$
The first is the expectation of the level index — that one the docs almost give away, since the Score page walks the arithmetic once (`0 × 0.0 + 1 × 0.70 + 2 × 0.30 = 1.30`). It holds on **6 of 6** published Score answers, exactly. The second is the top probability rescaled so a uniform distribution reads 0 and a point mass reads 1, and it is not stated anywhere. It holds on **every example on `/primitives/choice` and `/primitives/score`**, to within half a rounding step of the published two-decimal probabilities.
**Receipts.** TypeSafe documents that `score` is "probability-weighted" and that `confidence` is "derived from probabilities", but publishes neither formula. Both are recoverable from its own worked examples: score is exactly the expectation of the level index under the returned distribution (6 of 6 published Score answers, to the last digit), and confidence is the top probability rescaled so that a uniform distribution reads 0 and a point mass reads 1 — (p_max - 1/N) / (1 - 1/N) — on every example the two primitive reference pages publish. The three examples that do not fit are all on other pages.
| docs page | type | N | published probabilities | published value | recomputed | delta | conf fits |
| :--- | :--- | ---: | :--- | :--- | :--- | ---: | :--- |
| /api | choice | 3 | 0.08, 0.85, 0.07 | conf 0.82 | 0.775 | 0.045 | NO |
| /api | score | 3 | 0.05, 0.3, 0.65 | conf 0.78; score 1.6 | 0.475; 1.6 | 0.305; 0.000 | NO |
| /introduction/quickstart | choice | 3 | 0.84, 0.159, 0.001 | conf 0.596 | 0.760 | 0.164 | NO |
| /primitives/choice | choice | 3 | 0, 1, 0 | conf 1 | 1.000 | 0.000 | YES |
| /primitives/choice | choice | 3 | 0.02, 0.38, 0.6 | conf 0.39 | 0.400 | 0.010 | YES |
| /primitives/choice | choice | 5 | 1, 0, 0, 0, 0 | conf 1 | 1.000 | 0.000 | YES |
| /primitives/choice | choice | 5 | 0.63, 0.37, 0, 0, 0 | conf 0.53 | 0.537 | 0.007 | YES |
| /primitives/choice | choice | 4 | 0.1, 0.37, 0.24, 0.29 | conf 0.16 | 0.160 | 0.000 | YES |
| /primitives/choice | choice | 3 | 0.08, 0.92, 0 | conf 0.88 | 0.880 | 0.000 | YES |
| /primitives/choice | choice | 2 | 0, 1 | conf 1 | 1.000 | 0.000 | YES |
| /primitives/score | score | 3 | 0, 0.7, 0.3 | conf 0.54; score 1.3 | 0.550; 1.3 | 0.010; 0.000 | YES |
| /primitives/score | score | 3 | 0, 0.76, 0.24 | conf 0.63; score 1.24 | 0.640; 1.24 | 0.010; 0.000 | YES |
| /primitives/score | score | 3 | 0, 0.55, 0.45 | conf 0.33; score 1.45 | 0.325; 1.45 | 0.005; 0.000 | YES |
| /primitives/score | score | 4 | 0, 0, 0, 1 | conf 1; score 3 | 1.000; 3 | 0.000; 0.000 | YES |
| /primitives/score | score | 3 | 0, 0.94, 0.06 | conf 0.91; score 1.06 | 0.910; 1.06 | 0.000; 0.000 | YES |
Every row from /primitives/choice and /primitives/score fits. All three misses are on /api (two) and /introduction/quickstart (one). The quickstart miss is the interesting one: its confidence of 0.596 on a 3-option distribution of (0.84, 0.159, 0.001) matches normalised entropy confidence, 1 - H(p)/ln 3 = 0.594, which no other example does. So the docs appear to carry two different confidence conventions rather than one convention and a typo. I have no API key, so I cannot check which one the live endpoint returns; that is the falsifier.
> method: Full text of docs.typesafe.ai pulled as llms-full.txt (835,504 bytes, fetched 2026-09-18). Every JSON answer object carrying both `probabilities` and `confidence`, or both `probabilities` and `score`, was extracted by parser, not by hand. delta is |published - recomputed|; published probabilities are rounded to 2 decimals on most pages, so half a rounding step is 0.005 and the fit threshold used is 0.011.
> source: https://docs.typesafe.ai/llms-full.txt
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/rlcd-calibrated-decisions/data/output-contract.json (15 rows)
Three examples do not fit, and they are all on other pages — two on `/api`, one on `/introduction/quickstart`. The quickstart one is the interesting failure rather than a typo: its three-decimal distribution `(0.84, 0.159, 0.001)` is published with `confidence: 0.596`, and normalised entropy confidence, $1 - H(p)/\ln 3$, gives 0.594. No other example in the docs matches that formula and this one matches it to two thousandths. **Measured:** the docs carry two different confidence conventions. **Reasoned:** the primitive pages are the current one and the quickstart page is stale, because the primitive pages are where the captured responses live. I have no API key, so I cannot settle it; that is a falsifier below.
This matters more than a documentation nit, because [confidence-gated routing](https://docs.typesafe.ai/patterns/confidence-routing) is TypeSafe's own headline pattern and every threshold anyone tunes is a threshold on this number. Under the rescaled-max formula a two-option Noul-like Choice at 75/25 reads `confidence 0.50`; under normalised entropy the same distribution reads `0.19`. Those are different production systems.
One smaller loose end while I was in there. The `usage.output_tokens` field — which everyone noticed is odd, since the model emits no text — does not scale with anything in TypeSafe's own published examples. A single Noul question reports 48; a two-Noul request reports 39. A five-question, twenty-option Choice request reports 212; a three-question, ten-level Score request reports 43. Whatever it counts, it is not questions and it is not options, and the docs never say. It costs nothing, so nobody has to care; it is just not evidence of anything.
## What the objective would have to be
Here the reporting runs out and the reasoning starts. The question is narrow: what class of training objective produces a model whose reported probabilities track the frequency of being right?
The answer is not mysterious. It is a **proper scoring rule** — a loss over a reported distribution that is uniquely minimised, in expectation, by reporting your true belief. Log loss (cross-entropy) and the Brier score are the two everyone uses. The property is worth seeing rather than being told:
Slide the belief anywhere. The two proper rules bottom out on the dashed line every time; the linear rule — "average distance from the outcome", which sounds perfectly reasonable — always pays more for claiming certainty than for telling the truth. That is the mechanism by which an objective *creates* overconfidence, and it is not a metaphor for what RLHF does, it is a precise miniature of it: a preference reward is a function of which answer a rater liked, not of how well the probability was placed, so it has no term that penalises saying 0.99 when you mean 0.7.
So the family is settled and it is old. Gneiting and Raftery's proper-scoring-rules framing is from 2007; Guo et al.'s [On Calibration of Modern Neural Networks](https://arxiv.org/abs/1706.04599) (arXiv 1706.04599, 2017) is the paper that established that a deep classifier trained on cross-entropy still ends up overconfident and that one fitted scalar temperature mostly fixes it. Every open reproduction in this article that reports a calibration number uses one or both, because there is nothing else to use.
Which leaves the genuinely interesting question: **where does the RL come in?**
A per-example proper scoring rule is supervised learning. It needs a labelled example and a gradient; it does not need a policy, a rollout, a reward model, or an advantage estimate. If RLCD were only "train the decision head on cross-entropy plus Brier," the name would be false advertising, and TypeSafe would have picked a name that did not invite comparison to the two methods either side of it on their own diagram. Three readings survive contact with what is published. I find the first two technically substantive; the third is uncomfortable and I cannot dismiss it either.
**One: calibration is a property of a group, and a per-example loss cannot see a group.** ECE is defined over a *set* of predictions — bin them by confidence, compare each bin's mean confidence to its accuracy. No single example has an ECE. TypeSafe says this themselves, twice, in almost the same words ("Calibration is measured across groups of predictions; it does not guarantee that an individual answer is correct"). A reward defined over a batch — group-level ECE, or a reliability gap — is not differentiable per example and is exactly the shape a policy-gradient method eats happily. That is a real technical reason to reach for RL machinery, and it is the reason I would bet on if I had to pick one.
**Two: "optimized against outcomes."** The System One concept page has a phrase the primer does not: *"their probabilities are optimized against outcomes to reflect uncertainty."* Outcomes, not labels. If the target is the thing that happened downstream — the ticket did get refunded, the account was fraudulent, the escalation was warranted — then the label arrives late, from the world, attached to a decision the model already made. That is a bandit, not a supervised dataset, and RL is the standard way to learn from it. It also fits the data FAQ's "we make all the data ourselves" better than a hand-annotation story does.
**Three: it may substantially be lineage.** RLHF and RLVR are both named for their reward source. Naming the third branch for its *objective* rather than its reward source is a slight category error that the diagram papers over, and a company whose cofounder is on the InstructGPT author list has an obvious reason to place its method as the third entry in that sequence. This is the uncharitable reading and I hold it loosely; it is also the one that best explains why the acronym is being misread as "constrained decoding" by half the people repeating it.
On that cofounder claim, since it is load-bearing for "why should you believe these people can do this": the docs state RLHF "was [co-invented by Diogo Almeida](https://scholar.google.com/citations?user=0T4y07QAAAAJ&hl=en), cofounder of TypeSafe," and link his Scholar profile as the evidence. I checked the profile. He is the **fourth of twenty authors** on *Training language models to follow instructions with human feedback* (Ouyang et al., NeurIPS 2022, arXiv 2203.02155, 31,667 citations), and a credited author on the GPT-4 Technical Report and the "Introducing ChatGPT" post. He is **not** an author of *Deep reinforcement learning from human preferences* (Christiano et al., arXiv 1706.03741, 2017), which is the paper that introduced RLHF. So "co-invented RLHF" overstates it; "co-authored the paper that made RLHF the standard post-training recipe for chat models" is exactly right, and is what the founder himself writes in the launch post: *"At OpenAI, I helped build the methods that made language models useful at following instructions and talking with people."* The founder's own wording is the accurate one. The docs page is the one that stretched.
## The uncomfortable part: base models are already calibrated
If RLCD is a new algorithm for producing calibrated probabilities over a constrained answer set, it is worth asking what the starting point is, because the starting point is not zero.
That is Anthropic's 2022 result, and the right-hand panel is the one to sit with: expected calibration error on multiple-choice BIG Bench falls from roughly 0.14 to roughly 0.018 as the model grows, while the flat grey line — the same models answering in a free-form format — never improves at any scale. The paper's own framing: *"language models can produce well-calibrated probabilities when they are asked to choose the correct answer from among several explicit options."* Constraining the answer set is most of what makes calibration achievable; it is not a consequence of a new RL algorithm, it is a consequence of the question having a finite answer set.
And then the other half of the story, from a paper Diogo Almeida is himself credited on:
Put those two figures next to each other and the honest description of the problem RLCD solves changes shape. Pretraining gives you a calibrated chooser. Preference optimization destroys it — by an order of magnitude, measured, published by the lab that did it. **Reasoned:** a large part of "a new training method for calibrated decisions" is *not doing the thing that breaks calibration*, plus a proper scoring rule to sharpen what pretraining already put there. That is a real and valuable engineering decision. It is a smaller claim than a new branch of post-training, and TypeSafe's own diagram, which draws RLCD as a sibling of RLHF rather than as a path that declines RLHF, makes it look bigger than it is.
## How I would train one
Enough reasoning. Here is the recipe, and where an open reproduction has already run it I report what it got rather than what I think it would get.
**Base model.** An encoder is a defensible choice and the measured evidence is stronger than I expected. `kotoba-lang/typed-decisions` swept five encoders and separately LoRA-tuned a 7B diffusion LM on the same corpus. Its DeBERTa-v3-large run reaches 0.855 accuracy over 3,508 test questions at 42 ms end-to-end for ten questions; the LLaDA-MoE-7B-A1B run reaches 0.835 at 676 ms. Two points behind at sixteen times the latency, on one H100, with every run report committed — the comparison is theirs, not mine. If the model never generates text, the causal mask is pure overhead.
**Head.** One scalar per option, with the softmax applied within a question's option group. Do not put the score on a fresh marker token — this is the single most useful negative result in the whole corpus, and it is stated as an ablation rather than a hunch:
`typed_decisions/encoder.py`, in its own docstring: scoring a fresh `[OPT]` marker's hidden state **"does not learn"** in one epoch of 3,000 states, *"at any learning rate (1e-5…1e-4), head lr (3e-5, 1e-3), Brier weight (0, 1, 3), autocast on/off, sdpa/eager, `reference_compile` on/off, on base and on large: every run ends at the label prior."* The same loop overfits 16 states to loss 0.000 in 50 steps, so the mechanics were fine. Reading the **mean of the option's own text tokens** instead makes it learn immediately. A fresh token has no pretrained structure to score; the option's words do.
**Data shape.** One state, many questions, gold as an index into the option list. The cheapest honest corpus is public datasets re-shaped into typed questions — `typed-decisions` builds 18,000 states and 42,000 questions out of banking77, SST-5 and BoolQ, taking the gold from each dataset's own label, with no teacher model and no synthetic answers. Total cost of the run that produced the published checkpoint: 229 seconds on one H100, about \$0.25.
**Loss.** Cross-entropy plus Brier on the same softmax. Both are proper; adding them is the standard cheap way to get log loss's gradient behaviour with Brier's boundedness. The whole objective is a dozen lines:
```python
# typed_decisions/encoder.py — the entire training objective
def decision_loss(logits, gold, brier_weight: float = 1.0):
"""logits (B, Qm, Om) with -inf on padded options; gold (B, Qm) with -100 on padded questions."""
valid = gold >= 0
lg = logits[valid] # (n, Om)
g = gold[valid]
ce = F.cross_entropy(lg, g)
if brier_weight > 0:
p = lg.softmax(-1)
onehot = F.one_hot(g, lg.size(-1)).to(p.dtype)
finite = torch.isfinite(lg)
brier = (((p - onehot) ** 2) * finite).sum(-1).mean()
else:
brier = torch.zeros((), device=lg.device)
return ce + brier_weight * brier, {"ce": ce.item(), "brier": brier.item(), "n": int(valid.sum())}
```
`heman10x/rlcd-modernbert-151m` independently arrives at the identical objective and writes it into its `train_manifest.json` as `"loss_function": "CE + 1.0 * Brier (Proper Scoring Rule)"`. Two unconnected people reproducing the same shape reached for the same loss, which is what you would expect, because there is not much else to reach for.
**Select the checkpoint on a proper scoring rule, not on accuracy.** This is the one place the theory pays off visibly, and `heman10x`'s committed training history happens to contain a perfect worked example of it:
| epoch | val accuracy | val NLL | val Brier |
|---|---|---|---|
| 1 | 0.912 | 0.2698 | 0.1365 |
| **2 (selected)** | 0.916 | **0.2216** | 0.1192 |
| 3 | **0.930** | 0.2388 | 0.1105 |
Epoch 3 is the most accurate checkpoint by 1.4 points and the *worse* one by NLL. Its `train_manifest.json` records `"selection_metric": "val_nll"` and `"selected_epoch": 2` — the run deliberately shipped the less accurate model. If you select on accuracy you ship epoch 3, your headline number improves and your probabilities get worse, and nothing tells you. Read the third column and it gets more awkward still: epoch 3 has the **best Brier** of the three. Two proper scoring rules, on the same three checkpoints, disagree about which one to ship. Pick your selection metric before you look at the numbers, not after.
**Measure calibration properly.** ECE over binned confidence, Brier, NLL, and the reliability diagram itself — never one of them alone. Reference implementation, with the detail that matters (a bin with `n = 0` is skipped, not counted as perfect):
```python
# typed_decisions/metrics.py
def ece(confs: list[float], correct: list[bool], bins: int = 15) -> float | None:
if not confs:
return None
tot, acc, con = [0] * bins, [0.0] * bins, [0.0] * bins
for c, ok in zip(confs, correct):
b = min(bins - 1, int(c * bins))
tot[b] += 1
acc[b] += float(ok)
con[b] += c
n = len(confs)
return sum(abs(acc[b] / tot[b] - con[b] / tot[b]) * tot[b] / n for b in range(bins) if tot[b])
```
**Hold out the question, not the row.** This is the part every reproduction that measured it got hurt by, and it is the part I would build the whole evaluation around. Splitting states into train and test tells you almost nothing, because the model can memorise "this slot, this option list, this wording." The test that matters is a *new instruction with a new option set on the same states*. `typed-decisions` built exactly that split and the gap is brutal. On the run it measured that split against, in-domain 0.852 falls to **0.622** out-of-domain. Three seeds of its best configuration land at in-domain 0.847 and OOD 0.678, with a spread across seeds of 0.011 and 0.025 respectively — so the OOD number moves two and a half points on the seed alone, and anything smaller than that is noise.
Two specific failures inside that gap are worth naming because they are the failures a decision model must not have. A **negated** yes/no question — "is this claim false according to the passage" — was answered below the majority baseline by two of three backbones (0.40 and 0.39 against a 0.64 majority): they read past the negation and answered the un-negated question. And a **Score with an unseen set of level names** stays at or below majority for every model tried; the expectation read-out never learned that the levels are ordered, because nothing in the architecture told it they were. TypeSafe discloses the same weakness about Jev, in its own words: *"`jev-1.13`'s score levels are weak in numerical calibration. It will not be able to help you reconstruct the exact number by interpolating between the nearest two levels."* Independent architectures, same crack in the same place.
**Fix it with augmentation, and measure that it worked.** The measured fix is cheap and it is not a loss function — it is making memorisation impossible by showing every question in several surface forms with the gold preserved by construction: shuffle a Choice's options and remap the gold index, paraphrase the instruction from a template, drop half the distractors, rename a Score's levels with a same-length synonym set, negate a Noul and flip its label.
| DeBERTa-v3-large, 18k states | in-domain | OOD | OOD ECE | negated noul | cost |
|---|---|---|---|---|---|
| baseline, 1 epoch | 0.852 | 0.622 | 0.116 | 0.671 | \$0.26 |
| **augment p=0.7, 1 epoch** | 0.846 | **0.648** | **0.088** | **0.83** | \$0.26 |
| augment 0.7 + consistency KL 0.5 | 0.852 | 0.635 | 0.110 | 0.83 | \$0.50 |
| augment 0.5 + consistency 0.2, 2 epochs | 0.855 | 0.621 | 0.130 | 0.84 | \$0.99 |
Augmentation alone buys 2.6 points of out-of-domain accuracy, sixteen points on negated questions, and drops out-of-domain ECE from 0.116 to 0.088, for exactly the same money and six tenths of a point of in-domain accuracy. The obvious next idea — a symmetric-KL consistency loss between two surface forms of the same question, which is the textbook answer to openjev's finding that reversing the displayed option order flips 10 of 36 argmaxes — **does not help at this scale** and costs twice as much, because it needs a second forward pass. A second epoch raises in-domain and lowers OOD, which is memorisation with a nicer name. I would not have predicted either of those, which is why the row exists.
## The calibration reality check
Temperature scaling is where every reproduction ends up, so it is worth being precise about what it can and cannot do. It divides every logit by one positive scalar. It cannot change which option wins. It can only make the winner's probability sharper or flatter.
Those bars are real committed data, and as far as I can tell this is the first reliability diagram anyone has published for openjev's own predictions. openjev publishes per-row `option_logits` in `results/raw/predictions/*.jsonl` and its own gold labels in `benchmarks/data/*.jsonl`, but deliberately lists "calibrated probabilities suitable for operational thresholds" under what it has **not** established, so it never joins the two. Joining them gives you a completely untrained Qwen3.5-4B, read through direct option logits, at **79.4% accuracy with 85.5% mean confidence and ECE 0.071** over 252 rows.
**Receipts.** openjev publishes raw per-row option logits and its own gold labels but no calibration result — it lists "calibrated probabilities suitable for operational thresholds" under what it has NOT established. Joining the two files gives one anyway. A completely untrained Qwen3.5-4B read through direct option logits lands at ECE 0.071 over 252 rows, and one fitted scalar temperature takes it to 0.080 on held-out perturbed rows and 0.060 on held-out clean ones. Accuracy never moves, because dividing every logit by one positive number cannot reorder them.
| evaluated on | temperature fitted on | T | accuracy | mean conf | ECE | Brier | NLL |
| :--- | :--- | ---: | ---: | ---: | ---: | ---: | ---: |
| authored144 | — | 1.00 | 0.8056 | 0.8628 | 0.0678 | 0.2449 | 0.4221 |
| authored144 | perturbations108 (held out) | 1.42 | 0.8056 | 0.8106 | 0.0599 | 0.2401 | 0.4173 |
| perturbations108 | — | 1.00 | 0.7778 | 0.8454 | 0.1209 | 0.3240 | 0.5581 |
| perturbations108 | authored144 (held out) | 1.23 | 0.7778 | 0.8113 | 0.0801 | 0.3119 | 0.5339 |
| both, 252 | — | 1.00 | 0.7937 | 0.8553 | 0.0709 | 0.2788 | 0.4804 |
| both, 252 | same rows, min NLL (in-sample) | 1.32 | 0.7937 | 0.8124 | 0.0565 | 0.2696 | 0.4642 |
| both, 252 | same rows, min ECE (in-sample) | 1.19 | 0.7937 | 0.8295 | 0.0480 | 0.2720 | 0.4664 |
The 252 rows come from 36 authored cases (144 = 36 groups × 4 variants; 108 = the same 36 groups × 3 perturbations), so rows inside a group are correlated and the effective sample is far below 252. No confidence intervals are computed here; openjev's own evaluation code does bootstrap by source group and mine does not. The two rows that matter most are the last two: minimising NLL and minimising ECE on the very same data pick different temperatures (1.32 against 1.19) and neither is best on the other's metric.
> method: results/raw/predictions/direct-{authored144,perturbations108}.jsonl joined by row id to benchmarks/data/{authored144,perturbations108}.jsonl; each row's option_logits reordered from its option_ids into the gold file's options order; softmax at temperature T; ECE over 10 equal-width confidence bins; Brier is the multi-class sum of squared errors; NLL is -log p(gold). Temperatures fitted by grid search over 0.20 to 12.00 in steps of 0.01, minimising the stated objective on the stated split.
> source: https://github.com/TheoLeeCJ/openjev
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/rlcd-calibrated-decisions/data/openjev-calibration.json (7 rows)
Three things fall out of that table, and all three are load-bearing for anyone building on this.
**Where the miscalibration lives is the opposite of where you want it.** Slide the widget to T = 1 and look at the bins. The top bin holds 140 of 252 rows — more than half the data — at 0.979 mean confidence against 0.979 accuracy, which is as close to the diagonal as a ten-bin diagram can measure. Every bit of the error is in the middle: the 0.4–0.5 bin is 0.476 confident and 0.286 accurate, the 0.6–0.7 bin is 0.653 confident and 0.444 accurate. The model is honest when it is sure and badly overconfident when it is not, which is exactly backwards for confidence-gated routing, because the entire purpose of a threshold is to catch the cases the model is unsure about.
**Calibration degrades under distribution shift before accuracy does.** The perturbed rows — same semantic questions, option order reversed or wording paraphrased or irrelevant context appended — lose 2.8 points of accuracy (0.806 → 0.778) and nearly double their ECE (0.068 → 0.121). If you monitor accuracy in production you will not see this coming.
**One temperature is one knob and your traffic has several dials.** Fitting on the clean rows gives T = 1.23; fitting on the perturbed ones gives T = 1.42. Both transfer usefully — held out, ECE goes 0.121 → 0.080 and 0.068 → 0.060 — but they are not the same number, and the direction of the miss is set by which slice you happened to fit on. Worse, fitting to minimise NLL and fitting to minimise ECE on the *identical* 252 rows pick different temperatures (1.32 against 1.19), and neither wins on the other's metric. "Calibrated" is not one thing you can be.
Two other reproductions ran into the same wall from other directions, and both published the damage:
- `DavidHatley/system-one-mini` fits a temperature **per head** to minimise mean NLL, and reports that on its final renderer NLL improved from 0.8479 to 0.7133 while **ECE got worse, 0.1273 to 0.2673**, with mean confidence collapsing from 98.89% to 63.19% at fixed 86.25% accuracy. Its committed `temperatures.json` is even more telling: the five fitted values are 3.257, 7.389, 20.085, 3.731, 20.085, against a stated `log_temperature_bounds` of `[-3.0, 3.0]`. Two of the five sit at $\ln T = 2.999996$ and $2.999994$ — pinned against the bound. The fit wanted more damping than it was allowed to apply.
- `pngwn/open-jev` is the first place in this story with a published before/after ECE (7.3% → 2.2% on 5,087 held-out questions with a single T = 2.35), and it is also the place where one global temperature makes a specific slice worse — `severity` goes from 3.8% to 12.6%. That result is worked through in [What the Jev ecosystem actually built](/articles/jev-ecosystem); I am linking it rather than re-deriving it.
One artifact-level inconsistency while I was reading: `heman10x/rlcd-modernbert-151m`'s model card states post-hoc L-BFGS temperature scaling at $T = 1.0716$, while the `calibrator.json` committed in the same repository says `"temperature": 1.4265148639678955`, with a matching `"log_temperature": 0.3552342653274536` ($e^{0.35523} = 1.4265$, so the JSON is internally consistent and the README is the outlier). Anyone loading this model gets 1.4265; anyone reading the card and reimplementing gets 1.0716. It is a small thing and it is the kind of small thing that makes a calibration number unreproducible.
### And for Jev itself?
There is no published calibration number for Jev. Not in the docs, not in the launch post, not on `evals.typesafe.ai`, not in the manifesto — I searched all four for ECE, Brier, reliability diagrams and calibration curves and found zero. For a model whose entire differentiating claim is calibration, and whose vendor built a dedicated evaluation microsite, that absence is the most striking single fact in this article. **Updated 2026-09-19:** the absence at TypeSafe is unchanged, but "the only number that exists" below has not aged well, and I am leaving the sentence visible rather than quietly editing it. Outsiders have since measured Jev's ECE at 0.246 and its option-order instability at 0.13, and there is now a [fifteen-task independent benchmark](/articles/jev-scores-zero) — including a 0/100 on relational choice that bears directly on the architecture argued for above. None of it came from TypeSafe, which was the point; but the claim should have been about who is publishing, not about what exists.
The only number that existed when this was written is one outsider's black-box behavioural probe (a ten-bin ECE of 0.0313 on a 1,200-item MMLU sample), which is measuring Jev's outputs from outside, not TypeSafe's training; it is covered in [Jev's receipts, itemized](/articles/jev-system-one-models).
## What is actually unique
Here is the honest accounting, which is less flattering than the launch post and more flattering than the cynical read.
**Not unique, and not close.** A constrained output space is constrained decoding, which predates all of this. A softmax over a candidate set is a classifier. Probabilities that track frequency are what any classifier trained on a proper scoring rule gives you, and Guo et al. showed how to patch the rest with one scalar in 2017. One forward pass per state is what every encoder has always done. "Cannot emit a type error" is a property of the decoding constraint, not of the training — the PCD article builds it from scratch on stock weights in forty lines.
**Unique, or at least a real design point.** Three things survive.
The first is that **the answer set is chosen by the caller, at request time, at a cardinality of 255**, and the model has never seen it. That is what forces the one-scalar-per-option head — a K-wide classifier cannot do it, as `system-one-mini` demonstrates by being unable to answer a sixth question — and it is the requirement that makes everything else hard. The only project that built a held-out-*question* split found a 16-to-24-point drop on every backbone it tried. If Jev does not have that drop, that is the achievement — and it is precisely the number nobody outside TypeSafe can check.
The second is that **calibration is a first-class field in the API contract rather than a research metric**. `confidence` is returned on every Choice and Score, thresholding on it is the documented pattern, and the docs correctly refuse to promise more than it can deliver ("Calibration is measured across groups of predictions; it does not guarantee that an individual answer is correct"). Shipping a number your customers will build `if` statements on, and documenting its limits accurately, is a product decision more than a research one, and it is the decision that makes the rest of the system composable.
The third is negative and probably the most valuable: **not generating text is a constraint you have to hold to get any of this.** The failure-modes page is candid that Jev cannot count, cannot compare dates, degrades on indirection, and is not adversarially robust ("State is data, and `jev-1.13` does not treat it as hostile by default"). Those are the costs of a model that only ever places probability over a set you supplied. They are real costs and TypeSafe publishes them, which is more than the "can't hallucinate" headline suggests.
**Soft.** "A new model architecture" is fourteen words of denial. "A new training method" is an output contract with no method attached. And the framing that RLCD is a third branch beside RLHF and RLVR gets the relationship backwards in at least one important respect: the property it optimises for is one pretraining already provides on constrained answer sets, and RLHF is what removes it. A branch that declines a known harm and then sharpens with a proper scoring rule is a good engineering decision. It is not obviously a new kind of post-training, and there is currently no published evidence that would let anyone tell the difference from outside.
A paper, a technical report, a model card, or a docs page from TypeSafe that names an objective, a reward, a dataset, or a procedure. My search was `llms-full.txt` at 835,504 bytes fetched 2026-09-18, the launch post including its collapsed FAQ accordion, the manifesto, the antibenchmaxxing post and `evals.typesafe.ai`. If any of those changed after that date, or if there is a page not reachable from the docs index, this claim expires.
Any disclosure of the head shape. Or a behavioural probe: send the same question with the same option set in two requests where one option's *description* is lengthened without changing its meaning, and see whether the other options' probabilities move. Under a per-option scalar scorer they should barely move; under a shared vocabulary softmax the renormalisation couples them. I could not run this — no API key — and it is the cleanest experiment in this article that someone with access could do in an afternoon.
One live API response whose `confidence` disagrees with that formula by more than the rounding of its own probabilities. I fit it to fifteen published examples and it holds on the twelve from the primitive reference pages; two `/api` examples and one `/introduction/quickstart` example disagree, and the quickstart one matches normalised entropy instead. If the live endpoint returns the entropy version, I have the stale page and the current page backwards.
A request that succeeds with a single question whose state-plus-question exceeds 32k tokens, or a published context-window figure. My evidence is entirely the shape of two numbers on the jaggedness page; a vendor who simply set two independent quotas for unrelated reasons would produce the same page.
A published Jev ECE materially below what a well-fitted temperature gets you on a pretrained model of comparable scale over the same constrained answer sets — or any demonstration of calibration that *transfers to unseen question and option sets*, which is where the one reproduction that measured it measurably falls apart (0.852 in-domain to 0.622 out-of-domain before augmentation, 0.678 after, with a 0.025 spread over three seeds). That number, published, would settle it in TypeSafe's favour immediately.
More data. This is 252 rows drawn from 36 authored cases, so rows within a case are correlated and the effective sample is far smaller than 252; I computed no confidence intervals and openjev's own evaluation code bootstraps by source group where mine does not. The pattern is stark enough that I would be surprised to see it invert, but a wider evaluation could easily move the bin boundaries.
Evidence of authorship on, or documented origination of, the pre-InstructGPT RLHF line of work — Christiano et al. 2017 (arXiv 1706.03741), or the earlier preference-learning literature. I checked the Scholar profile the docs themselves link and found InstructGPT (4th of 20 authors), the GPT-4 report, and the ChatGPT announcement, all of which are real and substantial, and none of which is the invention of the method.
---
*Primary sources, all fetched or cloned 2026-09-18: TypeSafe's [launch post](https://typesafe.ai/blog/introducing-system-one-models-and-jev) (text and the collapsed FAQ answers extracted from the page's own serialized component tree, since it renders client-side), `docs.typesafe.ai/llms-full.txt` (835,504 bytes), `evals.typesafe.ai`, and the TypeSafe manifesto and antibenchmaxxing posts. Open reproductions read as source, not as READMEs: [kotoba-lang/typed-decisions](https://github.com/kotoba-lang/typed-decisions) and its published checkpoint [com-kotobalabs/open-jev-deberta-v3-large](https://huggingface.co/com-kotobalabs/open-jev-deberta-v3-large); [heman10x/rlcd-modernbert-151m](https://huggingface.co/heman10x/rlcd-modernbert-151m); [DavidHatley/system-one-mini](https://huggingface.co/DavidHatley/system-one-mini); [Foodoo1/Qwen3-14B-RLCD-Decision-LoRA](https://huggingface.co/Foodoo1/Qwen3-14B-RLCD-Decision-LoRA); [TheoLeeCJ/openjev](https://github.com/TheoLeeCJ/openjev), which has since renamed itself SemIf and which I keep calling openjev here for continuity with the companion pieces. Papers, every arXiv ID checked against the arXiv API rather than assumed: [2203.02155](https://arxiv.org/abs/2203.02155) (InstructGPT), [1706.03741](https://arxiv.org/abs/1706.03741) (Christiano et al.), [1706.04599](https://arxiv.org/abs/1706.04599) (Guo et al.), [2207.05221](https://arxiv.org/abs/2207.05221) (Kadavath et al.), [2303.08774](https://arxiv.org/abs/2303.08774) (GPT-4 report), and for the "ask it in words" line of work [2205.14334](https://arxiv.org/abs/2205.14334) and [2305.14975](https://arxiv.org/abs/2305.14975). Checkpoint shapes were read from safetensors headers by HTTP range request; the confidence and score formulas, the reliability measurement and the temperature fits are my own, computed from the committed artifacts named in each receipts block. Companion pieces: the inference mechanism in [Parallel constrained decoding](/articles/parallel-constrained-decoding), the product claims in [Jev's receipts, itemized](/articles/jev-system-one-models), the downstream repos in [What the Jev ecosystem actually built](/articles/jev-ecosystem).*