# ChessLFM: 408 of the 552 Elo are not in the weights

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/chesslfm
> date: 2026-09-18
> tags: llm, chess, tree-search, benchmarks, evaluation, explainer
"ChessLFM: 2000 Elo in 230M params." Chess is the rare subject where a headline like that is
fully checkable, because chess strength is a measured quantity with a well-understood
methodology, and because the whole system shipped: a model, a 4-bit ONNX export, and a browser
app you can read. So I read it. I rebuilt the action space from scratch, summed the safetensors
header, ported the prompt builder out of the minified bundle, and ran the exact artifact the demo
downloads with its legal-move mask taken off.

Two things come out of that. The encoding is better than the headline suggests — the model really
has learned legality, and I can put a number on it. And the strength number is a wrapper number:
of the 552 Elo between the first trained checkpoint and the final one, 408 come from code that
never touched a weight, and 106 of those 408 live in a row of the project's own results table that
the write-up never mentions.

<ModelCard
  repo="mlabonne/LFM2.5-230M-Chess"
  claimed="230M"
  note="The bf16 checkpoint. The demo does not run this file — it runs a vocab-pruned, 4-bit ONNX export of it, which is 164.8M parameters. See the parameter accounting below."
/>

## One move, one token

The design question the project opens with is the right one: how should a language model emit a
chess move? Free-form text makes the model spend capacity tracking the board, spreads a move over
several tokens, and leaves nothing to stop an illegal one. ChessLFM takes DeepMind's ChessBench
action space instead — every geometrically possible move becomes one vocabulary entry, like
`<m:e2e4>` — and keeps a decoder-only model so the thing exports to ONNX and runs under WebGPU.

The claim underneath is that chess has 1,968 geometrically possible moves. That is a
self-contained assertion about an 8×8 board, so I rebuilt the set from geometry alone: every
queen-like and knight-like `(from, to)` pair, plus every promotion move for all four pieces on both
back ranks. Then I diffed it against the `<m:*>` keys in the released `token_ids.json`.

**Receipts.** ChessLFM says chess has 1,968 geometrically possible moves and turns each into one vocabulary token. I rebuilt that action space from scratch — every queen-like and knight-like (from, to) pair on an 8x8 board, plus every promotion move for all four pieces on both back ranks — and diffed it against the released tokenizer. The 1,968 is exactly right and the sets are identical, move for move. The tokenizer ships 1,969 move tokens, not 1,968: the extra one is <m:0000>, the UCI null move, which is not a move at all but the left-padding for the 8-ply history window. Every other token group matches the shipped vocabulary to the unit, and 1,969 + 64 + 73 = 2,106, the count surgery_meta.json reports.

| token group | rebuilt from geometry | in the released tokenizer | match |
| :--- | ---: | ---: | :--- |
| queen-like and knight-like (from,to) pairs | 1,792 | 1,792 | yes |
| promotion moves (q, r, b, n; both colours) | 176 | 176 | yes |
| total geometric move space | 1,968 | 1,968 | yes |
| UCI null move <m:0000> (history padding) | 0 | 1 | no — extra |
| <m:*> tokens in the released tokenizer | 1,968 | 1,969 | no — extra |
| <v:*> win-probability bins | 64 | 64 | yes |
| board-cell tokens <c:*> (12 pieces + empty) | 13 | 13 | yes |
| castling-combination tokens <cast:*> | 16 | 16 | yes |
| en-passant file tokens <ep:*> (- plus a-h) | 9 | 9 | yes |
| halfmove-clock buckets <hm:*> (floor(min(hm,100)/4)) | 26 | 26 | yes |
| repetition-count tokens <rep:*> | 3 | 3 | yes |
| side-to-move tokens <stm:*> | 2 | 2 | yes |
| structural tokens (pos, hist, eval, bestmove) | 4 | 4 | yes |
| TOTAL added by the tokenizer surgery | 2,105 | 2,106 | no — the null move |

Castling and en passant need no special tokens: the king's two-square move (e8g8) and the pawn's diagonal capture are already members of the geometric set. That is why an action space this small is closed under the rules of chess.

> method: Reconstructed the move set in Python from board geometry alone, then set-differenced it against the <m:*> keys in token_ids.json from the model repo. Group counts taken by prefix from the same file; totals cross-checked against surgery_meta.json (base_vocab_len 64402, num_added 2106, vocab_len 66508) and against config.json's vocab_size.
> source: https://huggingface.co/mlabonne/LFM2.5-230M-Chess/blob/main/token_ids.json
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/chesslfm/data/action-space.json (14 rows)

It matches, move for move, with exactly one extra: the tokenizer ships 1,969 move tokens, and the
extra one is `<m:0000>`, the UCI null move. That is not a move — it is the left-padding for the
history window, which I will come back to. Every other group in the surgery lines up to the unit,
and `1969 + 64 + 73 = 2106`, which is the `num_added` that `surgery_meta.json` reports against a
base vocabulary of 64,402 and a final `vocab_size` of 66,508.

What I like about this action space is what it does *not* need. Castling is not a special token:
`e8g8` is already a member of the king's move set. En passant is not a special token either; it is
an ordinary diagonal pawn capture. The set is closed under the rules of chess without a single
exception, which is why 1,968 entries is enough.

The prompt is the other half. This is a port of `Gy()` out of the shipped Space bundle — same field
order, same halfmove bucketing, same padding — rendered for a real position:

<BoardPrompt />

Eighty tokens, always. The model never sees free text, and the position of every field is fixed, so
the network can learn "the square at index 37" rather than parsing. Note `<hm:0>`: the halfmove
clock is bucketed into fours (`floor(min(hm,100)/4)`, 26 buckets), so the model can tell a fresh
clock from a 50-move-rule scare but not 3 plies from 2. And note that the history window is eight
plies, padded left with the null move — which is exactly enough to see a repetition coming and not
much more.

<Figure
  src="/articles/chesslfm/fig1.png"
  alt="Diagram of ChessLFM inference. An 80-token prompt of position, 64 board squares, state tokens, last 8 moves and an eval marker feeds forward pass 1, which emits a value token v:38. That token plus a bestmove marker feeds forward pass 2, producing logits over the 1,968 move tokens. A legal-move mask then hatches out the illegal entries, leaving one legal move, m:e2e4, selected."
  caption="Two forward passes per move: value first, then policy, then the host's legal-move mask (ChessLFM announcement, inference diagram)."
/>

## Legal moves were never the hard part

The pitch opens with "LLMs are famously bad at chess... sooner or later they play an illegal move."
The system's answer is a mask: "illegal moves are impossible, because we mask them before selecting
our move." The model card is more precise about who does it — "the host masks the move logits to
the legal moves" — and the released code is more precise still. `evaluateBoard` calls
`t.moves({verbose:true})` on a chess.js board, looks up each legal move's index in the action
space, gathers only those logits, and softmaxes over that subset. Legality never passes through the
network at all.

So: how much work is that mask actually doing? Nobody published the number, and it is the exact
number that separates "the encoding taught the model chess" from "the wrapper is holding the board
together." I measured it.

I ran `onnx/model_q4.onnx` — the 4-bit export the browser downloads — under onnxruntime on CPU,
two forward passes per position exactly as `evaluateBoard` does, and took the plain argmax over all
2,106 vocabulary entries with no mask of any kind. Positions came from 150 master games, sampled
across all phases, plus a second stratum of sparse endgames.

**Receipts.** ChessLFM's framing is that language models play illegal moves, and its answer is that the host masks the move logits to the legal set so 'illegal moves are impossible'. I ran the exact artifact the demo downloads — the 4-bit ONNX export — over 1,700 real positions with the mask removed, taking the plain argmax over all 2,106 vocabulary entries. It picked a move token 1,700 times out of 1,700, and a legal move 1,694 times: 99.65%. The mask changed the move it would have played 6 times. Legality is not what the mask is buying; the encoding already bought it. The mask is a safety net that catches roughly one move in 280, and every catch in my sample was a tactical miss (a king walking into a defended capture) rather than a model that had lost the board.

| corpus | positions | argmax is a move token | argmax is legal | mask changed the move | mean legal mass | note |
| :--- | ---: | ---: | ---: | ---: | ---: | :--- |
| master games, all phases | 1,200 | 100.00% | 99.75% | 3 | 0.987 | 1,200 positions sampled from 150 games in the PGN Mentor Carlsen archive |
| master games, <=12 men | 500 | 100.00% | 99.40% | 3 | 0.968 | 500 sparse endgames — the phase the training mixture needed synthetic drills for |
| combined | 1,700 | 100.00% | 99.65% | 6 | 0.981 | 6 mask interventions in 1,700 positions |
| worked example — Rxb5, 9-man endgame | 1 | 100.00% | 0.00% | 1 | 0.158 | 8/8/p3r3/1k3pp1/1P6/1R4K1/6P1/8 w - - 12 55 — the top move captures the black king; not even pseudo-legal |
| worked example — Kxf7, 7-man endgame | 1 | 100.00% | 0.00% | 1 | 0.041 | 8/4KQk1/7p/6p1/r5P1/8/8/8 b - - 16 84 — king grabs a queen defended by the enemy king; 96% of the move mass is on illegal moves |

'Mean legal mass' is the softmax over the 1,969 move tokens, summed over the legal ones. This measures legality only. It says nothing about move quality, and I have no Stockfish binary here, so I make no independent claim about strength.

> method: Ported Gy() from the Space bundle, ran onnx/model_q4.onnx under onnxruntime 1.30.0 on CPU, two forward passes per position (80-token prompt for the value bin, then 82 tokens for the move logits) exactly as evaluateBoard does. Legality judged with python-chess 1.11.2. Positions sampled from the PGN Mentor Carlsen archive, seeds 7 and 11.
> source: https://huggingface.co/mlabonne/LFM2.5-230M-Chess-ONNX
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/chesslfm/data/legal-move-probe.json (5 rows)

Over 1,700 positions the unmasked argmax was a move token 1,700 times out of 1,700 — it never once
tried to emit a value token or a structural token in the move slot — and it was a *legal* move
1,694 times. 99.65%. The mask changed the move it would have played six times in seventeen hundred.
Mean probability mass on the legal set was 0.981.

That is a real result for the encoding, and I want to state it plainly because this site usually
ends up on the other side of these: the "LLMs play illegal moves" problem is solved here by the
board encoding and the one-token action space, not by the mask. The mask is a safety net that fires
about once every 280 moves.

It is also worth looking at what the six failures are, because they are not what the framing
predicts. Not one of them is a model that lost track of the position. All six are tactical:

```text
8/8/p3r3/1k3pp1/1P6/1R4K1/6P1/8 w - - 12 55
  top move: b3b5  (Rxb5 — the rook captures the black king)
  not even pseudo-legal; 84% of the move mass sits on illegal moves

8/4KQk1/7p/6p1/r5P1/8/8/8 b - - 16 84
  top move: g7f7  (Kxf7 — grabs the queen, which the king on e7 defends)
  pseudo-legal, illegal; 96% of the move mass sits on illegal moves

2rnqb2/1b1n1N1k/p2PB1Pp/1p6/4Q3/8/PP4PP/R4RK1 b - - 0 29
  top move: h7g6  (Kxg6 — the pawn is defended by the queen on e4, down the diagonal)
  pseudo-legal, illegal; the masked pick is the model's own second choice
```

Two are kings walking into a defended capture; one is a rook trying to take the enemy king outright.
These are misjudgements about *who defends what*, which is a strength failure, not a board-state
failure — and the mask turns a strength failure into a legal move, which is a different and smaller
service than "it stops the model losing the position." The illegal rate roughly doubles in sparse
endgames (0.25% → 0.60%), which is where the training mixture needed synthetic drills in the first
place, so that tracks.

Keep the two axes separate, because the rest of this piece depends on it. Legal-move rate: 99.65%
unmasked, 100% masked, and the mask is nearly free. Playing strength: a different claim entirely,
measured a different way, and this is where it gets interesting.

## 2004 against what?

An Elo number is meaningless without the pool. The announcement says "2004 Elo (95% CI 1972-2039)
on a Stockfish-anchored ladder" and leaves it there — no opponent settings, no time control, no
game count in the prose. The recipe table's footnote adds "the two full 720-game ladders." The
actual methodology is in the demo, in a tooltip string sitting in the bundle:

```js
// assets/index-BPxYr76G.js — the Elo badge's tooltip
"Anchored Elo from a ~720-game ladder vs node-limited Stockfish (UCI_Elo 1320-2400),
 with test-time tree search (depth 3, width 6) + repetition-aware backup.
 The demo plays that exact configuration. Engine-anchored scale, not directly
 comparable to FIDE or Lichess ratings."
```

That is an honest disclosure, and it is buried in a minified JavaScript bundle rather than in the
post whose title is the number. Unpacking it:

- **The pool is one engine, dialled down.** Not a field of engines, not humans on a server. Every
  opponent is Stockfish with `UCI_LimitStrength` on and `UCI_Elo` set somewhere between 1320 and
  2400.
- **`UCI_Elo` is a CCRL Blitz scale, not a human one.** This is checkable at the source: the
  Stockfish commit that set the current parameterisation (`a08b8d4`) describes the range as "CCRL
  Blitz Elo from 1320 to 3190, approximately," anchored "± 100 Elo to CCRL Blitz," fitted from games
  against ranked versions of the Stash engine. So the anchor itself carries ±100 Elo of stated
  slop, roughly three times the half-width of the 1972-2039 interval ChessLFM reports around 2004.
- **The dial was calibrated at a time control the ladder did not use.** That same commit pins the
  calibration to "a time control of 120s+1.0s." The ladder ran Stockfish node-limited, and the node
  budget is not published anywhere I can find. A node limit is a reasonable choice for
  reproducibility, but it is off-calibration, and without the number nobody can rebuild the ladder.
- **There is no public rating history.** Neither the post nor the model card links a bot account, and
  I found nothing under ChessLFM, LFM2 or mlabonne in Lichess's bot listing. The only strength
  evidence anywhere is the project's own harness, which is not released.

To be fair to the project: the tooltip and the model card both refuse the comparison the headline
invites. "Engine-anchored scale, not directly comparable to FIDE or Lichess ratings" is exactly
right, and the model card leads with "roughly 2004 Elo with a shallow depth-3 search on top, and
roughly 1500 with the raw one-pass policy," which is the most useful sentence anybody wrote about
this model. The headline is "2000 Elo in 230M params." A reader who stops there will come away
thinking a 230M file plays like a strong club player. Both halves of that sentence are doing work
they have not earned.

The one strength claim that travels well is the head-to-head: "the final model scores 49%
head-to-head against the 2000-rated anchor." That is a direct, interpretable statement — it played
the 2000 dial roughly even. Whether 2000 on that dial is 2000 anywhere else is a separate question
with a published answer of ±100.

<Callout type="note">
  The announcement opens with an animated game captioned "Fable 5.1 (white) getting destroyed by our
  230M model." One game. Against one opponent. It is a fun thing to show and it is not a measurement,
  and the post does not present it as one.
</Callout>

## Where the 552 Elo came from

The last figure in the announcement is the honest one, and it is the figure the prose contradicts by
omission. Six rows, each stacked on the one above:

<Figure
  src="/articles/chesslfm/fig3.png"
  alt="Table of six ChessLFM configurations with Elo and delta Elo. Row 1, raw SFT policy, 1452. Row 2, plus depth-3 tree search, 1754, plus 302. Row 3, plus wider search and repetition fix, about 1860, plus 106. Row 4, plus value-head warm-start, 1927, plus 67. Row 5, plus HL-Gauss value retrain, 1988 with interval 1951 to 2024, plus 61. Row 6, plus GRPO polish, 2004 with interval 1972 to 2039, plus 16. A footnote states that rows 4 and 5 are measured without GRPO and that the confidence intervals come from the two full 720-game ladders."
  caption="The project's own recipe table. The arithmetic is internally consistent: 1452 + 302 + 106 + 67 + 61 + 16 = 2004 (ChessLFM announcement, recipe table)."
/>

I checked the arithmetic first, because a stacked table is the easiest place for a rounding error to
hide: `1452 + 302 + 106 + 67 + 61 + 16 = 2004`. Exact. The table is internally consistent, and it is
the only place the full accounting appears.

Now read the prose next to it. The write-up describes SFT (about 1450), then search (about +300),
then a rerun of SFT over all 126M positions (+67), then HL-Gauss (+60), then GRPO (+16). Add the
prose's own numbers up and you land near 1893, not 2004. Most of the gap is row 03 — **"+ Wider
search, repetition fix", +106** — which appears in the table, in the figure, and nowhere in the text;
the rest is the prose rounding 1452 to 1450, 302 to 300 and 61 to 60. That +106 is the second largest
single gain in the entire project, larger than HL-Gauss and GRPO combined, and it is a search-width
increase plus a bug fix. It is also the only row written with a tilde (`~1860`).

While I am in the table: only two of the six rows carry a confidence interval, and the footnote says
those intervals "come from the two full 720-game ladders." So four of the six configurations were
not measured on a full ladder, which is a reasonable way to spend engine time and a reason not to
read the intermediate deltas as precisely as they are printed.

Attributing every row to what it changed:

<EloLadder />

Two rows changed the search wrapper and bought 408 Elo. Three rows changed the network and bought
144. And the 144 is softer than it looks: rows 04 and 05 both train the *value head*, which the
one-pass policy never consults — it exists to score leaves — and the post says GRPO "barely changes
greedy play (within the confidence interval)" and contributes its +16 "whenever the policy is used
as a search prior." So every Elo point earned after the first SFT checkpoint is realised through the
search loop. The model card's own framing agrees from the other direction: the final network, raw,
is "roughly 1500."

That is not a criticism of the work. It is a well-known result being re-derived cleanly at small
scale: a policy-plus-value network is a search prior, and the search is where the strength comes
out. AlphaZero is the same shape. The criticism is of the sentence "2000 Elo in 230M params," which
puts the strength inside the parameters. A truer version is "1452 Elo in 230M params, and 2004 with
432 leaf evaluations per move."

<Figure
  src="/articles/chesslfm/fig2.png"
  alt="Tree diagram of the depth-3 search. A root node branches to depth 1, labelled top 12 moves with 4 shown; each of those branches to depth 2, labelled top 6 replies with 3 shown; each of those branches to depth 3, labelled up to 432 leaves scored by the value token. One path through the tree is highlighted."
  caption="The search the +302 and +106 rows are about: 12 root moves, 6 replies thereafter, depth 3 (ChessLFM announcement, search diagram)."
/>

## What the demo actually runs

Three things in the released bundle differ from the description, and one of them matters.

<DemoPath />

**The device gate.** On load, the app resolves a device and then does this:

```js
// assets/index-BPxYr76G.js — immediately after the model loads
te(rt => ({ ...rt,
  useSearch:   Ne.device === "webgpu",
  searchDepth: Ne.device === "webgpu" ? 3 : 0
}))
```

`device` is `"webgpu"` if `navigator.gpu.requestAdapter()` resolves and `"wasm"` otherwise. So on
any browser without a WebGPU adapter, the demo turns the search off and plays the raw policy — the
~1500 configuration — while the tooltip in the same bundle says "the demo plays that exact
configuration." The UI does change its mode chip to "Greedy," so an attentive reader can notice, but
nothing says the Elo badge no longer applies. There is a second path to the same place: if the
search throws, the handler retries with `{useSearch: false, searchDepth: 0, topK: 1}` and reports it
to the console only.

This is the pattern I keep finding in this class of release: not a fabricated number, but a default
that quietly switches off the mechanism the number depends on. If you have played ChessLFM in the
browser and thought it felt weak, check whether your browser has WebGPU before concluding anything
about the model.

**The search is cheaper and more expensive than described.** Cheaper: the code is alpha-beta, not
plain minimax — `negamax` breaks out of its child loop when `alpha >= beta` and marks the node
`cutoff: true` — so the real leaf count is below 432, helped further by a FEN-keyed value cache.
More expensive: the post says the value head "scores the ~430 positions this reaches," but 432 is
only the *leaves*. Every interior node needs two forward passes, because the move logits are only
available after the value token has been appended to the prompt. Full budget at the shipped defaults
is 517 nodes and 602 forward passes per move, against 2 for the raw policy.

Which raises a question the code makes unavoidable: the value head has 64 levels, and the search
compares leaves by the bin's midpoint. How often can it actually tell two lines apart? I measured
the depth-1 frontier — 116 positions, the policy's top 12 legal moves each, one `evaluateValue` pass
per child, which is exactly how a leaf gets scored. Those roughly 12 candidates came back with a
mean of **4.4 distinct value bins**, and in **56.9%** of positions the best bin was shared by two or
more moves (2.9 on average; in one position all twelve tied). The deeper backup breaks some of those
ties, but when it does not, the decision falls through to the policy prior, because the root
candidate sort is a plain descending sort and JavaScript's `Array.prototype.sort` is stable. The
network's two heads are more entangled than the diagram suggests.

**The value decode throws away what HL-Gauss bought.** HL-Gauss replaces a one-hot target with a
Gaussian over neighbouring bins, and the point of it in *Stop Regressing* is to read the scalar back
out as the *expectation* of the softmax. The shipped decode is `bestValueBin`, a hard argmax, and
the win probability is the winning bin's midpoint: `(k + 0.5) / 64`. So inference resolves position
value to 1.6 percentage points and discards the distribution the training objective was shaped to
produce. It still bought +61 Elo — better-ranked argmax bins are worth having — but there may be
free Elo sitting in a two-line change.

## The 230M is not the 164.8M

One more thing worth checking, because the parameter count is half the headline. I range-read the
safetensors header rather than trusting the label:

**Receipts.** The headline is 230M parameters, and the bf16 checkpoint really is 230,688,512 — I summed the safetensors header rather than trusting the label. But 68,104,192 of them (29.52%) are the embedding table, which is tied and therefore also the output head, and 64,402 of its 66,508 rows belong to the base model's English vocabulary. The chess prompt is made only of the 2,106 added tokens and the only outputs read are value and move tokens, so those 65,947,648 parameters — 28.59% of the model — are never addressed at inference. The published ONNX export agrees: it prunes the table to 2,106 rows, and its fp32 data file is 659,025,920 bytes, i.e. 164,756,480 float32 values. The artifact you actually play against in the browser is a 4-bit, 164.8M-parameter model.

| artifact | parameters | bytes on disk | what it is |
| :--- | ---: | ---: | :--- |
| mlabonne/LFM2.5-230M-Chess, model.safetensors (bf16) | 230,688,512 | 461,391,768 | the checkpoint the 2004 Elo is attributed to |
|   of which model.embed_tokens.weight [66508, 1024] | 68,104,192 | — | 29.52% of the model; tied, so it is also the output head |
|   of which the 2,106 chess rows [2106, 1024] | 2,156,544 | — | 0.93% of the model — the entire chess vocabulary |
|   of which the 64,402 inherited base-vocab rows | 65,947,648 | — | 28.59% of the model, unreachable: no English token is ever an input or an output |
| everything that is not the embedding table | 162,584,320 | — | the part that actually computes |
| LFM2.5-230M-Chess-ONNX, onnx/model.onnx_data (fp32) | 164,756,480 | 659,025,920 | vocab pruned to 2,106 rows; 659,025,920 / 4 float32 values |
| LFM2.5-230M-Chess-ONNX, onnx/model_q4.onnx_data | 164,756,480 | 223,439,008 | 4-bit; this is what the browser demo downloads and runs |

164,756,480 is 15,616 more than 162,584,320 + 2,156,544; the remainder is graph-level constants the exporter materialises. No claim is made that the q4 export scores 2004 Elo — nothing published re-measures the quantised model.

> method: HTTP range-read the first 8 bytes of model.safetensors for the header length, then the header itself; parsed the JSON and summed the product of every tensor shape (132 tensors, no lm_head.weight, so embeddings are tied). ONNX sizes are Content-Length from the Hub file listing; the fp32 parameter count is that size divided by 4.
> source: https://huggingface.co/mlabonne/LFM2.5-230M-Chess/blob/main/model.safetensors
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/chesslfm/data/parameters.json (7 rows)

The checkpoint really is 230,688,512 parameters. But there is no `lm_head.weight` among its 132
tensors, so the embedding is tied, and `model.embed_tokens.weight` is `[66508, 1024]` — 68,104,192
parameters, 29.52% of the model, serving as both input table and output head. Of its 66,508 rows,
64,402 are the base model's English vocabulary. The prompt is built only from the 2,106 added
tokens; the only outputs ever read are value and move tokens. Those 64,402 rows are unreachable in
both directions.

The project agrees, in code. The ONNX export's `surgery_meta.json` records `"vocab_len": 2106` and
`"pruned_from": ".../checkpoints/grpo-hlg/step-1000"`, and its fp32 data file is 659,025,920 bytes —
164,756,480 float32 values. The browser downloads the 4-bit version: 223,439,008 bytes. So the thing
you actually play against is a 164.8M-parameter, 4-bit model with 2,106 tokens in its vocabulary,
which cannot emit a single English word and is not, in any operational sense, a language model any
more. It is a chess network that inherited a transformer's file format.

That is a good engineering decision — and it makes "230M params" the number for a file nobody runs.
The honest version is smaller and more impressive: a 165M network, quantised to 4 bits, playing a
strong club game in a browser tab.

I will also note that nothing published re-measures the quantised export. The 2004 came from the
bf16 checkpoint; the demo runs q4. Quantisation to 4 bits usually costs a little strength. My
legality probe ran on q4 and found 99.65%, which tells you the quantisation did not break the
encoding, but says nothing about the Elo.

<Callout type="warning">
  A third-party GGUF conversion picked up 305 downloads within two days of release. Loaded into a normal chat
  runner, that file gives you the network and nothing else — no board tokeniser, no legal-move mask,
  no search. That is the 1452 configuration at best, and only if you hand-build the 80-token prompt.
  Nothing in that repo says so.
</Callout>

## What is not published

For a project whose headline is a measured quantity, the measuring apparatus is the part I most
wanted and the part that does not exist publicly. There is no training code, no data-preparation
code, no ladder harness, no `UCI_Elo` schedule, no node budget, no time control, no PGN archive of
the 720-game ladders, and no bot account with a rating history. The GitHub repository named in the
older Chess-LLM collection was last updated in January 2024 and is a different project. The only
code released for ChessLFM is the inference side: a minified browser bundle, which to its credit is
complete enough to answer most of the questions above.

That means the 2004 is not independently reproducible today. It is not unreproducible in principle —
the ingredients are all standard, and anyone with Stockfish and a weekend could build a comparable
ladder — but they would be building a different ladder, and the two numbers would not be comparable.
This is a personal project published on a Substack, not a paper, and it is more forthcoming than
most papers. I am recording the gap, not demanding a tech report.

## What I would take from this

The parts of ChessLFM that deserve to be copied are the encoding and the honesty of the recipe
table. A fixed-length prompt, a closed one-token action space, and a value token in the same
vocabulary give you a model that is 99.65% legal without a mask, exports to ONNX cleanly, and can be
wrapped in a search loop without a custom head. That is a genuinely clean design and the
measurements back it.

The part that should not travel is the headline. "2000 Elo in 230M params" compresses four separate
things — an engine-anchored scale with ±100 of stated slop, a search wrapper worth 408 of 552 Elo, a
bug fix the prose omits, and a parameter count for a file the demo does not run — into a sentence
that reads as a property of the weights. The recipe table, the model card and the demo tooltip each
say the truer thing. The title says the louder one, and the title is what gets quoted.

<ChangeMyMind>
  <Falsifier claim="408 of the 552 Elo come from the search wrapper, not from training.">
    This is read straight off the project's own recipe table, attributing rows 02 and 03 to search
    and rows 04–06 to the network. If the "+ Wider search, repetition fix" row turns out to be
    mostly the repetition fix — a correctness bug in how repetitions were scored during evaluation,
    which would arguably be a measurement artefact rather than either category — the 408 should be
    split. A per-component ablation of that row would settle it. Separately, if the raw final policy
    measures well above the model card's "roughly 1500" on the same ladder, the network's share is
    larger than 144 and my attribution is too harsh.
  </Falsifier>

  <Falsifier claim="The unmasked policy is legal 99.65% of the time, so the legal-move mask is not what makes ChessLFM work.">
    1,700 positions, two corpora, one artifact (the q4 ONNX), sampled from master games. If the same
    probe on positions drawn from the search tree's own interior — which drift off the master-game
    distribution by construction — came back materially worse, say below 97%, the mask would be
    doing real work where it actually runs and my framing would be wrong. Running it on the bf16
    checkpoint instead of q4 would also be worth doing; quantisation could plausibly cost legality
    that the mask then hides.
  </Falsifier>

  <Falsifier claim="The browser demo does not always play the configuration its tooltip claims.">
    Read from the bundle: `useSearch: Ne.device === "webgpu"`. If a later build gates on something
    else, sets a non-zero WASM search depth, or surfaces the downgrade next to the Elo badge, this
    stops being true. It is a one-line fix and it may already be fixed by the time you read this;
    the commit I read is `c3e1a80`, the Space's main as of 2026-09-18.
  </Falsifier>

  <Falsifier claim="2004 on this ladder is not 2004 on a human scale.">
    Rests on Stockfish's own documentation of `UCI_Elo` as approximately CCRL Blitz, ±100, calibrated
    at 120s+1s. If ChessLFM were put on Lichess as a bot and settled near 2000 in blitz over a few
    hundred games against humans, that would be direct evidence the two scales land in the same
    place for this engine, and I would drop the caveat. A public bot account is the single cheapest
    thing that would resolve it.
  </Falsifier>
</ChangeMyMind>

---

**Primary sources.** The announcement,
[maximelabonne.substack.com/p/chesslfm-2000-elo-in-230m-params](https://maximelabonne.substack.com/p/chesslfm-2000-elo-in-230m-params);
the model, [mlabonne/LFM2.5-230M-Chess](https://huggingface.co/mlabonne/LFM2.5-230M-Chess); the
browser export,
[mlabonne/LFM2.5-230M-Chess-ONNX](https://huggingface.co/mlabonne/LFM2.5-230M-Chess-ONNX); the demo
and its code, [hf.co/spaces/mlabonne/ChessLFM](https://huggingface.co/spaces/mlabonne/ChessLFM);
and, for the anchor, Stockfish commit
[a08b8d4](https://github.com/official-stockfish/Stockfish/commit/a08b8d4). Figures are the
announcement's own, reproduced here for commentary.
