{
 "claim": "ChessLFM's framing is that language models play illegal moves, and its answer is that the host masks the move logits to the legal set so 'illegal moves are impossible'. I ran the exact artifact the demo downloads \u2014 the 4-bit ONNX export \u2014 over 1,700 real positions with the mask removed, taking the plain argmax over all 2,106 vocabulary entries. It picked a move token 1,700 times out of 1,700, and a legal move 1,694 times: 99.65%. The mask changed the move it would have played 6 times. Legality is not what the mask is buying; the encoding already bought it. The mask is a safety net that catches roughly one move in 280, and every catch in my sample was a tactical miss (a king walking into a defended capture) rather than a model that had lost the board.",
 "method": "Ported Gy() from the Space bundle, ran onnx/model_q4.onnx under onnxruntime 1.30.0 on CPU, two forward passes per position (80-token prompt for the value bin, then 82 tokens for the move logits) exactly as evaluateBoard does. Legality judged with python-chess 1.11.2. Positions sampled from the PGN Mentor Carlsen archive, seeds 7 and 11.",
 "source": "https://huggingface.co/mlabonne/LFM2.5-230M-Chess-ONNX",
 "captured": "2026-09-18",
 "note": "'Mean legal mass' is the softmax over the 1,969 move tokens, summed over the legal ones. This measures legality only. It says nothing about move quality, and I have no Stockfish binary here, so I make no independent claim about strength.",
 "columns": [
  {
   "key": "corpus",
   "label": "corpus"
  },
  {
   "key": "positions",
   "label": "positions",
   "align": "right"
  },
  {
   "key": "argmax_is_move_token",
   "label": "argmax is a move token",
   "align": "right"
  },
  {
   "key": "argmax_is_legal",
   "label": "argmax is legal",
   "align": "right"
  },
  {
   "key": "mask_changed_the_move",
   "label": "mask changed the move",
   "align": "right"
  },
  {
   "key": "mean_legal_mass",
   "label": "mean legal mass",
   "align": "right"
  },
  {
   "key": "note",
   "label": "note"
  }
 ],
 "rows": [
  {
   "corpus": "master games, all phases",
   "positions": 1200,
   "argmax_is_move_token": "100.00%",
   "argmax_is_legal": "99.75%",
   "mask_changed_the_move": 3,
   "mean_legal_mass": 0.9866,
   "note": "1,200 positions sampled from 150 games in the PGN Mentor Carlsen archive"
  },
  {
   "corpus": "master games, <=12 men",
   "positions": 500,
   "argmax_is_move_token": "100.00%",
   "argmax_is_legal": "99.40%",
   "mask_changed_the_move": 3,
   "mean_legal_mass": 0.9677,
   "note": "500 sparse endgames \u2014 the phase the training mixture needed synthetic drills for"
  },
  {
   "corpus": "combined",
   "positions": 1700,
   "argmax_is_move_token": "100.00%",
   "argmax_is_legal": "99.65%",
   "mask_changed_the_move": 6,
   "mean_legal_mass": 0.981,
   "note": "6 mask interventions in 1,700 positions"
  },
  {
   "corpus": "worked example \u2014 Rxb5, 9-man endgame",
   "positions": 1,
   "argmax_is_move_token": "100.00%",
   "argmax_is_legal": "0.00%",
   "mask_changed_the_move": 1,
   "mean_legal_mass": 0.1575,
   "note": "8/8/p3r3/1k3pp1/1P6/1R4K1/6P1/8 w - - 12 55 \u2014 the top move captures the black king; not even pseudo-legal"
  },
  {
   "corpus": "worked example \u2014 Kxf7, 7-man endgame",
   "positions": 1,
   "argmax_is_move_token": "100.00%",
   "argmax_is_legal": "0.00%",
   "mask_changed_the_move": 1,
   "mean_legal_mass": 0.0414,
   "note": "8/4KQk1/7p/6p1/r5P1/8/8/8 b - - 16 84 \u2014 king grabs a queen defended by the enemy king; 96% of the move mass is on illegal moves"
  }
 ]
}