{
 "claim": "How big is a hard four-number Countdown problem? Every distinct state is listed below, for the 30 hard problems whose numbers Bad Theory Labs published in its Phase 0 and Phase 1 raw results, and for the 4 showcase problems from the 30 used in the paper's headline tables. On the 30, a problem has 114 distinct non-terminal states on average (73 to 151). The paper's headline comparison gives each search 200 judged positions, which is more than the whole space of every one of them. The last two columns are the chance that the paper's frontier, with its judge replaced by a random ranking, solves the problem in one pass at width 6 (the configuration behind the 23 of 30) and at width 16 (the configuration behind the 30 of 30).",
 "method": "My own enumerator, written from the rules in section 3 of the paper: a state is the sorted multiset of available numbers; a move combines two of them with +, -, x or /, and the result must be a positive whole number (division by 1 is skipped, as in the paper's move lister, because it only produces duplicates). It reproduces the recorded showcase trace in results/llm/viz_slim.json exactly (19 first-level states; 110 raw children merging into 82 distinct second-level candidates). Alive = the target is still reachable, by exhaustive search. Non-terminal states = levels 0 to 2. Complete paths = distinct move sequences to a single number. Random-ranking columns: 4,000 trials per problem, one pass, no restarts, matching probe.py's enum_frontier at width 6 and core.search's first pass at width 16. None of Bad Theory Labs' code was executed.",
 "source": "https://github.com/Badtheorylabs/interference-search",
 "captured": "2026-09-26",
 "note": "Two things hold across all 34 rows. A breadth-first pass over merged states with no judge at all reaches every second-level state after judging at most 150 positions, so it solves every row. And the ranking matters: at width 6 a random order solves 14.4% of the seed-0 set in expectation (about 4.3 of 30), which is where the paper's 'ask the model yes or no' judge lands (4 of 30), and far below the trained judge's 23. The seed-0 problems are a different draw from the same generator as the paper's 30; the paper does not publish the numbers of those 30.",
 "columns": [
  {
   "key": "problem",
   "label": "numbers → target"
  },
  {
   "key": "set",
   "label": "set"
  },
  {
   "key": "l1",
   "label": "level 1",
   "align": "right"
  },
  {
   "key": "l2",
   "label": "level 2",
   "align": "right"
  },
  {
   "key": "l2alive",
   "label": "alive at 2",
   "align": "right"
  },
  {
   "key": "nonterm",
   "label": "all states",
   "align": "right"
  },
  {
   "key": "paths",
   "label": "paths",
   "align": "right"
  },
  {
   "key": "sols",
   "label": "solution paths",
   "align": "right"
  },
  {
   "key": "w6",
   "label": "random, W=6",
   "align": "right"
  },
  {
   "key": "w16",
   "label": "random, W=16",
   "align": "right"
  }
 ],
 "rows": [
  {
   "problem": "25, 11, 22, 14 → 451",
   "set": "hard, seed 0, #0",
   "l1": 19,
   "l2": 96,
   "l2alive": 1,
   "nonterm": 116,
   "paths": 587,
   "sols": 1,
   "w6": "4.3%",
   "w16": "15.2%"
  },
  {
   "problem": "34, 12, 1, 8 → 434",
   "set": "hard, seed 0, #1",
   "l1": 16,
   "l2": 75,
   "l2alive": 1,
   "nonterm": 92,
   "paths": 497,
   "sols": 1,
   "w6": "5.5%",
   "w16": "20.6%"
  },
  {
   "problem": "2, 5, 22, 27 → 226",
   "set": "hard, seed 0, #2",
   "l1": 19,
   "l2": 99,
   "l2alive": 1,
   "nonterm": 119,
   "paths": 585,
   "sols": 1,
   "w6": "4.1%",
   "w16": "15.3%"
  },
  {
   "problem": "13, 92, 1, 16 → 299",
   "set": "hard, seed 0, #3",
   "l1": 16,
   "l2": 75,
   "l2alive": 3,
   "nonterm": 92,
   "paths": 506,
   "sols": 4,
   "w6": "17.6%",
   "w16": "53.1%"
  },
  {
   "problem": "16, 15, 19, 57 → 582",
   "set": "hard, seed 0, #4",
   "l1": 19,
   "l2": 105,
   "l2alive": 1,
   "nonterm": 125,
   "paths": 566,
   "sols": 1,
   "w6": "3.6%",
   "w16": "14.3%"
  },
  {
   "problem": "3, 69, 2, 19 → 270",
   "set": "hard, seed 0, #5",
   "l1": 19,
   "l2": 110,
   "l2alive": 1,
   "nonterm": 130,
   "paths": 577,
   "sols": 3,
   "w6": "8.2%",
   "w16": "15.8%"
  },
  {
   "problem": "20, 11, 7, 30 → 390",
   "set": "hard, seed 0, #6",
   "l1": 18,
   "l2": 96,
   "l2alive": 1,
   "nonterm": 115,
   "paths": 503,
   "sols": 1,
   "w6": "3.9%",
   "w16": "15.8%"
  },
  {
   "problem": "24, 23, 18, 88 → 153",
   "set": "hard, seed 0, #7",
   "l1": 18,
   "l2": 95,
   "l2alive": 7,
   "nonterm": 114,
   "paths": 496,
   "sols": 18,
   "w6": "47.1%",
   "w16": "75.9%"
  },
  {
   "problem": "12, 17, 54, 16 → 338",
   "set": "hard, seed 0, #8",
   "l1": 18,
   "l2": 95,
   "l2alive": 3,
   "nonterm": 114,
   "paths": 496,
   "sols": 4,
   "w6": "14.5%",
   "w16": "43.3%"
  },
  {
   "problem": "70, 24, 12, 16 → 123",
   "set": "hard, seed 0, #9",
   "l1": 19,
   "l2": 107,
   "l2alive": 1,
   "nonterm": 127,
   "paths": 581,
   "sols": 1,
   "w6": "4.0%",
   "w16": "14.7%"
  },
  {
   "problem": "25, 89, 25, 5 → 531",
   "set": "hard, seed 0, #10",
   "l1": 13,
   "l2": 70,
   "l2alive": 3,
   "nonterm": 84,
   "paths": 652,
   "sols": 4,
   "w6": "20.4%",
   "w16": "54.3%"
  },
  {
   "problem": "3, 13, 41, 19 → 589",
   "set": "hard, seed 0, #11",
   "l1": 18,
   "l2": 99,
   "l2alive": 1,
   "nonterm": 118,
   "paths": 514,
   "sols": 3,
   "w6": "9.2%",
   "w16": "16.8%"
  },
  {
   "problem": "47, 14, 18, 15 → 875",
   "set": "hard, seed 0, #12",
   "l1": 18,
   "l2": 94,
   "l2alive": 3,
   "nonterm": 113,
   "paths": 508,
   "sols": 4,
   "w6": "14.8%",
   "w16": "43.0%"
  },
  {
   "problem": "15, 13, 64, 10 → 344",
   "set": "hard, seed 0, #13",
   "l1": 18,
   "l2": 98,
   "l2alive": 1,
   "nonterm": 117,
   "paths": 516,
   "sols": 1,
   "w6": "4.5%",
   "w16": "14.6%"
  },
  {
   "problem": "74, 4, 25, 9 → 888",
   "set": "hard, seed 0, #14",
   "l1": 18,
   "l2": 98,
   "l2alive": 1,
   "nonterm": 117,
   "paths": 510,
   "sols": 3,
   "w6": "9.0%",
   "w16": "17.6%"
  },
  {
   "problem": "11, 92, 9, 16 → 505",
   "set": "hard, seed 0, #15",
   "l1": 18,
   "l2": 100,
   "l2alive": 1,
   "nonterm": 119,
   "paths": 521,
   "sols": 2,
   "w6": "7.0%",
   "w16": "17.5%"
  },
  {
   "problem": "10, 4, 35, 20 → 280",
   "set": "hard, seed 0, #16",
   "l1": 20,
   "l2": 113,
   "l2alive": 5,
   "nonterm": 134,
   "paths": 627,
   "sols": 10,
   "w6": "27.1%",
   "w16": "56.0%"
  },
  {
   "problem": "15, 9, 1, 65 → 201",
   "set": "hard, seed 0, #17",
   "l1": 16,
   "l2": 75,
   "l2alive": 3,
   "nonterm": 92,
   "paths": 514,
   "sols": 4,
   "w6": "16.3%",
   "w16": "51.1%"
  },
  {
   "problem": "6, 10, 44, 9 → 308",
   "set": "hard, seed 0, #18",
   "l1": 18,
   "l2": 100,
   "l2alive": 2,
   "nonterm": 119,
   "paths": 526,
   "sols": 4,
   "w6": "13.6%",
   "w16": "30.5%"
  },
  {
   "problem": "62, 13, 7, 8 → 656",
   "set": "hard, seed 0, #19",
   "l1": 18,
   "l2": 97,
   "l2alive": 1,
   "nonterm": 116,
   "paths": 515,
   "sols": 3,
   "w6": "9.1%",
   "w16": "16.6%"
  },
  {
   "problem": "21, 11, 15, 28 → 75",
   "set": "hard, seed 0, #20",
   "l1": 18,
   "l2": 97,
   "l2alive": 7,
   "nonterm": 116,
   "paths": 507,
   "sols": 18,
   "w6": "47.7%",
   "w16": "74.2%"
  },
  {
   "problem": "60, 24, 20, 10 → 270",
   "set": "hard, seed 0, #21",
   "l1": 21,
   "l2": 129,
   "l2alive": 1,
   "nonterm": 151,
   "paths": 722,
   "sols": 1,
   "w6": "2.9%",
   "w16": "10.1%"
  },
  {
   "problem": "20, 11, 49, 18 → 467",
   "set": "hard, seed 0, #22",
   "l1": 18,
   "l2": 98,
   "l2alive": 1,
   "nonterm": 117,
   "paths": 526,
   "sols": 1,
   "w6": "4.0%",
   "w16": "15.9%"
  },
  {
   "problem": "51, 3, 3, 8 → 714",
   "set": "hard, seed 0, #23",
   "l1": 13,
   "l2": 71,
   "l2alive": 1,
   "nonterm": 85,
   "paths": 671,
   "sols": 3,
   "w6": "10.2%",
   "w16": "22.2%"
  },
  {
   "problem": "21, 25, 20, 3 → 315",
   "set": "hard, seed 0, #24",
   "l1": 19,
   "l2": 104,
   "l2alive": 5,
   "nonterm": 124,
   "paths": 555,
   "sols": 8,
   "w6": "24.3%",
   "w16": "58.4%"
  },
  {
   "problem": "3, 10, 15, 64 → 514",
   "set": "hard, seed 0, #25",
   "l1": 19,
   "l2": 104,
   "l2alive": 1,
   "nonterm": 124,
   "paths": 576,
   "sols": 3,
   "w6": "8.3%",
   "w16": "16.8%"
  },
  {
   "problem": "7, 4, 56, 5 → 116",
   "set": "hard, seed 0, #26",
   "l1": 20,
   "l2": 115,
   "l2alive": 1,
   "nonterm": 136,
   "paths": 641,
   "sols": 1,
   "w6": "3.6%",
   "w16": "12.4%"
  },
  {
   "problem": "9, 20, 29, 18 → 133",
   "set": "hard, seed 0, #27",
   "l1": 19,
   "l2": 94,
   "l2alive": 3,
   "nonterm": 114,
   "paths": 594,
   "sols": 4,
   "w6": "13.7%",
   "w16": "42.0%"
  },
  {
   "problem": "19, 19, 74, 16 → 249",
   "set": "hard, seed 0, #28",
   "l1": 12,
   "l2": 60,
   "l2alive": 3,
   "nonterm": 73,
   "paths": 551,
   "sols": 8,
   "w6": "25.6%",
   "w16": "62.8%"
  },
  {
   "problem": "14, 54, 19, 17 → 104",
   "set": "hard, seed 0, #29",
   "l1": 18,
   "l2": 99,
   "l2alive": 7,
   "nonterm": 118,
   "paths": 515,
   "sols": 18,
   "w6": "47.8%",
   "w16": "74.2%"
  },
  {
   "problem": "24, 98, 19, 3 → 361",
   "set": "showcase (seed 11)",
   "l1": 19,
   "l2": 106,
   "l2alive": 3,
   "nonterm": 126,
   "paths": 556,
   "sols": 4,
   "w6": "12.8%",
   "w16": "38.6%"
  },
  {
   "problem": "14, 17, 5, 36 → 737",
   "set": "showcase (seed 11)",
   "l1": 18,
   "l2": 99,
   "l2alive": 1,
   "nonterm": 118,
   "paths": 527,
   "sols": 1,
   "w6": "4.2%",
   "w16": "14.5%"
  },
  {
   "problem": "7, 5, 8, 55 → 880",
   "set": "showcase (seed 11)",
   "l1": 19,
   "l2": 106,
   "l2alive": 3,
   "nonterm": 126,
   "paths": 564,
   "sols": 4,
   "w6": "13.4%",
   "w16": "37.8%"
  },
  {
   "problem": "65, 12, 14, 18 → 308",
   "set": "showcase (seed 11)",
   "l1": 18,
   "l2": 98,
   "l2alive": 1,
   "nonterm": 117,
   "paths": 507,
   "sols": 2,
   "w6": "7.5%",
   "w16": "18.4%"
  }
 ]
}