{
  "claim": "TypeSafe documents that `score` is \"probability-weighted\" and that `confidence` is \"derived from probabilities\", but publishes neither formula. Both are recoverable from its own worked examples: score is exactly the expectation of the level index under the returned distribution (6 of 6 published Score answers, to the last digit), and confidence is the top probability rescaled so that a uniform distribution reads 0 and a point mass reads 1 — (p_max - 1/N) / (1 - 1/N) — on every example the two primitive reference pages publish. The three examples that do not fit are all on other pages.",
  "method": "Full text of docs.typesafe.ai pulled as llms-full.txt (835,504 bytes, fetched 2026-09-18). Every JSON answer object carrying both `probabilities` and `confidence`, or both `probabilities` and `score`, was extracted by parser, not by hand. delta is |published - recomputed|; published probabilities are rounded to 2 decimals on most pages, so half a rounding step is 0.005 and the fit threshold used is 0.011.",
  "source": "https://docs.typesafe.ai/llms-full.txt",
  "captured": "2026-09-18",
  "note": "Every row from /primitives/choice and /primitives/score fits. All three misses are on /api (two) and /introduction/quickstart (one). The quickstart miss is the interesting one: its confidence of 0.596 on a 3-option distribution of (0.84, 0.159, 0.001) matches normalised entropy confidence, 1 - H(p)/ln 3 = 0.594, which no other example does. So the docs appear to carry two different confidence conventions rather than one convention and a typo. I have no API key, so I cannot check which one the live endpoint returns; that is the falsifier.",
  "columns": [
    {
      "key": "page",
      "label": "docs page"
    },
    {
      "key": "kind",
      "label": "type"
    },
    {
      "key": "n",
      "label": "N",
      "align": "right"
    },
    {
      "key": "probs",
      "label": "published probabilities"
    },
    {
      "key": "published",
      "label": "published value"
    },
    {
      "key": "recomputed",
      "label": "recomputed"
    },
    {
      "key": "delta",
      "label": "delta",
      "align": "right"
    },
    {
      "key": "fits",
      "label": "conf fits"
    }
  ],
  "rows": [
    {
      "page": "/api",
      "kind": "choice",
      "n": 3,
      "probs": "0.08, 0.85, 0.07",
      "published": "conf 0.82",
      "recomputed": "0.775",
      "delta": "0.045",
      "fits": "NO"
    },
    {
      "page": "/api",
      "kind": "score",
      "n": 3,
      "probs": "0.05, 0.3, 0.65",
      "published": "conf 0.78; score 1.6",
      "recomputed": "0.475; 1.6",
      "delta": "0.305; 0.000",
      "fits": "NO"
    },
    {
      "page": "/introduction/quickstart",
      "kind": "choice",
      "n": 3,
      "probs": "0.84, 0.159, 0.001",
      "published": "conf 0.596",
      "recomputed": "0.760",
      "delta": "0.164",
      "fits": "NO"
    },
    {
      "page": "/primitives/choice",
      "kind": "choice",
      "n": 3,
      "probs": "0, 1, 0",
      "published": "conf 1",
      "recomputed": "1.000",
      "delta": "0.000",
      "fits": "YES"
    },
    {
      "page": "/primitives/choice",
      "kind": "choice",
      "n": 3,
      "probs": "0.02, 0.38, 0.6",
      "published": "conf 0.39",
      "recomputed": "0.400",
      "delta": "0.010",
      "fits": "YES"
    },
    {
      "page": "/primitives/choice",
      "kind": "choice",
      "n": 5,
      "probs": "1, 0, 0, 0, 0",
      "published": "conf 1",
      "recomputed": "1.000",
      "delta": "0.000",
      "fits": "YES"
    },
    {
      "page": "/primitives/choice",
      "kind": "choice",
      "n": 5,
      "probs": "0.63, 0.37, 0, 0, 0",
      "published": "conf 0.53",
      "recomputed": "0.537",
      "delta": "0.007",
      "fits": "YES"
    },
    {
      "page": "/primitives/choice",
      "kind": "choice",
      "n": 4,
      "probs": "0.1, 0.37, 0.24, 0.29",
      "published": "conf 0.16",
      "recomputed": "0.160",
      "delta": "0.000",
      "fits": "YES"
    },
    {
      "page": "/primitives/choice",
      "kind": "choice",
      "n": 3,
      "probs": "0.08, 0.92, 0",
      "published": "conf 0.88",
      "recomputed": "0.880",
      "delta": "0.000",
      "fits": "YES"
    },
    {
      "page": "/primitives/choice",
      "kind": "choice",
      "n": 2,
      "probs": "0, 1",
      "published": "conf 1",
      "recomputed": "1.000",
      "delta": "0.000",
      "fits": "YES"
    },
    {
      "page": "/primitives/score",
      "kind": "score",
      "n": 3,
      "probs": "0, 0.7, 0.3",
      "published": "conf 0.54; score 1.3",
      "recomputed": "0.550; 1.3",
      "delta": "0.010; 0.000",
      "fits": "YES"
    },
    {
      "page": "/primitives/score",
      "kind": "score",
      "n": 3,
      "probs": "0, 0.76, 0.24",
      "published": "conf 0.63; score 1.24",
      "recomputed": "0.640; 1.24",
      "delta": "0.010; 0.000",
      "fits": "YES"
    },
    {
      "page": "/primitives/score",
      "kind": "score",
      "n": 3,
      "probs": "0, 0.55, 0.45",
      "published": "conf 0.33; score 1.45",
      "recomputed": "0.325; 1.45",
      "delta": "0.005; 0.000",
      "fits": "YES"
    },
    {
      "page": "/primitives/score",
      "kind": "score",
      "n": 4,
      "probs": "0, 0, 0, 1",
      "published": "conf 1; score 3",
      "recomputed": "1.000; 3",
      "delta": "0.000; 0.000",
      "fits": "YES"
    },
    {
      "page": "/primitives/score",
      "kind": "score",
      "n": 3,
      "probs": "0, 0.94, 0.06",
      "published": "conf 0.91; score 1.06",
      "recomputed": "0.910; 1.06",
      "delta": "0.000; 0.000",
      "fits": "YES"
    }
  ]
}