{
 "claim": "The UltraData-Code card says L2 is ~400B tokens and L3 is ~150B, across 11 languages. Tokenised with MiniCPM5-2B's own tokenizer, L2 is 431.6B — 7.9% ABOVE the round number it advertises — and L3's full_content field is 137.1B, 8.6% below. The 150B is the full_content column specifically: read the `content` column, which is what the default config hands you and what the card calls \"a serialization containing task and solution\", and L3 is 52.4B tokens, a third of the headline.",
 "method": "No bulk download. Exact row counts per split from the Hugging Face datasets-server /size endpoint. Mean character length per column from /statistics (exact for L2/r, L3/r and L3/sh; a 1.3–60% leading-shard sample for the other 19 splits). Extrapolated total characters = mean x exact rows; that extrapolation reproduces the corpus's full Arrow byte total (3,061,255,972,282 B) to within 0.07%, and every split to within 0.7%, so the leading-shard samples are representative. Characters-per-token measured by tokenising 500 rows per split (5 windows of 100, spread across each split via /rows) with tokenizer.json from openbmb/MiniCPM5-2B — 11,000 rows in all. Tokens = extrapolated characters / measured characters-per-token. A different tokenizer moves every row here by roughly the same factor.",
 "source": "https://huggingface.co/datasets/openbmb/UltraData-Code",
 "captured": "2026-09-18",
 "note": "Token counts are estimates with a measured, validated error term — not exact counts. The row counts and byte totals are exact. chars/tok is the measured ratio for that split's cited field.",
 "columns": [
  {
   "key": "split",
   "label": "split"
  },
  {
   "key": "rows",
   "label": "rows",
   "align": "right"
  },
  {
   "key": "gb",
   "label": "parquet GB",
   "align": "right"
  },
  {
   "key": "field",
   "label": "field counted"
  },
  {
   "key": "cpt",
   "label": "chars/tok",
   "align": "right"
  },
  {
   "key": "tokens",
   "label": "tokens (measured)",
   "align": "right"
  }
 ],
 "rows": [
  {
   "split": "L2/cpp",
   "rows": 23290266,
   "gb": 32.4,
   "field": "content",
   "tokens": 24052002126,
   "cpt": 3.229
  },
  {
   "split": "L2/cs",
   "rows": 25779706,
   "gb": 55.5,
   "field": "content",
   "tokens": 39277230065,
   "cpt": 4.4707
  },
  {
   "split": "L2/go",
   "rows": 3778910,
   "gb": 6.1,
   "field": "content",
   "tokens": 4265862474,
   "cpt": 3.0429
  },
  {
   "split": "L2/java",
   "rows": 43480052,
   "gb": 68.0,
   "field": "content",
   "tokens": 45456827449,
   "cpt": 3.9843
  },
  {
   "split": "L2/js",
   "rows": 57088927,
   "gb": 175.2,
   "field": "content",
   "tokens": 119976223657,
   "cpt": 3.7783
  },
  {
   "split": "L2/php",
   "rows": 27474889,
   "gb": 125.0,
   "field": "content",
   "tokens": 101587141433,
   "cpt": 3.444
  },
  {
   "split": "L2/py",
   "rows": 72844227,
   "gb": 127.8,
   "field": "content",
   "tokens": 82331233678,
   "cpt": 3.7189
  },
  {
   "split": "L2/r",
   "rows": 843971,
   "gb": 1.5,
   "field": "content",
   "tokens": 1086633206,
   "cpt": 3.2024
  },
  {
   "split": "L2/rb",
   "rows": 6480797,
   "gb": 8.7,
   "field": "content",
   "tokens": 5343099084,
   "cpt": 3.7979
  },
  {
   "split": "L2/rust",
   "rows": 2746449,
   "gb": 5.7,
   "field": "content",
   "tokens": 4338479618,
   "cpt": 3.593
  },
  {
   "split": "L2/sh",
   "rows": 3070182,
   "gb": 5.3,
   "field": "content",
   "tokens": 3888928178,
   "cpt": 3.0278
  },
  {
   "split": "L3/cpp",
   "rows": 22600686,
   "gb": 181.5,
   "field": "full_content",
   "tokens": 39945026734,
   "cpt": 3.4071
  },
  {
   "split": "L3/cs",
   "rows": 3614454,
   "gb": 29.7,
   "field": "full_content",
   "tokens": 6196622824,
   "cpt": 3.7481
  },
  {
   "split": "L3/go",
   "rows": 1948148,
   "gb": 12.9,
   "field": "full_content",
   "tokens": 2897055180,
   "cpt": 3.2902
  },
  {
   "split": "L3/java",
   "rows": 21415405,
   "gb": 147.5,
   "field": "full_content",
   "tokens": 32305366414,
   "cpt": 3.8916
  },
  {
   "split": "L3/js",
   "rows": 6131527,
   "gb": 31.1,
   "field": "full_content",
   "tokens": 6757233858,
   "cpt": 3.7277
  },
  {
   "split": "L3/php",
   "rows": 2306009,
   "gb": 25.2,
   "field": "full_content",
   "tokens": 6137074277,
   "cpt": 2.8461
  },
  {
   "split": "L3/py",
   "rows": 20750202,
   "gb": 159.3,
   "field": "full_content",
   "tokens": 39123061320,
   "cpt": 2.8784
  },
  {
   "split": "L3/r",
   "rows": 304016,
   "gb": 2.9,
   "field": "full_content",
   "tokens": 547653878,
   "cpt": 3.4243
  },
  {
   "split": "L3/rb",
   "rows": 846872,
   "gb": 4.6,
   "field": "full_content",
   "tokens": 1017355944,
   "cpt": 3.615
  },
  {
   "split": "L3/rust",
   "rows": 922564,
   "gb": 7.9,
   "field": "full_content",
   "tokens": 1616150587,
   "cpt": 3.5623
  },
  {
   "split": "L3/sh",
   "rows": 365222,
   "gb": 2.0,
   "field": "full_content",
   "tokens": 536902201,
   "cpt": 3.5586
  },
  {
   "split": "L2 total",
   "rows": 266878376,
   "gb": 611.3,
   "field": "content",
   "tokens": 431603660968,
   "cpt": null
  },
  {
   "split": "L3 total",
   "rows": 81205105,
   "gb": 604.7,
   "field": "full_content",
   "tokens": 137079503217,
   "cpt": null
  }
 ]
}