# UltraData-Code: counting a 1.2 TB corpus without downloading it
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/ultradata-code
> date: 2026-09-18
> tags: datasets, pretraining, code-llm, data-provenance, licensing
A dataset is the most checkable thing anyone ships. A model card can claim a benchmark I have no cluster to reproduce; a corpus is a pile of files with a row count, and the row count is either right or it isn't. OpenBMB released [`openbmb/UltraData-Code`](https://huggingface.co/datasets/openbmb/UltraData-Code) on 7 September 2026 with two headline numbers — "**UltraData-Code-L2** (~**400B tokens**) and **UltraData-Code-L3** (~**150B tokens**) across **11 programming languages**" — so I went and counted.
The whole thing is 1,215,994,166,161 bytes of parquet. I have 1.3 GB of free disk. Everything below was measured through Hugging Face's datasets-server APIs and about 11,000 sampled rows, and I say which method produced which number every time.
## What got released, exactly
Two configs, eleven splits each, one split per language. The `/size` endpoint gives exact row counts and byte totals for every one of them, computed by the Hub over the full parquet set rather than sampled:
```bash
curl -sS 'https://datasets-server.huggingface.co/size?dataset=openbmb%2FUltraData-Code' \
| python3 -c 'import json,sys; d=json.load(sys.stdin)["size"];
print(f"{d[\"dataset\"][\"num_rows\"]:,} rows, {d[\"dataset\"][\"num_bytes_parquet_files\"]:,} bytes")'
# 348,083,481 rows, 1,215,994,166,161 bytes
```
348,083,481 rows: 266,878,376 in L2 and 81,205,105 in L3. The split is almost exactly even on disk — 611.3 GB for L2, 604.7 GB for L3 — which is the first thing that looks wrong, because L2 is advertised at 400B tokens and L3 at 150B. Nearly equal bytes for a 2.7× token gap means something is being stored more than once.
## Counting tokens without the tokens
Row counts are free. Token counts are not, and the card never says which tokenizer produced the 400B and 150B. Here is the method I used, because the method is the number:
1. **Exact rows** per split from `/size`, as above.
2. **Mean character length per column** from `/statistics`, which returns `min`, `max`, `mean`, `median` and a histogram for every column. For three splits — `L2/r`, `L3/r`, `L3/sh` — the server reads the whole split and the response carries `"partial": false`. For the other nineteen it reads the leading shards up to a size cap, between 1.3% and 60% of the rows.
3. **Extrapolate** total characters as mean × exact rows.
4. **Measure characters-per-token** by pulling 500 rows per split through `/rows` (five windows of 100 at offsets 0/20/40/60/80%) and tokenising them with `tokenizer.json` from [`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B) — the model this corpus exists to feed. 11,000 rows in total.
Step 2 is the one that should worry you, because a leading-shard sample is not a random sample. So I checked it. `/size` also returns `num_bytes_memory`, the Arrow in-memory size of the **entire** corpus, computed over all 348 million rows — a number my extrapolation has no access to and must nonetheless predict:
```
predicted 3,059,031,440,460 B (sum over splits of mean-chars x exact-rows + offsets)
actual 3,061,255,972,282 B (datasets-server, full corpus)
ratio 0.9993
```
0.07% off across the corpus, and every individual split within 0.7%. The leading shards are representative. That is the only reason I am willing to publish the next chart.
L2 measures 431.6B tokens against an advertised ~400B. That is 7.9% **more** than claimed, which is the rare direction. L3's `full_content` measures 137.1B against ~150B, 8.6% under. Both are inside what a tokenizer swap could plausibly explain, and I would not write a paragraph about either.
The number worth a paragraph is the third bar. `content` — the column of that name, the one the default config loads, the one the card's own Data Formats section describes as "serialization containing task and solution" — is **52.4B tokens**. If you `load_dataset` L3 and train on `content`, believing you have the 150B corpus the card advertises, you have about a third of it.
**Receipts.** The UltraData-Code card says L2 is ~400B tokens and L3 is ~150B, across 11 languages. Tokenised with MiniCPM5-2B's own tokenizer, L2 is 431.6B — 7.9% ABOVE the round number it advertises — and L3's full_content field is 137.1B, 8.6% below. The 150B is the full_content column specifically: read the `content` column, which is what the default config hands you and what the card calls "a serialization containing task and solution", and L3 is 52.4B tokens, a third of the headline.
| split | rows | parquet GB | field counted | chars/tok | tokens (measured) |
| :--- | ---: | ---: | :--- | ---: | ---: |
| L2/cpp | 23,290,266 | 32.4 | content | 3.229 | 24,052,002,126 |
| L2/cs | 25,779,706 | 55.5 | content | 4.471 | 39,277,230,065 |
| L2/go | 3,778,910 | 6.1 | content | 3.043 | 4,265,862,474 |
| L2/java | 43,480,052 | 68 | content | 3.984 | 45,456,827,449 |
| L2/js | 57,088,927 | 175.2 | content | 3.778 | 119,976,223,657 |
| L2/php | 27,474,889 | 125 | content | 3.444 | 101,587,141,433 |
| L2/py | 72,844,227 | 127.8 | content | 3.719 | 82,331,233,678 |
| L2/r | 843,971 | 1.5 | content | 3.202 | 1,086,633,206 |
| L2/rb | 6,480,797 | 8.7 | content | 3.798 | 5,343,099,084 |
| L2/rust | 2,746,449 | 5.7 | content | 3.593 | 4,338,479,618 |
| L2/sh | 3,070,182 | 5.3 | content | 3.028 | 3,888,928,178 |
| L3/cpp | 22,600,686 | 181.5 | full_content | 3.407 | 39,945,026,734 |
| L3/cs | 3,614,454 | 29.7 | full_content | 3.748 | 6,196,622,824 |
| L3/go | 1,948,148 | 12.9 | full_content | 3.29 | 2,897,055,180 |
| L3/java | 21,415,405 | 147.5 | full_content | 3.892 | 32,305,366,414 |
| L3/js | 6,131,527 | 31.1 | full_content | 3.728 | 6,757,233,858 |
| L3/php | 2,306,009 | 25.2 | full_content | 2.846 | 6,137,074,277 |
| L3/py | 20,750,202 | 159.3 | full_content | 2.878 | 39,123,061,320 |
| L3/r | 304,016 | 2.9 | full_content | 3.424 | 547,653,878 |
| L3/rb | 846,872 | 4.6 | full_content | 3.615 | 1,017,355,944 |
| L3/rust | 922,564 | 7.9 | full_content | 3.562 | 1,616,150,587 |
| L3/sh | 365,222 | 2 | full_content | 3.559 | 536,902,201 |
| L2 total | 266,878,376 | 611.3 | content | — | 431,603,660,968 |
| L3 total | 81,205,105 | 604.7 | full_content | — | 137,079,503,217 |
Token counts are estimates with a measured, validated error term — not exact counts. The row counts and byte totals are exact. chars/tok is the measured ratio for that split's cited field.
> method: No bulk download. Exact row counts per split from the Hugging Face datasets-server /size endpoint. Mean character length per column from /statistics (exact for L2/r, L3/r and L3/sh; a 1.3–60% leading-shard sample for the other 19 splits). Extrapolated total characters = mean x exact rows; that extrapolation reproduces the corpus's full Arrow byte total (3,061,255,972,282 B) to within 0.07%, and every split to within 0.7%, so the leading-shard samples are representative. Characters-per-token measured by tokenising 500 rows per split (5 windows of 100, spread across each split via /rows) with tokenizer.json from openbmb/MiniCPM5-2B — 11,000 rows in all. Tokens = extrapolated characters / measured characters-per-token. A different tokenizer moves every row here by roughly the same factor.
> source: https://huggingface.co/datasets/openbmb/UltraData-Code
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/ultradata-code/data/token-ledger.json (24 rows)
## Why L3 is 604 GB
Pull one row and the storage question answers itself. This is `UltraData-Code-L3`, split `py`, offset 0, straight off `/rows`, with the long fields cut:
```json
{
"uuid": "02ef0d3e-968f-4246-9def-862554b44b2c",
"content": "Write a Python function `draw_star_square(n)` that takes a positive …",
"content_format": "markdown",
"raw_content": "import sys\n\ninput = lambda: sys.stdin.readline().rstrip()\nn = int(input())\n…",
"task": "Write a Python function `draw_star_square(n)` that takes a positive …",
"analysis": "The approach is to directly simulate the pattern. First, compute side …",
"solution": "def draw_star_square(n):\n \"\"\"\n Generate a square pattern of …",
"test": "assert draw_star_square(1) == [\"*\"]\nassert draw_star_square(2) == [ …",
"full_content": "Write a Python function `draw_star_square(n)` that takes a positive …",
"full_content_format": "standard order"
}
```
Measure the fields on that row and the arithmetic is exact: `task` 746 + `analysis` 1,645 + `solution` 1,004 + `test` 680 = 4,075 bytes, and `full_content` is 4,078 — the four fields concatenated with three separators. `content` is 1,766, and `task` + `solution` is 1,750. Every generated token is on disk two or three times.
Across the whole of L3 that comes to 402.5B tokens of stored text for a corpus described as 150B — 2.94× redundancy. In bytes of parquet per delivered token, L2 costs 1.416 and L3 costs 4.411 — L3 is 3.1× more expensive to store per token you can actually train on. Plan disk from the card's 150B and you will be short.
This is a defensible design, not a bug. Shipping the parts separately means you can train on `solution` alone, or on `task` + `test` for an eval set, without re-parsing a serialized blob. But it means "approximately 150B tokens" is a statement about one column out of ten, and the card never says which one.
`raw_content` is the other 79.0B tokens, and it is not generated at all — it is the original GitHub source file the exercise was derived from, carried along verbatim. Hold that thought; it comes back in the licence section.
## "Algorithmically relevant" is a claim about composition
L2's pitch is that it selects "algorithmically relevant code" out of L1 using a role model and a relevance model. Both models' outputs ship inside the data — `category` and `algo_rel_score` are columns — which makes the claim checkable without running anything.
Corpus-wide, weighting each split's `/statistics` frequency table by its exact row count, L2 is:
| label | rows | share |
|---|---:|---:|
| TOOL | 111,232,432 | 41.68% |
| ALGO | 71,038,674 | 26.62% |
| WEB | 39,635,179 | 14.85% |
| CONFIG | 21,034,432 | 7.88% |
| TEST | 15,664,131 | 5.87% |
| DATA | 8,273,528 | 3.10% |
The plurality label in "algorithmically relevant code" is TOOL. ALGO is a quarter of it.
The card is not hiding this — it says the framework learns "algorithmic relevance beyond explicit ALGO files", which is exactly what a 26.6% ALGO share looks like. But per language the spread is enormous, and the spread is where the interesting part is. Pick a language:
C++ is 86.7% ALGO and genuinely looks like the thing on the tin. Ruby is 43.1% CONFIG and 10.2% ALGO. PHP is 4.0% ALGO. Shell is 86.3% TOOL. The name "UltraData-Code-L2" describes C++ well and Ruby not at all.
The second panel is the part the release never wrote down anywhere. A corpus cannot hide its own thresholds: the minimum surviving score **is** the cut. Across all eleven languages no file has a `quality_score` below 3.0 — the lowest surviving score ranges from 3.00000 in C++ to 3.00019 in JavaScript, which is a `>= 3.0` gate seen from the inside. And `algo_rel_score` has a per-language floor:
```
cpp 0.85 php 0.85 sh 0.85
js 0.75 rb 0.75
java 0.70
cs 0.60
go 0.50 r 0.50 rust 0.50
py 0.00
```
That ladder is what "language-adaptive" means in practice, and it is a reasonable thing to do. Nine of the eleven floors sit at a round number. Python's is zero.
**Receipts.** UltraData-Code-L2 is sold as "algorithmically relevant code". By the project's own role classifier, shipped in the `category` column of the data itself, 26.6% of its 266,878,376 rows are ALGO. The plurality label is TOOL at 41.7%. Two thresholds the card never states are visible in the released scores: no file in any of the eleven languages has a `quality_score` below 3.0 — the lowest surviving score ranges from 3.00000 to 3.00019, and `algo_rel_score` has a per-language floor of 0.50 to 0.85 — except Python, whose floor is 0.00.
| language | rows | algo_rel floor | quality floor | ALGO share | plurality label | its share |
| :--- | ---: | ---: | ---: | ---: | :--- | ---: |
| C++ | 23,290,266 | 0.85 | 3.00000 | 86.7% | ALGO | 86.7% |
| C# | 25,779,706 | 0.60 | 3.00001 | 11.2% | TOOL | 76.9% |
| Go | 3,778,910 | 0.50 | 3.00000 | 44.1% | TOOL | 46.9% |
| Java | 43,480,052 | 0.70 | 3.00012 | 47.1% | ALGO | 47.1% |
| JavaScript | 57,088,927 | 0.75 | 3.00019 | 7.9% | TOOL | 44.9% |
| PHP | 27,474,889 | 0.85 | 3.00002 | 4.0% | TOOL | 47.8% |
| Python | 72,844,227 | 0.00 | 3.00001 | 25.1% | TOOL | 31.0% |
| R | 843,971 | 0.50 | 3.00000 | 19.2% | TOOL | 45.4% |
| Ruby | 6,480,797 | 0.75 | 3.00002 | 10.2% | CONFIG | 43.1% |
| Rust | 2,746,449 | 0.50 | 3.00001 | 30.8% | TOOL | 47.9% |
| Shell | 3,070,182 | 0.85 | 3.00000 | 8.1% | TOOL | 86.3% |
R is the one split measured end to end: 162,047 ALGO files out of 843,971 rows, exactly 19.2%. Python is the outlier on every column — its relevance floor is zero, 11.2% of sampled rows score below 0.1, and it is both the largest split and the one carrying the card's headline Python results table.
> method: Hugging Face datasets-server /statistics, one call per split. The `category` frequency table and the score min/max are computed by the server over the rows it reads: the whole split for R (843,971 rows, partial=false), a 1.3–60% leading-shard sample elsewhere. ALGO share is that sample's share; rows is the exact count from /size.
> source: https://datasets-server.huggingface.co/statistics?dataset=openbmb%2FUltraData-Code&config=UltraData-Code-L2&split=r
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/ultradata-code/data/selection-floors.json (11 rows)
## Python is not like the other ten
Python is the largest split — 72,844,227 rows, 82.3B tokens — and it is the one the card's headline results table is built on. It is also the only split whose relevance filter effectively isn't one.
In the 1,224,287 rows the statistics server read, 21.76% score below 0.5 and 11.21% score below 0.1. Extrapolated against the exact row count that is roughly 15.9M and 8.2M files. In my 500-row samples every other language bottoms out at its floor in every one of the six categories; Python's lowest ALGO-labelled file scores `3.69e-10`, and its lowest non-ALGO file scores 0.103, so the bypass is not the whole story — Python's whole relevance gate is set an order of magnitude lower than anyone else's.
Here is the concrete case, from my sample. Three rows, byte-identical, 5,467 characters each:
```
row 29,137,716 gokuls999/exam_portal
venv/Lib/site-packages/pip/_vendor/rich/_ratio.py
row 43,706,618 ChiragJiwnani/DataCraft---End-to-End-Data-Science-Project-Tool
venv/lib/python3.12/site-packages/pip/_vendor/rich/_ratio.py
row 58,275,388 knagaki1225/memoApp
venv/lib/python3.12/site-packages/pip/_vendor/rich/_ratio.py
category ALGO
quality_score 8.447600364685059
algo_rel_score 2.6479629013920203e-05 / 1.3834218407282606e-05
```
`rich/_ratio.py` is a genuinely nice piece of algorithmic code — it solves a proportional layout allocation. It is also pip's vendored copy of it, sitting inside three different people's committed virtualenvs. The role model calls it ALGO. The relevance model scores it at 0.000026. Two models that are supposed to agree on the same file, shipped side by side in the same row, disagreeing by five orders of magnitude — and in Python the ALGO label is what carries the file through.
## What survived deduplication
L1 is where the card says near-duplicate removal happens: "MinHash signatures are generated and LSH retrieves near-duplicate candidates. Similarity edges define connected components. For groups containing multiple near-duplicates, the highest ranked 50% are retained." L2 is selected from L1, so exact byte-for-byte repeats in L2 are the easiest possible case for that stage — identical files have identical signatures and Jaccard 1.0.
I hashed the `content` column of all 5,500 sampled L2 rows.
178 of 5,500, or 3.24%, have a byte-identical twin inside the same small sample. PHP is 12.0%, Ruby 10.4%; Java and Rust are zero. And the shape of the duplicates is consistent: **all 178 pairs cross a repository boundary**. Not one is a within-repo copy. Whatever ran at L1 clearly caught duplicates inside a repository and did not catch the same generated or vendored file appearing in thousands of unrelated ones.
To be fair to the pipeline: "the highest ranked 50% are retained" means a group of four near-duplicates is *supposed* to keep two. Retaining exact copies is the documented behaviour, not a failure of it. The honest framing is that "near-duplicate deduplication" in this pipeline halves duplicate groups rather than collapsing them, and the card does say so if you read the sentence carefully. What the card does not say is that the residue is 3.2% of rows in a 5,500-row sample, or that it is concentrated in framework boilerplate: Flutter's generated `windows/runner/utils.cpp` five times, Laravel's stock `config/database.php` seven times.
There is one more thing in that data I did not go looking for. Of the 106 duplicate groups, **67 carry byte-identical content with different scores attached**:
```
windows/runner/utils.cpp, 1,788 bytes, 5 identical copies
row 3 ambedgar777/quiz_flutter q 6.763549327850342 algo 0.9988085031509399
row 4,658,133 Emad-Pro/Mini-Wall-Social-App q 6.760730266571045 algo 0.9987998008728027
row 13,974,204 samir802/fileformatting q 6.760730266571045 algo 0.9987998008728027
row 13,974,239 jackson338/gemini_goals q 6.760730266571045 algo 0.9987998008728027
row 18,632,256 dev-ravan/hive_database_flutter q 6.760730266571045 algo 0.9987998008728027
```
Same bytes in, different score out. The card says the framework "reuses precomputed semantic embeddings across role, relevance, and quality models", which makes this a statement about the embedding pass: identical inputs did not produce identical embeddings. The usual cause is batch-shape-dependent reduction order on a GPU, and the magnitude here — 3e-3 on a 0–10 scale, 9e-6 on a 0–1 scale — is exactly float noise, not a different model. It does not move any file across a threshold. It does mean the selection is not reproducible: re-run the pass and a file sitting on the 3.0 quality boundary could land on either side.
No group disagreed about the `category` label, so the role model at least is stable at the granularity that matters.
**Receipts.** L1 of the pipeline is where "near-duplicate deduplication" happens, and L2 is selected from L1. 178 of 5,500 sampled L2 rows — 3.24% — are byte-for-byte identical to another row in the same 5,500. Every one of the 178 duplicate pairs crosses a repository boundary, so within-repo deduplication clearly worked and cross-repo copies of vendored and template files did not. PHP is 12.0% duplicate rows in-sample, Ruby 10.4%, Java and Rust 0.0%.
| language | rows sampled | duplicate rows | rate | groups | largest group |
| :--- | ---: | ---: | ---: | ---: | ---: |
| C++ | 500 | 5 | 1.00% | 2 | 5 |
| C# | 500 | 21 | 4.20% | 19 | 3 |
| Go | 500 | 1 | 0.20% | 1 | 2 |
| Java | 500 | 0 | 0.00% | 0 | 1 |
| JavaScript | 500 | 7 | 1.40% | 6 | 3 |
| PHP | 500 | 60 | 12.00% | 29 | 7 |
| Python | 500 | 7 | 1.40% | 6 | 3 |
| R | 500 | 2 | 0.40% | 2 | 2 |
| Ruby | 500 | 52 | 10.40% | 24 | 6 |
| Rust | 500 | 0 | 0.00% | 0 | 1 |
| Shell | 500 | 23 | 4.60% | 17 | 4 |
| all 11 | 5,500 | 178 | 3.24% | 106 | 7 |
67 of the 106 duplicate groups carry byte-identical content with different scores attached, so the scoring pass is not deterministic. No group disagreed about the `category` label.
> method: 500 rows per split (5 windows of 100 at offsets 0%, 20%, 40%, 60% and 80%) pulled through the datasets-server /rows endpoint, 11,000 rows over 22 splits, of which these 5,500 are L2. SHA-256 over the UTF-8 bytes of the `content` column; a group is a set of rows sharing one digest. This is a floor, not the corpus rate: a 5,500-row sample of 266,878,376 rows will only catch duplicates common enough to hit twice.
> source: https://datasets-server.huggingface.co/rows?dataset=openbmb%2FUltraData-Code&config=UltraData-Code-L2&split=php&offset=0&length=100
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/ultradata-code/data/duplicates.json (12 rows)
## L3 is not one exercise per ALGO file
The overview figure labels L3's first stage "ALGO Seed" and the card says the protocol "converts **each** algorithmically relevant implementation selected by L2 into a programming exercise". That reads as a map from ALGO files to exercises, and it isn't one.
R is the split where both sides of the comparison are exact. `/statistics` reads all 843,971 rows of `L2/r` (`"partial": false`) and reports 162,047 ALGO files, 19.2005%. `/size` reports 304,016 rows in `L3/r`. That is **1.876 exercises per ALGO file**, from two exact counts with nothing extrapolated.
Every language runs the same way, from 1.05× in Java to 2.08× in PHP. And my samples show why: across the 500 sampled `L3/php` rows, only 347 distinct `raw_content` values appear, while all 500 `full_content` values are distinct. The same source file was turned into several different exercises. That is a sensible thing to do — and "each implementation" is the wrong description of it.
## The benchmark numbers, all of them
Since I have been unkind about the prose, the arithmetic deserves the same treatment. UltraData-Code has no tech report — the card's own header link reads "Tech Report (Coming Soon)" — so every benchmark number in this release exists only as prose next to a PNG of a table. That is a bad place for numbers to live, and it makes them very easy to check.
Thirteen numbers quoted in the card's prose, thirteen subtractions against its own tables:
| claim on the card | check | result |
|---|---|---|
| L2 beats L1 by 7.80 on EvalPlus | 31.42 − 23.62 | 7.80 ✓ |
| L2 beats L1 by 5.13 on MultiPL-E | 20.38 − 15.25 | 5.13 ✓ |
| L2 beats Stack-Edu by 4.37 / 3.05 | 31.42 − 27.05 · 20.38 − 17.33 | 4.37 / 3.05 ✓ |
| L3 mix beats L2 by 8.42 / 8.07 | 39.84 − 31.42 · 28.45 − 20.38 | 8.42 / 8.07 ✓ |
| L3 mix beats best synthetic by 5.57 / 7.80 | 39.84 − 34.27 · 28.45 − 20.65 | 5.57 / 7.80 ✓ |
| L2-py beats L1-py by 17.91 | 43.13 − 25.22 | 17.91 ✓ |
| L2-py beats Stack-Edu-py by 8.46 | 43.13 − 34.67 | 8.46 ✓ |
| L3-py mix reaches 46.43 avg | table row | 46.43 ✓ |
| 3.30 above L2-py alone | 46.43 − 43.13 | 3.30 ✓ |
| 4.46 above the L2+SwallowCode-v2 mix | 46.43 − 41.97 | 4.46 ✓ |
Thirteen for thirteen. Nothing rounded up, nothing quoted from a different table. That is worth stating plainly, because it is the part of this release that is done properly and the part nobody will check.
Two footnotes on the same tables. First, the L2 block of the multilingual table contains `Ultra-Stack-v3`, which beats `UltraData-Code-L2` on MultiPL-E by 2.37 points (22.75 vs 20.38) — the table even bolds it. The card's L2 bullet compares against Stack-Edu and does not mention it. Reporting the win over the older external baseline and skipping the loss to the in-house one is a choice, and it is the one place the card selects its comparison.
Second, the table's bottom row is labelled `UltraData-Code-L3` while the prose calls the same number "the equal-token L2 + L3 mixture". The figure's legend says `L2+L3`. Two of the three agree and the odd one out is the row label, so the prose is right and the table is mislabelled — but you need the figure to know that.
## The licence says two things
The card YAML declares `license: apache-2.0`. The repo ships an 11,335-byte `LICENSE` that is the verbatim Apache License 2.0, whose §2 grants "a perpetual, worldwide, non-exclusive, no-charge, royalty-free, **irrevocable** copyright license to reproduce … and distribute the Work".
Three paragraphs after citing that licence, the README says:
> **No unauthorized unchanged redistribution:** Without prior written permission from the original authors (or this organization), any institution, organization, or third-party platform is strictly prohibited from directly reposting, mirroring, re-hosting, or commercially repackaging and republishing any artifacts of this project in any form.
The Chinese card carries the same sentence, so it is not a translation slip. Whatever that paragraph is, it is not Apache-2.0, and the Hub badge, every mirror, and every automated licence scan will read the YAML and see a permissive corpus.
The provenance section is more careful, and more honest than most: "Public availability of a repository is not a grant of redistribution, commercial use, or training rights", and "Users must also comply with the **LICENSE of each source repository**." That is the right thing to say. It is also impossible to act on for half the release. Compare the two schemas:
```
L2 row: uuid, repo_name, relative_path, content, category,
algo_rel_score, quality_score
L3 row: uuid, content, content_format, raw_content, task, analysis,
solution, test, full_content, full_content_format
```
L2 carries `repo_name` and `relative_path`, so "go read the source repository's licence" is a thing you can do — the duplicate ledger above names a repository for every row it lists. L3 carries `raw_content`, which is the verbatim source file, and drops both. That is 79.0B tokens of third-party source code redistributed with the attribution removed, under a card that instructs you to comply with each source file's licence. The `uuid` column exists in both and might join them; I could not test it, because the datasets-server `/filter` endpoint returns HTTP 500 for this dataset even though `/is-valid` advertises `"filter": true`.
One more gap. L3 is roughly 137B tokens of model-generated text. The card's Acknowledgements name exactly one model — `Qwen3-Embedding-0.6B`, used for L2's file embeddings — and never name the model that generated L3. Without that, nobody can check whether the generating model's terms of use permit redistributing its output, which is the single most common way a synthetic corpus becomes unusable.
**Receipts.** UltraData-Code declares Apache-2.0 in its card YAML and ships the verbatim Apache-2.0 text as LICENSE. Its README then prohibits, in English and Chinese, the reposting and mirroring that Apache-2.0 §2 grants irrevocably. Separately, the card instructs users to comply with each source repository's licence — which L2's schema makes possible and L3's schema makes impossible, because L3 keeps the verbatim source file in raw_content and drops repo_name and relative_path.
| where it is stated | what it says | what that means |
| :--- | :--- | :--- |
| dataset card YAML, line 3 | license: apache-2.0 | Hub renders an Apache-2.0 badge; every mirror and index reads this field |
| LICENSE (11,335 bytes, repo root) | Verbatim Apache License 2.0, §2: “each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce … and distribute the Work” | Grants redistribution outright |
| README.md § License and Data Sources | “This project is released under the Apache 2.0 license.” | Agrees with the YAML |
| README.md § License and Data Sources, next paragraph | “Users must also comply with the LICENSE of each source repository. Apache 2.0 does not override those terms.” | Correct, and impossible to act on for L3 — see below |
| README.md § License and Data Sources, final paragraph | “Without prior written permission … any institution, organization, or third-party platform is strictly prohibited from directly reposting, mirroring, re-hosting, or commercially repackaging and republishing any artifacts of this project in any form.” | Withdraws exactly the right Apache-2.0 §2 grants, three paragraphs after granting it |
| README_ZH.md, same section | “未经原作者（或本组织）事先书面许可，任何机构、组织或第三方平台严禁以任何形式直接转载、镜像、重新托管” | The Chinese card carries the same prohibition, so it is not a translation slip |
| L2 row schema | uuid, repo_name, relative_path, content, category, algo_rel_score, quality_score | Provenance present: you can find the source repo and read its licence |
| L3 row schema | uuid, content, content_format, raw_content, task, analysis, solution, test, full_content, full_content_format | No repo_name, no relative_path — raw_content carries 79.0B tokens of verbatim third-party source with the attribution removed |
| README.md § Acknowledgements | “The UltraData-Code-L2 file-level semantic embeddings are computed with Qwen3-Embedding-0.6B.” | The only model named anywhere on the card. The model that generated L3's 137.1B tokens is never named, in either language |
None of this is a licence-compliance opinion; it is a list of what the release says about itself, in the order it says it. Anyone redistributing this corpus has to reconcile rows 2 and 5 themselves.
> method: Read at commit ebea626 / 86a7233 on 2026-09-18: the card YAML via the Hub API (cardData.license), README.md and README_ZH.md via /raw/main, the LICENSE blob via /resolve/main, and the two row schemas from the README's own Data Formats section, cross-checked against the feature list the datasets-server /rows endpoint returns for each config.
> source: https://huggingface.co/datasets/openbmb/UltraData-Code/blob/main/README.md
> captured: 2026-09-18
> data: https://ai.thesatyajit.com/articles/ultradata-code/data/licence.json (9 rows)
## What I could not check
Three things, stated so nobody mistakes them for measured:
- **L0's "approximately 192 million public GitHub repositories."** Nothing is released at L0 or L1. There is no artifact, so there is no check. The number is not implausible for a full public-GitHub crawl; it is simply unverifiable from this release.
- **The exact corpus token count.** My token numbers are extrapolations with a validated error term (0.07% on the byte cross-check), not exact counts, and they depend on the MiniCPM5-2B tokenizer. A different tokenizer moves every number in the same direction by roughly the same factor. The row counts and byte totals **are** exact.
- **The true duplicate rate.** 3.24% is what a 5,500-row sample of 266,878,376 rows caught. It is a floor. The real rate could be higher; it cannot be lower.
The arXiv link the card uses for its framework, [2602.09003](https://arxiv.org/abs/2602.09003), I did verify independently rather than trusting the card: it resolves to "Data Science and Technology Towards AGI Part I: Tiered Data Management", submitted 9 February 2026, seventeen authors including Xu Han, Zhiyuan Liu and Maosong Sun. It is the L0–L4 framework paper, not a paper about this corpus. (The sibling `openbmb/UltraData-Math` card still carries `arxiv: xxxx.xxxxx` in its YAML, which is the placeholder somebody forgot.)
## The verdict
This is a serious release. 348 million rows, 1.2 TB, eleven languages, both selection models' outputs shipped inside the data so the selection can be audited by anyone with a `curl` and an afternoon — which is more transparency than most corpora of this size offer, and the reason this article was possible at all. Every benchmark delta it quotes is exactly right.
The failures are all of description. "~150B tokens" is one column out of ten and the card never says which. "Algorithmically relevant" is 26.6% ALGO. "Each algorithmically relevant implementation" is 1.88 exercises per implementation in the one split where both numbers are exact. "Near-duplicate deduplication" leaves Laravel's default config in seven times. And a corpus cannot be Apache-2.0 and also prohibited from being mirrored.
None of that makes the data worse. It makes the card worse than the data, which is the easier of the two problems to fix.
A statement from OpenBMB that the 150B counts a different column, or a
published tokenizer + count that reproduces 150B from `content` alone. My
measurement puts `content` at 52.4B and `full_content` at 137.1B with the
MiniCPM5-2B tokenizer; a tokenizer that closes a 2.6× gap does not exist.
A full-corpus count of the `category` column disagreeing with the
per-split frequencies `/statistics` reports. Nineteen of my twenty-two
splits are leading-shard samples; if the later shards are sorted differently
from the early ones, the share moves. The byte cross-check says they are not,
but it constrains lengths, not labels.
A hash over the full `content` column of any split coming back with a
materially lower rate, or any within-repository duplicate pair at all — I
found zero in 178 pairs, and one counterexample kills the "cross-repo only"
half of the claim outright.
An explanation that the differing rows are not in fact byte-identical —
e.g. a normalisation step applied before scoring but after the stored
`content`. I compared SHA-256 over the stored UTF-8 bytes, which is the only
content the release ships; if scoring ran on something else, my premise is
wrong and the scores are fine.
A revision of the README that removes the redistribution prohibition, or a
change of the YAML to a licence that is compatible with it. Both statements
are currently live at the same commit; either edit resolves it.
Every number in this piece is downloadable next to it. The four `` blocks above each link the JSON they render from, and the components read the same committed table. If an arithmetic error is in here, it is in a file you can open.