# FineEnvs multilingual: 12 million RL tasks from two datasets, and what a character-error reward teaches Gemma in Kannada
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.thesatyajit.com/articles/fineenvs-multilingual
> date: 2026-10-14
> tags: reinforcement-learning, rl-environments, reward-design, grpo, ocr, asr, speech, datasets, huggingface, lora
Adithya Kolavi [posted](https://x.com/adithya_s_k/status/2110364361231651321) what he calls "the largest open-source multilingual, multimodal RL environments": OCR over "1M+ document pages in 22 languages", speech recognition "in 102 languages", and "12M+ tasks across two OpenEnv Environments". The second post in the thread carries the idea: "You don't need new data to build RL environments." He took two open datasets, [Nayana](https://huggingface.co/datasets/Cognitive-Lab/NayanaOCR_Corpus_2025) by CognitiveLab for OCR and [FLEURS](https://huggingface.co/datasets/google/fleurs) by Google for speech, and wrapped them with verifiable rewards. The third post is the test: Gemma 4 E4B, trained with GRPO on Kannada on one A100, makes "16% fewer character errors on @SarvamAI's held-out OCR benchmark" and "45% fewer character errors across 838 held-out speech clips".
This is the third FineEnvs release the site has read, after the [GeoGuessr environment](/articles/geoguessr-rl-environment) and [PortSimEnv](/articles/portsimenv). Like those, it ships its evidence. The [collection](https://huggingface.co/collections/FineEnvs/multilingual-multimodal-envs) holds the two environment Spaces, two LoRA adapters, a Trackio dashboard, and a dataset, [`FineEnvs/multilingual-multimodal-rl-runs`](https://huggingface.co/datasets/FineEnvs/multilingual-multimodal-rl-runs), with every held-out prediction at every checkpoint and the exact scripts that ran.
So I did what that dataset invites. I cloned both Spaces and read their graders. I re-implemented the reward in Python. I pulled the base and final checkpoints' predictions for both runs through the Hugging Face datasets-server, along with FLEURS' own Kannada split files, and recomputed every number. Each figure below is labelled **measured** (I computed it from a file), **reported** (the publisher's figure, which I did not re-run) or **reasoned** (my arithmetic on the other two).
| | |
|---|---|
| OCR environment | [FineEnvs/nayana-ocr-env](https://huggingface.co/spaces/FineEnvs/nayana-ocr-env), code Apache-2.0, served data CC BY-NC 4.0 |
| ASR environment | [FineEnvs/fleurs-asr-env](https://huggingface.co/spaces/FineEnvs/fleurs-asr-env), code Apache-2.0, served data CC BY 4.0 |
| Adapters | [kannada-ocr-grpo](https://huggingface.co/FineEnvs/gemma-4-E4B-it-kannada-ocr-grpo) (CC BY-NC 4.0), [kannada-asr-grpo](https://huggingface.co/FineEnvs/gemma-4-E4B-it-kannada-asr-grpo) (Apache-2.0) |
| Evidence | [multilingual-multimodal-rl-runs](https://huggingface.co/datasets/FineEnvs/multilingual-multimodal-rl-runs): 20,112 ASR and 6,300 OCR per-item rows |
## What the environment adds to a dataset
A supervised dataset is a list of (input, target) pairs, and the training loop owns the loss. An RL environment inverts that. The model never sees the target. It gets an observation, emits an action, and gets back a number.
Both Spaces implement the [OpenEnv](https://github.com/huggingface/OpenEnv) interface, `reset` and `step` over HTTP or WebSocket. An episode here is one step long. `reset(task_id=…)` returns the observation: a crop of a page with the prompt "Transcribe the text in this cropped document region (language: kn)", or a 16 kHz clip with "Transcribe this Kannada speech". `step(action)` grades the answer and returns the reward with `done=True` (`server/environment.py` in each Space, measured). Task discovery and the observation never include the reference. The OCR README states it plainly: "The reference is revealed after scoring; the API never sends it."
That separation is the whole point. The grader lives on a server, the trainer is a client, and the same server scores training rollouts, the playground in the browser, and the held-out evaluation. A dataset can't promise that the number in a paper is the same function the model was trained against. A versioned server can, and both evaluation files name the grading policy they were scored with (`fleurs-asr-cer-v1` for the Kannada speech run).
For GRPO the environment is exactly what the algorithm needs: a scalar per sample. The trainer draws a group of answers for one input, sends each to `step`, and turns the spread of rewards into advantages. In the speech run that is 16 transcripts per clip at temperature 0.9 (reported, model card). The OCR run uses 8 per crop.
One trainer detail in that figure deserves its own line. The speech model card reports that TRL 1.13 "drops audio features" from the forward pass the GRPO loss is computed on, so a stock run "trains on how likely each transcript is *without* the clip". Their subclass, `AudioGRPOTrainer`, carries the audio into every log-prob pass. The card puts the gap at −0.38 nats per token with the clip and −6.22 without (reported), and says earlier Kannada runs without the fix "raised training reward and barely moved held-out scores". I did not re-run that. It is the most useful sentence in the release for anyone training an audio model with TRL.
## The reward
Both environments use the same shape:
$$
r = 0.8 \cdot \max(0,\ 1 - e) + 0.2 \cdot \mathbb{1}[\text{exact}]
$$
where $e$ is an edit distance divided by the reference length. Most of the reward is continuous, so GRPO always has a gradient to follow, and a fifth of it pays for getting the text exactly right. The error rate is unbounded above, since a model that loops can insert far more characters than the reference has. The `max` clamps that to zero reward.
The details differ between the two graders, and they matter more than the formula.
**OCR** (`nayana-ocr-env/server/rewards.py`, measured). Both strings are normalised to Unicode NFC and their whitespace collapsed, and nothing else changes. Case, punctuation, vowel signs and the zero-width joiners stay, as the code comment says: "Preserve case, punctuation, vowel signs, ZWJ/ZWNJ, and RTL text order." The edit distance runs over code points. An answer longer than `max(1024, 4 × len(reference))` scores 0, checked on the raw length "so whitespace padding cannot bypass the guard".
**ASR** (`fleurs-asr-env/server/rewards.py` and `data/schema.py`, measured). For the `transcription` task, both strings go through NFKC, every Unicode punctuation character becomes a space, and the text is casefolded, because FLEURS' normalised references carry no punctuation or case. The unit then depends on the script. Seven languages that don't put spaces between words (Mandarin, Cantonese, Japanese, Thai, Lao, Burmese and Khmer) are scored per character. Every other language, Kannada included, is scored per **word** by default. A server started with `ASR_REWARD_UNIT=cer` scores characters everywhere, and that is how the Kannada run was trained. The code comment explains why: "A GRPO run on Kannada cut character errors by 14% while word errors moved 2%, and the reward hardly registered the improvement" (reported). Kannada words are long and inflected, so one wrong vowel sign makes the whole word wrong.
That points at the real question for an Indic script: what is a character? The grader counts Unicode code points. A reader of Kannada counts aksharas, the written syllables. ಕನ್ನಡ ("Kannada") is five code points, ಕ ನ ್ ನ ಡ, but three aksharas, ಕ, ನ್ನ and ಡ. The virama ್ glues the two ನ into one conjunct, and a vowel sign such as ಾ or ೆ rides on the consonant before it.
The two units disagree about which mistakes are expensive. In the explorer below, against the reference ಕನ್ನಡ ಭಾಷೆ ("the Kannada language", 10 code points with the space):
- Dropping the conjunct (ಕನಡ) costs two code points, `್` and `ನ`, for an error rate of 0.200 and a reward of 0.640. As aksharas it is one substitution, 1 of 6.
- Hearing ಭ as ಬ, writing the wrong vowel sign, or slipping in a Devanagari क each cost one code point: rate 0.100, reward 0.720.
- Dropping the space costs one code point too, still 0.720. Under the default word rate it is a total miss: two words become one, the rate is 1.0, and the reward is 0.
All of those are measured, by running the environment's own normalisation and distance on each pair.
Two more things the explorer shows. First, normalisation is not decoration. The vowel sign ೋ has a composed code point, U+0CCB, and a canonical decomposition into three: ೆ, ೂ and the length mark ೕ. The two spellings of ಕೋಟೆ ("fort") look identical on screen. Without NFC, the decomposed one is 3 edits off a 4-code-point reference and scores 0.200. With it, the two strings are equal and score 1.0 (measured). Both graders normalise, so a model can't be punished for the encoding its tokenizer happens to emit. A home-rolled grader that forgets this would be.
Second, the reward has opinions you might not expect. The FLEURS sentence in the explorer says "11 12 ಮತ್ತು 13ನೇ ಶತಮಾನಗಳಲ್ಲಿ" (in the 11th, 12th and 13th centuries) with digits, because that is how FLEURS transcribes it. The base model wrote the numbers out as Kannada words, ಹನ್ನೊಂದು ಹನ್ನೆರಡು, which is a correct transcription of what was said. It also wrote ಗಟನೆಗಳು for ಘಟನೆಗಳು and slipped in the English word "hundred". Its error rate is 0.393 and its reward 0.486. After training, the model writes the digits and scores 0.770 (measured, from the stored rows). The reward is teaching FLEURS' house style along with listening. That isn't wrong, since the test set uses the same convention, but it is part of what "45% fewer character errors" measures. I come back to it below.
## Counting the 12 million
Every count in the thread is in the Spaces' `corpus-manifest.json` files, so I summed them.
| | OCR (Nayana) | ASR (FLEURS) |
|---|---:|---:|
| Languages | 22 | 102 |
| Source items | 1,006,170 page renderings | 383,980 clips |
| Task families | section OCR, page OCR, layout, MCQ VQA, descriptive VQA | transcription, verbatim, language ID |
| Indexed tasks | 11,020,101 | 1,151,940 |
All of it is measured. Together that is 12,172,041 tasks, so "12M+" holds. The OCR tasks split into 4,477,445 section-OCR, 4,024,592 descriptive VQA, 1,006,168 layout, 872,448 multiple-choice and 639,448 full-page OCR. On the speech side, each of the 383,980 clips appears three times, once per task family. The 102 languages are every FLEURS config, as the dataset was designed.
The "1M+ document pages" needs one qualification. The manifest gives every one of the 22 languages exactly 45,735 pages, and 22 × 45,735 = 1,006,170 (reasoned). That is the design: the Nayana card calls it "fully-parallel synthetic", "the same ~45,700 source pages are re-rendered into all 22 languages". The text in each region is a machine translation of the English original, rendered with a font such as `NotoSansKannada-Regular.ttf`. So there are a million page images but 45,735 layouts, and the reference text is exact by construction, since the renderer drew it, but its language is machine-generated.
That is not a flaw for OCR training. It does make the OCR evaluation the more interesting one. The model trains on clean synthetic renders and is tested on [Sarvam Indic OCR Bench](https://huggingface.co/datasets/sarvamai/indic-ocr-bench), human-verified crops from printed books and periodicals. That is a real distribution shift, not a held-out slice of the training set.
The splits are leak-proof by construction. Nayana pages are bucketed by a hash of their document id, 80/10/10, so "all its pages, languages, regions, and questions stay together" (`data/schema.py`, measured). FLEURS keeps its published splits, on purpose: "a re-split would silently break comparison with every reported FLEURS score."
## The Kannada runs, recomputed
| | OCR run | ASR run |
|---|---|---|
| Training data | 4,000 crops from the 164,366 Kannada section-OCR train tasks | all 2,282 FLEURS `kn_in` train clips |
| Rollouts | 8 per crop, 500 steps | 16 per clip, 575 steps × 4 clips |
| LoRA | r=16, α=32, language model and vision tower, 98 modules | r=16, α=32, language model, 66 modules |
| Hardware | one A100 80GB, 10.5 h | one A100 80GB, 4.7 h |
| Held-out set | Sarvam Indic OCR Bench test, Kannada, 300 crops | FLEURS `kn_in` test, 838 clips |
The training configuration is reported from the model cards and `run.json`. The two counts I could check against the manifests, 164,366 train tasks and 2,282 train clips, match (measured). 575 steps × 4 clips is 2,300 draws, a shade over the one epoch the card describes (reasoned).
**The reward reproduces exactly.** I ran my re-implementation of each grader over the base and final checkpoints' stored predictions: 838 × 2 speech rows and 300 × 2 OCR rows. Every reward and every character error rate matches the stored value to within 1e-9, all 2,276 of them (measured). The numbers in the evaluation files are the environment's function, applied as written.
### OCR: 16% holds, by Sarvam's metric
Sarvam CER, from the benchmark's own `metrics.py` vendored unmodified, goes from 0.4277 to 0.3602 (measured). That is 15.8% lower (reasoned), which the thread rounds to 16%. 226 crops improve and 59 get worse (measured). Outputs flagged as loops or catastrophic fall from 16 to 6. Crops that contain letters from another Indic script fall from 37 to 14 (measured, by script range). No crop is transcribed exactly, before or after.
Sarvam's metric and the environment's reward don't measure the same thing. Sarvam strips quotes, bullets, ZWJ and ZWNJ, among other "content" folds, while the environment keeps them. 59 of the 300 Kannada references contain a ZWJ or ZWNJ (measured). On the environment's own capped code-point CER the drop is 0.4311 to 0.3592, 16.7% (measured). Pooled over all characters, edits fall from 83,096 to 68,799 on 144,683 reference code points, 17.2% (measured). The three ways of counting agree within a point and a half.
### ASR: 45% holds, on its chosen average
Mean per-clip CER, with each clip capped at 1, goes from 0.1047 to 0.0571 (measured). That is 45.5% lower (reasoned). 565 clips improve and 139 get worse (measured). Exact transcripts rise from 28 to 67 of 838 (measured), the card's 3.3% and 8.0%.
Three other ways of counting the same predictions give smaller numbers:
| Measure | Base | Step 575 | Change |
|---|---:|---:|---:|
| Capped per-clip CER (the headline) | 0.1047 | 0.0571 | −45.5% |
| Uncapped per-clip CER | 0.1335 | 0.0828 | −38.0% |
| Pooled edits over all 111,035 reference code points | 14,155 | 8,270 | −41.6% |
| Capped per-clip CER over aksharas | 0.1323 | 0.0780 | −41.0% |
All measured; the percentages are reasoned from the columns. The card explains its choice of the capped mean: one looping transcript "can add five units of error to the mean". That is defensible, but "45% fewer character errors" read literally is the pooled count, and the pooled count is 41.6%. The akshara row uses my own segmentation, the one in the explorer. Kannada references average 1.76 code points per akshara (measured), and the gain is a little smaller when vowel signs and conjuncts are counted as the units a reader sees.
**Is the test set disjoint from training?** Yes. I downloaded FLEURS' `kn_in` TSVs. Train has 1,319 distinct sentence ids and test has 344, and the two sets share none. No normalised transcript string appears in both either (measured). The environment also skips one train clip for its duration, which is why the manifest has 2,282 where the TSV has 2,283 rows (measured).
**But 838 clips are 344 sentences.** FLEURS has several speakers read each sentence, so each test sentence appears about 2.4 times (reasoned). Sentence 1732, the one in the explorer, is read by three different speakers in the test set (measured). The card's 95% interval, −0.057 to −0.039 on the capped CER, treats the 838 clips as independent (reported). Clustering by sentence would widen it. It would not erase it, since 565 of 838 clips improve.
**How much of it is digits?** 165 of the 838 references contain a digit (measured). On those clips, capped CER halves, 0.172 to 0.082. On the other 673 it falls 0.088 to 0.051, 42% (measured and reasoned). The fifth of the clips with digits supplies 37.4% of the total drop (measured). Transcripts containing any digit rise from 127 to 177 (measured). Part of that is the model learning FLEURS' convention of writing numerals rather than words. Any CER-trained ASR system learns its target's normalisation, so that's fair, but it means the 45% is part listening, part formatting. The cleaner signal is the script one: transcripts with letters from another Indic script fall from 174 to 13 (measured, matching the card's 174 and 13).
**Variance.** Each result is one run with one seed. The repository holds no second seed and no retrained baseline. Checkpoint selection is not a worry here. Both cards pick the final checkpoint, and the held-out curves are flat at the end: capped CER sits between 0.0571 and 0.0585 from step 350 to 575, and Sarvam CER between 0.3601 and 0.3619 from step 300 to 500 (measured, from `curves`). Picking the best checkpoint on the test set would change the third decimal.
## Licences
| Piece | Licence | Source |
|---|---|---|
| Environment code (both Spaces) | Apache-2.0 | Space READMEs, `LICENSE` |
| Nayana pages, crops and annotations as served | CC BY-NC 4.0 plus CognitiveLab's supplemental terms | Nayana card and `LICENSE.md` |
| FLEURS audio and transcripts | CC BY 4.0 | FLEURS card |
| Sarvam Indic OCR Bench | Apache-2.0 | collection note |
| OCR adapter | CC BY-NC 4.0, because of its training data | adapter card |
| ASR adapter | Apache-2.0 | adapter card |
The one to read closely is Nayana. Its `LICENSE.md` combines CC BY-NC 4.0 with supplemental terms, and "where the two conflict, this document controls". The grant is "revocable", and commercial use "requires a separate licence". So the OCR environment's data, and the adapter trained from it, are research-only unless you talk to CognitiveLab. The speech environment, its data and its adapter carry no such restriction.
## What holds
| Claim | Verdict |
|---|---|
| 12M+ tasks across two environments | **Holds.** 12,172,041 indexed tasks (measured) |
| OCR over 1M+ pages in 22 languages | **Holds, with a qualifier.** 1,006,170 renderings of 45,735 synthetic pages, one per language |
| ASR in 102 languages | **Holds.** Every FLEURS config, 383,980 clips |
| 16% fewer character errors on Sarvam's held-out benchmark | **Holds.** Sarvam CER 0.4277 to 0.3602, 15.8% (measured); the benchmark is never offered to the training sampler |
| 45% fewer character errors on 838 held-out clips | **Holds as a capped per-clip mean, 45.5%.** Pooled edits fall 41.6%; 37% of the gain is on the 165 clips with digits; one seed |
| Built from existing data with verifiable rewards | **Holds.** Both rewards are deterministic functions of the reference, and my re-implementation reproduces every stored one |
## What I'd take from it
The environment idea here is simple, and that is why it works. A dataset with a reference becomes an RL environment once you put a deterministic grader behind a server and keep the reference on the server's side. What makes these two good is not their size. It is the dozen small decisions in the graders: NFC so encoding can't cost reward, NFKC and casefolding only where the reference has neither, a character rate where words aren't spaced, a length guard read before whitespace is collapsed, an empty reference rejected rather than scored, and a comment explaining each one.
The Kannada runs say something modest and true. The run files can't tell you whether RL beats supervised fine-tuning on the same 2,282 clips, and nobody here claims it does. They do show a 4B multimodal model learning to stay in its script and to write the way its references are written, from a reward of a few lines, on one GPU. If you want to train on another language, the playgrounds let you score answers before you spend a GPU-hour. And if your script uses conjuncts, decide on purpose whether a code point is the unit you want to pay for. The explorer above is a quick way to see what each choice rewards.
For more on the interface itself, the [multi-harness RL guide](/articles/multi-harness-rl) reads OpenEnv's capture proxy, and the [MiMo-V2.6 environments](/articles/mimo-rl-environments) are the opposite design point: 7,780 hand-built tasks with five graders and judges you supply. The base model is covered in [Gemma 4](/articles/gemma-4), and the site's last FLEURS check was in [Cactus Whistle](/articles/whistle-stt).
## How I checked
- Cloned both Spaces and both adapter repos (`git clone --depth 1`, weights skipped) and read `rewards.py`, `schema.py`, `environment.py`, `bench_rewards.py` and the vendored Sarvam `metrics.py`.
- Summed `counts` and `pages` in both `corpus-manifest.json` files for the task, page and language totals.
- Pulled the `base`, `step-575` and `step-500` rows of `asr_kannada_evals` and `ocr_kannada_evals` through the datasets-server `filter` endpoint, then recomputed rewards, CERs, pooled edits and the akshara rates with a stdlib Levenshtein.
- Downloaded `data/kn_in/{train,dev,test}.tsv` from `google/fleurs` and intersected sentence ids and transcripts.
- Read per-checkpoint means from the `curves` config.
- Not done: running either environment server, any training, or the step-300 OCR revision. Every training detail in the tables is reported, not reproduced.