TeleOCR: the warped page is never unwarped

mdjsonmcpcopy↗ai

2026-09-26 · 20 min · ocr · document-parsing · vision-language-models · small-models · benchmarks · licensing

ModelScope's post about TeleOCR fits in one breath: digital PDFs or warped phone photos, one lightweight 1.2B vision-language model, Apache 2.0, 96.87 on OmniDocBench v1.6, first in the ICDAR 2026 Sci-ImageMiner Challenge, and "without a separate dewarping model." TeleOCR comes from China Telecom's AI lab. It shipped in August as NaviDC-OCR and was renamed on 10 September. I read the paper (arXiv:2608.12898, v3), the checkpoint's safetensors header, the config and modeling files on both hubs, the inference code on GitHub, and the leaderboards of the benchmarks it cites.

The interesting part is the answer to "no dewarping model". TeleOCR never flattens the photo. Training teaches it where a page bends. At inference it outlines each curved region as a polygon, masks everything outside it black, and reads the crop as it is, still curved. Below: how that works, what the checkpoint contains, and which headline numbers survive a second look.

Two passes over one model

TeleOCR is a decoupled parser, the MinerU2.5 pattern: the same weights run twice per page with different instructions. Appendix A of the paper lists eight tasks. Each is selected by nothing but a line of text after the image:

Pass one, layout. The page is resized to 1036 by 1036 pixels and asked for its layout. The answer is one line per block, in the format the paper gives, with coordinates on a 0 to 999 grid:

<box:x1 y1 x2 y2><label:category><rotate_dir>            digital page
<box:x1 y1 x2 y2 x3 y3 ...><label:category><rotate_dir>  camera page: a polygon

The order of those lines is the reading order. The pipeline stores each block's position in the list as its index and sorts by it when it assembles the page (vlm_magic_model.py); there is no separate reading-order model. The label is one of 24 block types in structs.py: text, title, table, image, code, equation, header, footer, caption, footnote, seal, chart and so on. The rotation is up, right, down or left.

Pass two, content. Each block is cropped from the full-resolution page (pages over 64 million pixels are scaled down first), rotated upright, and sent back to the same model with its type's instruction. Images and lists are passed through without a model call. Tables come back as OTSL (Optimized Table Structure Language), a grid written in six tokens: <fcel> a cell with text, <ecel> an empty cell, <lcel> merged with the cell to its left, <ucel> merged with the one above, <xcel> both, <nl> a new row. A converter turns that into an HTML <table> with rowspan and colspan. Formulas come back as LaTeX, code as a fenced block, and the page is stitched into Markdown by code whose file names (vlm_magic_model.py, vlm_middle_json_mkcontent.py) are MinerU's. The README credits MinerU, and the paper says its layout format "adapts the representation introduced in MinerU".

Decoding is greedy (temperature 0, top-k 1), content calls add a presence penalty of 1.0, and the client ships its own vLLM logits processor that forbids repeating any 100-token span (no_repeat_ngram_size=100). That guard exists for a reason the paper admits, which comes up again under PureDocBench.

What is in the checkpoint

model.safetensors is one file of 2,830,223,488 bytes. Its header is 77,944 bytes of JSON listing 701 tensors, all BF16. I read it with two range requests and summed the shapes by module:

visual.patch_embed                1,505,280
visual.blocks      (32)         630,470,400   19,702,200 per block
visual.merger                    31,464,704   the "Aligner", trained from scratch
model.layers       (28)         440,466,432   15,730,944 per layer
model.embed_tokens              155,582,464   151,936 x 1,024
model.norm                            1,024
lm_head                         155,582,464   151,936 x 1,024
                          total 1,415,072,768

Three things come out of that listing.

The decoder is Qwen3 wearing a Qwen2.5-VL config. All 28 decoder layers carry a q_norm and a k_norm (128 weights each) and no query, key or value bias: Qwen3's attention, not Qwen2.5's. But config.json declares "model_type": "qwen2_5_vl", so a loader that trusts the config builds the wrong decoder. The repo's modeling_naviocr.py is the Transformers Qwen2.5-VL file with Qwen3's query-key norm grafted in, hence trust_remote_code=True. The vLLM plugin builds the text model as Qwen3ForCausalLM and registers it under the name Qwen2_5_VLForConditionalGeneration from an auto-loaded vllm.general_plugins entry point, so installing it next to a real Qwen2.5-VL swaps that architecture for every model in the process (reasoned from the code; I did not run it).

The vision half is Qwen2.5-VL's encoder. 32 blocks, width 1,280, a SwiGLU MLP of 3,420, windowed attention with full-attention layers at 7, 15, 23 and 31. The aligner takes a 2 by 2 group of 1,280-wide patches (5,120 values) down to 1,024, the width of Qwen3-0.6B. No Qwen2.5-VL merger outputs 1,024 (the 3B's outputs 2,048, the 7B's 3,584), which is why it had to be trained from scratch.

"1.2B" counts the output head once. text_config sets tie_word_embeddings: true, and both the Transformers class and vLLM's Qwen3 implementation then point the head at the embedding and ignore the stored lm_head.weight (reasoned from the loaders; whether that copy is byte-identical to the embedding needs tensor data, which I did not read). So the Hub's 1,415,072,768 includes a 155,582,464-parameter copy. The model you run has 1,259,490,304: 663,440,384 of vision encoder and aligner, 596,049,920 of language model. That second figure is exactly the Hub's count for Qwen3-0.6B-Base, and it is how Qwen gets to "0.6B": the Qwen3-0.6B repo stores the same duplicate and shows 751,632,384. So "approximately 1.2B" is Qwen's convention, rounded down from 1.26B. It mirrors Jina-OCR-v1, whose active-parameter figure assumed a tie its config switched off; here the config ties and the file stores both. Vision is 53% of the model.

Resolution: no tiles, only crops

There is no tiling here, unlike DeepSeek-OCR's global view plus local tiles. Qwen2.5-VL reads an image at its own resolution: both sides are rounded to a multiple of 28 pixels, the encoder cuts 14-pixel patches, and each 2 by 2 group of patches becomes one language-model token. So one token covers a 28 by 28 square, and a picture of $H \times W$ pixels costs

tokens after rounding. preprocessor_config.json bounds each image between 3,136 and 12,845,056 pixels: 4 to 16,384 tokens.

The two passes use this differently. The layout pass always sees 1036 by 1036, which is 37 by 37 = 1,369 tokens. An A4 page is squashed square, and on a 300 DPI scan the layout pass works from under a third of the page's pixel height. The crops are read at full resolution, each as its own image.

Take an A4 page at 200 DPI, the resolution the pipeline renders PDF pages at: 1,654 by 2,338 pixels. Read in one pass, the same encoder would spend 59 by 84 = 4,956 tokens on it. TeleOCR spends 1,369 on layout plus the crops. If 60% of the page is inside 25 blocks, that is 25 crops of about 118 tokens, 2,950 in all, for 4,319 visual tokens.

The tokens are not the cost that matters. Per visual token, the vision encoder pushes four 14-pixel patches through 631,975,680 parameters, about 5.06 GFLOP; the Qwen3 decoder spends 0.88 GFLOP on the same token. The encoder holds 53% of the parameters and does about 85% of the prefill matrix arithmetic. Attention is where two passes pay off. The encoder's four full-attention layers cost the square of each image's patch count: the whole A4 page is 19,824 patches, about 8.0 TFLOP in those four layers, while the 5,476-patch layout image costs about 0.6 and 25 small crops about 0.1. In total, about 27.8 TFLOP of prefill against 40.8 for one pass over the page. All of this is reasoned from the shapes, not measured.

What the calculator cannot show is decode, and neither can I. The paper has no speed number at all: no pages per second, no latency, no inference hardware. "Lightweight" is a parameter count.

Warped photos: geometry as a training signal

The paper's motivating experiment runs a dewarping model over Wild-OmniDocBench photos before parsing, and two-stage parsers improve "substantially"; no number is given. The diagnosis: layout detection assumes rectangles, and a curved page breaks that before any text is read. So instead of a dewarping module, TeleOCR is taught the deformation.

The labels come free from synthesis. Take a clean digital page whose labels passed the consensus vote described below. Sample boundary points clockwise around each layout region, and an M by M grid of control points over the page. Take a backward map from Doc3D, a dataset of 3D paper deformations, derive the forward map, and apply it to the image, the boundary points and the control points alike, following the recipe of ForCenNet, the lead author's earlier dewarping paper. The result is a crumpled page with exactly known warped polygons and control points. Two kinds of supervision come out of it:

• Point level. The model predicts where the control points went. The deformation field is downsampled to $N^2 = 1{,}024$ control points, where a dedicated dewarping model like ForCenNet uses 82,944.
• Region level. Polygons replace rectangles. Layout detection becomes boundary-point prediction.

How many points a polygon gets is the subtle part. Uniform sampling wastes points on straight edges and under-samples corners and creases. Douglas-Peucker simplification picks points by their distance $d$ from the chord, and for a smooth curve $d \approx \kappa L^2 / 8$ with curvature $\kappa$ and span $L$: it catches big bends and misses short, sharp creases. Curvature-Guided Douglas-Peucker (CGDP) scores each point by

where $\hat{\kappa}_i$ is normalised local curvature, and recurses on a point when $\max_i S_i > \tau$. With low curvature it is plain Douglas-Peucker; creases get extra points. The paper does not give $\lambda$ or $\tau$.

Stage 2 of training is built from this: 4M digital layout samples, 2M synthetic camera samples (1.2M region-level, 0.8M point-level), plus about 120K parsing samples restyled with distortion, shadows and blur.

expand

At inference, the camera-layout instruction returns polygons with a variable number of points. The pipeline takes each polygon's bounding rectangle, fills everything outside the polygon with black (cv2.fillPoly), turns the crop upright by the predicted rotation, and sends it for recognition. The text recogniser was trained on exactly that input; the paper's text task says non-text areas of camera crops are "masked with black pixels". Nothing is unwarped. The recogniser reads curved lines directly.

Two things I would not have guessed from the announcement. The point-level control points are not one of the eight tasks and have no instruction in the released client, so they are a training-time signal only; the model never outputs a deformation field you can use (reasoned from Appendix A and the code). And the polygon path is not the default: TeleOCR/config.py sets LAYOUT_MODE = "Detection", rectangles. You opt in:

# from the TeleOCR repo; I read this code, I did not run it
python infer.py --image_sub_path ./photos --result_save_path ./out \
  --override LAYOUT_MODE=Segmentation --use_async

The paper does not say which mode produced its Wild-OmniDocBench and PureDocBench numbers.

expand

Labels by vote, checked by picture

Nobody hand-labels millions of pages, so the data engine rests on two filters.

Multi-node Consensus Voting. Run $N$ different parsers on the same page. Score every pair of predictions with the task's own metric, $S(y_i, y_j) \in [0, 1]$: IoU for layout, edit distance for text, TEDS for tables, CDM for formulas. Each prediction's consensus is its mean agreement with the others:

The most-agreed-with prediction becomes the label if its $C_i$ clears a threshold $\tau$; otherwise the page goes to correction or a human. The point is that no single teacher's systematic errors become ground truth. The paper does not name the voters; Figure 2 draws a five-by-five agreement matrix of logos.

A judge that compares pictures. Checking a Markdown table against a photo is a cross-modal judgement, and general models are bad at it: Qwen3-VL-235B's zero-shot recall on the authors' own verification set stayed below 40%. So TeleOCR renders the prediction back into an image (layout boxes on a canvas, text re-laid out, the table as rendered HTML, the formula as rendered LaTeX) and compares image to image. The judge is Qwen2.5-VL-7B-Instruct, fine-tuned on correct renderings against deliberately broken ones: missing or merged regions, wrong row-column structure, formula syntax errors.

The four training stages:

Stage 3 teaches structure apart from content. For formulas, a regex-and-parser pass extracts the LaTeX skeleton, the commands without their symbols. For tables, the OTSL is kept with every cell's text deleted, so the model learns topology and merges alone. Figure 3 draws this as a <think> block of structure followed by the content. The released client, though, defines two structure-only instructions, Table Recognition: and Formula Recognition:, and never calls them. A table crop gets one OTSL request, and nothing in the post-processing strips a think block. As shipped, the decoupling is a training curriculum, structure-only samples alongside full ones, not a two-step decode (reasoned from the code). The ICDAR organisers describe the team's chart method the same way: "separate training on structure-only and full-content samples".

expand

The evidence

Four benchmarks and one competition, and it matters who ran each:

OmniDocBench v1.6. The benchmark's owners added TeleOCR to their v1.6 leaderboard on 11 September; their entry links the Hub weights, which have not changed since the 17 August upload (the Hub's last commit to model.safetensors). Their run: 96.91 overall, text edit distance 0.0267, formula CDM 96.5895, table TEDS 96.8183, reading-order edit 0.1184. That is first, 0.44 ahead of OvisOCR2 at 96.47. The headline holds.

Now split it. The overall is the mean of three parts, $((1 - \text{TextEdit}) \times 100 + \text{FormulaCDM} + \text{TableTEDS}) / 3$. TeleOCR's table TEDS is 96.8183 against 94.7619 for PaddleOCR-VL-1.6 and 94.5842 for OvisOCR2. On formulas, all five of the next five systems score higher. On text, OvisOCR2 edges it, 0.0265 to 0.0267. On reading order, OvisOCR2 (0.1120) and Youtu-Parsing (0.116) are ahead. Give TeleOCR the runner-up's 94.7619 for tables and its overall drops to 96.23, third behind OvisOCR2 and PaddleOCR-VL-1.6 (96.34). The first place is table structure, and the structure-first training above is the plausible reason.

The paper's prose claims more than its table shows. It says TeleOCR gets "the lowest normalized edit distance for text and reading order evaluation", but its own Table 1 has OvisOCR2 at 0.025 for text and 0.111 for reading order against TeleOCR's 0.027 and 0.122, and Youtu-Parsing (0.116) and MinerU2.5-Pro (0.120) are also lower on reading order. The README's copy of the table marks this correctly. Table 1 also has two transcription slips. HunyuanOCR-1.5's text, formula, table and reading-order cells repeat PaddleOCR-VL-1.6's exactly; they average to 96.32, yet the row prints 94.74. The Jina-OCR-v1 report prints HunyuanOCR-1.5 as 0.039, 94.50 and 93.67, which are consistent with 94.74 within rounding. And TeleOCR's own components average to 96.90 against a printed 96.87. The owners' run is the cleaner number to quote.

Wild-OmniDocBench v1.5 is 1,350 photos of printed-and-deformed pages and screen re-captures, with OmniDocBench v1.5's annotations. TeleOCR scores 88.53 to OvisOCR2's 87.91. On photos it leads text edit (0.1173 against 0.129) as well as tables (89.05 against 85.76), and formulas are its worst column: 88.26, sixth of nine. The benchmark's repository publishes no leaderboard, and the paper does not say whether the baselines were re-run or copied.

PureDocBench. The 78.41 is the mean of three tracks: $(86.90 + 77.47 + 70.85) / 3 = 78.41$. The text says TeleOCR "achieves state-of-the-art performance on the Degraded track", but Table 3 itself has OvisOCR2 at 77.77 on Digital-Degraded and Gemini-3.1-Pro at 71.98 on Real-Degraded. The PureDocBench leaderboard lists TeleOCR, still as NaviDC-OCR and marked author-reported, first on Clean and on the average, and second on both degraded tracks, behind WeVisDoc-4B (77.74) and Gemini-3.1-Pro (71.98). Real photographed pages are the case the deformation training targets, and a general model is ahead there. Appendix D also discloses that six TeleOCR outputs, one Clean and five Real-Degraded, fell into "severe repetitive generation" and were scored as zero. That is the failure the 100-token no-repeat rule exists for.

ICDAR 2026 Sci-ImageMiner. The organisers' report (arXiv:2607.26848) lists "TeleOCR-VL" first on Task 2, data extraction from charts, and third on Task 3, summarisation, out of four task leaderboards. Task 2 ranks by the mean of RMS and TEDS: 41.81 against VLMinators' 40.80, won on TEDS (66.39 against 64.31) while VLMinators had the higher RMS (17.29 against 17.23). The organisers describe the winning entry's inference as "a heterogeneous ensemble". The first place belongs to an ensemble built around TeleOCR, on one task of four, not to the released checkpoint alone.

expand

The release, and the licence

Released: the weights (one BF16 safetensors file, byte-identical on both hubs), the tokenizer, modeling_naviocr.py, and on GitHub the pipeline with a vLLM 0.11.0 plugin, pinned to Transformers 4.57.1. A community GGUF for llama.cpp exists. Not released: training code, any of the training data, the voting setup, the judge model, and the ICDAR ensemble.

The licence is thinner than the post suggests. The Hub card's metadata says license: apache-2.0, and its badge links to a LICENSE file that is not in the repo. The GitHub repo has no licence file. ModelScope's licence field is empty and its README never mentions one. So the Apache-2.0 claim rests on one metadata field.

Upstream is the harder question. Qwen3-0.6B is Apache-2.0. The vision encoder is "inherited from Qwen2.5-VL", with no size named. Its shapes narrow it to two parents: the MLP width of 3,420 matches Qwen2.5-VL-3B and -7B, while the 32B and 72B configs use 3,456. Qwen2.5-VL-7B is Apache-2.0. Qwen2.5-VL-3B ships under the Qwen Research License, which does not allow commercial use. The header cannot say which one TeleOCR started from, and stages 1 and 2 retrained the encoder anyway, so there is no byte comparison to settle it. If it was the 3B, Apache-2.0 on the derivative is not the authors' to grant. I would ask them before shipping it commercially.

Where it fits

TeleOCR's pipeline renders every PDF page to pixels, text layer or not, so the cheap first step is pdf-inspector's question of whether a page needs a vision model at all. If you want a page as few tokens as possible rather than as a transcript, LensVLM-9B is the opposite design.

What holds up: first place on OmniDocBench v1.6, reproduced by the benchmark's owners on the released weights; a warped-page method that is a real idea, geometry learned from synthetic warps with exact labels and applied as polygons and masks; and a parameter count that is honest by Qwen's convention. What does not: the prose claims of best text and reading order, the degraded-track claim, and an ICDAR first place that belongs to an ensemble on one task of four. The lead itself is table structure. Without it, TeleOCR is a very good third-place parser, which at 1.26B is still worth running.