2026-08-03 · 4 min · open-source · rust · pdf · heuristics · explainer
Most PDF-to-text pipelines start with a coin flip: run OCR on everything and pay for it, or guess which files need it and get burned when you guess wrong. pdf-inspector (Firecrawl, MIT, 6.3k stars) skips the guess. It classifies a PDF as text-based, scanned, or mixed — page by page — and converts the text-based pages to Markdown, in under 200ms, with one dependency (lopdf) and no ML models, no OCR, no external services. Firecrawl's own number for why this matters: roughly 54% of PDFs don't need OCR at all, and this is how you find out which 54% without running an OCR engine to check.
That's the whole pitch: a bounded, well-understood problem — is this page's text selectable? — solved with parsing and arithmetic instead of a model.
The heuristics, not the pitch
The interesting part of pdf-inspector isn't that it's fast. It's what it checks. Reading src/detector.rs, three heuristics stand out, each with an inline rationale in the source:
Path-op density. A page can have plenty of drawing operators and still not have real text — some PDFs render glyphs as outlined vector paths rather than as selectable characters. The detector flags this specifically: massive path-drawing volume, almost no text-showing operators, almost no distinct characters actually rendered. Try the three conditions live:
Font decodability, with a fallback chain. Not every font that's present in a PDF is used — the detector only inspects fonts actually invoked via a text-show operator. If those fonts are Type0/Identity-H without a ToUnicode CMap, decoding produces garbage characters. Rather than flag that immediately, it tries two fallbacks first — CID values that happen to look like passthrough Unicode, then an embedded TrueType cmap table lookup — before finally giving up and marking the page suspected_garbled_text.
A newspaper-layout detector. Even a page classified TextBased can still need OCR: dense multi-column prose with a low font-change-to-text-op ratio reads badly as extracted text even when every character decodes correctly. The thresholds for this one are, per the source comments, calibrated against a named 50-page Wall Street Journal test PDF plus DPA/contract PDFs and SEC filings — a heuristic tuned against specific real documents, not an abstract rule.
None of this is a neural network. It's operator-stream scanning over the page's content stream, with page sampling (8 evenly-spread pages by default, not "bail on the first bad page" — the source comment explains why: an image-only cover page followed by dense text, like most annual reports, would trip an early-exit strategy into over-flagging OCR).
Where it lands on a benchmark
Firecrawl's own July 31, 2026 benchmark, run on an Apple M4 Pro against the 200-PDF opendataloader-bench corpus, with OCR disabled and only non-ML local engines in the comparison:
pdf-inspector edges liteparse by 0.002 overall, but wins tables decisively (TEDS 0.814 vs 0.693) and is roughly 1.6× faster (0.470s vs 0.750s for the full corpus) — while losing on headings (MHS 0.788 vs 0.811). pymupdf4llm and markitdown aren't close on tables or speed. This is a genuinely tight three-way race at the top, not a rout.
Honest gaps
This is Firecrawl's own benchmark, on Firecrawl's own hardware, published in Firecrawl's own README — there's no independent re-run I could find, though the corpus and evaluator are public and a reproducible-results branch is published, so a third party could check it (I did not). It's single-machine (one Apple M4 Pro), so there's no cross-platform or server-CPU number to point to. And the version story is a little tangled: the README's benchmark says it tested "pdf-inspector 0.2.6," which matches none of the three independently-versioned language bindings cleanly (Rust crate 0.1.7, npm package 1.11.2) — normal for a multi-target Rust project, but worth knowing if you go looking for "the" version number.
The take
There's no dramatic headline number here and no diagram to embed — the README's own "figure" is a Markdown table. What's worth taking from pdf-inspector is smaller and more useful: three specific, well-reasoned heuristics (path-op density, a font-decodability fallback chain, a newspaper-layout detector tuned against named documents) that collectively do a job people increasingly reach for an ML model to do, at a fraction of the cost, on the specific slice of the problem where a heuristic is the right tool. Not every classification problem needs a model. This one apparently doesn't.
Source: the pdf-inspector README and benchmark (Firecrawl, MIT, refreshed 2026-07-31) and src/detector.rs. The heuristic thresholds and benchmark numbers are the project's own; the interactive is my reconstruction of the vector-text condition for explanation, not a copy of the crate's code.