Nobody tells you that OCR benchmark numbers are close to useless for picking a model, and i found that out the slow way this weekend. I had a large pile of old papers i wanted to make searchable. Some were reasonably clean scans, but a few were phone photos i had taken of a library monitor because the PDF was no longer online, so they came with a tilt and heavy glare. Every one of them has to pass through OCR before a model sees the text, so this single step decides whether anything downstream works. I opened the leaderboards, sorted by score, and planned to take the top model. That plan failed by the end of the first day, because the scores cant be read side by side.

MinerU reports its accuracy on OmniDocBench. PaddleOCR reports a 94.5, but that figure is on OmniDocBench v1.5, and the same team then released a separate benchmark called Real5 on the argument that the common sets go easy on degraded real-world pages. Thats a reasonable thing to care about. The problem is that a team scoring its own model on a benchmark it wrote is not a number i can hold up against MinerU and learn anything from. And the metric my use case actually depends on, formula and table accuracy on a damaged scan, doesnt appear on anyones product page.

MinerU is the one i kept. Its built for scientific papers, so formulas come out as LaTeX and tables as HTML instead of scrambled text, and it strips the running headers and page numbers on its own, which matters more than it sounds since that junk is what breaks retrieval later. The backends are the real reason to choose it. The CPU path runs in the low 80s on OmniDocBench and the GPU path clears 90, so you choose based on your hardware and accept reduced accuracy if its weak. It was the only model where i eventually stopped checking every formula against the source PDF by hand.

The other three didnt fit my inputs. Docling is barely an OCR tool. It extracts the text layer out of digital-native PDFs and does nothing useful with a photograph. PaddleOCR-VL was the one i almost kept, a small vision-language model under a billion parameters built for degraded input like phone-photo skew and glare, and it handles that well. My inputs fail in a different way, since a flatbed scan doesnt warp the way a photo taken at an angle does. GOT-OCR-2.0 is a 580M end-to-end model that handles formulas and LaTeX, but a large part of its purpose is scene text, reading words off signs and storefronts, so its wasted on flat printed pages.

Testing four models in a weekend rather than a week came down to one thing: i didnt install any of them myself. MinerU alone offers around five backends, and PaddleOCR-VL requires a vLLM server that takes about two minutes to warm up. I ran them from preconfigured setups on HyperAI, passed the same batch of pages through each one, and moved between them by swapping notebooks. Doing that setup work by hand, I would've tested exactly one model, decided it was good enough, and never found out whether it actually was.

So dont shortlist these by benchmark score. Take your most difficult pages, run them through two or three models yourself, and watch which one fails. That was the only signal that told me anything useful.

I tried to pick an OCR model by benchmark score and learned the numbers do not compare