The Three Paradigms Shaping Modern OCR

Anyone who has worked with PDFs, forms, or scanned reports knows this feeling: extracting text is easy until structure starts to matter.

The real shift is this: OCR is no longer just a reading task, but a document understanding problem.

Modern OCR Is About Understanding Documents

Figure 1: OCR Beyond Text: The Rise of Document Intelligence. Image by the author.

In the past, OCR usually meant one thing: recognizing text from an image.

Today, the scope of OCR has expanded far beyond basic character recognition. Real-world document processing rarely stops at reading text. It often involves layout analysis, table parsing, chart understanding, question answering, and extracting key information from complex pages. Documents are no longer just lines of text. They contain structure, hierarchy, and meaning that must be understood as a whole.

Figure 2: Rapid growth of document parsing methods since June 2025. [Source].

Because of this shift, OCR has gradually evolved from a character recognition tool into something much closer to a document intelligence system. Instead of only asking what characters are on the page, modern OCR systems are expected to understand how the page is organized and what the content actually represents.

Three Paradigms of Modern OCR

Current OCR paradigms can roughly be grouped into three categories:

• Pipeline OCR systems
• End-to-end OCR models
• General vision-language models.

These three approaches reflect different stages in the evolution of OCR, and each comes with its own set of trade-offs.

Traditional Pipeline OCR: Strong in Control, Weak in Fragmentation

The most classic design is the pipeline OCR system. A typical workflow starts with layout detection, followed by element-level recognition, and finally a rule-based step that assembles everything into the final output.

Figure 3: The overall pipeline of MonkeyOCR v1.5, which first detect all layout elements with order index and then recognize contents in a parallel way using a VLM. [Source].

Systems such as PaddleOCR-VL, MonkeyOCR (MonkeyOCR v1.5: Making Complex PDFs Parseable — AI Innovations and Insights 90), and MinerU 2.5 (From Big Picture to Details: MinerU 2.5 Redefines Document Parsing — AI Innovations and Insights 77) follow this paradigm.

Figure 4: The framework of MinerU2.5. In stage I, MinerU2.5 performs rapid, global layout analysis on a downsampled page. In stage II, MinerU2.5 leverages the layout results to crop key regions from the original high-resolution document, performing fine-grained content recognition (e.g., text, table, and formula recognition) within these native-resolution local regions. [Source].

They usually combine lightweight detection models with a VLM-based recognizer, which makes the overall architecture modular and efficient in production settings. Another important advantage is that these systems can expose intermediate results such as bounding boxes, element types, and reading order.

The real strength of pipeline OCR is not that it feels more like AI, but that it behaves more like an industrial system. Each step is explicit. The model can tell exactly which region is a title, which region is a table, and in what order the content should be read.

In many real applications, this kind of layout-level information is essential. Tasks like form processing, contract review, and receipt extraction require more than plain text. Developers often need to know where the content appears on the page and how different parts relate to each other structurally.

The downside is equally clear. Once the workflow is split into multiple stages, error propagation becomes unavoidable. If the layout detection step makes a mistake, every downstream stage inherits that error. Another limitation is the loss of global visual context. When the page is cut into smaller regions before recognition, the model no longer sees the full document at once, and that lost context cannot be recovered later.

End-to-End OCR: More Unified, But Often Only the Answer, Not the Process

The second direction is end-to-end OCR models. Instead of breaking the task into separate stages, this approach maps a document image directly to structured output in a single model.

Figure 5: The architecture of DeepSeek-OCR. DeepSeek-OCR consists of a DeepEncoder and a DeepSeek-3B-MoE decoder. DeepEncoder is the core of DeepSeek-OCR, comprising three components: a SAM for perception dominated by window attention, a CLIP for knowledge with dense global attention, and a 16× token compressor that bridges between them. [Source].

There are several representative systems along this line:

• Nougat (Demystifying PDF Parsing 03: OCR-Free Small Model-Based Method) showed that end-to-end OCR can work well for academic papers.
• GOT-OCR 2.0 (Demystifying PDF Parsing 05: Unifying Separate Tasks into a Small Model) extended the idea to support multiple output formats such as Markdown, LaTeX, and TikZ.
• DeepSeek-OCR (DeepSeek-OCR: See Less, Remember More — AI Innovations and Insights 83) focused on efficiency through context optical compression.
• olmOCR (AI Innovations and Insights 31: olmOCR , HippoRAG 2, and RAG Web UI) and its later versions introduced large-scale SFT together with GRPO reinforcement learning guided by unit-test rewards.
• More recent work such as Dolphin v2 (Two Stages, One Breakthrough: Dolphin’s New Paradigm for Document Parsing — AI Innovations and Insights 44), Logics-Parsing (Taming Chaotic Layouts: SFT + Layout-Centric RL for Document Understanding — AI Innovations and Insights 75), and Infinity-Parser (Rethinking Scanned Document Parsing with Layout-Aware RL — AI Innovations and Insights 67) has started exploring more structured modeling strategies, including analyze-then-parse pipelines inside a single model and layout-aware reinforcement learning.

Figure 6: Overview of Dolphin’s two-stage document image parsing paradigm. Left: Stage 1 and Stage 2. Right: Examples of input-output pairs, including page-level layout analysis and element-level content parsing for text paragraphs, tables, and formulas. “P∗” denotes different prompts. [Source].

The key idea behind this paradigm is to unify recognition, parsing, and understanding inside one model instead of handling them as separate steps.

One immediate benefit is that the model can keep more of the original visual context while processing the document. It also makes true end-to-end optimization possible.

That said, current end-to-end systems have their own limitations. Many of them focus heavily on recognition accuracy or inference efficiency, but do not provide explicit layout analysis outputs. In practice, the model can give the final result directly, yet it often cannot clearly describe what structural elements exist on the page, where they are located, or in what order they should be read. This is exactly the kind of information that pipeline-based systems have traditionally provided, and many real applications still depend on it.

General VLMs: Capable of Many Things, But Not Always the Best at Documents

The third direction is the one most people are familiar with today, the large multimodal model (LMM) approach, usually referred to as general vision-language models. Examples include Qwen-VL, InternVL, and Gemini.

Figure 7: The Qwen2.5-VL framework demonstrates the integration of a vision encoder and a language model decoder to process multimodal inputs, including images and videos. [Source].

These models have OCR ability mostly because they were trained on large-scale multimodal data. OCR is more of a side effect of that training, not the primary objective the models were designed to optimize.

The strengths of general VLMs are easy to see. They have strong reasoning ability, handle open-ended tasks well, and respond naturally to prompts. This makes them useful for many scenarios that traditional OCR systems could never handle.

However, when these models are used directly for structured document parsing, several limitations start to appear. Inference cost is higher, layout control is less precise, and performance on structure-sensitive metrics is often worse than specialized OCR systems. Two good examples of such metrics are table TEDS scores and reading order accuracy.

This reflects a very practical reality in current AI systems. LMMs often look like they can do everything, but that does not mean they are the best choice for tasks with strong structural and spatial constraints. Document parsing falls exactly into that category.

A document is not just an image, and OCR is not just visual question answering. Titles, footnotes, tables, formulas, multi-column layouts, and reading order all require the model to understand not only the text itself, but also the structure of the page.

Thoughts

In my view:

• Pipeline OCR is often underrated in academic discussions. Research tends to care more about aggregate scores, while production teams care more about accountability, rollback paths, and safe staged releases. As long as OCR continues to serve high-stakes scenarios like contracts, receipts, identity documents, and bank records, pipeline systems will remain relevant for a long time.
• For End-to-end OCR, its real value is not simply reducing engineering steps, but preserving full-document context.
• For General-purpose VLMs, they are best at understanding exceptions, not handling the everyday bulk. They are a strong fit for complex question answering and multi-hop semantic extraction after OCR, but not for large-scale, high-concurrency, low-cost document parsing, especially in core systems with strict requirements around coordinates, reading order, and table structure. The issue is not that these models are not capable enough. The issue is that they are too expensive, too unconstrained, and too difficult to control.

There is also a hard truth in industrial OCR: the real business value rarely comes from solving the hardest 5 percent. It comes from processing the most ordinary 95 percent at low enough cost and with enough stability.

In my view, the mainstream architecture for industrial OCR will not be a pure pipeline or a pure VLM. It will be a layered routing system, where each sample is handled not by the most powerful model, but by the most appropriate one.