Surya OCR 2: The Lightweight Open Source Model That Is Redefining Document Intelligence

Document understanding has long been one of the most frustrating challenges in applied machine learning. Raw text extraction from scanned pages, handwritten notes, complex forms, and mathematical equations has historically required either expensive proprietary platforms or brittle pipelines stitched together from multiple specialized models. Datalab has changed that calculus with the release of Surya OCR 2, a single open source vision language model that handles layout analysis, full text recognition, table extraction, and reading order detection in one unified architecture, all within 650 million parameters.

The benchmark result that immediately catches attention is 83.3% on olmOCR-bench, the standard quality benchmark for document parsers. That score places Surya OCR 2 at the top of the under 3 billion parameter category, ahead of models with considerably more computational weight. It is a result that challenges the assumption that document intelligence at production quality requires massive model scale or expensive cloud infrastructure.

What Surya OCR 2 Actually Does

It is worth being precise about the scope of what Surya OCR 2 covers, because the term OCR undersells it considerably.

Traditional OCR systems extract raw text from images. They identify characters and string them together into lines. What they typically cannot do is understand the structure of a document: which text is a heading versus a body paragraph, where a table begins and ends, how columns relate to rows, which elements are captions for images, or what the correct reading sequence is across a complex multi-column layout.

Surya OCR 2 handles all of that through a single shared vision language model. The same model that reads the text also identifies layout regions, classifies them by type, assigns reading order, recognizes table structure at the row and column level, and handles mathematical notation inline as part of the regular text output. Mathematical expressions are returned inside standard tags in KaTeX-compatible LaTeX format, embedded naturally within the surrounding prose without requiring a separate equation recognition pass.

The layout labels the model can assign cover a comprehensive range of document element types including captions, footnotes, equations, list groups, page headers, page footers, pictures, section headers, tables, body text, figures, code blocks, forms, tables of contents, chemical structures, diagrams, bibliography sections, and blank pages. This level of semantic granularity means downstream systems can make intelligent decisions about how to process each region rather than treating the entire page as undifferentiated text.

Table recognition goes beyond simple cell extraction. The model can operate in a standard mode that returns geometric row and column intersections, or in a full mode that produces complete HTML table markup including support for spanning cells and header rows. For complex financial documents, scientific papers, or government forms where table structure carries meaning, this distinction matters enormously.

The Architecture Behind the Score

Surya OCR 2 is built on a Qwen3.5-style vision language architecture with approximately 650 million parameters. The model is trained on diverse document images to emit either a layout JSON or a full-page HTML output depending on the prompt it receives. This unified approach means that layout, OCR, and table recognition all route through the same model weights, sharing computation and avoiding the overhead of coordinating separate specialized models.

Text line detection is handled by a separate, smaller torch model based on a modified EfficientViT segformer architecture trained from scratch on document line annotations. This separation is deliberate: the detection model runs purely on PyTorch without requiring the inference backend needed by the main VLM, which means it can operate even in environments where the heavier serving infrastructure is not available.

The inference backend for the main model is flexible. On NVIDIA hardware, the model is served through vllm, which enables high-throughput batched inference with configurable concurrency. On CPU and Apple Silicon, llama.cpp provides Metal-accelerated inference. The SuryaInferenceManager handles backend selection automatically, defaulting to vllm when NVIDIA hardware is detected and falling back to llama.cpp otherwise. Users can also point the manager at an already-running OpenAI-compatible server using the SURYA_INFERENCE_URL environment variable, which is useful in production deployments where the model server is managed separately from the client application.

The throughput numbers are striking for a model in this parameter range. On an RTX 5090 with vllm at 128 concurrent requests, Surya OCR 2 processes approximately 5.35 pages per second with a median latency of around 19 seconds per page and output token throughput of nearly 13,000 tokens per second. On Apple Silicon with llama.cpp using Metal acceleration at 8 parallel slots, the model processes roughly 0.1 pages per second at approximately 30 watts of power consumption, which makes it genuinely viable for local, offline use on a laptop without specialized hardware.

Multilingual Coverage at Scale

One of the most significant capabilities of Surya OCR 2 is its multilingual reach. The model achieves an overall pass rate of 87.2% across an internal benchmark covering 91 languages, spanning text accuracy, layout recognition, table handling, mathematical notation, and reading order across documents drawn from each language.

Of the 91 languages evaluated, 38 score at or above 90% accuracy and 76 score at or above 80%, which means the overwhelming majority of the language coverage is genuinely high quality rather than marginal support. Looking at widely spoken languages specifically, English reaches 92.3%, Italian 93.0%, Spanish 90.7%, German 89.7%, French 89.3%, Russian 88.8%, Korean 86.7%, Japanese 86.2%, Portuguese 86.1%, Bengali 82.7%, Chinese 82.5%, Hindi 82.2%, and Persian 82.3%. Arabic and Vietnamese score lower at 72.7% and 73.2% respectively, which likely reflects the additional complexity of right-to-left scripts and tonal languages with dense diacritic systems.

This breadth of language support without requiring language-specific model variants is a significant engineering achievement. Most document intelligence pipelines require separate models or fine-tuned checkpoints for non-Latin scripts. Surya OCR 2 handles the full range within a single set of weights.

Installation and Getting Started

Installation is straightforward through pip:

pip install surya-ocr

The inference backend must be set up separately depending on the available hardware. For NVIDIA GPUs, Docker and the NVIDIA Container Toolkit are required to run vllm. For CPU or Apple Silicon, the llama.cpp server binary is needed:

brew install llama.cpp

Alternatively, a release binary can be downloaded directly from the llama.cpp GitHub releases page for non-macOS systems.

The SuryaInferenceManager handles server lifecycle automatically on first use, spawning the appropriate backend based on detected hardware. For workflows that involve running multiple commands sequentially, the server restart overhead on each command can be eliminated by keeping the server running across invocations:

surya_ocr    DATA_PATH --keep_server
surya_layout DATA_PATH
surya_table  DATA_PATH

The first command spawns the backend and leaves it running. Subsequent commands attach to the already-running server rather than re-spawning it. The same behavior can be made permanent by setting the SURYA_INFERENCE_KEEP_ALIVE environment variable to 1.

For development and manual installation, the repository can be cloned and set up using uv:

git clone https://github.com/datalab-to/surya.git
cd surya
uv sync --group dev
uv run surya_ocr ...

Running OCR, Layout, and Table Recognition

The command line interface exposes each capability as a separate command, all operating on images, PDFs, or folders of either:

surya_ocr DATA_PATH
surya_detect DATA_PATH
surya_layout DATA_PATH
surya_table DATA_PATH

Each command writes a results.json file containing structured output. For OCR, that output includes per-block results in reading order, with each block carrying a canonicalized layout label, the block content as HTML, polygon and bounding box coordinates, a confidence score, and flags indicating whether the block was skipped or encountered an error.

From Python, the same capabilities are accessible with a clean API. A full-page OCR pass requires only a few lines:

from PIL import Image
from surya.inference import SuryaInferenceManager
from surya.recognition import RecognitionPredictor
manager = SuryaInferenceManager()
recognition_predictor = RecognitionPredictor(manager)
predictions = recognition_predictor([Image.open("document.png")])

When more granular control is needed, layout analysis can be run first and then passed to the recognition predictor to trigger per-block OCR mode:

from surya.layout import LayoutPredictor
layout = LayoutPredictor(manager)
layouts = layout([Image.open("document.png")])
predictions = recognition_predictor([Image.open("document.png")], layouts)

Table recognition follows the same pattern. The simple mode returns geometric row and column structure. The full mode generates complete HTML output suitable for direct rendering or downstream parsing:

from surya.table_rec import TableRecPredictor
table_rec_predictor = TableRecPredictor(SuryaInferenceManager())
table_predictions = table_rec_predictor([Image.open("table_image.png")])
# Full HTML output with spanning cells and header rows
table_predictions = table_rec_predictor.predict_full([image])

Text line detection, which uses the separate torch model rather than the VLM, has an even simpler interface:

from surya.detection import DetectionPredictor
det_predictor = DetectionPredictor()
predictions = det_predictor([Image.open("document.png")])

An interactive Streamlit application is also available for exploratory use without writing any code:

pip install streamlit pdftext
surya_gui

This launches a browser-based interface where documents can be uploaded and processed interactively, making it easy to evaluate the model’s behavior on new document types before committing to a programmatic integration.

Performance Tuning

Several configuration knobs affect throughput and accuracy tradeoffs. The most impactful for throughput is input DPI. The model defaults to 192 DPI, which maximizes accuracy but increases token counts and therefore latency. Reducing DPI to 96 roughly halves the average token count per page and can significantly increase pages per second at the cost of some accuracy on very small text.

On GPU with vllm, concurrency is the primary throughput lever. Raising the max number of sequences and batched tokens in vllm configuration, or increasing SURYA_INFERENCE_PARALLEL on the client side, keeps more pages in flight simultaneously and improves overall throughput at the cost of individual request latency.

On CPU with llama.cpp, the SURYA_INFERENCE_PARALLEL setting should match the parallel value passed to llama-server to ensure client concurrency aligns with server capacity.

For challenging documents, a few preprocessing steps can substantially improve results. Increasing image resolution before passing to the model helps when source document quality is low. For very old or blurry documents, binarization and deskewing as preprocessing steps before OCR can improve character recognition accuracy meaningfully. The detection thresholds DETECTOR_BLANK_THRESHOLD and DETECTOR_TEXT_THRESHOLD can also be adjusted when line detection is producing incorrect segmentation. The debug output from the detector includes a heatmap that makes it straightforward to diagnose whether thresholds need to move up or down for a specific document type.

Licensing and Commercial Use

The Surya codebase is released under the Apache 2.0 license, which permits broad use including commercial applications. The model weights carry a modified AI Pubs Open Rail-M license that covers research use, personal use, and startups with under 5 million dollars in combined funding and revenue without restriction. Organizations that fall outside those boundaries or require broader commercial terms can access the weights under a commercial license through Datalab’s pricing page.

For higher accuracy requirements, Datalab also offers Chandra OCR 2, a 5.3 billion parameter model scoring 85.9% on olmOCR-bench, accessible through their managed platform alongside Surya. The managed platform offers 5 dollars in free credits for new accounts and a public playground for trying the models without any signup.

Where Surya Fits in the Document Intelligence Landscape

The olmOCR-bench results tell an interesting story about the current state of document intelligence. The top score in the benchmark belongs to a 35 billion parameter model at 87.6%. Datalab’s own Chandra OCR 2 at 5.3 billion parameters reaches 85.9%. Surya OCR 2 at 650 million parameters reaches 83.3%, which is a difference of only about 4 percentage points from the state of the art at roughly 54 times fewer parameters.

That parameter efficiency gap matters in practice. Running a 35 billion parameter model requires substantial GPU resources that many organizations cannot justify or afford for document processing workloads. Running a 650 million parameter model on a single consumer GPU, or even on an Apple Silicon laptop at acceptable throughput, opens document intelligence to a much broader range of use cases and deployment environments.

The specific per-source breakdown of Surya OCR 2’s olmOCR-bench performance also reveals where the model excels and where headroom remains. Standard text documents score 99.7%, header and footer handling reaches 92.5%, tiny text recognition reaches 93.7%, and multi-column layouts reach 82.4%. The most challenging categories are old scanned documents at 41.8% and old mathematical content at 81.4%, both of which reflect the genuine difficulty of degraded historical material rather than a fundamental limitation of the architecture.

Practical Applications

The combination of capabilities in Surya OCR 2 opens up a wide range of practical applications that would previously have required assembling and maintaining multiple specialized models.

Legal and compliance teams processing large volumes of scanned contracts, regulatory filings, or historical records can extract structured text with layout context preserved, making downstream search and analysis substantially more reliable than raw text dumps.

Research institutions digitizing scientific literature benefit from inline mathematical notation handling, which eliminates the need for a separate equation recognition step and preserves the semantic relationship between equations and surrounding prose.

Financial services organizations extracting data from complex reports, statements, and forms gain access to table recognition that understands spanning cells and header rows, making structured data extraction from financial documents considerably more reliable.

Multilingual document processing pipelines serving global audiences can use a single model deployment rather than routing documents through language-specific specialized systems, simplifying infrastructure significantly.

Software development teams working with legacy codebases often need to process printed or scanned technical documentation. Surya OCR 2’s code block label and high accuracy on technical text makes it well-suited for that use case.

Conclusion

Surya OCR 2 represents a meaningful step forward in what is achievable with a compact, open source document intelligence model. At 650 million parameters, it delivers accuracy competitive with models many times its size, covers 91 languages with high fidelity, runs on consumer hardware without specialized infrastructure, and handles the full range of document understanding tasks from raw text extraction through complex table recognition and mathematical notation in a single unified architecture.

The practical implications are significant. Organizations that previously could not justify the infrastructure cost or API dependency of high quality document intelligence now have a credible open source path to production-grade capability. Researchers and developers who need to process documents at scale on modest hardware have a model that runs locally without requiring cloud connectivity.

The 83.3% olmOCR-bench score is not just a benchmark number. It is evidence that document intelligence is becoming genuinely accessible, not just to organizations with deep infrastructure budgets, but to anyone with a standard development environment and a document processing problem worth solving.

The repository is available at: https://github.com/datalab-to/surya