DeepSeek-OCR: Rethinking Text Compression Through Vision

Introduction

Inefficiency related to tokenization and transformer mechanism:

Large Language Models are highly capable of understanding and generating human language, but this ability comes with a significant computational cost. These systems rely on tokenization, a process that breaks text into small units such as subwords. Each token must be processed individually, which means that longer texts directly translate into larger computational workloads. As the number of tokens increases, memory use and processing time grow rapidly, often at a rate that makes large-scale applications inefficient.

This limitation becomes most evident when handling long documents or extended conversations. A model that easily manages a few hundred words may struggle with thousands, not because it lacks comprehension but because the underlying computation expands quadratically with sequence length. In practical terms, analyzing a 10,000-word document requires far more resources than processing a 1,000-word one. This challenge limits how effectively these models can scale without significant infrastructure or optimization.

How we approach texts:

When we read a page from a book, our brain processes the entire visual scene almost instantly. We recognize words, structure, and meaning without focusing on each individual character. A language model, however, must process each token sequentially.

This contrast raises an important question: can we find a way to compress textual information more efficiently, reducing computation while preserving meaning and accuracy?

DeepSeek-OCR explores this idea by rethinking text compression through a visual perspective. Instead of treating text purely as a sequence of tokens, it represents written language as an image to make text processing and understanding more efficient.

DeepSeek-OCR: More Than Just OCR

Despite its name, DeepSeek-OCR is not just another optical character recognition system. It is a proof of concept designed to explore a broader idea: whether visual representations can compress text more efficiently than conventional token-based methods.

The concept is straightforward but powerful. When a page containing bunch of words is converted into an image, the same information can be represented with far fewer vision tokens than text tokens. DeepSeek-OCR demonstrates that this approach can achieve compression ratios between seven and twenty times, depending on the nature and depth of the context being processed. This technique points toward a promising way to overcome the long-context limitations that continue to challenge large language models.

Think of it this way: vision-language models are typically used to describe images, for example, “This photo shows a large sea beneath scattered clouds.” In the case of DeepSeek-OCR, the goal is reversed. Instead of analyzing pictures of natural scenes, the model is asked to interpret an image that contains text (of course it can also contain images, graphs etc). Rather than feeding a language model the text of a book and asking for a summary, we can provide “images” of the pages and ask for the same result. This approach allows the model to perform the same task while using far fewer tokens to understand the given context.

A simple visualization illustrating the proof of concept that DeepSeek-OCR aims to demonstrate. The number of tokens is hypothetical.

How Does This Differ From Similar Models?

You might already be familiar with vision-language models such as PaliGemma2, which combine visual and textual understanding. DeepSeek-OCR, however, takes a fundamentally different direction.

Traditional vision-language models like PaliGemma2 focus on interpreting images that humans create and find meaningful, such as photographs, charts, or diagrams. They are designed to answer questions like “What is shown in this image?” or “Describe what you see.”

DeepSeek-OCR approaches the problem from the perspective of large language models. It asks a different question: “How can vision help language models process text more efficiently?” Instead of emphasizing human-style visual understanding, it treats images as a medium for compressing textual information.

Note: DeepSeek-OCR can also perform image understanding tasks. However, its main contribution lies elsewhere. The researchers use it as an experimental framework to explore a key question “Can large amounts of textual information be represented as an image using fewer tokens than traditional text-based methods?”

In this framework, OCR tasks serve as a bridge between vision and language. They provide an ideal test environment for studying vision-based text compression, since they naturally create a compression–decompression relationship between visual and textual representations.

Architecture

DeepSeek-OCR consists of two main components:

1. DeepEncoder

The DeepEncoder is responsible for understanding the semantic information contained in images. It is designed to be as simple, fast, and content-aware as possible. While existing vision encoders used in popular vision-language models can perform similar tasks, they often have limitations in efficiency or semantic precision. To address these shortcomings, the authors developed a new encoder architecture specifically optimized for visual text understanding.

Some examples of popular encoders:

The figure is from the original paper.

DeepEncoder approach

Deep Encoder Architecture (The figure is from the original paper.)

The heart of the system is DeepEncoder, a carefully engineered vision encoder with approximately 380 million parameters. Its architecture is designed to efficiently process high-resolution images while minimizing GPU memory usage.

The DeepEncoder connects two main components in sequence, linked by a 16x convolutional compressor:

SAM-base (80M): Perception Stage

• Acts as the visual perception module, responsible for low-level feature extraction.
• Uses window-based attention to efficiently handle high-resolution images (for example, a 1024x1024 image with 4,096 tokens).
• Keeps activation memory low through localized attention.

16x Convolutional Compressor: Token Reduction

• Reduces token count by a factor of 16 (for example, from 4,096 to 256 tokens).
• Minimizes memory use and computation before global processing.

CLIP-large (300M): Knowledge Stage

• Serves as the visual knowledge extraction module, leveraging CLIP’s large-scale pretraining.
• Uses dense global attention to capture high-level semantic relationships.
• Processes the compressed tokens efficiently to achieve deep contextual understanding.

Note: CLIP is originally designed to take raw image tokens as input. In DeepSeek-OCR, the authors remove CLIP’s embedding layer so that it can directly process the compressed tokens produced by the convolutional module.

The architecture supports multiple resolution modes that allow flexible experimentation with compression and detail. For example, the Tiny mode (512x512) uses only 64 vision tokens, making it suitable for testing efficient compression. The Large mode (1280x1280) increases fidelity with 400 tokens, offering greater detail when needed. There is also a dynamic Gundam mode that combines multiple resolutions to handle ultra-high-resolution inputs such as newspaper scans or densely formatted documents.

The figure is from the original paper.

Note: “Gundam” name is coming from an anime about giant humanoid robots known as Mobile Suits, with the most iconic models referred to as Gundams. In the context of DeepSeek-OCR, the term Gundam describes a dynamic-resolution vision model configuration. The name is likely inspired by (I hope so) the way the model combines multiple image segments, much like how a Gundam mecha is assembled from interconnected components.

2. DeepSeek-3B-MoE Decoder

The figure is from the original paper.

The decoder uses a Mixture-of-Experts architecture with 3B total parameters but only 570M activated during inference. This makes it suitable for domain-centric research, obtaining the expressive capability of a 3B model while enjoying the inference efficiency of a 500M small model.

The decoder’s job is to take those compressed vision tokens and reconstruct the original text. It learns a non-linear mapping that decompresses the visual representation back into readable text.

Benchmarks: Proving the Concept

Note: I will go through this part quickly, since it is generally understood that a published research paper already demonstrates some improvement :). While it may not achieve the best results across all text, formula, table, and OCR benchmarks, it performs competitively with dots.ocr. However, this blog post mainly focuses on the text compression aspect, so I won’t dive into the OCR results. If you’re interested in those, you can check them out in the paper.

The experimental results validate DeepSeek-OCR’s compression capabilities impressively:

Compression Ratio Tests

When testing on diverse document layouts, the model achieves 96%+ OCR decoding precision at 9–10x text compression, approximately 90% at 10–12x compression, and roughly 60% at 20x compression.

To put this in perspective: imagine a document page containing 1,000 text tokens. DeepSeek-OCR can represent this using just 100 vision tokens while maintaining over 96% accuracy. Even at extreme compression (20x, or 50 vision tokens for 1,000 text tokens), accuracy remains at 60%.

Practical Performance

On OmniDocBench, DeepSeek-OCR surpasses GOT-OCR2.0 (which uses 256 tokens per page) using only 100 vision tokens, and outperforms MinerU2.0 (which requires 6000+ tokens per page on average) while utilizing fewer than 800 vision tokens.

Real-World Capabilities

Beyond compression, DeepSeek-OCR demonstrates practical utility:

• Deep Parsing: The model can perform secondary analysis on documents, parsing charts into structured HTML tables, converting chemical formulas to SMILES format, and even understanding geometric figures.
• Multilingual Support: It handles nearly 100 languages, from English and Chinese to Arabic and Sinhala.

Note: The huge amount of data is Chinese and English. About ~16% of training data is other languages.

• Flexible Output: Through different prompts, users can get either raw text extraction or formatted output with layout information preserved.

Discussion

The most exciting aspect of DeepSeek-OCR isn’t just what it achieves today, but what it suggests for tomorrow. Innovation often comes from asking unconventional questions, and this project does exactly that by reframing a fundamental assumption about how AI should process text.

Probable Use Cases and Future Directions

1. Historical Document Compression

Consider a conversational AI system that maintains context across many turns of dialogue. Currently, storing all previous conversation history as text tokens becomes computationally expensive. With optical compression, older conversation rounds could be rendered as images and compressed by 10x, maintaining most information while dramatically reducing token count.

The researchers propose an intriguing parallel: just as human memories fade over time and distant objects appear blurrier, older context could be progressively downsampled to smaller resolutions. Recent conversation stays crystal clear (high resolution), while older exchanges become increasingly compressed (lower resolution). This mimics biological memory decay while managing computational resources efficiently.

The figure is from the original paper.

2. Memory Mechanisms in LLMs

Current language models treat all tokens equally, whether they represent something from the first sentence or the last. The optical compression approach can simulate memory forgetting mechanisms by rendering previous rounds of historical text onto images for initial compression, then progressively resizing older images to achieve multi-level compression, where token counts gradually decrease and text becomes increasingly blurred.

This approach enables theoretically unlimited context architectures that effectively balance information retention and computational efficiency. Recent information is preserved with high fidelity, while distant memories gradually compress, fading in detail as their relevance diminishes.

3. Enhanced Document Parsing

Traditional OCR systems extract text but lose valuable formatting information. DeepSeek-OCR’s visual approach inherently preserves formatting cues:

• Font emphasis: Bold, italic, and colored text remain distinguishable in the visual representation
• Size variations: Headings versus body text retain their relative importance
• Spatial relationships: Tables, columns, and layout structure are naturally preserved
• Visual elements: Charts, formulas, and diagrams can be parsed alongside text

This holistic understanding could enhance how AI systems interpret documents where formatting conveys meaning, such as academic papers, legal texts, technical manuals, and other structured materials.

4. Efficient Data Generation

The practical throughput of 200k+ pages per day makes DeepSeek-OCR valuable for creating training data. As language models grow larger and require more diverse training data, efficient document processing becomes crucial. The ability to parse and structure documents at scale, while using fewer computational resources than traditional approaches, has immediate practical value.

Thinking Differently About Modalities

DeepSeek-OCR encourages us to rethink how different modalities such as vision and language can work together. Instead of viewing vision as something separate that language models must learn to interpret, it asks whether vision itself could be a more efficient way to represent certain types of linguistic information.

Final Words

DeepSeek-OCR represents more than a technical achievement; it exemplifies the value of open research and development in AI. The team has made their code and model weights publicly accessible, allowing researchers worldwide to build upon this work, validate the findings, and explore new applications.

Why does this matter?

We cannot know whether large enterprises have already developed similar technologies behind closed doors. Major AI labs have significant resources and may have explored comparable approaches internally. However, without open publications and shared code, that knowledge remains isolated. The wider research community, including academics, independent researchers, and developers, cannot learn from it, build upon it, or apply it to address real-world problems.

The research also demonstrates something equally important: innovation doesn’t always require the biggest models or the most resources. Sometimes it requires asking different questions and challenging assumptions. By reframing OCR as a compression problem and treating vision as a computational tool rather than just a perceptual task, DeepSeek-OCR opens new research directions that might not have emerged from conventional approaches.

As we continue to push the boundaries of what AI systems can achieve, projects like this remind us that progress comes not only from scaling up but also from thinking differently.

References / Sources

[embed]DeepSeek-OCR: Contexts Optical Compression We present DeepSeek-OCR as an initial investigation into the feasibility of compressing long contexts via optical 2D…www.arxiv.org

[embed]DeepSeek AI | Leading AI Language Models & Solutions DeepSeek AI is the leading provider of advanced AI language models and enterprise solutions. Experience…deepseek.ai

[embed]