End-to-end OCR has seen a new wave of enthusiasm in the past two years. Solutions like DeepSeek OCR, which utilize large language models (LLMs) as decoders, have significantly improved recognition accuracy by leveraging language priors.
However, this type of solution has an inherent flaw: as the output sequence grows, the KV cache continuously expands, consuming substantial GPU memory and causing the latency of each decoding step to gradually increase.
Existing models generally adopt a for-loop-like approach, processing one page at a time, resetting memory after each page, and then moving to the next. This approach is essentially an engineering patch, not genuine long-range understanding.
In contrast, when humans copy a book, we don't constantly look back at everything we've written. Instead, we focus on three things: the original text we're copying, the small amount of characters we've just written, and the next character we're about to write. For the already written content, humans adopt a kind of "soft forgetting" that gradually fades from memory. This mechanism allows people to complete hundreds of pages of continuous transcription with very low cognitive load.
Inspired by this, the Baidu team proposed "Unlimited OCR," which replaces all attention layers in the decoder with "Reference Sliding Window Attention" (R-SWA). This reduces attention computation costs while maintaining a constant KV cache throughout the decoding process.
Paper Link: https://arxiv.org/pdf/2606.23050
GitHub: https://github.com/baidu/Unlimited-OCR
According to the paper, by combining the high compression rate of the DeepSeek OCR encoder with a constant KV cache design, Unlimited OCR can transcribe dozens of document pages in a single forward pass, all within the standard 32K maximum length limit.
More importantly, R-SWA is a general-purpose sparse attention mechanism. Beyond OCR, it is also applicable to tasks like ASR and translation.
The Design Logic of R-SWA
Standard multi-head attention (MHA) uses a full KV cache that grows linearly with the number of output tokens, causing memory and computation time to increase correspondingly. For a multi-page document requiring the output of 100,000 tokens, this means the KV cache would expand infinitely, and inference speed would continuously degrade. This is the fundamental reason why all current end-to-end OCR models cannot achieve "one-shot parsing of an entire book." R-SWA divides attention into two segments: the reference segment and the decoding sliding window.
The reference segment contains visual tokens and prompts. It remains globally visible throughout the decoding process and does not participate in state transitions. Visual features are encoded only once and remain static afterward. This design avoids the "gradual blurring of visual features" problem that occurs in ordinary sliding window attention (SWA) when visual tokens are also included in the sliding loop.
The width of the decoding sliding window is fixed at n (default 128) and slides to the right in a causal manner. Each time a new token is generated, the KV of the token that entered the window earliest is evicted from the queue. Consequently, the size of the KV cache on the decoding side is always limited to a fixed capacity, rather than increasing linearly with output length.
Figure: Comparison of KV cache between R-SWA and standard full attention.
For long sequence decoding, the difference in total KV cache is particularly pronounced. When the number of output tokens is much larger than the window width, the cache usage of R-SWA tends towards a bounded constant, while the cache of MHA grows without limit. This gap widens as the output length increases, and it is the theoretical foundation for R-SWA's near-infinite parsing capability.
Unlimited OCR uses the DeepEncoder from DeepSeek OCR. DeepEncoder cascades SAM-ViT with CLIP-ViT and implements 16x token compression at the bridging point, so that a 1024x1024 PDF page image ultimately produces only 256 visual tokens. This extremely high compression rate means that even when dozens of document pages are input simultaneously, the total number of visual tokens generated during the prefill phase remains relatively manageable.
The combination of DeepEncoder compressing the visual side and R-SWA stabilizing the decoding side supports Unlimited OCR's ability to parse dozens of document pages in one go within a 32K maximum length limit.
Figure: Schematic of the overall architecture of Unlimited OCR.
Experimental Results
1. Single-Page Documents: Achieving End-to-End SOTA
The research team evaluated Unlimited OCR on OmniDocBench v1.5 and v1.6, and systematically compared it with current mainstream end-to-end VLM (Vision-Language Model) solutions.
On OmniDocBench v1.5, Unlimited OCR (3B-A0.5B) achieved an Overall score of 93.23, a 6.22 improvement over DeepSeek-OCR. On OmniDocBench v1.6, Unlimited OCR achieved end-to-end SOTA with an Overall score of 93.92%, surpassing concurrent works such as Logics-Parsing-v2 (93.33%) and FireRed-OCR (93.26%).
The research team also conducted a category-by-category comparison across the 9 document types covered in OmniDocBench v1.5 (academic papers, books, exam papers, magazines, newspapers, notes, etc.).
The results show that Unlimited OCR outperforms DeepSeek OCR in all categories and on all metrics. In 7 out of 9 text editing distance and reading order metrics, it even surpassed DeepSeek OCR 2, indicating that the introduction of R-SWA brings lossless gains.
2. Long-Range Parsing: One-Shot Transcription of Multiple Pages
This is the key capability that differentiates Unlimited OCR from all existing models. The research team constructed an internal test set containing novels, academic documents, and papers, divided into six tiers based on page count: 2, 5, 10, 15, 20, and 40+ pages, with at least 10 documents per tier, to evaluate multi-page one-shot parsing performance.
Distinct-n measures the proportion of non-repetitive n-grams in the generated text and is an important metric for detecting whether a model falls into cyclic repetition during long-range generation. Even in scenarios with over 40 pages, Distinct-35 still reaches 96.90%, and the edit distance remains within 0.11.
The research team pointed out that recognition errors in such scenarios mainly stem from small font text in PDF images being difficult to read at 1024x1024 resolution, rather than R-SWA losing its way in long-range inference.
3. Inference Efficiency
Regarding inference throughput (TPS), as the output sequence becomes longer, DeepSeek OCR's TPS continuously decreases: it drops to 5822.87 TPS when outputting 6000 tokens, while Unlimited OCR maintains 7847.71 TPS, a lead of approximately 35%.
In the single-page OmniDocBench evaluation scenario, Unlimited OCR's throughput is also 12.7% higher than DeepSeek OCR (5580 vs. 4951 TPS).
The call latency of the Flash Attention v3 kernel at each step remains consistently stable in Unlimited OCR, while in DeepSeek OCR it increases linearly with the decoding step, with occasional sudden drops when the KV cache crosses a certain alignment boundary.
Limitations and Future Directions
The research team acknowledges that the current Unlimited OCR cannot achieve truly "infinite" parsing within a limited context length (e.g., 32K), because the length of the prefill side is still constrained by the context window. As the number of input pages increases, the total number of visual tokens during the prefill phase accumulates.
In the near term, their plan is to extend the maximum training context to 128K to support one-shot prefill of more pages. The long-term goal is to build a prefill pool that allows the model to automatically read KV segments on demand, simulating the memory retrieval behavior of a human turning a page, thereby achieving truly infinite parsing.
Additionally, they plan to transfer R-SWA to other long-range tasks with reference inputs, such as automatic speech recognition (ASR) and translation.
The core conclusion revealed by Unlimited OCR—that in an end-to-end model's decoder, completely replacing full attention with R-SWA causes no loss for parsing tasks—provides a practical path to leverage long-range reasoning capabilities with limited resources.
Transforming the KV cache from an infinitely growing quantity tied to decoding length into a bounded constant has the potential to be applied beyond OCR to a wider range of reference-type long-range generation tasks.
Editor: George
For reprint or submission, please leave a comment directly in this article.