Jina AI Releases jina-ocr-v1: A 3.4B MoE Document Parser
Jina AI launches jina-ocr-v1, a 3.4-billion parameter MoE document parser featuring built-in speculative decoding for low-budget GPUs, achieving 2.57 pages per second.
Stock photo for illustration only, not from the actual event
- Jina AI introduces jina-ocr-v1, a 3.4B MoE document parser designed for research and non-commercial use
- Delivers a top throughput of 2.57 pages per second on an A100 40GB at concurrency 32
- Integrates speculative decoding via a FastMTP head to boost generation speed without accuracy loss
- Supports deployment via vLLM, Transformers, and an OpenAI-compatible API endpoint
The artificial intelligence and document processing landscape welcomes a powerful new tool as Jina AI officially releases jina-ocr-v1. This Mixture of Experts (MoE) document parser packs approximately 3.4 billion total parameters, specifically engineered to run efficiently on low-budget or consumer-grade GPUs. Open weights in BF16 format, totaling roughly 6.8 GB, are currently available for research and non-commercial applications under the CC BY-NC 4.0 license.
The architecture builds upon a post-trained DeepSeek-OCR backbone while retaining its core efficiency components. The DeepEncoder contains about 380 million parameters, chaining SAM, a 16x convolutional compressor, and CLIP-L. This setup converts a 1024x1024 page view from 4,096 patches into 256 visual tokens, supported by a dynamic-resolution mode that adds up to 9 local tiles at 100 tokens each, capping a page at 1,156 visual tokens.
The decoder utilizes DeepSeek-3B-MoE, structured with 12 layers, 64 routed experts, and 2 shared experts. Utilizing Top-6 routing, it activates roughly 570 million parameters per token and supports a position limit of 32,768 tokens. The model outputs structured Markdown text, with tables rendered in HTML and formulas expressed in LaTeX.
Stock photo for illustration only, not from the actual event
A standout innovation in this release is the integration of Speculative Decoding through a FastMTP head, applying 1 dense draft block recursively for K=3 steps. This mechanism accelerates generation speed while ensuring the committed text remains identical to standard greedy decoding. When tested on an NVIDIA L4 at batch size 1, eager decoding speed jumps from 42.7 to 83.1 tokens per second, marking a 1.95x speedup.
"Throughput is the main result. On 1 A100 40 GB at concurrency 32, jina-ocr-v1 parses 2.57 pages per second."
Jina AI
Jina AI's strategic focus on parsing throughput rather than leading raw accuracy benchmarks addresses a critical bottleneck for developers handling massive document pipelines. By incorporating speculative decoding into an OCR pipeline, the model achieves high-speed text extraction, making large-scale PDF ingestion and document parsing highly practical even on constrained hardware environments.
For immediate integration, users can leverage Jina Reader by sending URLs to r.jina.ai with custom headers, or connect directly via the OpenAI-compatible endpoint at api.jina.ai/v1/chat/completions. Self-hosting options are also fully supported through Transformers and vLLM repositories.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment