MonkeyOCR: a small, multimodal document-parsing model

Parsing PDFs with complex layouts is a mess: modular OCR pipelines are fiddly to assemble, and general-purpose multimodal LLMs aren’t efficient enough for the job. MonkeyOCR is a document-parsing model built on a lightweight multimodal LLM, with a paper already accepted by Science China Information Sciences. It parses PDFs and images in three stages—structure, recognition, relation—and outputs Markdown, layout-annotated PDFs, and block-level JSON. GitHubDaily recommended it in June 2025; on the OmniDocBench benchmark alone, the README claims it beats larger models such as Gemini 2.5 Pro and GPT-4o on both Chinese and English documents.

Core features

• SRR three-stage paradigm: layout detection, content recognition, and order prediction replace a chain of separate tools.
• Two model sizes: pro-3B and the lighter pro-1.2B—the README reports the latter is about 36% faster with roughly a 1.6% performance gap.
• Chinese and English documents: evaluation covers nine page types, including papers, financial reports, exam papers, and magazines.
• Local deployment: Gradio, FastAPI, Docker, and AWQ quantization support local GPU inference with no external API dependency.

Typical use cases

• Batch-converting formula-heavy papers and textbook PDFs into Markdown.
• Structured extraction and reading-order reconstruction for multi-column layouts like financial reports and newspapers.
• Turning the parser into a service to mass-produce RAG corpora.

Quick start

1. See docs/install_cuda_pp.md for environment setup.
2. Download the weights and parse a document:

python tools/download_model.py -n MonkeyOCR-pro-3B
python parse.py input_path

Summary

This suits developers, researchers, and data teams who need to parse Chinese and English PDFs locally, especially for preparing RAG source material. A few caveats: the source code is Apache 2.0, but the v1 weights are licensed for academic research and non-commercial evaluation only—commercial use requires written authorization; inference requires an NVIDIA GPU, and some Docker images need patching for certain cards; support for scanned photos, handwriting, Traditional Chinese, and other languages is still incomplete; a v2 shipped in July 2026, while this repository is the v1 mainline and still maintained. Overall, it’s a controllable, locally deployable document-parsing solution.