Model Catalog

This document provides a comprehensive overview of all models and inference engines available in Docling, organized by processing stage.

Overview

Docling's document processing pipeline consists of multiple stages, each using specialized models and inference engines. This catalog helps you understand:

• What stages are available for document processing
• Which model families power each stage
• What specific models you can use
• Which inference engines support each model

Stages and Models Overview

The following table shows all processing stages in Docling, their model families, and available models.

Inference Engines by Model Family

Object Detection Models (Layout)

Note: Layout models run through AutoModelForObjectDetection, reading their label map from the model repository's own config.json. Only docling-layout-heron also supports ONNXRuntime.

Deprecated: LayoutOptions and docling-layout-v2 are superseded by LayoutObjectDetectionOptions. LayoutOptions still works — it is translated onto the object-detection path — but selecting DOCLING_LAYOUT_V2 warns and falls back to Heron.

TableFormer Models (Table Structure)

Note: MPS is currently disabled for TableFormer due to performance issues.

Image Classifier (Picture Classifier)

OCR Engines

Languages are given as BCP-47 tags for every engine; see OCR engines.

Vision-Language Models (VLM)

VLM Convert Stage

nanonets_ocr2 includes preset API overrides for OpenAI-compatible runtimes and LM Studio, and can also be used with vLLM runtimes.

minicpm_ocr runs OpenBMB's MiniCPM-OCR 1.0 through Transformers 5.21 or later. It reads English and Chinese pages and asks the model for Markdown with one page image per request. The preset declares Transformers and OpenAI-compatible API engines; no MLX or in-process vLLM export is configured.

mineru2_pro runs MinerU's native two-step flow: page-level layout detection followed by type-specific recognition of the detected region crops. Its Transformers and API configurations use the official opendatalab/MinerU2.5-Pro-2604-1.2B repository. The MLX override uses the community-published carlesonielfa/MinerU2.5-Pro-2604-1.2B-mlx-bf16 conversion because no official OpenDataLab MLX checkpoint is currently available. Image/chart analysis and cross-page table merging are not enabled by this preset.

nemotron_parse_v3_5 runs code from the nvidia/Nemotron-3.5-Parse model repository, pinned to revision e7ea79242a87ea0ad3919b672bfedb429a683206. Docling does not trust that code by default: review it, then set trust_remote_code=True on the VLM engine options (for example AutoInlineVlmEngineOptions(trust_remote_code=True)) or pass --trust-remote-code to the CLI. Without this opt-in, the Transformers engine stops with an error before it downloads the model. In-process vLLM is not supported because the model needs NVIDIA's out-of-tree vLLM plugin. To use vLLM, start NVIDIA's server (vllm serve ... --served-model-name nemotron-parse, see the model card) and use ApiVlmEngineOptions(engine_type=VlmEngineType.API, url=...). The API override sends top_k=1, repetition_penalty=1.1, and skip_special_tokens=False. If you set your own params, the preset parameters are not sent, so include these values yourself. The preset maps charts to pictures with a chart class and data table, with one picture for each chart panel.

The default is single-pass. To re-parse Table and Chart crops, set VlmConvertOptions.nemotron_parse in the Python SDK:

from docling.datamodel.pipeline_options import (
    NemotronParseOptions,
    VlmConvertOptions,
    VlmPipelineOptions,
)
from docling.datamodel.vlm_engine_options import AutoInlineVlmEngineOptions

pipeline_options = VlmPipelineOptions(
    vlm_options=VlmConvertOptions.from_preset(
        "nemotron_parse_v3_5",
        engine_options=AutoInlineVlmEngineOptions(trust_remote_code=True),
        nemotron_parse=NemotronParseOptions(
            passes="double",  # "single", "double", or "triple"
            triple_regions="union",  # "union" or "detector"; triple-pass only
            crop_batch_size=8,
        ),
    ),
)

Double-pass uses the Table and Chart boxes from the page parse. Triple-pass adds a separate no-text detection request. triple_regions="detector" uses detector boxes; "union" also keeps parser boxes that do not overlap detector boxes of the same class. Both Transformers and API engines use bounded crop batches. Each page has at most 80 crop regions. Crops have a 4% + 12 pixel margin, use JPEG quality 95, and are skipped if either padded dimension is less than 8 pixels. Crop coordinates map back to the page frame.

Multi-pass page and detection requests use temperature 0.2 and repetition penalty 1.05. Crop requests use 0.1 and 1.0. All stages use top_k=50 and top_p=1.0. Detection has a 4096 token limit; page and crop requests have an 8744 token limit. Explicit API params override these request settings. Single-pass keeps the original preset settings. For every Transformers VLM preset, output that reaches the token limit without EOS is now reported as incomplete. The VLM pipeline returns PARTIAL_SUCCESS for these pages. Output that reaches EOS at the token limit remains complete.

Only guarded merging is supported. A crop table must have a valid grid and compatible column count. A replacement also needs at least 0.5 box intersection over union and 0.65 text similarity. It keeps the page box and reading order. New objects must not overlap an existing object of the same class. Failed, empty, truncated, and invalid crop output keeps the page result. Invalid detection output skips the crop stage. These guards can retain the less detailed page result. NVIDIA's published multi-pass scores use its alternatives merge policy and do not measure this guarded policy.

Each page prediction has nemotron_parse metadata with the mode and model request counts. The exported document keeps these values at body.meta.docling__nemotron_parse.pages, keyed by source page number. requests counts page, detection, and crop requests, including unsuccessful crop requests; it is not a stage or engine-batch count. rejected_crop_requests counts unsuccessful or invalid crop generations before merging. Token counts include all stages, and usage keeps each stage response's provider usage when available.

unlimited_ocr is served through an OpenAI-compatible endpoint: point ApiVlmEngineOptions.url at your own runtime. Its API override sets skip_special_tokens=False, without which the layout annotations are stripped from the completion.

unlimited_ocr2 runs through an OpenAI-compatible server that you start yourself. Docling runs no model code in-process for this preset. It has no in-process vLLM configuration. The Transformers engine cannot run it, because the model needs its own generation loop and transformers==4.52.1. vLLM support needs vllm-project/vllm#64016, which is not merged yet.

On Apple Silicon, use unlimited_ocr2_mlx instead. It runs the community mlx-community/Unlimited-OCR-2-bf16 conversion in-process with the MLX engine, at a pinned revision, and needs mlx-vlm>=0.7.11. The docling-slim[models-vlm-inline] extra allows mlx-vlm>=0.7.0, so upgrade mlx-vlm if your environment has an older release. The preset sends the prompt Multi page parsing., and the MLX chat template adds the image token. It does not set trust_remote_code. Docling still sends one page per request.

Today the server is the customized SGLang wheel from the model repository. It needs an NVIDIA GPU. The command below is the model card command at the pinned revision, without --enable-custom-logit-processor, because Docling does not send a logit processor. The model card also lists memory settings for the server.

hf download baidu/Unlimited-OCR-2 \
    --revision c14071b549f4fcab10909f3c4acc6a2edb366947 --local-dir Unlimited-OCR-2
uv venv --python 3.10 && source .venv/bin/activate
uv pip install Unlimited-OCR-2/wheel/sglang-0.5.20.dev13+g75b9b06b02-py3-none-any.whl \
    fastokens==0.3.2

python -m sglang.launch_server \
    --model-path Unlimited-OCR-2 \
    --served-model-name Unlimited-OCR-2 \
    --chat-template unlimited-ocr \
    --tokenizer-backend fastokens \
    --tokenizer-worker-num 8 \
    --mem-fraction-static 0.7 \
    --max-running-requests 64 \
    --max-prefill-tokens 2048 \
    --enable-streaming-session \
    --allow-auto-truncate \
    --disable-radix-cache \
    --mm-preprocess-cache-size-mb 0 \
    --cuda-graph-backend-decode full --cuda-graph-max-bs-decode 64 \
    --cuda-graph-backend-prefill full \
    --cuda-graph-max-bs-prefill 16384 \
    --cuda-graph-bs-prefill 1024 2048 \
    --cuda-graph-config '{"prefill": {"backend": "full", "full_prefill_max_req": 64}}' \
    --host 127.0.0.1 \
    --port 10000

Keep the CUDA Graph flags. The vendor reports that eager mode can repeat box coordinates. Then point the preset at the server in the Python SDK. The CLI has no option for the API URL, so --vlm-model unlimited_ocr2 uses the default OpenAI URL and is not useful without the SDK.

from docling.datamodel.pipeline_options import VlmConvertOptions, VlmPipelineOptions
from docling.datamodel.vlm_engine_options import ApiVlmEngineOptions, VlmEngineType

pipeline_options = VlmPipelineOptions(
    enable_remote_services=True,
    vlm_options=VlmConvertOptions.from_preset(
        "unlimited_ocr2",
        engine_options=ApiVlmEngineOptions(
            engine_type=VlmEngineType.API_OPENAI,
            url="http://127.0.0.1:10000/v1/chat/completions",
            timeout=600,
        ),
    ),
)

Limits of this preset:

• One page per request. Docling sends each page alone. The main feature of the model is to read all pages of a document in one request, and Docling does not use this mode yet. The vendor's paper reports only a small gain on single pages: 94.02 against 93.92 for Unlimited-OCR on OmniDocBench.
• No merge across pages. Docling reads cross_ and append_ blocks as normal blocks. A table or paragraph that continues on the next page stays split. <MERGE> joins text on the same page.
• No repetition guard. The vendor client sends a pickled logit processor, and Docling does not. A page that repeats ends at max_tokens (8000) with the length stop reason.
• Your own params. If you set params, the preset parameters are not sent, so include model, max_tokens, and skip_special_tokens=False. Without the last value, the server removes <td> and <tr> from tables.

surya_ocr2_5 runs datalab-to/surya-ocr-2.5 at a pinned revision. Docling decodes the compact output of the model with its own code, so the surya-ocr package is not needed. This preset is not the docling-surya OCR plugin. All engines load the same Hugging Face repository:

• Transformers needs transformers>=5.12.1 for the Qwen3.5 architecture. The Docling Transformers engine runs on CPU, CUDA, or XPU, not on MPS.
• MLX loads the original weights and needs mlx-vlm>=0.7.6. Auto-inline selects it on Apple Silicon. No converted MLX checkpoint is needed.
• vLLM needs vllm>=0.25.1, the release that upstream tests. The preset sets max_model_len=24576.
• OpenAI-compatible API: any server that serves the repository, for example vllm serve datalab-to/surya-ocr-2.5 or mlx_vlm.server --model datalab-to/surya-ocr-2.5. The default model parameter is the repository ID. With llama-server and the GGUF build datalab-to/surya-ocr-2.5-gguf, start the server with --special; otherwise the block labels and boxes are removed from the reply.

The preset follows the upstream full-page settings: pages render at 192 DPI (scale=192/72, at most 3072 pixels on the long edge), greedy decoding, and at most 12288 new tokens. The upstream client also retries a page at a higher temperature when it repeats. Docling does not.

The model weights use a modified OpenRAIL-M license. It is free for research, personal use, and organizations under USD 5M funding or revenue. Read the model card before commercial use.

Picture Description Stage

Code & Formula Stage

Usage Examples

Layout Detection

from docling.datamodel.pipeline_options import LayoutObjectDetectionOptions

# Use Heron layout model (default)
layout_options = LayoutObjectDetectionOptions.from_preset("layout_heron_default")

# Or a higher-accuracy variant
layout_options = LayoutObjectDetectionOptions.from_preset("layout_egret_large")

Table Structure Recognition

from docling.datamodel.pipeline_options import TableStructureOptions, TableFormerMode

# Use accurate mode for best quality
table_options = TableStructureOptions(
    mode=TableFormerMode.ACCURATE,
    do_cell_matching=True
)

Picture Classification

from docling.models.stages.picture_classifier.document_picture_classifier import (
    DocumentPictureClassifierOptions
)

# Use default picture classifier
classifier_options = DocumentPictureClassifierOptions.from_preset("document_figure_classifier_v2")

OCR

from docling.datamodel.pipeline_options import TesseractOcrOptions

# Use Tesseract with English and German. Languages are BCP-47 tags;
# `eng`/`deu` still work and canonicalize to `en-Latn`/`de-Latn`.
ocr_options = TesseractOcrOptions(lang=["en", "de"])

VLM Convert (Full Page)

from docling.datamodel.pipeline_options import VlmConvertOptions

# Use SmolDocling with auto-selected engine
options = VlmConvertOptions.from_preset("smoldocling")

# Or force specific engine
from docling.datamodel.vlm_engine_options import MlxVlmEngineOptions
options = VlmConvertOptions.from_preset(
    "smoldocling",
    engine_options=MlxVlmEngineOptions()
)

Picture Description

from docling.datamodel.pipeline_options import PictureDescriptionVlmEngineOptions

# Use Granite Vision for detailed descriptions
options = PictureDescriptionVlmEngineOptions.from_preset("granite_vision")

Code & Formula Extraction

from docling.datamodel.pipeline_options import CodeFormulaVlmOptions

# Use specialized CodeFormulaV2 model
options = CodeFormulaVlmOptions.from_preset("codeformulav2")

Additional Resources

• Vision Models Usage Guide - VLM-specific documentation
• Advanced Options - Advanced configuration
• GPU Support - GPU acceleration setup
• Supported Formats - Input format support

Notes

• DocTags Format: Structured XML-like format optimized for document understanding
• Markdown Format: Human-readable format for general-purpose conversion
• Model Updates: New models are added regularly. Check the codebase for latest additions
• Engine Compatibility: Not all engines work on all platforms. AUTO_INLINE handles this automatically
• Performance: Actual performance varies by hardware, document complexity, and model size