ExtractOperator

Converts raw document files to markdown text and optionally extracts structured entities.

• Short Name: extract_operator
• Category: Extract

Overview

The operator follows hexagonal architecture principles, separating business logic from infrastructure concerns:

• Domain Layer: Core models (TextExtractionMode, EntityExtractionMode, extraction requests/results)
• Port Layer: Interfaces defining extraction contracts (TextExtractionPort, EntityExtractionPort)
• Adapter Layer: Concrete implementations for different extraction strategies
• Factory Layer: Creates appropriate adapters based on configuration

This architecture enables:
- Easy addition of new extraction strategies
- Clear separation of concerns
- Testability through dependency injection
- Flexibility in switching between implementations

Key Features

• Dual-Mode Operation: Supports both text extraction and entity extraction in a single operator
• Multiple Text Extraction Strategies: Docling Library (with optional VLM and ASR pipeline) and Docling Serve API
• Multiple Entity Extraction Strategies: LiteLLM (including Ollama via openai/ prefix), Docling template-based, and WatsonX
• Streaming Pipeline: When entity extraction is enabled, each document is submitted for entity extraction immediately after its text extraction completes, without waiting for the full batch. The two stages run concurrently, reducing end-to-end latency on large batches.
• Estimated Page Count Calculation: Automatically calculates estimated page counts for extracted text
• Parallel Processing: Automatic worker optimization based on CPU count
• Flexible Configuration: Provider-specific parameters with sensible defaults
• Consistent Error Handling: Unified error handling and metadata across all modes

Operator Configuration

{
  "type": "extract_operator",
  "name": "extract_documents",
  "config": {
    "text_extraction": {
      "provider": "docling_library",
      "doc_column": "content",
      "provider_config": {}
    },
    "entity_extraction": {
      "provider": "none"
    }
  },
  "depends_on": ["ingest_documents"]
}

Text Extraction Providers

1. Docling Library Provider (Default)

Standard document extraction using the Docling library locally. Supports optional VLM (Vision-Language Model) pipeline for enhanced extraction and ASR (Automatic Speech Recognition) pipeline for audio/video processing.

Basic Configuration:

{
  "max_workers": 4,
  "text_extraction": {
    "provider": "docling_library",
    "doc_column": "content",
    "provider_config": {
    }
  },
  "entity_extraction": {
    "provider": "none"
  }
}

VLM Pipeline Configuration:

Enable VLM pipeline for enhanced extraction with vision-language models:

{
  "max_workers": 1,
  "text_extraction": {
    "provider": "docling_library",
    "doc_column": "content",
    "provider_config": {
      "vlm_pipeline": {
        "preset": "granite_docling",
        "engine": "transformers",
        "engine_options": {}
      }
    }
  },
  "entity_extraction": {
    "provider": "none"
  }
}

GPU Acceleration Configuration:

Enable GPU-accelerated standard pipeline processing for PDF and image documents. The adapter builds one DocumentConverter at initialization and reuses it across all documents — model weights are loaded onto the GPU once per adapter execution.

Note: Cannot be combined with vlm_pipeline. Requires max_workers: 1 and use_processes: false.

When device is omitted, the best available GPU is auto-detected at runtime via torch (CUDA → MPS → XPU). Specify device explicitly to pin a particular GPU.

{
  "max_workers": 1,
  "use_processes": false,
  "text_extraction": {
    "provider": "docling_library",
    "doc_column": "content",
    "provider_config": {
      "standard_pipeline": {
        "accelerator": {
          "num_threads": 6
        }
      }
    }
  },
  "entity_extraction": {
    "provider": "none"
  }
}

Or with an explicit device:

{
  "max_workers": 1,
  "use_processes": false,
  "text_extraction": {
    "provider": "docling_library",
    "doc_column": "content",
    "provider_config": {
      "standard_pipeline": {
        "accelerator": {
          "device": "cuda",
          "num_threads": 6
        }
      }
    }
  },
  "entity_extraction": {
    "provider": "none"
  }
}

Supported GPU Devices:
- mps — Apple Metal Performance Shaders (Apple Silicon: M1/M2/M3/M4)
- cuda — NVIDIA CUDA (automatic device selection)
- cuda:<index> — NVIDIA CUDA on a specific device (e.g. cuda:0, cuda:1)
- xpu — Intel XPU

Supported VLM Engines:
- transformers: Local inference using Transformers library
- mlx: Local inference optimized for macOS (Apple Silicon)
- api_ollama: Ollama API
- api_openai: OpenAI API
- api_watsonx: IBM watsonx.ai API
- api_lmstudio: LM Studio API
- api: Generic API endpoint

VLM Pipeline Configuration Examples:

Ollama:

{
  "text_extraction": {
    "provider": "docling_library",
    "provider_config": {
      "vlm_pipeline": {
        "preset": "granite_docling",
        "engine": "api_ollama",
        "engine_options": {
          "api_base": "http://localhost:11434",
          "model_id": "ibm/granite-docling:258m"
        }
      }
    }
  }
}

OpenAI:

{
  "text_extraction": {
    "provider": "docling_library",
    "provider_config": {
      "vlm_pipeline": {
        "preset": "qwen",
        "engine": "api_openai",
        "engine_options": {
          "api_base": "https://api.openai.com",
          "model_id": "gpt-4-vision-preview",
          "api_key": "<your-api-key>"
        }
      }
    }
  }
}

ASR Pipeline Configuration:

Enable ASR pipeline for audio and video file transcription:

{
  "max_workers": 2,
  "text_extraction": {
    "provider": "docling_library",
    "doc_column": "content",
    "provider_config": {
      "asr_pipeline": {
        "model_id": "whisper_turbo"
      }
    }
  },
  "entity_extraction": {
    "provider": "none"
  }
}

Use Cases:
- Simple document conversion to markdown (without VLM)
- Complex document layouts (with VLM)
- Documents with mixed content types (with VLM)
- Audio/video transcription (with ASR)
- Local processing without external dependencies
- High-accuracy extraction requirements (with VLM)
- Quick prototyping and testing

Sample Flows:
- Basic: sample_flows/quickstart/basic_ingest_extract.json
- Complete: sample_flows/quickstart/complete_pipeline_ollama.json
- Audio/Video: sample_flows/use_cases/audio_video_extraction.json

2. Docling Serve Provider

REST API-based extraction using the Docling-Serve service for scalable, production-ready document processing.

Configuration:

{
  "text_extraction": {
    "provider": "docling_serve",
    "provider_config": {
      "base_url": "http://localhost:5001",
      "timeout": 300,
      "poll_interval": 2,
      "max_retries": 3,
      "ocr": {
        "enabled": true,
        "engine": "easyocr",
        "engine_options": {
          "lang": ["en"]
        }
      },
      "pdf_backend": "dlparse_v4",
      "table_mode": "accurate",
      "image_export_mode": "embedded"
    }
  },
  "entity_extraction": {
    "provider": "none"
  }
}

File Handling:
- .txt files are processed locally using basic text extraction (not sent to Docling Serve)
- Binary content submissions preserve the original filename for proper MIME type detection
- Supported MIME types are automatically detected based on file extension

MIME Type Mapping:

Prerequisites:

# Start docling-serve locally
docker run -p 5001:5001 ds4sd/docling-serve:latest

# Or use docker-compose
docker-compose -f docker-compose.docling-serve.yml up -d

Features:
- OCR Support: Process scanned documents and images
- EasyOCR engine (supports 80+ languages)
- Tesseract engine
- Multi-language support
- Multiple PDF Backends:
- dlparse_v4: Latest Docling parser (recommended)
- dlparse_v3: Legacy Docling parser
- pypdfium2: PyPDFium2 backend
- Table Extraction Modes:
- accurate: High accuracy (slower)
- fast: Fast processing (less accurate)
- Image Export Options:
- placeholder: Replace images with a placeholder (default)
- embedded: Embed images in output
- referenced: Reference images by path
- none: Skip image export

Use Cases:
- Production document processing pipelines
- High-volume document ingestion
- Scanned document processing with OCR
- Multi-language document processing
- Distributed processing architectures

Sample Flow: sample_flows/quickstart/complete_pipeline_ollama.json

Entity Extraction Providers

Entity extraction can be combined with any text extraction provider to extract structured data from the extracted text.

1. None Provider (Default)

No entity extraction is performed. Only text extraction is executed.

Configuration:

{
  "text_extraction": {
    "provider": "docling_library"
  },
  "entity_extraction": {
    "provider": "none"
  }
}

2. Docling Provider (VLM-Based)

Vision-Language Model (VLM) based entity extraction using Docling's VLM pipeline for structured data extraction from documents.

Basic Configuration:

{
  "text_extraction": {
    "provider": "docling_library"
  },
  "entity_extraction": {
    "provider": "docling",
    "custom_schema": {
      "invoice_number": "string",
      "invoice_date": "string",
      "total_amount": "number",
      "line_items": [
        {
          "description": "string",
          "quantity": "number",
          "unit_price": "number"
        }
      ]
    }
  }
}

Custom Model Configuration:

Users can configure custom inline VLM models for entity extraction using the vlm_pipeline parameter. Only inline models (HuggingFace) are supported as DocumentExtractor does not support remote API endpoints.

Inline Model (HuggingFace with Transformers):

{
  "text_extraction": {
    "provider": "docling_library"
  },
  "entity_extraction": {
    "provider": "docling",
    "provider_config": {
      "vlm_pipeline": {
        "model_type": "inline",
        "inline_model": {
          "repo_id": "numind/NuExtract-2.0-2B",
          "inference_framework": "transformers",
          "scale": 2.0,
          "temperature": 0.0,
          "max_new_tokens": 4096,
          "load_in_8bit": true,
          "torch_dtype": "bfloat16"
        }
      }
    },
    "custom_schema": {
      "invoice_number": "string",
      "total_amount": "number"
    }
  }
}

Note: API model configuration is not supported. For API-based entity extraction, use entity_extraction.provider: "litellm" instead.

Supported Model Types:
- Inline Models: HuggingFace models with Transformers, vLLM, or MLX backends
- API Models: Ollama, vLLM server, OpenAI-compatible endpoints

Supported Backends (for inline models):
- transformers: HuggingFace Transformers library
- vllm: vLLM inference engine
- mlx: Apple MLX framework (macOS only)

Use Cases:
- Extracting structured data from standardized forms
- Processing documents with known schema
- Vision-based entity extraction from complex layouts
- Custom model integration for specialized domains
- Template-driven workflows with VLM enhancement

Sample Flow: sample_flows/quickstart/complete_pipeline_ollama.json

3. LiteLLM Provider

Multi-provider LLM extraction using LiteLLM for accessing 100+ LLM providers (OpenAI, Anthropic, Cohere, Ollama, etc.).

Configuration (OpenAI):

{
  "text_extraction": {
    "provider": "docling_library"
  },
  "entity_extraction": {
    "provider": "litellm",
    "provider_config": {
      "model_id": "openai/gpt-3.5-turbo",
      "api_key": "your-api-key",  # pragma: allowlist secret
      "api_base": "https://api.openai.com/v1",
      "temperature": 0.0,
      "max_tokens": 2000
    },
    "custom_schema": {
      "invoice_number": "string",
      "total_amount": "number"
    }
  }
}

Configuration (Ollama via LiteLLM):

{
  "text_extraction": {
    "provider": "docling_library"
  },
  "entity_extraction": {
    "provider": "litellm",
    "provider_config": {
      "model_id": "openai/llama3.2",
      "api_base": "http://localhost:11434/v1",
      "api_key": "<ollama_key>",
      "temperature": 0.0,
      "max_tokens": 4096
    },
    "custom_schema": {
      "invoice_number": "string",
      "total_amount": "number"
    }
  }
}

Configuration (Remote vLLM with Streaming & Extended Timeout):

For high-concurrency scenarios with remote vLLM clusters processing large documents, use streaming and extended timeouts to prevent connection drops:

{
  "text_extraction": {
    "provider": "docling_library"
  },
  "entity_extraction": {
    "provider": "litellm",
    "provider_config": {
      "model_id": "openai/granite4:latest",
      "api_base": "https://your-vllm-route/v1",
      "api_key": "YOUR_API_KEY",  # pragma: allowlist secret
      "temperature": 0.0,
      "max_tokens": 5000,
      "stream": true,
      "timeout": 1800
    },
    "custom_schema": {
      "invoice_number": "string",
      "total_amount": "number"
    }
  }
}

Advanced Provider Configuration Parameters:
- stream (boolean, default: false): Enable HTTP chunked transfer encoding to keep connections alive during long-running requests. Recommended for remote vLLM clusters processing large documents.
- timeout (integer, default: 60): HTTP client read timeout in seconds. Set to 1800 (30 minutes) for large documents that require extended generation time.

Why Streaming & Extended Timeout?

During high-concurrency scalability testing with remote vLLM clusters, connection issues were identified:
1. Infrastructure Idle Timeout: Load balancers (IBM Cloud Edge, HAProxy) reset idle connections after ~100 seconds without data transmission. With stream=false, vLLM waits until entire generation completes (150+ seconds for large documents) before sending response, causing connections to be dropped mid-generation.
2. Extended Processing Time: Large documents requiring 5000+ tokens at ~88 tokens/sec take 150+ seconds to generate, exceeding typical load balancer idle timeouts.

Solution: Combining stream=true with timeout=1800 ensures:
- Continuous packet flow (streaming chunks) prevents idle timeout detection by load balancers
- Extended timeout (30 minutes) allows completion of large document processing
- Connections remain stable under high concurrency (10,000+ documents)

Supported Providers:
- OpenAI (GPT-3.5, GPT-4, GPT-4o)
- Anthropic (Claude 3 Opus, Sonnet, Haiku)
- Cohere (Command, Command-R)
- Google (Gemini Pro, Gemini Ultra)
- Azure OpenAI
- AWS Bedrock
- Ollama (via openai/ model prefix)
- And 100+ other providers via LiteLLM

Prerequisites for Ollama:
- Ollama server running on http://localhost:11434
- Model pulled: ollama pull llama3.2

Use Cases:
- Multi-provider LLM support without code changes
- Enterprise LLM deployments (Azure, AWS Bedrock)
- Cost optimization by switching between providers
- Fallback strategies across multiple providers
- Schema-based and schema-free entity extraction
- Local LLM processing via Ollama

Sample Flows:
- sample_flows/operators/entity_extraction_litellm.json

4. WatsonX Provider

IBM WatsonX.ai LLM-based entity extraction for enterprise deployments.

Configuration:

{
  "text_extraction": {
    "provider": "docling_library"
  },
  "entity_extraction": {
    "provider": "watsonx",
    "provider_config": {
      "model_id": "ibm/granite-13b-chat-v2",
      "api_key": "${WATSONX_API_KEY}",
      "container_id": "${WATSONX_CONTAINER_ID}",
      "api_base": "https://us-south.ml.cloud.ibm.com",
      "container_kind": "project",
      "temperature": 0.0,
      "max_tokens": 2000
    },
    "custom_schema": {
      "invoice_number": "string",
      "total_amount": "number"
    }
  }
}

Environment Variables:
- WATSONX_API_KEY: WatsonX API key (required)
- WATSONX_CONTAINER_ID: WatsonX project or space ID (required)
- WATSONX_API_BASE_URL: WatsonX API base URL (optional, defaults to us-south)
- WATSONX_CONTAINER_KIND: Container type - "project" or "space" (optional, defaults to "project")

Use Cases:
- Enterprise LLM deployments with IBM WatsonX.ai
- Regulated industries requiring on-premises or private cloud LLM
- Schema-based entity extraction with IBM Granite models
- Integration with existing IBM Cloud infrastructure

Sample Flow: sample_flows/operators/entity_extraction_watsonx.json

Configuration Parameters

Common Parameters

Docling Library Provider Parameters

Docling Serve Provider Parameters

OCR Configuration

Both docling_library and docling_serve providers accept an ocr block inside provider_config. Omitting the block entirely uses docling-pipelines defaults (OCR enabled, RapidOCR engine, default mode).

"provider_config": {
  "ocr": {
    "enabled": true,
    "engine": "easyocr",
    "mode": "pdf_aware_layout_regions",
    "engine_options": {
      "lang": ["en", "fr"],
      "use_gpu": false,
      "confidence_threshold": 0.5
    }
  }
}

OCR Fields

Supported Engines

Installation Guidance by OS

RapidOCR is included in the default PyPI install, and ocr.engine: "rapidocr" is the default runtime behaviour for first-run OCR.

Supported Modes

engine_options Reference

Docling Entity Extraction Parameters

vlm_pipeline Structure:

Only "inline" models (HuggingFace) are supported. API model types are not supported by Docling's DocumentExtractor; use entity_extraction.provider: "litellm" for API-based extraction instead.

Example (all fields shown with their defaults; only repo_id is required):

{
  "model_type": "inline",
  "inline_model": {
    "repo_id": "numind/NuExtract-2.0-2B",
    "inference_framework": "transformers",
    "scale": 2.0,
    "temperature": 0.0,
    "max_new_tokens": 4096,
    "load_in_8bit": true,
    "torch_dtype": "bfloat16",
    "prompt": "",
    "response_format": "markdown"
  }
}

LiteLLM Entity Extraction Parameters

All parameters are nested under entity_extraction.provider_config:

WatsonX Entity Extraction Parameters

All parameters are nested under entity_extraction.provider_config:

Input/Output Data Formats

Input columns (from IngestSourceOperator)

Output Columns

All input columns are preserved. The operator appends:

When expand_extracted_data: true, entity fields are expanded into individual columns.

Examples

Example 1: Basic Text Extraction Only

{

  "operator_params": {
    "text_extraction": {
      "provider": "docling_library",
      "doc_column": "content",
      "provider_config": {
      }
    },
    "entity_extraction": {
      "provider": "none"
    }
  }
}

Example 2: Text + Entity Extraction with Ollama (via LiteLLM)

{

  "operator_params": {
    "max_workers": 2,
    "text_extraction": {
      "provider": "docling_library",
      "doc_column": "content"
    },
    "entity_extraction": {
      "provider": "litellm",
      "provider_config": {
        "model_id": "openai/llama3.2",
        "api_base": "http://localhost:11434/v1",
        "api_key": "<ollama_key>",
        "temperature": 0.0,
        "max_tokens": 4096
      },
      "custom_schema": {
        "invoice_number": "string",
        "vendor_name": "string",
        "total_amount": "number"
      }
    }
  }
}

Example 3: VLM Pipeline Text Extraction

{

  "operator_params": {
    "max_workers": 1,
    "text_extraction": {
      "provider": "docling_library",
      "doc_column": "content",
      "provider_config": {
        "vlm_pipeline": {
          "preset": "granite_docling",
          "engine": "transformers",
          "engine_options": {}
        }
      }
    },
    "entity_extraction": {
      "provider": "none"
    }
  }
}

Example 4: Docling Serve with OCR

{
  "operator_params": {
    "max_workers": 4,
    "text_extraction": {
      "provider": "docling_serve",
      "doc_column": "content",
      "provider_config": {
        "base_url": "http://localhost:5001",
        "ocr": {
          "enabled": true,
          "engine": "tesseract",
          "mode": "layout_regions",
          "engine_options": {
            "lang": ["eng", "spa"]
          }
        },
        "pdf_backend": "dlparse_v4",
        "table_mode": "accurate"
      }
    },
    "entity_extraction": {
      "provider": "none"
    }
  }
}

Example 5: Template-Based Entity Extraction

{

  "operator_params": {
    "max_workers": 2,
    "text_extraction": {
      "provider": "docling_library",
      "doc_column": "content"
    },
    "entity_extraction": {
      "provider": "docling",
      "custom_schema": {
        "invoice_number": "string",
        "invoice_date": "string",
        "vendor_name": "string",
        "vendor_address": "string",
        "total_amount": "number",
        "currency": "string",
        "line_items": [
          {
            "description": "string",
            "quantity": "number",
            "unit_price": "number",
            "total": "number"
          }
        ]
      }
    }
  }
}

Example 6: VLM Pipeline + Ollama Entity Extraction (via LiteLLM)

{

  "operator_params": {
    "max_workers": 1,
    "text_extraction": {
      "provider": "docling_library",
      "doc_column": "content",
      "provider_config": {
        "vlm_pipeline": {
          "preset": "granite_docling",
          "engine": "transformers",
          "engine_options": {}
        }
      }
    },
    "entity_extraction": {
      "provider": "litellm",
      "provider_config": {
        "model_id": "openai/llama3.2",
        "api_base": "http://localhost:11434/v1",
        "api_key": "<ollama_key>",
        "temperature": 0.0
      }
    }
  }
}

Example 7: ASR Pipeline for Audio/Video Transcription

{

  "operator_params": {
    "max_workers": 2,
    "text_extraction": {
      "provider": "docling_library",
      "doc_column": "content",
      "provider_config": {
        "asr_pipeline": {
          "model_id": "whisper_turbo"
        }
      }
    },
    "entity_extraction": {
      "provider": "none"
    }
  }
}

Integration Requirements

ASR Dependencies (for Audio/Video Processing)

Requirement: ASR (Automatic Speech Recognition) dependencies must be installed to process audio and video files

Installation:

# Install ASR dependencies
uv pip install -e '.[asr]'

Supported Audio/Video Formats (when ASR is installed):
- Audio: MP3, WAV, M4A, FLAC, OGG
- Video: MP4, AVI, MOV, MKV

Note: If ASR dependencies are not installed, the operator will only support standard document formats (PDF, DOCX, PPTX, etc.) and will log a warning if use_asr_pipeline=true is configured.

Used By: Text extraction with ASR when use_asr_pipeline=true

ffmpeg (for Audio/Video Processing)

Requirement: ffmpeg must be installed and available on your PATH for processing certain audio and video formats

Required For:
- Audio formats: M4A, AAC, OGG, FLAC
- All video formats: MP4, AVI, MOV, etc.

Not Required For:
- Audio formats: WAV, MP3
- Document formats: PDF, images, etc.

Installation:

macOS (using Homebrew):

brew install ffmpeg

Linux (Ubuntu/Debian):

sudo apt update
sudo apt install ffmpeg

Linux (RHEL/CentOS/Fedora):

sudo dnf install ffmpeg

Verify Installation:

ffmpeg -version

Used By: Text extraction with ASR (Automatic Speech Recognition) when processing audio/video files

Ollama Integration (for LiteLLM entity extraction with Ollama)

Requirement: Ollama server must be running on http://localhost:11434

Setup:

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull required model
ollama pull llama3.2

# Verify Ollama is running
curl http://localhost:11434/api/tags

Used By: Entity extraction when entity_extraction.provider="litellm" with entity_extraction.provider_config.model_id="openai/llama3.2" and entity_extraction.provider_config.api_base="http://localhost:11434/v1"

Docling Serve Integration (for docling_serve text extraction)

Requirement: Docling Serve must be running (default: http://localhost:5001)

Setup:

# Using Docker
docker run -p 5001:5001 ds4sd/docling-serve:latest

# Or using docker-compose
docker-compose -f docker-compose.docling-serve.yml up -d

# Verify service is running
curl http://localhost:5001/health

Used By: Text extraction when text_extraction.provider="docling_serve"

Mode Comparison

Best Practices

When to Use Each Text Extraction Provider

Use Docling Library Provider (Basic) When:
- Processing simple documents locally
- No OCR required
- Quick prototyping
- Minimal setup needed

Use Docling Library Provider (VLM Pipeline) When:
- Complex document layouts
- High accuracy requirements
- Local processing preferred
- GPU available for inference

Use Docling Serve Provider When:
- Production deployment
- OCR required for scanned documents
- Multi-language support needed
- Horizontal scaling required
- Processing high document volumes

When to Use Each Entity Extraction Provider

Use None Provider When:
- Only text extraction is needed
- Entity extraction will be done in a separate step

Use LiteLLM Provider When:
- Multi-provider LLM support needed
- Cloud-based or local (Ollama) LLM processing
- Cost optimization by switching between providers
- Flexible entity extraction without predefined templates
- Schema-based or schema-free extraction needed

Use Docling Provider When:
- Extracting structured data from standardized forms
- Processing documents with known schema
- Fast, deterministic extraction required
- Template-driven workflows

Use WatsonX Provider When:
- Enterprise LLM deployments with IBM WatsonX.ai
- Regulated industries requiring private cloud LLM
- Integration with existing IBM Cloud infrastructure
- IBM Granite models preferred

Performance Optimization

1. Worker Configuration:
2. Text extraction: Use default auto-detection (typically 4-8 workers)
3. Entity extraction with LLM: Reduce to 1-2 workers to avoid overwhelming the LLM

4. VLM extraction: Use 1 worker due to high memory requirements

5. Document Size:

6. For large documents with entity extraction, adjust entity_max_doc_chars to control LLM input size

7. Consider chunking very large documents before extraction

8. Parallel Processing:

9. Set top-level use_processes: true for CPU-intensive tasks
10. Use use_processes: false (default) for I/O-bound tasks

Execution Metadata

The operator provides the following metadata after execution:

• page_type_stats (dict): Aggregate estimated pages grouped by source document format (e.g., {"pdf": 120, "docx": 45})
• total_pages_converted (int): Total estimated pages across all successfully processed documents

These metrics are available through the operator's metadata and can be used for tracking document processing volume and performance analysis.

Sample Flows

Complete sample flows are available in sample_flows/:

• sample_flows/quickstart/basic_ingest_extract.json - Basic text extraction without entity extraction
• sample_flows/quickstart/complete_pipeline_ollama.json - Complete extraction with VLM and entity extraction (Ollama)
• sample_flows/quickstart/complete_pipeline_watsonx.json - Complete extraction with WatsonX
• sample_flows/operators/entity_extraction_litellm.json - LiteLLM entity extraction
• sample_flows/operators/entity_extraction_watsonx.json - WatsonX entity extraction
• sample_flows/use_cases/audio_video_extraction.json - Audio and video extraction

Troubleshooting

Common Issues

Issue: "Failed to initialize text extraction adapter"
- Verify the text_extraction.provider value is valid: "docling_library" or "docling_serve"
- For VLM pipeline (when vlm_pipeline is configured), ensure required model files are available
- For Docling Serve provider, verify the service is running and accessible

Issue: "Failed to initialize entity extraction adapter"
- Verify the entity_extraction.provider value is valid: "litellm", "docling", "watsonx", or "none"
- For LiteLLM provider with Ollama, ensure Ollama server is running and the model is pulled, and use openai/ model prefix
- For WatsonX provider, ensure environment variables WATSONX_API_KEY and WATSONX_CONTAINER_ID are set
- Check that required parameters (model_name, etc.) are provided

Issue: "Ollama connection refused"
- Verify Ollama is running: curl http://localhost:11434/api/tags
- Check that the specified model is available: ollama list
- Ensure no firewall is blocking port 11434

Issue: "Docling Serve timeout"
- Increase timeout value
- Check Docling Serve service health: curl http://localhost:5001/health
- Verify network connectivity to the Docling Serve endpoint

Issue: "Entity extraction returns empty results"
- Verify document content is not empty after text extraction
- Check entity_max_doc_chars is not too restrictive
- For schema-based extraction, ensure the schema matches the document structure
- Review LLM model capabilities for the extraction task

Issue: "Audio/video processing fails with codec errors"
- Ensure ffmpeg is installed: ffmpeg -version
- Verify ffmpeg is in your PATH: which ffmpeg (macOS/Linux) or where ffmpeg (Windows)
- Install ffmpeg if missing:
- macOS: brew install ffmpeg
- Linux: sudo apt install ffmpeg or sudo dnf install ffmpeg
- Supported formats requiring ffmpeg: M4A, AAC, OGG, FLAC (audio), MP4, AVI, MOV (video)
- WAV and MP3 audio files do not require ffmpeg

Issue: "ffmpeg not found" error during audio/video extraction
- Verify ffmpeg installation: ffmpeg -version
- Add ffmpeg to your PATH if installed but not found
- Restart your terminal/shell after installing ffmpeg
- On macOS, ensure Homebrew's bin directory is in PATH: export PATH="/opt/homebrew/bin:$PATH"

Architecture Details

Hexagonal Architecture

The ExtractOperator follows hexagonal architecture (ports and adapters pattern) with clear separation of concerns:

ExtractOperator (Orchestrator)
    ↓
┌─────────────────────────────────────────────────────────────┐
│ Domain Layer                                                 │
│  - EntityExtractionService (business logic)                  │
│  - Domain Models (TextExtractionMode, EntityExtractionMode)  │
└─────────────────────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────────────────────┐
│ Port Layer (Interfaces)                                      │
│  - TextExtractionPort                                        │
│  - EntityExtractionPort                                      │
└─────────────────────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────────────────────┐
│ Adapter Layer (Implementations)                              │
│  Text Extraction:                                            │
│   - DoclingAdapter (docling_library provider, optional VLM/ASR)  │
│   - DoclingServeAdapter (docling_serve provider)                 │
│  Entity Extraction:                                          │
│   - LLMEntityAdapter (litellm and watsonx providers - unified)   │
│   - DoclingEntityAdapter (docling provider)                      │
└─────────────────────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────────────────────┐
│ Factory Layer                                                │
│  - TextExtractionAdapterFactory                              │
│  - EntityExtractionAdapterFactory                            │
└─────────────────────────────────────────────────────────────┘

Architecture Components:

• Domain Layer: EntityExtractionService handles business logic (prompt building, schema validation, response parsing)
• Port Layer: Interfaces define extraction contracts without implementation details
• Adapter Layer: Concrete implementations for different extraction strategies
• Factory Layer: Creates appropriate adapters based on configuration mode

Key Benefits:
- Easy addition of new extraction strategies by implementing ports
- Clear separation between business logic, interfaces, and implementations
- Testability through dependency injection and mocking
- Unified LLM support: Both litellm and watsonx use the same LLMEntityAdapter

Execution Flow

1. Initialization:
2. Parse extraction providers from configuration
3. Create text extraction adapter via TextExtractionAdapterFactory
4. Create entity extraction adapter via EntityExtractionAdapterFactory (if enabled)

5. Initialize EntityExtractionService with the entity adapter

6. Text Extraction:

7. Delegate to text extraction adapter
8. Process documents in parallel using worker pool

9. Collect extracted content and metadata

10. Entity Extraction (if enabled):

11. EntityExtractionService builds prompts with schema
12. Delegate to entity extraction adapter
13. Process extracted text in parallel
14. Extract structured entities based on schema or LLM

15. Service parses and validates responses

16. Result Assembly:

17. Combine text and entity extraction results
18. Update metadata with processing statistics
19. Return transformed PyArrow table

Related Documentation

• Docling Documentation
• Ollama Documentation
• Sample Flows