v2.0 IBM/docling-pipelines

Changelog

All notable changes to docling-pipelines will be documented in this file.

The format follows Keep a Changelog.
Versioning follows Semantic Versioning 2.0.0.

Unreleased

Added

  • React frontend UI with Elyra-based pipeline canvas, per-operator properties panels, project/flow management, and Node.js BFF layer; bundled into the wheel and served at /ui/ (#58)
  • StorageOutputOperator — writes processed documents to a configurable destination with processed_content, refetch_original, and comprehensive_export modes (#58)
  • S3DestinationAdapter — writes to S3 / IBM COS / MinIO with env-var credentials and bucket pre-flight validation (#58)
  • SharePointDestinationAdapter — writes to SharePoint document libraries via Microsoft Graph API (#58)
  • GoogleDriveDestinationAdapter — writes to Google Drive with resumable uploads and Service Account / OAuth2 auth (#58)
  • onedrive provider alias for SharePointDestinationAdapter (#58)
  • ibm_cos provider alias for S3DestinationAdapter (#58)
  • Hierarchical multi-source path namespacing when ingest_source is configured with multiple paths (#58)
  • Dropbox ingest source adapter with OAuth2 auth, cursor pagination, and extension/size filters (#44)
  • Milvus Lite support (auth_type: "lite") for container-free local vector storage in VectorDBOperator (#56)
  • Full OCR engine exposure — ocr.engine, ocr.mode, ocr.enabled, and ocr.engine_options in ExtractOperator provider config (#58)
  • GPU acceleration for ExtractOperator via standard_pipeline.accelerator in provider_config (#58)
  • HashiCorp Vault integration — vault:// URI scheme for resolving secrets in flow configs at runtime (#58)
  • docling-pipelines-slim package variant excluding heavyweight operator dependencies (#58)
  • Comprehensive Vitest unit test suite for the frontend (149 files, 2155+ tests) (#81)
  • Flow execution defaults enable_micro_batching to true when not set in the flow (#58)
  • OpenLineage dependencies, domain models, and port interfaces (#62)
  • Deprecation policy (docs/guides/DEPRECATION_POLICY.md) and migration guide template (#58)
  • Release process documentation (RELEASE_PROCESS.md) (#58)

Changed

  • Breaking: IngestSourceOperator migrated from connection_params + credentials to unified provider_config; all 8 source adapters and all sample flows updated (#152)
  • PIIAndHAPAnnotator provider selection replaced with decorator-based PIIAndHAPDetectionFactory registry (#76)
  • 33 self-free instance methods converted to @staticmethod across FlowValidator, AuthoringCompiler, FlowExecutionReporter, MetadataAggregator, and IncrementalUpdateService (#139)
  • 18 self-free per-document hot-path methods converted to @staticmethod across ReadabilityMetrics, JobReportGenerator, EmbeddingsOperator, and DuckDBTableStorage (#139)

Fixed

  • Storage output now accepts cloud destination credentials and S3 key_prefix saved by the UI, while retaining support for the separate credentials field and legacy S3 prefix, and reads ingest source paths from the normalized provider_config.
  • Updated JupyterLab to 4.6.4 in the full and slim notebooks extras to resolve the security alerts tracked in #51.
  • Notification panel now propagates action_type from backend validation and fixes stale alert detection (#91)
  • Markdown chunking: send extracted content with .md filename to preserve source name in chunk metadata (#148)
  • Job runs list response now populates the actual flow name instead of a blank value (#118)
  • VectorDB enrichment: hydrate connection defaults into provider_config before API call (#80)
  • Milvus auth fields, advanced JSON validation, and SSL default corrected in the frontend (#80)
  • Frontend wheel build is now mandatory; UI included in the distributed package (#125)
  • PII and HAP operator validates provider_config and model_id during flow validation (#105)
  • Node execution logs aligned with DAG pipeline order; active duration calculation fixed (#107)
  • scripts/test_examples.py --dry-run skips prerequisites before probing Ollama or environment (#31)
  • S3 benchmark flows use configured prefix and max_files without embedding a corpus-wide exclusion list

1.0.1 - 2026-09-07

Fixed

  • docling-pipelines-api console command — The entry point previously pointed at the FastAPI app object (docpipe.api.main:app), causing a TypeError on invocation. A run() launcher function has been added to src/docpipe/api/main.py and pyproject.toml now registers docpipe.api.main:run as the entry point. Running docling-pipelines-api now correctly starts a Uvicorn server on 127.0.0.1:8080.

  • Ruff PTH123 compliance — Replaced open() calls with Path.open() across the codebase.

Changed

  • Bumped vulnerable dependencies flagged by Dependabot.

1.0.0 - 2026-08-28

Added

  • Modular operator-based data processing framework for building document curation pipelines
  • Extract operators: ExtractOperator with support for Docling document extraction, entity extraction, and multiple output modes
  • Ingest operators: IngestLocalOperator, IngestSourceOperator with support for local filesystem, S3, Azure Blob, Google Cloud Storage, and Box
  • ACL operators: ACLOperator for access control list management
  • Functional operators: ChunkerOperator, EmbeddingsOperator, BranchingOperator, NOOPOperator, MergeOperator, EntityCurationOperator, DocIdHashOperator
  • Quality operators: EdedupOperator, RedactionOperator, LanguageDetect, SQLFilterOperator, DocumentClassifierOperator, ReadabilityOperator, PIIAndHAPAnnotator, MLEnrichmentOperator, DocQuality
  • Storage operators: DocumentSetOperator for document set management
  • VectorDB operators: VectorDBOperator with OpenSearch adapter
  • DAG-based flow execution model defined via JSON flow configuration files
  • Prefect orchestration layer for parallel execution and task dependency management
  • FastAPI REST service with OpenAPI / Swagger documentation
  • CLI entry point (docling-pipelines) for flow execution, validation, and operator listing
  • Python library interface via DocpipeFlowManager
  • PyArrow table format for all inter-operator data transfer
  • JSON structured logging (DS_LOG_JSON=True) via ConditionalFormatter
  • Sensitive data sanitisation (sanitize_sensitive_data()) in REST API calls
  • Job run tracking with metadata aggregation
  • detect-secrets pre-commit hook integration
  • SonarQube, ruff, mypy, and Mend CI quality gates
  • Comprehensive documentation under docs/