EmbeddingsOperator

Generates dense vector embeddings from document text using a configurable AI provider.

• Short Name: embeddings
• Category: Functional

Overview

EmbeddingsOperator converts document text (or pre-chunked content) into float vector
embeddings suitable for similarity search and RAG pipelines. It supports HuggingFace (local
and API), LiteLLM (100+ providers including Ollama, OpenAI, Azure, Cohere, AWS), and WatsonX
through a unified adapter architecture. Long text is automatically split and averaged when it
exceeds the model token limit.

Key Features

• Three provider backends: HuggingFace, LiteLLM, WatsonX
• Local inference (HuggingFace) — no API key, no cost, offline capable, tested to 600+ parallel processes
• Automatic chunking of text that exceeds token_limit
• Per-document error handling — failed documents are logged and skipped
• Configurable output column name (embeddings_column)
• Works with both content (full text) and chunked_content (pre-chunked) inputs

Operator Configuration

{
  "type": "embeddings",
  "name": "generate_embeddings",
  "config": {
    "provider": "litellm",
    "provider_config": {
      "model_id": "openai/nomic-embed-text",
      "api_base": "http://localhost:11434"
    },
    "embeddings_column": "embeddings",
    "doc_column": "content"
  },
  "depends_on": ["chunk_documents"]
}

Parameters

Common parameters

HuggingFace provider_config

LiteLLM provider_config

WatsonX provider_config

Output Columns

All input columns are preserved. The operator appends:

Examples

Example 1 — Ollama via LiteLLM (local)

{
  "type": "embeddings",
  "name": "embed",
  "config": {
    "provider": "litellm",
    "provider_config": {
      "model_id": "openai/nomic-embed-text",
      "api_base": "http://localhost:11434"
    }
  },
  "depends_on": ["chunk"]
}

Example 2 — HuggingFace local inference

{
  "type": "embeddings",
  "name": "embed",
  "config": {
    "provider": "huggingface",
    "provider_config": {
      "model_id": "sentence-transformers/all-MiniLM-L6-v2",
      "use_local": true,
      "device": "cpu",
      "batch_size": 16
    }
  },
  "depends_on": ["chunk"]
}

Example 3 — WatsonX native

{
  "type": "embeddings",
  "name": "embed",
  "config": {
    "provider": "watsonx",
    "provider_config": {
      "model_id": "ibm/slate-125m-english-rtrvr",
      "api_key": "${WATSONX_API_KEY}",
      "api_base": "${WATSONX_API_BASE}",
      "container_id": "${WATSONX_PROJECT_ID}",
      "container_kind": "project",
      "batch_size": 800,
      "enable_rate_limiting": true
    }
  },
  "depends_on": ["chunk"]
}

Example 4 — OpenAI via LiteLLM

{
  "type": "embeddings",
  "name": "embed",
  "config": {
    "provider": "litellm",
    "provider_config": {
      "model_id": "openai/text-embedding-3-small",
      "api_key": "${OPENAI_API_KEY}"
    }
  },
  "depends_on": ["chunk"]
}

Troubleshooting

API key best practice

Store keys as environment variables — never in flow files committed to git:

export OPENAI_API_KEY=sk-...
export WATSONX_API_KEY=...
export HUGGINGFACE_API_KEY=hf_...

Architecture

Provider selection guide

LiteLLM model prefix reference

Typical pipeline position

Ingest → Extract → Chunker → EmbeddingsOperator → VectorDB

Sample flow

See sample_flows/use_cases/invoice_processing.json.