[Feature] Multi-provider embedding abstraction layer

[Siddhant-K-code]

Summary

Create a unified embedding provider interface that supports OpenAI, Ollama, Cohere, Azure OpenAI, and HuggingFace TEI. Users configure once, swap providers without code changes.

Motivation

OpenAI lock-in is the #1 objection in enterprise conversations:

• Azure-only shops can't use vanilla OpenAI endpoints
• Privacy-sensitive teams need local models (Ollama)
• Cost-conscious users want cheaper alternatives (Cohere)
• The memory store ([Product] Context Memory Store - persistent deduplicated memory across sessions #29), graph ([Product] Code Change Impact Graph - blast radius analysis from codebase structure #30), and commit analysis ([Product] Semantic commit analysis - find similar past changes and predict incidents #32) all need embeddings - a single abstraction prevents provider-specific code from spreading

Provider interface

// pkg/embedding/provider.go
type Provider interface {
    Embed(ctx context.Context, text string) ([]float32, error)
    EmbedBatch(ctx context.Context, texts []string) ([][]float32, error)
    Dimension() int
    Name() string
}

// Factory function
func NewProvider(cfg ProviderConfig) (Provider, error)

Supported providers

Provider	Config key	Endpoint	Models
OpenAI	openai	api.openai.com	text-embedding-3-small, text-embedding-3-large
Azure OpenAI	azure	{deployment}.openai.azure.com	Same models, Azure-hosted
Ollama	ollama	localhost:11434	nomic-embed-text, mxbai-embed-large
Cohere	cohere	api.cohere.com	embed-english-v3.0, embed-multilingual-v3.0
HuggingFace TEI	huggingface	configurable	Any model via Text Embeddings Inference

Config file support

# distill.yaml
embedding:
  provider: "ollama"           # openai | azure | ollama | cohere | huggingface
  model: "nomic-embed-text"
  host: "http://localhost:11434"  # for ollama/huggingface
  api_key: "${EMBEDDING_API_KEY:-}"  # for cloud providers
  
  # Azure-specific
  azure_deployment: "my-embedding"
  azure_api_version: "2024-02-01"
  
  # Batch settings
  batch_size: 100              # max texts per batch call
  max_retries: 3
  timeout: "30s"

CLI flags

distill api --embedding-provider ollama --embedding-model nomic-embed-text
distill api --embedding-provider azure --embedding-model text-embedding-3-small
distill api --embedding-provider cohere --embedding-model embed-english-v3.0

Deliverables

• pkg/embedding/provider.go - Interface + factory
• pkg/embedding/openai/ - Existing, refactored to implement interface
• pkg/embedding/ollama/client.go - Ollama HTTP client
• pkg/embedding/azure/client.go - Azure OpenAI client
• pkg/embedding/cohere/client.go - Cohere client
• pkg/embedding/huggingface/client.go - HuggingFace TEI client
• Tests for each provider (mock HTTP)
• Update cmd/api.go, cmd/serve.go, cmd/mcp.go to use factory
• Update pkg/config/config.go with provider config
• docker-compose.yml example with Ollama sidecar
• README section on provider configuration

Acceptance Criteria

• Swap providers via config file without code changes
• All providers pass the same integration test suite
• Graceful error messages when provider is misconfigured
• Batch embedding works across all providers
• Dimension mismatch detected early (e.g., mixing 384-dim and 1536-dim embeddings)