Skip to content

feat: migrate embeddings from Gunicorn service to Ollama / BYOK providers #146

@jordanpartridge

Description

@jordanpartridge

Summary

Replace the custom Python/Gunicorn embedding service (knowledge-embeddings Docker container at :8001) with direct Ollama integration and support for bring-your-own-key (BYOK) providers like OpenAI.

Current State

  • Knowledge uses EmbeddingServiceInterface backed by a custom EmbeddingService that calls a Gunicorn Python server running BAAI/bge-large-en-v1.5 at http://localhost:8001/embed
  • This is the same model Ollama already serves locally (bge-large) on the same hardware
  • The Gunicorn container is a single point of failure with no scaling, no fallback, and extra operational overhead

Proposed Changes

  1. Add Ollama embedding provider — call Ollama's /api/embed endpoint directly (or via laravel/ai if feasible in Laravel Zero)
  2. Add OpenAI-compatible provider — support text-embedding-3-large or any OpenAI-compatible API with an API key
  3. Config-driven provider selection~/.knowledge/config.json or env vars to choose provider:
    {
      "embeddings": {
        "provider": "ollama",
        "model": "bge-large",
        "url": "http://100.68.122.24:11434"
      }
    }
  4. Kill knowledge-embeddings container — remove Docker service, update docs
  5. No vector re-indexing needed — same bge-large model produces identical 1024-dim vectors

Context

Acceptance Criteria

  • Embedding provider is configurable (ollama / openai / none)
  • Ollama provider works with existing bge-large model
  • OpenAI provider works with API key
  • knowledge-embeddings Docker container is removed
  • All existing tests pass, no vector migration needed

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions