Bug: Local LLM setup fails — missing env loading and vector dimension mismatch

[androemeda]

Bug Report

When following the SETUP.md instructions for local development using Ollama (Section 4), the backend fails to start and the knowledge base upload endpoint returns errors.

---

Bug 1: Backend fails to start with Field required errors

Steps to Reproduce

1. Follow SETUP.md to configure backend/.env with PROVIDER=local and Ollama settings.

2. Run the backend directly:

poetry run uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

3. Observe all settings fields reported as missing.

Expected Behavior

The server should start successfully, reading configuration from backend/.env.

Actual Behavior

AZURE_DEPLOYMENT_NAME
  Field required [type=missing, input_value={}, input_type=dict]
AZURE_EMBEDDING_DEPLOYMENT_NAME

Field required [type=missing, input_value={}, input_type=dict]
QDRANT_URL

Field required [type=missing, input_value={}, input_type=dict]

Root Cause

Settings(BaseSettings) in config.py does not configure env_file, so Pydantic only reads from OS environment variables, not from the .env file.

This works with Docker Compose (which injects env vars via env_file: in docker-compose.yml) but fails when running the backend directly.

---

Bug 2: Knowledge base PDF upload fails with vector dimension mismatch

Steps to Reproduce

1. Configure PROVIDER=local with
   AZURE_EMBEDDING_DEPLOYMENT_NAME=nomic-embed-text (as recommended in SETUP.md)

2. Start Qdrant and the backend.

3. Upload a PDF via:

POST /kb/upload-pdf

Expected Behavior

The PDF should be chunked, embedded, and stored in Qdrant successfully.

Actual Behavior

{

"status": "error",

"message": "Error uploading PDF file: Unexpected Response: 400 (Bad Request)\nRaw response content:\nb'{"status":{"error":"Wrong input: Vector dimension error: expected dim: 1536, got 768"}}'"

}

Root Cause

create_knowledge_base_collection_if_not_exists() in knowledge_base_service.py hardcodes

vector_size = 1536

This matches Azure OpenAI’s text-embedding-ada-002, but nomic-embed-text produces 768-dimensional vectors, which causes the mismatch.

---

Additional: qdrant_storage/ not ignored in git

When running Qdrant with the Docker command from SETUP.md : the generated .json, .dat, and .mmap storage files are not ignored and can accidentally be committed.

---

Proposed Fix

File	Change
backend/app/core/config.py	Add SettingsConfigDict(env_file=".env") to the Settings class
backend/app/services/knowledge_base_service.py	Set vector_size dynamically: 768 for local, 1536 for Azure
.gitignore	Add qdrant_storage/

---

Additional Context

These issues appear only when running the backend locally with Ollama, while the Docker-based setup works correctly because environment variables are injected through docker-compose.yml.

[androemeda]

@vcellmike I’ve opened a PR (#44 ) to fix this issue. Could you please take a look when you have time? Happy to make any changes if needed !

[M-V-RAGHUPATHI-SAI]

good catch, i ran into the exact same field required and dimension mismatch errors. appreciate you detailing the workaround.