Add LLM call tracing via Langfuse and OpenTelemetry (#139)

* Add LLM call tracing via Langfuse and OpenTelemetry

- All OpenAI calls are now automatically traced with model name, token
  counts, and message content via OpenInference instrumentation
- Langfuse (v3) available as an optional local dev service via
  docker compose profile; zero config needed to start tracing locally
- Traces route to Langfuse via standard OTEL env vars — works with
  Langfuse cloud or self-hosted (configured in vconic-deploy)
- docker-compose.yml is now committed and covers all optional services
  (postgres, elasticsearch, langfuse) via profiles

CON-180

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix CI: remove stale example_docker-compose.yml copy step

docker-compose.yml is now committed directly; the cp step is no longer needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update poetry.lock for openinference-instrumentation-openai dependency

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Author: pavanputhra

.env.example

@@ -1,14 +1,45 @@
+# ── Docker Compose profiles ──────────────────────────────────────────────────
+# Comma-separated list of optional services to run locally.
+# Available: redis, postgres, elasticsearch
+# Remove a service if you're pointing to an external instance instead.
+COMPOSE_PROFILES=postgres
+
+# ── Redis ─────────────────────────────────────────────────────────────────────
+# Use redis://redis when running the redis profile locally.
+# Use redis://<host> when pointing to an external instance.
 REDIS_URL=redis://redis
 
-# Leave this blank to disable API security
-# You set this before opening the port in your firewall
+# ── API ───────────────────────────────────────────────────────────────────────
+# Leave blank to disable API key security.
+# Set before opening the port in your firewall.
 CONSERVER_API_TOKEN=
 
-# to customize the config copy example_config.yml to config.yml
-# modify the values in config.yml as needed
-# and set CONSERVER_CONFIG_FILE to ./config.yml below
+# ── Config ────────────────────────────────────────────────────────────────────
+# Copy example_config.yml to config.yml, modify as needed, then set this path.
 CONSERVER_CONFIG_FILE=
 LOGGING_CONFIG_FILE=server/logging_dev.conf
 
-# Groq API key for Whisper transcription
-GROQ_API_KEY=your_groq_api_key_here
+
+# ── OpenTelemetry ─────────────────────────────────────────────────────────────
+# Leave OTEL_EXPORTER_OTLP_ENDPOINT blank to disable tracing.
+#
+# Local Langfuse (add 'langfuse' to COMPOSE_PROFILES, then use pre-seeded creds):
+#   OTEL_EXPORTER_OTLP_ENDPOINT=http://langfuse-server:3000/api/public/otel
+#   OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
+#   OTEL_EXPORTER_OTLP_HEADERS=Authorization=Basic cGstbGYtbG9jYWxkZXY6c2stbGYtbG9jYWxkZXY=
+#   (pre-seeded: pk-lf-localdev / sk-lf-localdev, UI at http://localhost:3000, admin@langfuse.local / admin1234)
+#
+# Langfuse cloud:
+#   OTEL_EXPORTER_OTLP_ENDPOINT=https://cloud.langfuse.com/api/public/otel
+#   OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
+#   OTEL_EXPORTER_OTLP_HEADERS=Authorization=Basic <base64(public_key:secret_key)>
+#   Generate: echo -n "pk-lf-xxx:sk-lf-xxx" | base64
+#
+# SigNoz / OTEL collector (gRPC):
+#   OTEL_EXPORTER_OTLP_ENDPOINT=http://signoz-otel-collector:4317
+#   OTEL_EXPORTER_OTLP_PROTOCOL=grpc
+#   OTEL_EXPORTER_OTLP_INSECURE=true
+OTEL_EXPORTER_OTLP_ENDPOINT=
+OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
+OTEL_EXPORTER_OTLP_INSECURE=false
+OTEL_EXPORTER_OTLP_HEADERS=

.github/workflows/run-tests.yml

@@ -1,5 +1,5 @@
 name: Python conserver unit test
-on: 
+on:
   push:
     branches: [ main ] # only run on pushes to main branch
   pull_request: # all PRs
@@ -17,10 +17,6 @@ jobs:
       run: |
         sudo apt-get update
         sudo apt-get install -y docker-compose
-    
-    - name: Copy example_docker-compose.yml to docker-compose.yml
-      run: |
-        cp example_docker-compose.yml docker-compose.yml
 
     - name: Create .env file
       run: |
@@ -35,7 +31,6 @@ jobs:
       run: |
         docker-compose up --build --detach
 
-    # Step 3: Run Docker container with additional commands
     - name: Run tests inside Docker container
       run: |
         docker-compose run --rm conserver bash -c "

.gitignore

@@ -6,7 +6,6 @@ venv
 
 .vscode
 docker-compose.override.yml
-docker-compose.yml
 docker-compose.dev.yml
 
 static