LiteLLM-X-Server-Config

A self-hosted LLM proxy stack built around LiteLLM, CLIProxyAPI, PostgreSQL, Netdata, Traefik, and optional Cloudflare Tunnel. It provides centralized API key management, model routing, access-group control, Claude Code request validation, and optional monitoring for a Docker Swarm deployment managed through Portainer.

Architecture

This repository is deployed as multiple Docker Swarm stacks:

Infrastructure stack (`portainer/portainer.yaml`)

traefik: Shared ingress proxy for Portainer, LiteLLM, CLIProxyAPI, and Netdata
cloudflared: Optional Cloudflare Tunnel connector that forwards public hostnames to Traefik
portainer: Docker Swarm management UI
agent: Portainer agent for Swarm node access

Application data stack (`llmproxy-data.yaml`)

db: PostgreSQL database for LiteLLM state, usage logs, and model configuration

Application stack (`llmproxy.yaml`)

cli-proxy-api: Anthropic-compatible proxy and auth service
litellm: Core routing layer and LiteLLM admin UI

Monitoring stack (`monitoring/netdata.yaml`)

netdata: Host and container monitoring dashboard
config-generator: Sidecar that watches Docker labels and generates Netdata collector configs

Networks

internal: private overlay network between application services and PostgreSQL
public: shared overlay network for Traefik, optional cloudflared, and routed services
monitoring: shared overlay network used by Netdata auto-discovery

Routing modes

The stacks support two ingress modes:

Default: Cloudflare Tunnel forwards requests to Traefik on http://traefik:80, so Let's Encrypt is not required.
Optional: Traefik can terminate HTTPS itself with Let's Encrypt by deploying portainer/portainer.letsencrypt.yaml and switching router env vars from web/false to websecure/true.

Shared routing variables

Use the same routing variables in portainer/.env.example, monitoring/.env.example, and .env.example:

Variable	Default	When to change it
`TRAEFIK_ROUTER_ENTRYPOINTS`	`web`	Set to `websecure` when Traefik terminates HTTPS
`TRAEFIK_ROUTER_TLS`	`false`	Set to `true` when Traefik terminates HTTPS
`LETSENCRYPT_RESOLVER`	`le`	Change only if you use a different Traefik resolver name

Prerequisites

# Install ptctools
uv tool install ptctools --from git+https://github.com/tamntlib/ptctools.git

Installation

1. Install Portainer CE, Traefik, and optional Cloudflare Tunnel

Choose how hostnames reach Traefik

Default Cloudflare Tunnel flow:

Create a remotely managed tunnel in Cloudflare Zero Trust.
Add public hostnames such as portainer.example.com, netdata.example.com, llm.example.com, and cli-proxy-api.llm.example.com.
Point each public hostname at http://traefik:80.

Optional direct HTTPS with Let's Encrypt:

Create DNS A/AAAA records for the same hostnames and point them to your server IP.

Create the Portainer config directory on the server

ssh root@<ip> 'mkdir -p /opt/portainer'

Copy the Portainer stack files to the server

scp portainer/portainer.yaml portainer/portainer.letsencrypt.yaml root@<ip>:/opt/portainer/
scp portainer/.env.example root@<ip>:/opt/portainer/.env

SSH to server

Install Docker

https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository

Configure the Portainer stack

docker swarm init

Edit /opt/portainer/.env and set at least:

PORTAINER_HOST
CLOUDFLARE_TUNNEL_REPLICAS=1 and CLOUDFLARE_TUNNEL_TOKEN=<token> if you want the built-in cloudflared service enabled
the shared routing variables from Routing modes if you want values other than the defaults

Deploy Portainer with the default non-Let's Encrypt setup

docker stack deploy -c /opt/portainer/portainer.yaml portainer

Optional: enable Let's Encrypt on Traefik

If you want Traefik to terminate HTTPS itself:

Set TRAEFIK_ROUTER_ENTRYPOINTS=websecure and TRAEFIK_ROUTER_TLS=true in /opt/portainer/.env.
Set LETSENCRYPT_EMAIL=<email> in /opt/portainer/.env.
Set the same TRAEFIK_ROUTER_ENTRYPOINTS=websecure and TRAEFIK_ROUTER_TLS=true values in monitoring/.env and .env before deploying those stacks.
Deploy with the override file:

docker stack deploy -c /opt/portainer/portainer.yaml -c /opt/portainer/portainer.letsencrypt.yaml portainer

2. Deploy the monitoring stack

Deploy this first so the shared monitoring overlay network exists before the application stacks join it.

Expose the Netdata hostname

Cloudflare Tunnel: add netdata.example.com as a public hostname to http://traefik:80
Direct Let's Encrypt: add netdata.example.com as a DNS record to your server IP

Set environment variables

Copy monitoring/.env.example to monitoring/.env and fill in the values:

cp monitoring/.env.example monitoring/.env

Required environment variables:

NETDATA_HOST: Hostname for the Netdata dashboard
NETDATA_BASIC_AUTH: Basic auth credentials for Traefik

If you use Traefik-managed HTTPS, also set the shared routing variables from Routing modes.

Create configs and deploy

ptctools docker config set -n monitoring_netdata-conf -f 'monitoring/configs/netdata.conf'
ptctools docker config set -n monitoring_config-generator-script -f 'monitoring/scripts/netdata-config-generator.sh'
ptctools docker stack deploy -n monitoring -f 'monitoring/netdata.yaml' --ownership team

3. Deploy the application stacks from your local machine

Expose the application hostnames

Cloudflare Tunnel: add llm.example.com and cli-proxy-api.llm.example.com as public hostnames to http://traefik:80
Direct Let's Encrypt: add the same hostnames as DNS records to your server IP

Set environment variables

Copy .env.example to .env and fill in the values:

cp .env.example .env

Required environment variables:

DB_USER, DB_PASSWORD, DB_NAME: PostgreSQL credentials
LITELLM_HOST, LITELLM_MASTER_KEY, LITELLM_SALT_KEY: LiteLLM configuration
CLI_PROXY_API_HOST: CLIProxyAPI hostname

Optional environment variables used by the stack:

the shared routing variables from Routing modes when using Traefik-managed HTTPS
CLAUDE_CODE_MODELS: Comma-separated model names that should enforce Claude Code checks
CLAUDE_CODE_MIN_VERSION: Minimum allowed Claude Code version for those models
SLACK_WEBHOOK_URL: LiteLLM Slack webhook

Upload configs and deploy

export PORTAINER_URL=https://portainer.example.com
export PORTAINER_ACCESS_TOKEN=<token>

ptctools docker config set -n llmproxy_litellm-config-yaml -f 'configs/litellm.yaml' --ownership team
ptctools docker config set -n llmproxy_litellm-claude-code-hook-py -f 'configs/claude_code_hook.py' --ownership team
ptctools docker config set -n llmproxy_cli-proxy-api-config-yaml -f 'configs/cli-proxy-api.yaml' --ownership team

ptctools docker stack deploy -n llmproxy-data -f 'llmproxy-data.yaml' --ownership team
ptctools docker stack deploy -n llmproxy -f 'llmproxy.yaml' --ownership team

LiteLLM management

cd litellm_scripts

# Generate a resolved config from config.json + config.local.json
python3 gen_config.py

# Full sync of credentials, models, aliases, fallbacks, and public model hub
python3 config.py --only credentials,models,aliases,fallbacks,public_model_hub --force --prune

# Sync specific components
python3 config.py --only models --force
python3 config.py --only aliases,fallbacks,public_model_hub
python3 config.py --only public_model_hub

# Create a LiteLLM user and API key
python3 create_api_key.py user@example.com
python3 create_api_key.py user@example.com --alias my-key

Required environment variables in litellm_scripts/.env:

LITELLM_API_KEY
LITELLM_BASE_URL

Configuration files

File	Description
`portainer/portainer.yaml`	Infrastructure stack with Traefik, Portainer, and optional Cloudflare Tunnel
`portainer/portainer.letsencrypt.yaml`	Optional Traefik override that enables Let's Encrypt ACME and HTTP->HTTPS redirect
`portainer/.env`	Environment variables for the Portainer/Traefik stack
`llmproxy-data.yaml`	PostgreSQL Docker Swarm stack
`llmproxy.yaml`	Application Docker Swarm stack for LiteLLM and CLIProxyAPI
`monitoring/netdata.yaml`	Monitoring stack with Netdata and the label-watching config generator
`configs/litellm.yaml`	LiteLLM runtime config (callbacks, DB batching, connection pool settings)
`configs/cli-proxy-api.yaml`	CLIProxyAPI runtime config
`configs/claude_code_hook.py`	LiteLLM callback that enforces Claude Code User-Agent and minimum version rules
`litellm_scripts/config.json`	Base provider/model/alias/fallback/public-model-hub config
`litellm_scripts/config.local.json`	Local overrides including API keys (gitignored, deep-merged with `config.json`)
`litellm_scripts/config.gen.json`	Generated resolved config output from `gen_config.py` with LiteLLM-ready credential and model request bodies
`.env`	Environment variables for the application stacks
`monitoring/.env`	Environment variables for the monitoring stack

Local configuration (`config.local.json`)

Create litellm_scripts/config.local.json to add API keys and local overrides:

{
  "providers": {
    "my-provider": {
      "api_key": "sk-your-api-key-here"
    },
    "another-provider": {
      "api_key": "sk-another-key"
    }
  }
}

This file is deep-merged with config.json, so you only need to specify overrides. Provider configs can also use $extend in config.json and override or disable inheritance in config.local.json.

Interface-level `api_base`

Each interface may override the provider-level api_base. This is useful when a single provider exposes different OpenAI-compatible and Anthropic-compatible endpoints.

{
  "providers": {
    "my-provider": {
      "api_base": "https://shared-gateway.example.com",
      "interfaces": {
        "anthropic": {
          "api_base": "https://custom-anthropic.example.com"
        },
        "openai": {
          "api_base": "https://custom-openai.example.com/v1"
        }
      }
    }
  }
}

Rules:

interface-level api_base overrides the provider-level api_base for credential generation and interface-specific model discovery
interface-level models_api_base may be set separately when the /models endpoint lives on a different base URL
if interface models_api_base is omitted, model discovery falls back to interface api_base, then provider-level models_api_base, then provider-level api_base

`public_model_hub` and `is_public_model_hub`

Use public_model_hub to add explicit model groups or aliases to LiteLLM's public model hub:

{
  "public_model_hub": [
    "claude-opus-4-7"
  ]
}

Use is_public_model_hub to derive public model hub entries from config defaults:

{
  "providers": {
    "my-provider": {
      "is_public_model_hub": true,
      "interfaces": {
        "openai": {
          "models": {
            "model-a": null,
            "model-b": {
              "is_public_model_hub": false
            }
          }
        }
      }
    }
  }
}

Rules:

provider-level is_public_model_hub is the default for all models under that provider
model-level is_public_model_hub overrides the provider default
if is_public_model_hub is omitted, it is treated as false
public_model_hub entries are combined from three sources by default: derived model entries, alias names, and the explicit public_model_hub array
set public_model_hub_autofill_disabled: true to disable derived model entry autofill
set public_model_hub_aliases_autofill_disabled: true to disable alias-name autofill
in config.local.json, the public_model_hub array replaces the base list instead of merging element-by-element

`model_name_prefix`

Each interface may define model_name_prefix to control derived model group names. When omitted, it defaults to the interface name.

Default examples:

interfaces.anthropic.models.claude-sonnet-4-6 resolves to anthropic/claude-sonnet-4-6
interfaces.openai.models.gpt-5.4 resolves to openai/gpt-5.4
interfaces.gemini.models.gemini-2.5-pro resolves to gemini/gemini-2.5-pro

{
  "providers": {
    "my-provider": {
      "interfaces": {
        "anthropic": {
          "model_name_prefix": "anthropic",
          "models": {
            "claude-sonnet-4-6": null
          }
        }
      }
    }
  }
}

With no explicit model_name, the generated model group name becomes <model_name_prefix>/<model-id>. In the example above, claude-sonnet-4-6 resolves to anthropic/claude-sonnet-4-6.

If model_name is set on a model, it still wins and fully overrides the derived prefix-based name.

These resolved prefixed names are the ones used by generated models and should be the names you reference in:

aliases targets
fallbacks
public_model_hub
model_name_base_model_map entries when you want to key by resolved model name instead of raw provider model ID

Model-level `access_groups`

Individual models can override the provider-level access_groups by specifying access_groups in their model config:

{
  "providers": {
    "my-provider": {
      "access_groups": ["General"],
      "models": {
        "model-a": null,
        "model-b": {
          "access_groups": ["Premium"]
        }
      }
    }
  }
}

model-a inherits the provider-level access_groups: ["General"]
model-b uses its own access_groups: ["Premium"]

Backup and restore

# Volume backup/restore (uses Duplicati)
ptctools docker volume backup -v vol1,vol2 -o s3://mybucket
ptctools docker volume restore -i s3://mybucket/vol1
ptctools docker volume restore -v vol1 -i s3://mybucket/vol1

# Database backup/restore (uses minio/mc for S3)
ptctools docker db backup -c container_id -v db_data \
  --db-user postgres --db-name mydb -o backup.sql.gz
ptctools docker db backup -c container_id -v db_data \
  --db-user postgres --db-name mydb -o s3://mybucket/backups/db.sql.gz

ptctools docker db restore -c container_id -v db_data \
  --db-user postgres --db-name mydb -i backup.sql.gz
ptctools docker db restore -c container_id -v db_data \
  --db-user postgres --db-name mydb -i s3://mybucket/backups/db.sql.gz

Monitoring

Netdata collects host, container, and PostgreSQL metrics.

Metrics retention

Netdata limits local metrics storage to 10 GiB in monitoring/configs/netdata.conf, which provides roughly 2-4 weeks of retention depending on metric volume.

Auto-discovery

Services can self-register for PostgreSQL monitoring by adding Docker labels:

deploy:
  labels:
    - netdata.postgres.name=my_database
    - netdata.postgres.dsn=postgresql://user:pass@host:5432/dbname

networks:
  - monitoring

The service must also join the shared monitoring network so the Netdata stack can reach it.

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

LiteLLM-X-Server-Config

Architecture

Infrastructure stack (portainer/portainer.yaml)

Application data stack (llmproxy-data.yaml)

Application stack (llmproxy.yaml)

Monitoring stack (monitoring/netdata.yaml)

Networks

Routing modes

Shared routing variables

Prerequisites

Installation

1. Install Portainer CE, Traefik, and optional Cloudflare Tunnel

Choose how hostnames reach Traefik

Create the Portainer config directory on the server

Copy the Portainer stack files to the server

SSH to server

Install Docker

Configure the Portainer stack

Deploy Portainer with the default non-Let's Encrypt setup

Optional: enable Let's Encrypt on Traefik

2. Deploy the monitoring stack

Expose the Netdata hostname

Set environment variables

Create configs and deploy

3. Deploy the application stacks from your local machine

Expose the application hostnames

Set environment variables

Upload configs and deploy

LiteLLM management

Configuration files

Local configuration (config.local.json)

Interface-level api_base

public_model_hub and is_public_model_hub

model_name_prefix

Model-level access_groups

Backup and restore

Monitoring

Metrics retention

Auto-discovery

Infrastructure stack (`portainer/portainer.yaml`)

Application data stack (`llmproxy-data.yaml`)

Application stack (`llmproxy.yaml`)

Monitoring stack (`monitoring/netdata.yaml`)

Local configuration (`config.local.json`)

Interface-level `api_base`

`public_model_hub` and `is_public_model_hub`

`model_name_prefix`

Model-level `access_groups`