Skip to content

Latest commit

 

History

History
467 lines (325 loc) · 16.7 KB

File metadata and controls

467 lines (325 loc) · 16.7 KB

Docker Compose Deployment

GoClaw ships a composable docker-compose setup: a base file, a compose.d/ directory of always-active overlays, and a compose.options/ directory of opt-in overlays you mix and match.

Auto-upgrade on start: The Docker entrypoint runs goclaw upgrade automatically before starting the gateway. This applies pending database migrations so you don't need a separate upgrade step for simple deployments. For production, consider running the upgrade overlay explicitly first.

Overview

The compose setup is modular. The base docker-compose.yml defines the core goclaw service. Active overlays live in compose.d/ and are assembled automatically. Optional overlays in compose.options/ can be copied into compose.d/ to activate them.

compose.d/ — always-active overlays

Files in compose.d/ are loaded automatically by prepare-compose.sh (sorted by filename):

compose.d/
  00-goclaw.yml        # Core service definition
  11-postgres.yml      # PostgreSQL 18 + pgvector
  12-selfservice.yml   # Web dashboard UI (nginx + React, port 3000)
  13-upgrade.yml       # One-shot DB migration runner
  14-browser.yml       # Headless Chrome sidecar (CDP, port 9222)
  15-otel.yml          # Jaeger for OpenTelemetry trace visualization
  16-redis.yml         # Redis 7 cache backend
  17-sandbox.yml       # Docker-in-Docker sandbox for agent code execution
  18-tailscale.yml     # Tailscale tsnet for secure remote access

compose.options/ — opt-in overlays

The compose.options/ directory holds the same overlay files as reference copies. Copy the ones you want into compose.d/ to activate them.

prepare-compose.sh — build the COMPOSE_FILE

Run this script once after changing compose.d/ to regenerate the COMPOSE_FILE variable in .env:

./prepare-compose.sh

The script reads all compose.d/*.yml files (sorted), validates the merged config with docker compose config, and writes the COMPOSE_FILE value to .env. Docker Compose reads COMPOSE_FILE automatically on every docker compose command.

# Flags
./prepare-compose.sh --quiet             # suppress output
./prepare-compose.sh --skip-validation   # skip docker compose config check

podman-compose: COMPOSE_FILE is not read automatically. Run source .env before each podman-compose command.


Recipes

First-time setup

Run the environment preparation script to auto-generate required secrets:

./prepare-env.sh

This creates .env from .env.example and generates GOCLAW_ENCRYPTION_KEY and GOCLAW_GATEWAY_TOKEN if not already set.

Optionally add an LLM provider API key to .env now, or add it later via the web dashboard:

GOCLAW_OPENROUTER_API_KEY=sk-or-xxxxx
# or GOCLAW_ANTHROPIC_API_KEY=sk-ant-xxxxx
# or any other GOCLAW_*_API_KEY

Docker vs bare metal: In Docker, configure providers via .env or through the web dashboard after first start. The goclaw onboard wizard is for bare metal only — it requires an interactive terminal and does not run inside containers.

Required vs optional .env variables (Docker)

Variable Required Notes
GOCLAW_GATEWAY_TOKEN Yes Auto-generated by prepare-env.sh
GOCLAW_ENCRYPTION_KEY Yes Auto-generated by prepare-env.sh
GOCLAW_*_API_KEY No LLM provider key — set in .env or add via dashboard. Required before chatting
GOCLAW_AUTO_UPGRADE Recommended Set to true to auto-run DB migrations on startup
POSTGRES_USER No Default: goclaw
POSTGRES_PASSWORD No Default: goclawchange for production

Important: All GOCLAW_* env vars must be set inside the .env file, not as shell prefixes (e.g. GOCLAW_AUTO_UPGRADE=true docker compose … will not work because compose reads from env_file).

Starting the stack

After running prepare-compose.sh, start the stack normally — COMPOSE_FILE in .env tells Docker Compose which files to load:

./prepare-compose.sh
docker compose up -d --build

To add or remove an optional component, copy the relevant file from compose.options/ into compose.d/ (or remove it), then re-run prepare-compose.sh.

Minimal — core + PostgreSQL only

Keep only the essential files in compose.d/:

compose.d/00-goclaw.yml
compose.d/11-postgres.yml
compose.d/13-upgrade.yml

Then:

./prepare-compose.sh && docker compose up -d --build

Standard — + dashboard + sandbox

compose.d/00-goclaw.yml
compose.d/11-postgres.yml
compose.d/12-selfservice.yml
compose.d/13-upgrade.yml
compose.d/17-sandbox.yml
# Build the sandbox image first (one-time)
docker build -t goclaw-sandbox:bookworm-slim -f Dockerfile.sandbox .

./prepare-compose.sh && docker compose up -d --build

Dashboard: http://localhost:3000

Full — everything including OTel tracing

Add compose.options/15-otel.yml to compose.d/, then:

./prepare-compose.sh && docker compose up -d --build

Jaeger UI: http://localhost:16686


Overlay Reference

docker-compose.postgres.yml

Starts pgvector/pgvector:pg18 and wires GOCLAW_POSTGRES_DSN automatically. GoClaw waits for the health check before starting.

Environment variables (set in .env or shell):

Variable Default Description
POSTGRES_USER goclaw Database user
POSTGRES_PASSWORD goclaw Database password — change for production
POSTGRES_DB goclaw Database name
POSTGRES_PORT 5432 Host port to expose

docker-compose.selfservice.yml

Builds the React SPA from ui/web/ and serves it via nginx on port 3000.

Variable Default Description
GOCLAW_UI_PORT 3000 Host port for the dashboard

docker-compose.sandbox.yml

Mounts /var/run/docker.sock so GoClaw can spin up isolated containers for agent shell execution. Requires the sandbox image to be built first.

Security note: Mounting the Docker socket gives the container control over host Docker. Only use in trusted environments.

Variable Default Description
GOCLAW_SANDBOX_MODE all off, non-main, or all
GOCLAW_SANDBOX_IMAGE goclaw-sandbox:bookworm-slim Image to use for sandbox containers
GOCLAW_SANDBOX_WORKSPACE_ACCESS rw none, ro, or rw
GOCLAW_SANDBOX_SCOPE session session, agent, or shared
GOCLAW_SANDBOX_MEMORY_MB 512 Memory limit per sandbox container
GOCLAW_SANDBOX_CPUS 1.0 CPU limit per sandbox container
GOCLAW_SANDBOX_TIMEOUT_SEC 300 Max execution time in seconds
GOCLAW_SANDBOX_NETWORK false Enable network access in sandbox
DOCKER_GID 999 GID of the docker group on the host

docker-compose.browser.yml

Starts zenika/alpine-chrome:124 with CDP enabled on port 9222. GoClaw connects via GOCLAW_BROWSER_REMOTE_URL=ws://chrome:9222.

docker-compose.otel.yml

Starts Jaeger (jaegertracing/all-in-one:1.68.0) and rebuilds GoClaw with the ENABLE_OTEL=true build arg to include the OTel exporter.

Variable Default Description
GOCLAW_TELEMETRY_ENABLED true Enable OTel export
GOCLAW_TELEMETRY_ENDPOINT jaeger:4317 OTLP gRPC endpoint
GOCLAW_TELEMETRY_PROTOCOL grpc grpc or http
GOCLAW_TELEMETRY_SERVICE_NAME goclaw-gateway Service name in traces

docker-compose.tailscale.yml

Rebuilds with ENABLE_TSNET=true to embed Tailscale directly in the binary (no sidecar needed).

Variable Required Description
GOCLAW_TSNET_AUTH_KEY Yes Tailscale auth key from the admin console
GOCLAW_TSNET_HOSTNAME No (default: goclaw-gateway) Device name on the tailnet

docker-compose.redis.yml

Rebuilds GoClaw with ENABLE_REDIS=true and starts a Redis 7 Alpine instance with AOF persistence enabled.

Variable Default Description
GOCLAW_REDIS_DSN redis://redis:6379/0 Redis connection string (auto-set)

Build arg: ENABLE_REDIS=true — compiles in the Redis cache backend.

Volume: redis-data/data (AOF persistence).

docker-compose.upgrade.yml

A one-shot service that runs goclaw upgrade and exits. Use it to apply database migrations without downtime.

# Preview what will change (dry-run)
docker compose \
  -f docker-compose.yml \
  -f docker-compose.postgres.yml \
  -f docker-compose.upgrade.yml \
  run --rm upgrade --dry-run

# Apply upgrade
docker compose \
  -f docker-compose.yml \
  -f docker-compose.postgres.yml \
  -f docker-compose.upgrade.yml \
  run --rm upgrade

# Check migration status
docker compose \
  -f docker-compose.yml \
  -f docker-compose.postgres.yml \
  -f docker-compose.upgrade.yml \
  run --rm upgrade --status

Build Arguments

These are compile-time flags passed during docker build. Each enables optional dependencies.

Build Arg Default Effect
ENABLE_OTEL false OpenTelemetry span exporter
ENABLE_TSNET false Tailscale networking
ENABLE_REDIS false Redis cache backend
ENABLE_SANDBOX false Docker CLI in container (for sandbox)
ENABLE_PYTHON false Python 3 runtime for skills
ENABLE_NODE false Node.js runtime for skills
ENABLE_FULL_SKILLS false Pre-install skill dependencies (pandas, pypdf, etc.)
ENABLE_CLAUDE_CLI false Install @anthropic-ai/claude-code npm package
VERSION dev Semantic version string

Privilege Separation (v3)

Starting in v3, the Docker image uses privilege separation via su-exec:

docker-entrypoint.sh (runs as root)
  ├── Installs persisted apk packages (reads /app/data/.runtime/apk-packages)
  ├── Starts pkg-helper as root (Unix socket /tmp/pkg.sock, permissions 0660 root:goclaw)
  └── su-exec goclaw → starts /app/goclaw serve (drops to non-root)

pkg-helper

pkg-helper is a small root-privileged binary that handles system package management on behalf of the goclaw process. It listens on a Unix socket and accepts requests to install/uninstall Alpine packages (apk). The goclaw user cannot call apk directly but can request it through this helper.

Required Docker capabilities when using pkg-helper (added by default in the compose setup):

cap_add:
  - SETUID
  - SETGID
  - CHOWN
  - DAC_OVERRIDE

If you override cap_drop: ALL in a security-hardened compose setup, you must explicitly add these four capabilities back, or pkg-helper will fail and package installs via the admin UI will not work.

Runtime Package Directories

On-demand packages (pip/npm) installed via the admin UI go to the data volume:

Path Owner Contents
/app/data/.runtime/pip goclaw pip-installed Python packages
/app/data/.runtime/npm-global goclaw npm global packages
/app/data/.runtime/pip-cache goclaw pip download cache
/app/data/.runtime/apk-packages root:goclaw persisted apk package list (0640)

These persist across container recreation because they live on the goclaw-data volume.


Volumes

Volume Mount path Contents
goclaw-data /app/data config.json and runtime data
goclaw-workspace /app/workspace or /app/.goclaw Agent workspaces
goclaw-skills /app/skills Skill files
postgres-data /var/lib/postgresql PostgreSQL data
tsnet-state /app/tsnet-state Tailscale node state
redis-data /data Redis AOF persistence

Base Container Hardening

The base docker-compose.yml applies these security settings to the goclaw service:

security_opt:
  - no-new-privileges:true
cap_drop:
  - ALL
read_only: true
tmpfs:
  - /tmp:rw,noexec,nosuid,size=256m
deploy:
  resources:
    limits:
      memory: 1G
      cpus: '2.0'
      pids: 200

The sandbox overlay (docker-compose.sandbox.yml) overrides cap_drop and security_opt because Docker socket access requires relaxed capabilities.


Update / Upgrade Procedure

# 1. Pull latest images / rebuilt code
docker compose pull

# 2. Run DB migrations before starting new binary
docker compose run --rm upgrade

# 3. Restart the stack
docker compose up -d --build

COMPOSE_FILE in .env (set by prepare-compose.sh) includes 13-upgrade.yml automatically, so no explicit -f flags are needed.


Installation Alternatives

Binary installer (no Docker)

Download the latest binary directly:

curl -fsSL https://raw.githubusercontent.com/nextlevelbuilder/goclaw/main/scripts/install.sh | bash

# Specific version
curl -fsSL https://raw.githubusercontent.com/nextlevelbuilder/goclaw/main/scripts/install.sh | bash -s -- --version v1.19.1

# Custom directory
curl -fsSL https://raw.githubusercontent.com/nextlevelbuilder/goclaw/main/scripts/install.sh | bash -s -- --dir /opt/goclaw

Supports Linux and macOS (amd64 and arm64).

Interactive Docker setup

The setup script generates .env and builds the right compose command:

./scripts/setup-docker.sh              # Interactive mode
./scripts/setup-docker.sh --variant full --with-ui   # Non-interactive

Variants: alpine (base), node, python, full. Add --with-ui for the dashboard, --dev for development mode with live reload.


Pre-built Docker Images

Official multi-arch images (amd64 + arm64) are published on every release to both registries:

Registry Gateway Web Dashboard
Docker Hub digitop/goclaw digitop/goclaw-web
GHCR ghcr.io/nextlevelbuilder/goclaw ghcr.io/nextlevelbuilder/goclaw-web

Tag variants

Images are split into runtime variants (what's pre-installed) and build-tag variants (compiled-in features):

Runtime variants:

Tag Node.js Python Skill deps Use case
latest / vX.Y.Z Minimal base (~50 MB)
node / vX.Y.Z-node JS/TS skills
python / vX.Y.Z-python Python skills
full / vX.Y.Z-full All skill dependencies pre-installed

Build-tag variants:

Tag OTel Tailscale Redis Use case
otel / vX.Y.Z-otel OpenTelemetry tracing
tsnet / vX.Y.Z-tsnet Tailscale remote access
redis / vX.Y.Z-redis Redis caching

Tip: Runtime and build-tag variants are independent. If you need Python + OTel, build locally with ENABLE_PYTHON=true and ENABLE_OTEL=true.

Pull example:

# Latest minimal
docker pull digitop/goclaw:latest

# With Python runtime
docker pull digitop/goclaw:python

# Full runtime (Node + Python + all deps)
docker pull digitop/goclaw:full

# With OTel tracing
docker pull ghcr.io/nextlevelbuilder/goclaw:otel

Common Issues

Problem Cause Fix
goclaw exits immediately on start PostgreSQL not ready The postgres overlay adds a health check dependency; ensure you include it
Sandbox containers not starting Docker socket not mounted or wrong GID Add the sandbox overlay and set DOCKER_GID to match stat -c %g /var/run/docker.sock
Dashboard returns 502 goclaw service not healthy yet Check docker compose logs goclaw; dashboard depends on goclaw being up
OTel traces not appearing in Jaeger Binary built without ENABLE_OTEL=true Add --build flag when using the otel overlay; it rebuilds with the build arg
Port 5432 already in use Local Postgres running Set POSTGRES_PORT=5433 in .env
database schema is outdated Migrations not applied after update Add GOCLAW_AUTO_UPGRADE=true to .env file (not as shell prefix — compose reads from env_file), or run the upgrade overlay before starting
network goclaw-net … incorrect label A goclaw-net Docker network already exists with conflicting labels Run docker network rm goclaw-net then retry — Compose creates its own goclaw-net network automatically

What's Next