Skip to content

chore(hosting): wire the agent runner sidecar into compose #4776

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mmabrouk wants to merge 2 commits into
base: main
Choose a base branch
from
+64 −22
Open

chore(hosting): wire the agent runner sidecar into compose #4776

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
143edd6
chore(hosting): wire the agent runner sidecar into the dev compose stack
mmabrouk Jun 19, 2026
3017977
chore(hosting): clarify agent sidecar dependency
mmabrouk Jun 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 64 additions & 22 deletions hosting/docker-compose/ee/docker-compose.dev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,6 @@ services:
- ../../../web/oss/src:/app/oss/src
- ../../../web/oss/public:/app/oss/public
- ../../../web/packages:/app/packages
- nextjs-ee-cache:/app/ee/.next/cache
- nextjs-oss-cache:/app/oss/.next/cache
- turbo-ee-cache:/app/.turbo
# === CONFIGURATION ======================================== #
env_file:
- ${ENV_FILE:-./.env.ee.dev}
Expand Down Expand Up @@ -82,12 +79,8 @@ services:
"--reload-dir",
"/app/ee/src",
"--reload-dir",
"/app/ee/databases",
"--reload-dir",
"/app/oss/src",
"--reload-dir",
"/app/oss/databases",
"--reload-dir",
"/app/entrypoints",
"--reload-dir",
"/sdks/python/agenta",
Expand Down Expand Up @@ -150,9 +143,7 @@ services:
image: agenta-ee-dev-api:latest
# === EXECUTION ============================================ #
command: >
watchmedo auto-restart --directory=/app/ee/src --directory=/app/ee/databases --directory=/app/oss/src
--directory=/app/oss/databases --directory=/app/entrypoints --directory=/sdks/python/agenta
--directory=/clients/python/agenta_client --pattern=*.py --recursive --ignore-patterns=*/tests/* --
watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive --ignore-patterns=*/tests/* --

@coderabbitai coderabbitai Bot Jun 19, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Inspect worker watch directories and mounted source directories.
# Expected: worker watchmedo commands include the mounted SDK/client paths or those mounts are intentionally not hot-reloaded.

sed -n '141,264p' hosting/docker-compose/ee/docker-compose.dev.yml

Repository: Agenta-AI/agenta

Length of output: 5516

🏁 Script executed:

# Check structure of SDK and client directories
find ../../../sdks/python -maxdepth 2 -type d 2>/dev/null | head -20
echo "---"
find ../../../clients/python -maxdepth 2 -type d 2>/dev/null | head -20

Repository: Agenta-AI/agenta

Length of output: 64

🏁 Script executed:

# Search for API service hot-reload configuration to understand expected pattern
rg -A 10 "watchmedo.*directory=" hosting/docker-compose/ee/ --no-heading

Repository: Agenta-AI/agenta

Length of output: 4521

🏁 Script executed:

# Check if there are any existing references to watching SDK/client paths
rg "sdks/python|clients/python" hosting/docker-compose/ -B 2 -A 2

Repository: Agenta-AI/agenta

Length of output: 13615

Add SDK and client paths to worker watchmedo scopes.

The four worker services watch only /app/, but they mount SDK and client code at /sdks/python and /clients/python. This means edits to those dependencies will not trigger restarts, unlike the API service in the same file which includes them. Align worker watchers with the OSS version and API service by adding both directories:

♻️ Proposed watcher scope fix 
-            watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive --ignore-patterns=*/tests/* --
+            watchmedo auto-restart --directory=/app/ --directory=/sdks/python/agenta --directory=/clients/python/agenta_client --pattern=*.py --recursive --ignore-patterns=*/tests/* --

Apply to lines 146, 183, 220, and 263 (worker-evaluations, worker-tracing, worker-webhooks, worker-events).

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive --ignore-patterns=*/tests/* --
watchmedo auto-restart --directory=/app/ --directory=/sdks/python/agenta --directory=/clients/python/agenta_client --pattern=*.py --recursive --ignore-patterns=*/tests/* --

python -m entrypoints.worker_evaluations
# === STORAGE ============================================== #
volumes:
Expand Down Expand Up @@ -189,9 +180,7 @@ services:
image: agenta-ee-dev-api:latest
# === EXECUTION ============================================ #
command: >
watchmedo auto-restart --directory=/app/ee/src --directory=/app/ee/databases --directory=/app/oss/src
--directory=/app/oss/databases --directory=/app/entrypoints --directory=/sdks/python/agenta
--directory=/clients/python/agenta_client --pattern=*.py --recursive --ignore-patterns=*/tests/* --
watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive --ignore-patterns=*/tests/* --
python -m entrypoints.worker_tracing
# === STORAGE ============================================== #
volumes:
Expand Down Expand Up @@ -228,9 +217,7 @@ services:
image: agenta-ee-dev-api:latest
# === EXECUTION ============================================ #
command: >
watchmedo auto-restart --directory=/app/ee/src --directory=/app/ee/databases --directory=/app/oss/src
--directory=/app/oss/databases --directory=/app/entrypoints --directory=/sdks/python/agenta
--directory=/clients/python/agenta_client --pattern=*.py --recursive --ignore-patterns=*/tests/* --
watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive --ignore-patterns=*/tests/* --
python -m entrypoints.worker_webhooks
# === STORAGE ============================================== #
volumes:
Expand Down Expand Up @@ -273,9 +260,7 @@ services:
image: agenta-ee-dev-api:latest
# === EXECUTION ============================================ #
command: >
watchmedo auto-restart --directory=/app/ee/src --directory=/app/ee/databases --directory=/app/oss/src
--directory=/app/oss/databases --directory=/app/entrypoints --directory=/sdks/python/agenta
--directory=/clients/python/agenta_client --pattern=*.py --recursive --ignore-patterns=*/tests/* --
watchmedo auto-restart --directory=/app/ --pattern=*.py --recursive --ignore-patterns=*/tests/* --
python -m entrypoints.worker_events
# === STORAGE ============================================== #
volumes:
Expand Down Expand Up @@ -409,6 +394,15 @@ services:
- ${ENV_FILE:-./.env.ee.dev}
environment:
DOCKER_NETWORK_MODE: ${DOCKER_NETWORK_MODE:-bridge}
# Agent workflow: reach the agent runner sidecar in-network.
AGENTA_AGENT_PI_URL: http://agent-pi:8765

@mmabrouk mmabrouk Jun 19, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the single line that routes runs to the sidecar. The runner reads AGENTA_AGENT_PI_URL and only goes over HTTP when it is set; unset means a local subprocess (services/oss/src/agent/app.py:63). The agent-pi host here must match the service name at line 421, or every run hits an unresolved host.

# Agent runtime: drive the harness over ACP via a rivet daemon.
# Harness (pi/claude) and sandbox (local/daytona) are independent axes.
# The agent-pi sidecar files are introduced by PR #4778, so this compose
# wiring must land after #4778 or be reviewed with that branch applied.
AGENTA_AGENT_RUNTIME: ${AGENTA_AGENT_RUNTIME:-rivet}
AGENTA_AGENT_HARNESS: ${AGENTA_AGENT_HARNESS:-pi}
AGENTA_AGENT_SANDBOX: ${AGENTA_AGENT_SANDBOX:-local}
Comment on lines +397 to +405

@coderabbitai coderabbitai Bot Jun 19, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Locate sidecar URL usage to decide whether service_started is enough or a healthcheck is required.
# Expected: startup-time probes should be guarded by depends_on + healthcheck; request-time calls should handle connection failures gracefully.

rg -n -C3 'AGENTA_AGENT_PI_URL|agent-pi|8765' services api hosting/docker-compose

Repository: Agenta-AI/agenta

Length of output: 12396

🏁 Script executed:

# Get the full services container definition to check if depends_on already exists
head -n 470 hosting/docker-compose/ee/docker-compose.dev.yml | tail -n 100

Repository: Agenta-AI/agenta

Length of output: 5329

🏁 Script executed:

# Search for where AGENTA_AGENT_PI_URL is actually used in Python code
rg -t py 'AGENTA_AGENT_PI_URL' --max-count 20

Repository: Agenta-AI/agenta

Length of output: 42

🏁 Script executed:

# Check the agent-pi service definition (lines 421-468)
sed -n '421,468p' hosting/docker-compose/ee/docker-compose.dev.yml

Repository: Agenta-AI/agenta

Length of output: 2995

🏁 Script executed:

# Search more broadly for AGENTA_AGENT_PI_URL usage across all file types
rg 'AGENTA_AGENT_PI_URL' --max-count 30

Repository: Agenta-AI/agenta

Length of output: 289

🏁 Script executed:

# Check if depends_on already exists in the services container definition
# by looking at the full services block
sed -n '370,420p' hosting/docker-compose/ee/docker-compose.dev.yml

Repository: Agenta-AI/agenta

Length of output: 2411

🏁 Script executed:

# Search for any healthcheck definition in the agent-pi service
grep -n 'healthcheck' hosting/docker-compose/ee/docker-compose.dev.yml

Repository: Agenta-AI/agenta

Length of output: 301

🏁 Script executed:

# Find where AGENTA_AGENT_PI_URL env var is consumed in the services application code
find services -type f -name '*.py' | xargs grep -l 'AGENTA_AGENT_PI_URL' 2>/dev/null

Repository: Agenta-AI/agenta

Length of output: 44

🏁 Script executed:

# Also search for 'agent_pi' or 'agent-pi' in services code
rg -t py 'agent.?pi|agent.?runner' services --max-count 20 -i

Repository: Agenta-AI/agenta

Length of output: 42

🏁 Script executed:

# Search more broadly for agent pi in all file types, not just Python
rg 'agent.?pi|AGENTA_AGENT_PI' services --max-count 30 -i

Repository: Agenta-AI/agenta

Length of output: 42

🏁 Script executed:

# Check if there's a TypeScript/JS implementation in services that uses the env var
rg 'AGENTA_AGENT_PI_URL|agent-pi:8765' services --type ts --type js --max-count 20

Repository: Agenta-AI/agenta

Length of output: 42

🏁 Script executed:

# Look at the services entrypoint to understand startup flow
find services -name '__main__.py' -o -name 'main.py' -o -name 'app.py' | head -5 | xargs -I {} sh -c 'echo "=== {} ===" && head -50 {}'

Repository: Agenta-AI/agenta

Length of output: 1733

🏁 Script executed:

# Search in the SDK code for agent-pi or AGENTA_AGENT_PI_URL usage
find services/sdk -type f \( -name '*.py' -o -name '*.ts' -o -name '*.js' \) 2>/dev/null | xargs grep -l 'AGENTA_AGENT_PI_URL\|agent.pi\|agent_pi' 2>/dev/null | head -10

Repository: Agenta-AI/agenta

Length of output: 42

🏁 Script executed:

# Look for any config or environment initialization that reads AGENTA_AGENT_PI_URL
rg 'getenv|environ\[' services/sdk -t py -B2 -A2 | grep -A5 -B5 'AGENT' | head -40

Repository: Agenta-AI/agenta

Length of output: 113

🏁 Script executed:

# Check if depends_on is present anywhere in the services block of the file
sed -n '340,420p' hosting/docker-compose/ee/docker-compose.dev.yml | grep -i 'depends_on'

Repository: Agenta-AI/agenta

Length of output: 80

🏁 Script executed:

# Check the full depends_on section for services to see what's listed
sed -n '340,425p' hosting/docker-compose/ee/docker-compose.dev.yml | grep -A 10 'depends_on'

Repository: Agenta-AI/agenta

Length of output: 405

🏁 Script executed:

# Look at the services structure more carefully - check lines before and after depends_on
sed -n '400,420p' hosting/docker-compose/ee/docker-compose.dev.yml

Repository: Agenta-AI/agenta

Length of output: 1230

🏁 Script executed:

# Look for where agent-pi URL is actually called - check for HTTP client initialization or requests
rg -t py 'http://' services --max-count 50 | grep -i 'agent\|8765' | head -20

Repository: Agenta-AI/agenta

Length of output: 42

🏁 Script executed:

# Check if agent-pi has a healthcheck
sed -n '421,510p' hosting/docker-compose/ee/docker-compose.dev.yml | grep -A 5 'healthcheck'

Repository: Agenta-AI/agenta

Length of output: 231

🏁 Script executed:

# Verify the exact structure: does services.depends_on only have postgres, or does it have agent-pi too?
sed -n '338,408p' hosting/docker-compose/ee/docker-compose.dev.yml

Repository: Agenta-AI/agenta

Length of output: 2888

🏁 Script executed:

# Verify the complete depends_on for the services container - look for agent-pi anywhere in it
sed -n '338,425p' hosting/docker-compose/ee/docker-compose.dev.yml | grep -A 20 'services:' | grep -A 15 'depends_on'

Repository: Agenta-AI/agenta

Length of output: 42

🏁 Script executed:

# Check if there's any other reference to agent-pi in the services container section
sed -n '338,425p' hosting/docker-compose/ee/docker-compose.dev.yml | grep -i 'agent.pi'

Repository: Agenta-AI/agenta

Length of output: 205

🏁 Script executed:

# Look at the full structure around lines 397-403 to confirm the exact context
sed -n '390,410p' hosting/docker-compose/ee/docker-compose.dev.yml

Repository: Agenta-AI/agenta

Length of output: 1139

🏁 Script executed:

# Check if the services container itself has a healthcheck or startup probe
sed -n '338,425p' hosting/docker-compose/ee/docker-compose.dev.yml | tail -40

Repository: Agenta-AI/agenta

Length of output: 2128

🏁 Script executed:

# Let's look at the complete file structure around the services definition to see if there's a depends_on after the environment section
sed -n '395,425p' hosting/docker-compose/ee/docker-compose.dev.yml

Repository: Agenta-AI/agenta

Length of output: 1757

🏁 Script executed:

# Search the entire docker-compose.dev.yml file for any reference to agent-pi in depends_on
grep -n 'agent-pi' hosting/docker-compose/ee/docker-compose.dev.yml

Repository: Agenta-AI/agenta

Length of output: 217

Add startup ordering for the new agent-pi sidecar dependency.

The services container references http://agent-pi:8765 via AGENTA_AGENT_PI_URL but has no depends_on entry for agent-pi. Docker Compose will not guarantee startup order, risking race conditions where the services container initializes before the sidecar is ready.

🔧 Startup-order fix 
         # === NETWORK ============================================== #
         networks:
             - agenta-network
         extra_hosts:
             - "host.docker.internal:host-gateway"
+        # === ORCHESTRATION ======================================== #
+        depends_on:
+            agent-pi:
+                condition: service_started
         # === LABELS =============================================== #

# === NETWORK ============================================== #
networks:
- agenta-network
Expand All @@ -426,6 +420,56 @@ services:
# === LIFECYCLE ============================================ #
restart: always

agent-pi:

@mmabrouk mmabrouk Jun 19, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The agent-pi service has no env_file by design. The sandbox must not inherit the stack secrets (Composio, Stripe, PostHog, Google). It gets only its port, the mounted Pi login, the OTLP fallback, and the Daytona keys for the daytona sandbox axis. Tools run server-side via /tools/call, so the sandbox never needs the broad secret set. Worth confirming this stays empty as the stack grows.

# === IMAGE ================================================ #
# Agent runner sidecar. The services container
# calls it in-network at http://agent-pi:8765 (AGENTA_AGENT_PI_URL).
# The build context and Dockerfile come from PR #4778.
build:
context: ../../../services/agent
dockerfile: docker/Dockerfile.dev
# === EXECUTION ============================================ #
# No file watcher (the box's inotify limit is shared across stacks). Copy the
# read-only mounted Pi login into a writable path so OAuth refresh stays
# in-container.
command: >
sh -c "mkdir -p /pi-agent && cp -a /pi-agent-ro/. /pi-agent/ 2>/dev/null || true;
exec node_modules/.bin/tsx src/server.ts"
# === CONFIGURATION ======================================== #
# Deliberately NO env_file: the Pi sandbox must not inherit the stack's
# secrets (COMPOSIO_API_KEY, STRIPE/POSTHOG/GOOGLE keys, ...). Tools run
# server-side via /tools/call, so the sandbox only needs its own port, the Pi
# login (mounted below), the OTLP export fallback, and the Daytona credentials
# the SDK reads for the rivet `daytona` sandbox axis.
environment:
PORT: "8765"
PI_CODING_AGENT_DIR: /pi-agent
# Tracing export fallback (used when a request carries no usable OTLP
# credential). Must be reachable from this container.
AGENTA_HOST: ${AGENTA_HOST:-http://144.76.237.122:8280}
AGENTA_API_KEY: ${AGENTA_API_KEY:-}
Comment on lines +447 to +450

@coderabbitai coderabbitai Bot Jun 19, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Do not default telemetry to a public HTTP IP.

Line 446 sends the sidecar’s fallback tracing export to http://144.76.237.122:8280 whenever AGENTA_HOST is unset. For a dev stack, that can leak local trace data/metadata to an external endpoint by default.

🛡️ Proposed safer default 
-            AGENTA_HOST: ${AGENTA_HOST:-http://144.76.237.122:8280}
+            # Set AGENTA_HOST explicitly when exporting traces outside the local stack.
+            AGENTA_HOST: ${AGENTA_HOST:-}

If tracing must work out of the box, default this to an in-stack service URL instead of a public IP.

# Daytona sandbox axis: the rivet daytona provider's `new Daytona()` reads
# these. Scoped to Daytona only (not the full stack secret set).
DAYTONA_API_KEY: ${DAYTONA_API_KEY:-}
DAYTONA_API_URL: ${DAYTONA_API_URL:-}
DAYTONA_TARGET: ${DAYTONA_TARGET:-}
# Pre-baked snapshot (rivet daemon + Pi + certs; Claude inherited from rivet's
# -full base) so Daytona runs skip the ~150s per-invoke `npm install pi`. We
# ship the builder (poc/build_rivet_snapshot.py), not the snapshot itself: each
# operator builds their own, so we never distribute a Claude-containing image.
# See services/agent/docker/README.md for the licensing posture.
AGENTA_RIVET_DAYTONA_SNAPSHOT: ${AGENTA_RIVET_DAYTONA_SNAPSHOT:-agenta-rivet-pi}
AGENTA_RIVET_DAYTONA_INSTALL_PI: ${AGENTA_RIVET_DAYTONA_INSTALL_PI:-false}
# === STORAGE ============================================== #
volumes:
- ../../../services/agent/src:/app/src
- ${HOME}/.pi/agent:/pi-agent-ro:ro
# === NETWORK ============================================== #
networks:
- agenta-network
# === LIFECYCLE ============================================ #
restart: always

postgres:
# === IMAGE ================================================ #
image: postgres:17
Expand Down Expand Up @@ -593,6 +637,4 @@ volumes:
postgres-data:
redis-volatile-data:
redis-durable-data:
nextjs-ee-cache:
nextjs-oss-cache:
turbo-ee-cache:
nextjs_cache:
Loading