feat(kubernetes): add agent execution mode, GKE Sandbox, and image pu…#36
Open
gafda wants to merge 5 commits intoaron-muon:mainfrom
Open
feat(kubernetes): add agent execution mode, GKE Sandbox, and image pu…#36gafda wants to merge 5 commits intoaron-muon:mainfrom
gafda wants to merge 5 commits intoaron-muon:mainfrom
Conversation
Contributor
Author
|
@aron-muon : Second PR from original PR #33 |
There was a problem hiding this comment.
Pull request overview
This PR upgrades KubeCodeRun’s Kubernetes execution architecture by introducing an agent-based execution mode (recommended/default) alongside the legacy nsenter mode, adds GKE Sandbox (gVisor) scheduling options, and significantly expands Redis configurability (cluster/sentinel/TLS/key-prefix) while updating Helm, docs, CI, and tests accordingly.
Changes:
- Add agent-based execution path (executor-agent + sidecar support) with optional legacy nsenter mode.
- Add GKE Sandbox (gVisor) runtimeClass/nodeSelector/tolerations/annotations + image pull secrets support.
- Refactor Redis configuration/pooling (standalone/cluster/sentinel, TLS, key-prefix) and enforce non-transactional pipelines for cluster compatibility (with new safety-net tests).
Reviewed changes
Copilot reviewed 41 out of 42 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/test_settings_validators.py | Adds unit tests for JSON parsing (GKE sandbox fields) and Redis config sanitizers. |
| tests/unit/test_session_service.py | Fixes Redis pipeline mocking to match redis.asyncio semantics. |
| tests/unit/test_kubernetes_client.py | Adds test coverage for execution modes, GKE Sandbox behavior, and image pull secrets. |
| tests/unit/test_job_executor.py | Updates default sidecar image and asserts PodSpec fields are forwarded into manifests. |
| tests/unit/test_core_pool.py | Updates Redis pool tests for new failure/cluster kwargs behavior and new pool_stats fields. |
| tests/unit/test_cluster_pipeline_compat.py | New tests ensuring all pipelines use transaction=False for Redis Cluster compatibility. |
| src/utils/logging.py | Uses runtime settings.service_version when present for log context versioning. |
| src/utils/config_validator.py | Validates Redis connectivity across standalone/cluster/sentinel and forwards TLS kwargs. |
| src/services/state.py | Prefixes state keys and uses non-transactional pipelines for cluster compatibility. |
| src/services/session.py | Prefixes session keys and switches pipelines to transaction=False (cluster compat). |
| src/services/metrics.py | Prefixes metrics keys via redis_pool. |
| src/services/kubernetes/pool.py | Passes new PodSpec fields (execution mode, sandbox, pull secrets) into pod creation. |
| src/services/kubernetes/models.py | Extends PodSpec/PoolConfig with execution mode, executor port, sandbox, pull secrets/policy. |
| src/services/kubernetes/manager.py | Wires new execution/sandbox/pull-secret options from settings into Job/Pool execution. |
| src/services/kubernetes/job_executor.py | Forwards new PodSpec fields into job manifests and updates default sidecar image. |
| src/services/kubernetes/client.py | Implements agent vs nsenter pod manifests, GKE Sandbox scheduling, and image pull secrets. |
| src/services/health.py | Prefixes Redis health-check key. |
| src/services/file.py | Uses shared redis_pool client and prefixes file metadata keys. |
| src/services/detailed_metrics.py | Converts key constants to prefixed keys computed at init time. |
| src/services/api_key_manager.py | Converts key constants to prefixed keys computed at init time + cluster-compatible pipelines. |
| src/main.py | Introduces effective_version, parses pull secrets, adds config consistency warnings, wires new K8S options. |
| src/core/pool.py | Major RedisPool refactor: mode support (standalone/cluster/sentinel), TLS kwargs, key prefix helper. |
| src/config/redis.py | New RedisConfig supporting mode/cluster/sentinel/TLS/key-prefix + sanitizing validators. |
| src/config/kubernetes.py | Adds execution mode, executor port, pull secrets/policy, and GKE Sandbox config fields. |
| src/config/init.py | Adds new env vars, redis/k8s fields, validators, and JSON parsing for sandbox selector/tolerations. |
| scripts/build-images.sh | Builds two sidecar targets (agent/nsenter), adds --list, improves help, passes VERSION build arg. |
| helm-deployments/kubecoderun/values.yaml | Adds Redis mode/TLS/key-prefix/cluster/sentinel config + execution mode, gVisor, pull secrets. |
| helm-deployments/kubecoderun/templates/secret.yaml | Adds REDIS_PASSWORD emission (all modes) alongside REDIS_URL. |
| helm-deployments/kubecoderun/templates/configmap.yaml | Exposes new Redis/K8S/GKE sandbox env vars and image pull secrets. |
| helm-deployments/kubecoderun/templates/_helpers.tpl | Makes Redis URL honor TLS (redis:// vs rediss://). |
| docs/SECURITY.md | Documents execution modes + agent-mode security posture + gVisor support and Redis TLS guidance. |
| docs/CONFIGURATION.md | Documents Redis modes/TLS/key-prefix and K8S execution modes/pull secrets/gVisor configuration. |
| docs/ARCHITECTURE.md | Updates architecture doc to describe agent vs nsenter pod designs. |
| docker/sidecar/main.py | Adds agent-mode execution via HTTP to executor-agent, keeps nsenter path, adds agent readiness check. |
| docker/sidecar/executor-agent/main.go | Adds the executor-agent HTTP server used in agent execution mode. |
| docker/sidecar/executor-agent/go.mod | Declares Go module for executor-agent. |
| docker/sidecar/Dockerfile | Multi-target sidecar build producing sidecar-agent and sidecar-nsenter images (+ Go build stage). |
| .gitignore | Ignores .pdm-python. |
| .github/workflows/docker-publish.yml | Publishes both sidecar images (agent + nsenter) and retags agent variant. |
| .github/workflows/docker-build-reusable.yml | Adds build_target input to support multi-target Docker builds. |
| .env.example | Adds Redis modes/TLS/key-prefix and K8S execution mode/pull-secrets/gVisor examples. |
| .editorconfig | Adds repo-wide formatting rules. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ll config Agent execution mode (default): - Add executor-agent Go binary for HTTP-based code execution - Multi-target sidecar Dockerfile (sidecar-agent + sidecar-nsenter) - GKE Sandbox (gVisor) support with runtime class and node selectors - Image pull policy and secrets for private registries - CI builds both sidecar-agent and sidecar-nsenter images Redis Cluster/Sentinel/TLS support: - Three deployment modes: standalone (default), cluster, sentinel - Full TLS/SSL configuration for managed Redis services - Key prefixing for multi-tenant deployments - Pipeline transaction=False for cluster cross-slot compatibility Infrastructure: - Add .editorconfig for consistent coding style - Update Helm chart with execution mode and Redis config - Update docs: ARCHITECTURE, SECURITY, CONFIGURATION
…tion - Import ClusterNode explicitly from redis.asyncio.cluster for better type safety - Add execution_mode validation to prevent silent security downgrades from typos - Derive GKE Sandbox values from runtime_class_name for consistency - Make make_key() pure string operation without triggering Redis initialization - Load Redis key prefix from settings in service __init__ to avoid pool init Fixes all 6 review comments from PR aron-muon#36: - Improves Redis cluster import reliability - Prevents execution_mode typos from silently enabling nsenter - Ensures runtime_class_name consistency across pod spec - Prevents unexpected Redis initialization in make_key() - Prevents unexpected Redis initialization in ApiKeyManagerService - Prevents unexpected Redis initialization in DetailedMetricsService All tests passing (1352 unit tests).
gafda
force-pushed
the
feat-agent-execution-mode
branch
from
ad0c0ec to
a12c7d8
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 40 out of 41 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Validate request.working_dir against WORKING_DIR in execute_via_agent and execute_via_nsenter before any file writes or process spawning, preventing path-traversal (e.g. working_dir=/etc would write code files outside /mnt/data) - Add agent_init_image parameter to create_pod_manifest() so callers can explicitly specify the agent sidecar image for the init container when the runtime sidecar image is the nsenter variant (which lacks the binary) - Switch Go builder stage in docker/sidecar/Dockerfile from golang:1.26-alpine to dhi.io/golang:1.26-debian13-dev to align with repo-wide DHI image policy and improve supply-chain consistency All 1352 unit tests passing.
Contributor
Author
|
@aron-muon, I've fixed all Copilot's complaints. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant improvements to the Docker sidecar architecture for KubeCodeRun, enabling support for two distinct execution modes: agent-based and legacy nsenter-based. It refactors the Docker build process to produce two separate images, adds a new Go-based executor agent for secure code execution, and updates configuration and CI workflows to support these changes.
Sidecar architecture and Dockerfile refactor:
docker/sidecar/Dockerfileto use multi-target builds, producing two distinct images:kubecoderun-sidecar-agent(agent mode, recommended) andkubecoderun-sidecar-nsenter(legacy nsenter mode). The agent image includes a statically compiled Go binary for secure code execution without Linux capabilities or privilege escalation, while the nsenter image retains legacy compatibility for clusters that require it. [1] [2]Executor agent implementation:
docker/sidecar/executor-agentwithmain.goimplementing a lightweight HTTP server for code execution. This agent allows the sidecar to execute code via HTTP requests inside the main container, inheriting the container's environment and enforcing directory restrictions for security. [1] [2]CI/CD workflow updates:
build_targetinput, and the publish workflow separately builds and tags both agent and nsenter images. [1] [2] [3] [4]Configuration improvements:
.env.examplewith new environment variables for Redis deployment modes (standalone, cluster, sentinel), TLS options, key prefixing, and detailed Kubernetes execution configuration for agent and nsenter modes. These changes improve flexibility and clarity for deployment in various environments. [1] [2] [3]Editor and code style consistency:
.editorconfigfile to standardize code formatting across languages and file types in the repository.