feat: Kubernetes deployment support (Dockerfile, manifests, docs)

[chaosreload]

Summary

Add first-class Kubernetes deployment support for Zeroboot, addressing all items in #9.

EKS validation: All manifests and scripts validated on EKS 1.31 / ap-southeast-1 / c8i.xlarge with nested virtualization.

Note: The serve bind address fix (127.0.0.1 → 0.0.0.0) required for K8s health probes is tracked separately in #14 and should be merged before or alongside this PR.

Changes

Dockerfile

Multi-stage build: Rust 1.86 compiler stage + Ubuntu 22.04 runtime. Firecracker binary bundled at build time. vmlinux and rootfs images are not baked into the image — they are mounted via PersistentVolume, keeping the image lean and allowing runtime upgrades without rebuilding.

docker/entrypoint.sh

• Validates /dev/kvm access on startup (fast-fail with clear error message)
• Creates the snapshot template on first boot (~15s), skips if already present on PVC
• Supports Python and optional Node.js runtimes via env vars

deploy/k8s/

Reference manifests ready to apply:

• namespace.yaml — dedicated zeroboot namespace
• pvc.yaml — 20 Gi gp3 PVC for snapshot persistence (eliminates 15s re-snapshot on restart)
• deployment.yaml — privileged + hostPath /dev/kvm (works on plain EKS without KubeVirt); podAntiAffinity to spread Pods across Nodes; tuned readiness/liveness probes (initialDelaySeconds: 120)
• service.yaml — ClusterIP service
• hpa.yaml — custom-metric HPA on zeroboot_concurrent_forks

deploy/eks/

EKS-specific deployment configs:

• eks-cluster-only.yaml — create cluster without node groups (Step 1)
• eks-self-managed-kvm.sh — end-to-end script for self-managed ASG with CpuOptions.NestedVirtualization=enabled; see EKS note below
• eks-with-kvm-nodegroup.yaml — eksctl managed node group config (⚠️ see caveat)
• eks-add-kvm-nodegroup.yaml — add KVM node group to existing cluster

docs/KUBERNETES.md

Comprehensive deployment guide covering:

• EC2 instance selection: c8i/m8i/r8i recommended
• EKS managed vs self-managed node groups — root cause and solution for CpuOptions being silently dropped
• KVM device access (privileged + hostPath, with note on kubevirt alternative)
• PVC storage guidance
• Autoscaling, Prometheus, configuration reference, known limitations

⚠️ EKS Node Group: Use Self-Managed

TL;DR: EKS managed node groups silently drop CpuOptions.NestedVirtualization — your nodes start without /dev/kvm even if the eksctl YAML looks correct.

Root cause: When you provide a Launch Template to a managed node group, EKS generates a new internal LT and merges only a subset of fields. CpuOptions is not in that subset — despite not being listed in the official blocked-fields list. This is a documentation gap.

Solution: Use deploy/eks/eks-self-managed-kvm.sh, which creates an ASG + Launch Template directly via AWS CLI, bypassing EKS's internal LT generation entirely.

Architecture note

Kubernetes manages the lifecycle of the zeroboot server process — it does not schedule individual sandboxes. Each /v1/exec request is handled entirely within the Pod via a KVM fork (~0.8 ms). K8s role is capacity management: health checks, rolling updates, and horizontal scaling.

Testing

Validated on EKS 1.31 / ap-southeast-1 / c8i.xlarge (self-managed node group with nested virt):

• ✅ /dev/kvm present on nodes
• ✅ Docker image builds successfully (rust:1.86)
• ✅ Template creation completes (~19s)
• ✅ Readiness probe passes after template creation
• ✅ /v1/health reachable from Service ClusterIP
• ✅ CODE:print(1+1) → 2 (~100ms)
• ✅ CODE:import numpy... → result (~265ms)
• ✅ cat /etc/os-release → content (~28ms)

Depends on: #14

Closes #9