feat(sandbox): add provider entity to support configuring tools such as claude, outlook, etc (!23)

## Summary
- Add `Provider` entity for managing 3p deps from a sandbox
- Add provider CRUD API/server persistence and new CLI workflows (`nav provider create/get/list/update/delete`), including `--from-existing` laptop discovery.
- Integrate providers into sandbox create flow: infer from command (`-- claude`), support repeatable `--provider <type>`, prompt before auto-create, and allow manual in-sandbox setup.
- Add a dedicated `navigator-providers` crate with per-provider modules and mockable discovery test helpers.

## Key UX Changes
- `nav sandbox create --provider gitlab -- claude`
- Missing provider prompt now asks before creating from local state.
- `nav provider list --names` for scripting/cleanup.

## Test Plan
- `mise run cluster:deploy`
- `mise run test:e2e:sandbox`
- `mise run pre-commit`

Closes NVIDIA#19
Closes NVIDIA#22
Closes NVIDIA#11

Author: drew

.agent/skills/debug-navigator-cluster/SKILL.md

@@ -20,7 +20,7 @@ Diagnose why a navigator cluster failed to start after `nav cluster admin deploy
 7. **Prepare local images** (if `NAVIGATOR_PUSH_IMAGES` is set): In `internal` registry mode, bootstrap waits for the in-cluster registry and pushes tagged images there. In `external` mode, bootstrap uses legacy `ctr -n k8s.io images import` push-mode behavior.
 8. Wait for cluster health checks to pass (up to 6 min):
    - k3s API server readiness (`/readyz`)
-   - `navigator` deployment available in `navigator` namespace
+    - `navigator` statefulset ready in `navigator` namespace
    - `navigator-gateway` Gateway programmed in `navigator` namespace
    - If TLS enabled: `navigator-cli-client` secret exists with cert data
 9. Extract mTLS credentials if TLS is enabled (up to 3 min)
@@ -129,19 +129,19 @@ If `/readyz` fails, k3s is still starting or has crashed. Check container logs (
 
 If pods are in `CrashLoopBackOff`, `ImagePullBackOff`, or `Pending`, investigate those pods specifically.
 
-### Step 4: Check Navigator Server Deployment
+### Step 4: Check Navigator Server StatefulSet
 
-The Navigator server is deployed via a HelmChart CR. Check its status:
+The Navigator server is deployed via a HelmChart CR as a StatefulSet with persistent storage. Check its status:
 
 ```bash
-# Deployment status
-docker exec navigator-cluster-<name> sh -lc 'KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n navigator get deployment/navigator -o wide'
+# StatefulSet status
+docker exec navigator-cluster-<name> sh -lc 'KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n navigator get statefulset/navigator -o wide'
 
 # Navigator pod logs
-docker exec navigator-cluster-<name> sh -lc 'KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n navigator logs deployment/navigator --tail=100'
+docker exec navigator-cluster-<name> sh -lc 'KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n navigator logs statefulset/navigator --tail=100'
 
-# Describe deployment for events
-docker exec navigator-cluster-<name> sh -lc 'KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n navigator describe deployment/navigator'
+# Describe statefulset for events
+docker exec navigator-cluster-<name> sh -lc 'KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n navigator describe statefulset/navigator'
 
 # Helm install job logs (the job that installs the Navigator chart)
 docker exec navigator-cluster-<name> sh -lc 'KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n kube-system logs -l job-name=helm-install-navigator --tail=200'
@@ -286,7 +286,7 @@ If DNS is broken, all image pulls from the distribution registry will fail, as w
 | Container exited, OOMKilled | Insufficient memory | Increase host memory or reduce workload |
 | Container exited, non-zero exit | k3s crash, port conflict, privilege issue | Check `docker logs` and `docker inspect` for details |
 | `/readyz` fails | k3s still starting or crashed | Wait longer or check container logs for k3s errors |
-| Navigator pods `Pending` | Insufficient CPU/memory for scheduling | Check `kubectl describe pod` for scheduling failures |
+| Navigator pods `Pending` | Insufficient CPU/memory for scheduling, or PVC not bound | Check `kubectl describe pod` for scheduling failures and `kubectl get pvc -n navigator` for volume status |
 | Navigator pods `CrashLoopBackOff` | Server application error | Check `kubectl logs` on the crashing pod |
 | Navigator pods `ImagePullBackOff` (push mode) | Images not imported or wrong containerd namespace | Check `k3s ctr -n k8s.io images ls` for component images (Step 6) |
 | Navigator pods `ImagePullBackOff` (pull mode) | Registry auth or DNS issue | Check `/etc/rancher/k3s/registries.yaml` credentials and DNS (Step 8) |
@@ -364,8 +364,8 @@ run docker exec "${CONTAINER}" sh -lc "${KCFG} kubectl get pods -A -o wide" 2>&1
 echo "=== Failing Pods ==="
 run docker exec "${CONTAINER}" sh -lc "${KCFG} kubectl get pods -A --field-selector=status.phase!=Running,status.phase!=Succeeded" 2>&1
 
-echo "=== Navigator Deployment ==="
-run docker exec "${CONTAINER}" sh -lc "${KCFG} kubectl -n navigator get deployment/navigator -o wide" 2>&1
+echo "=== Navigator StatefulSet ==="
+run docker exec "${CONTAINER}" sh -lc "${KCFG} kubectl -n navigator get statefulset/navigator -o wide" 2>&1
 
 echo "=== Navigator Gateway ==="
 run docker exec "${CONTAINER}" sh -lc "${KCFG} kubectl -n navigator get gateway/navigator-gateway" 2>&1

.gitlab-ci.yml

@@ -140,28 +140,7 @@ build_ci_image:
 # =============================================================================
 # Lint Jobs
 # =============================================================================
-fmt_check:
-  extends: .rust_job_rules
-  stage: lint
-  rules:
-    - if: $CI_COMMIT_TAG
-      when: never
-    - when: on_success
-  script:
-    - mise run fmt:check
-
-clippy:
-  extends: .rust_job_rules
-  stage: lint
-  rules:
-    - if: $CI_COMMIT_TAG
-      when: never
-    - when: on_success
-  script:
-    - mise run clippy
-
-python_lint:
-  extends: .python_job_rules
+lint:
   stage: lint
   rules:
     - if: $CI_COMMIT_TAG
@@ -170,7 +149,7 @@ python_lint:
   before_script:
     - uv sync --frozen
   script:
-    - mise run python:lint
+    - mise run lint
 
 # =============================================================================
 # Test Jobs
@@ -183,7 +162,7 @@ rust_test:
       when: never
     - when: on_success
   script:
-    - mise run test:rust
+    - PATH="/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" /root/.cargo/bin/cargo test --workspace
 
 python_test:
   extends: .python_job_rules
@@ -195,7 +174,53 @@ python_test:
   before_script:
     - uv sync --frozen
   script:
-    - mise run test:python
+    - |
+      uv run python -m grpc_tools.protoc \
+        -Iproto \
+        --python_out=python/navigator/_proto \
+        --pyi_out=python/navigator/_proto \
+        --grpc_python_out=python/navigator/_proto \
+        proto/inference.proto \
+        proto/navigator.proto \
+        proto/datamodel.proto \
+        proto/sandbox.proto
+    - |
+      uv run python - <<'PY'
+      from pathlib import Path
+      import re
+
+      line_rewrites = {
+          "python/navigator/_proto/inference_pb2_grpc.py": [
+              (r"^import inference_pb2 as inference__pb2$", "from . import inference_pb2 as inference__pb2"),
+          ],
+          "python/navigator/_proto/navigator_pb2_grpc.py": [
+              (r"^import navigator_pb2 as navigator__pb2$", "from . import navigator_pb2 as navigator__pb2"),
+              (r"^import sandbox_pb2 as sandbox__pb2$", "from . import sandbox_pb2 as sandbox__pb2"),
+          ],
+          "python/navigator/_proto/navigator_pb2.py": [
+              (r"^import datamodel_pb2 as datamodel__pb2$", "from . import datamodel_pb2 as datamodel__pb2"),
+              (r"^import sandbox_pb2 as sandbox__pb2$", "from . import sandbox_pb2 as sandbox__pb2"),
+          ],
+          "python/navigator/_proto/datamodel_pb2.py": [
+              (r"^import sandbox_pb2 as sandbox__pb2$", "from . import sandbox_pb2 as sandbox__pb2"),
+          ],
+          "python/navigator/_proto/datamodel_pb2_grpc.py": [
+              (r"^import datamodel_pb2 as datamodel__pb2$", "from . import datamodel_pb2 as datamodel__pb2"),
+          ],
+          "python/navigator/_proto/sandbox_pb2_grpc.py": [
+              (r"^import sandbox_pb2 as sandbox__pb2$", "from . import sandbox_pb2 as sandbox__pb2"),
+          ],
+      }
+
+      for path, rules in line_rewrites.items():
+          file_path = Path(path)
+          text = file_path.read_text()
+          text = text.replace("from . from . import", "from . import")
+          for pattern, replacement in rules:
+              text = re.sub(pattern, replacement, text, flags=re.MULTILINE)
+          file_path.write_text(text)
+      PY
+    - uv run pytest python/
 
 python_e2e_sandbox_test:
   extends: .e2e_job_rules
@@ -222,8 +247,8 @@ python_e2e_sandbox_test:
   script:
     - socat UNIX-LISTEN:/var/run/docker.sock,fork,reuseaddr TCP:docker:2375 &
     - sleep 1
-    - mise run cluster
-    - mise run test:e2e:sandbox
+    - mise run --no-prepare cluster
+    - mise run --no-prepare test:e2e:sandbox
 
 # =============================================================================
 # Publish Jobs

Cargo.lock

@@ -0,0 +1,150 @@
+# Providers
+
+## Overview
+
+Navigator uses a first-class `Provider` entity to represent external tool credentials and
+configuration (for example `claude`, `gitlab`, `github`, `outlook`).
+
+Providers exist as an abstraction layer for configuring tools that rely on third-party
+access. Rather than each tool managing its own credentials and service configuration,
+providers centralize that concern: a user configures a provider once, and any sandbox that
+needs that external service can reference it.
+
+At sandbox creation time, providers configure the sandbox environment with the
+credentials and settings the tool needs. Access is then enforced through the sandbox
+policy — the policy decides which outbound requests are allowed or denied based on
+the providers attached to that sandbox.
+
+Core goals:
+
+- manage providers directly via CLI,
+- discover provider data from the local machine automatically,
+- require providers during sandbox creation,
+- project provider context into sandbox runtime,
+- drive sandbox policy to allow or deny outbound access to third-party services.
+
+## Data Model
+
+Provider is defined in `proto/datamodel.proto`:
+
+- `id`: unique entity id
+- `name`: user-managed name
+- `type`: canonical provider slug (`claude`, `gitlab`, `github`, etc.)
+- `credentials`: `map<string, string>` for secret values
+- `config`: `map<string, string>` for non-secret settings
+
+The gRPC surface is defined in `proto/navigator.proto`:
+
+- `CreateProvider`
+- `GetProvider`
+- `ListProviders`
+- `UpdateProvider`
+- `DeleteProvider`
+
+## Components
+
+- `crates/navigator-providers`
+  - canonical provider type normalization and command detection,
+  - provider registry and per-provider discovery plugins,
+  - shared discovery engine and context abstraction for testability.
+- `crates/navigator-cli`
+  - `nav provider ...` command handlers,
+  - sandbox provider requirement resolution in `sandbox create`.
+- `crates/navigator-server`
+  - provider CRUD gRPC handlers,
+  - persistence using `object_type = "provider"`.
+
+## Provider Plugins
+
+Each provider has its own module under `crates/navigator-providers/src/providers/`.
+
+Current modules:
+
+- `claude.rs`
+- `codex.rs`
+- `opencode.rs`
+- `openclaw.rs`
+- `gitlab.rs`
+- `github.rs`
+- `outlook.rs`
+
+Each plugin defines:
+
+- canonical `id()`,
+- discovery spec (env vars + config paths),
+- `discover_existing()` behavior.
+
+The registry is assembled in `ProviderRegistry::new()` by registering each provider module.
+
+## Discovery Architecture
+
+Discovery behavior is split into three layers:
+
+1. provider module defines static spec (`ProviderDiscoverySpec`),
+2. shared engine (`discover_with_spec`) performs env/file scanning,
+3. runtime context (`DiscoveryContext`) supplies filesystem/environment reads.
+
+`DiscoveryContext` has:
+
+- `RealDiscoveryContext` for production runtime,
+- `MockDiscoveryContext` test helper for deterministic tests.
+
+This keeps provider tests isolated from host environment and filesystem.
+
+## CLI Flows
+
+### Provider CRUD
+
+`nav provider create --type <type> --name <name> [--from-existing] [--credential k=v]... [--config k=v]...`
+
+- `--from-existing` merges discovered laptop data into explicit CLI key-value args.
+- Explicit `--credential` / `--config` values take precedence.
+
+Also supported:
+
+- `nav provider get <name>`
+- `nav provider list`
+- `nav provider update <name> ...`
+- `nav provider delete <name> [<name>...]`
+
+### Sandbox Create
+
+`nav sandbox create --provider gitlab -- claude`
+
+Resolution logic:
+
+1. infer provider from command token after `--` (for example `claude`),
+2. union with explicit `--provider <type>` flags,
+3. ensure each required provider type exists,
+4. if interactive and missing, auto-create from existing local state,
+5. set `NAVIGATOR_PROVIDER_TYPES` in sandbox spec environment.
+
+Non-interactive mode fails with a clear missing-provider error.
+
+> **Note:** Providers can also be configured from within the sandbox itself. This allows
+> sandbox users to set up or update provider credentials and configuration at runtime,
+> without requiring them to be fully resolved before sandbox creation.
+
+## Persistence and Validation
+
+Server enforces:
+
+- `provider.type` must be non-empty,
+- name uniqueness for providers,
+- generated `id` on create,
+- id preservation on update.
+
+Providers are stored with `object_type = "provider"` in the shared object store.
+
+## Security Notes
+
+- Provider credentials are stored in `credentials` map and treated as sensitive.
+- CLI output intentionally avoids printing credential values.
+- CLI displays only non-sensitive summaries (counts/key names where relevant).
+
+## Test Strategy
+
+- Per-provider unit tests in each provider module.
+- Shared normalization/command-detection tests in `crates/navigator-providers/src/lib.rs`.
+- Mocked discovery context tests cover env and path-based behavior.
+- CLI and server integration tests validate end-to-end RPC compatibility.

architecture/providers.md

@@ -8,13 +8,21 @@ database URL scheme.
 
 Supported backends:
 - Postgres (`postgres://` or `postgresql://`)
-- SQLite (`sqlite:`)
+- SQLite (`sqlite:`) — file-backed (default) or in-memory
 
 The server requires a database URL. The CLI enforces `--db-url` / `NAVIGATOR_DB_URL`, and
 `run_server` will reject an empty value.
 
-Example in-memory SQLite URL:
-`sqlite::memory:?cache=shared`
+The default database URL is `sqlite:/var/navigator/navigator.db`, which stores data on a persistent
+volume. In-memory SQLite (`sqlite::memory:?cache=shared`) can be used for ephemeral environments
+but data will be lost on pod restart.
+
+## Deployment Storage
+
+The Navigator server runs as a **StatefulSet** with a `volumeClaimTemplate` that provisions a 1Gi
+`ReadWriteOnce` PersistentVolumeClaim mounted at `/var/navigator`. On k3s clusters this uses the
+built-in `local-path-provisioner` StorageClass. The SQLite database file is stored at
+`/var/navigator/navigator.db` and survives pod restarts and rescheduling.
 
 ## Data Model