Skip to content

feat: migrate sandbox operations to OpenShell gRPC#4329

Draft
ericksoa wants to merge 28 commits into
mainfrom
feat/openshell-grpc-sandbox-ops
Draft

feat: migrate sandbox operations to OpenShell gRPC#4329
ericksoa wants to merge 28 commits into
mainfrom
feat/openshell-grpc-sandbox-ops

Conversation

@ericksoa
Copy link
Copy Markdown
Contributor

@ericksoa ericksoa commented May 27, 2026

Summary

Migrates NemoClaw-managed sandbox lifecycle operations from SSH/SFTP/SSHFS-style paths to direct OpenShell gRPC transport. This adds a Node gRPC adapter for OpenShell gateway metadata, sandbox exec streams, sandbox logs/watch, and dashboard ForwardTcp bridges while preserving the privileged Docker exec path for root/shields mutations.

Related Issue

None.

Changes

  • Add vendored OpenShell protos, gRPC gateway metadata resolution, sync exec wrappers, sandbox log/watch helpers, and a Node-managed ForwardTcp bridge.
  • Move recovery, verification, version probes, skill install, snapshot/rebuild backup+restore, state-file transfer, status probes, connect route probes, and channel cleanup to gRPC exec streams.
  • Replace dashboard forward recovery with NemoClaw-managed gRPC ForwardTcp bridge state.
  • Disable live share mount under the gRPC-only model while retaining status/unmount cleanup for legacy mounts.
  • Add gateway metadata tests and a static migration guard against reintroducing SSH/SSHFS/upload/download/SSH-backed forward production paths.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

  • npx prek run --all-files passes
  • npm test passes
  • Tests added or updated for new or changed behavior
  • No secrets, API keys, or credentials committed
  • Docs updated for user-facing behavior changes
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Ran locally:

  • npm run build:cli
  • npm run typecheck:cli
  • npx vitest run src/lib/adapters/openshell/gateway-metadata.test.ts src/lib/adapters/openshell/grpc-migration-guard.test.ts src/lib/share-command.test.ts src/lib/sandbox/version.test.ts test/share-command-deps-probe-argv.test.ts src/lib/actions/sandbox/process-recovery.test.ts src/lib/skill-install.test.ts test/process-recovery.test.ts test/snapshot.test.ts test/snapshot-restore-existing-dest.test.ts test/rebuild-credential-preflight.test.ts test/rebuild-shields-auto-unlock.test.ts test/rebuild-credential-hydration.test.ts test/rebuild-policy-presets.test.ts test/share-command-remote-path.test.ts test/share-command-writable.test.ts
  • git diff --check

Not run locally:

  • Live local E2E, because the local OpenShell gateway is not running (openshell sandbox list returns connection refused).

Signed-off-by: Aaron Erickson aerickson@nvidia.com

Summary by CodeRabbit

  • New Features
    • Transitioned sandbox communication from SSH to gRPC protocol for improved reliability and performance
    • Implemented forward bridge infrastructure for enhanced port forwarding stability

Review Change Stack

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented May 27, 2026

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 27, 2026

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4b5d7936-0e27-4c89-9fa6-2f9181506553

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/openshell-grpc-sandbox-ops

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 27, 2026

E2E Advisor Recommendation

Required E2E: cloud-onboard-e2e, sandbox-operations-e2e, inference-routing-e2e, network-policy-e2e, openclaw-onboard-security-posture-e2e, issue-4462-scope-upgrade-approval-e2e, double-onboard-e2e, hermes-e2e, skill-agent-e2e, tunnel-lifecycle-e2e, vm-driver-privileged-exec-routing-e2e
Optional E2E: cloud-e2e, cloud-inference-e2e, state-backup-restore-e2e, diagnostics-e2e, snapshot-commands-e2e, hermes-dashboard-e2e, openclaw-inference-switch-e2e

Dispatch hint: cloud-onboard-e2e,sandbox-operations-e2e,inference-routing-e2e,network-policy-e2e,openclaw-onboard-security-posture-e2e,issue-4462-scope-upgrade-approval-e2e,double-onboard-e2e,hermes-e2e,skill-agent-e2e,tunnel-lifecycle-e2e,vm-driver-privileged-exec-routing-e2e

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • cloud-onboard-e2e (medium): Required because installer, build packaging, proto copying, onboarding, dashboard setup, gateway cleanup, and sandbox verification changed. This is the clean Ubuntu onboard smoke for first-run product behavior.
  • sandbox-operations-e2e (high): Required because connect/destroy/process-recovery/state/status/share and OpenShell transport code changed. This exercises real sandbox lifecycle operations, teardown, recovery, logs, and CLI operations against live OpenShell.
  • inference-routing-e2e (medium): Required because sandbox exec transport and connect inference-route probes moved through new gRPC paths. This validates credential isolation and inference route error classification in a real sandbox.
  • network-policy-e2e (medium): Required because policy-channel, sandbox proxy/runtime guard behavior, DNS/proxy-related files, and E2E residual capability defaults changed. This validates security/network policy enforcement in a live sandbox.
  • openclaw-onboard-security-posture-e2e (high): Required because the reusable E2E workflow now opt-ins residual caps and nemoclaw-start runtime guard/proxy-env behavior changed. This validates non-root onboard security posture and runtime guard assertions for OpenClaw.
  • issue-4462-scope-upgrade-approval-e2e (high): Required because scripts/nemoclaw-start.sh changes the in-sandbox openclaw devices approve wrapper and the corresponding E2E test was touched. This validates real scope-upgrade approval and that approved agent runs stay gateway-routed.
  • double-onboard-e2e (high): Required because onboard forward cleanup/start, stale gateway cleanup, preflight port recovery, install post-check forwarding, and the double-onboard E2E script changed. This validates repeated onboarding does not leave stale gateway/forwarding state.
  • hermes-e2e (high): Required because install.sh removed the legacy post-check OpenShell forward resurrection for Hermes and test-hermes-e2e.sh changed. This validates the real Hermes install/onboard/health/live-inference path with the new CLI-owned forwarding behavior.
  • skill-agent-e2e (medium): Required because skill install, sandbox skill action, and skill remote code changed. This validates installing and using skills in a live assistant sandbox.
  • tunnel-lifecycle-e2e (high): Required because tunnel service discovery and forward bridge state/runner code changed. This validates lifecycle of exposed tunnel services against a real sandbox.
  • vm-driver-privileged-exec-routing-e2e (low): Required because the OpenShell exec transport/gRPC migration can change how privileged exec is routed for VM-driver paths. This lightweight E2E guards the privileged exec routing contract.

Optional E2E

  • cloud-e2e (high): Useful broad confidence check for the complete OpenClaw install → onboard → live inference → CLI operations user journey after a wide transport/onboarding PR.
  • cloud-inference-e2e (medium): Additional live inference confidence after gRPC sandbox exec and inference probe changes; narrower than cloud-e2e if only inference route behavior is under review.
  • state-backup-restore-e2e (high): Adjacent confidence for changed sandbox state, destroy cleanup, and upgrade/recovery paths, but less directly targeted than sandbox-operations and double-onboard.
  • diagnostics-e2e (medium): Useful because status/share/metadata and sandbox logs/connect paths changed, but not merge-blocking if lifecycle and onboard tests pass.
  • snapshot-commands-e2e (medium): Adjacent because snapshot gateway guard/state tests changed and destroy/state code moved; optional unless reviewers see snapshot-specific changes in the non-truncated diff.
  • hermes-dashboard-e2e (high): Optional targeted dashboard/forwarding confidence for Hermes if the removed installer forward restoration has reviewer concern beyond the base Hermes E2E coverage.
  • openclaw-inference-switch-e2e (medium): Optional confidence for runtime inference reconfiguration after changes to verify deployment and sandbox exec transport; required inference-routing covers the core route checks.

New E2E recommendations

  • openshell-grpc-transport-and-protos (high): Existing E2Es exercise sandbox behavior but do not appear to explicitly assert that packaged dist proto files are present and that the real @openshell/sdk/native direct gRPC path is used rather than falling back to CLI/shell execution.
    • Suggested test: Add an E2E that installs from a packed artifact, verifies dist/lib/adapters/openshell/proto/*.proto are present, enables the direct OpenShell SDK/gRPC transport, runs sandbox exec/connect/destroy through it, and fails if the legacy CLI transport is used.
  • forward-bridge-dashboard-lifecycle (medium): Forward bridge runner/state and dashboard forward ownership changed across OpenClaw and Hermes; existing broad E2Es may not isolate stale forward cleanup, restart after gateway restart, and pid ownership behavior.
    • Suggested test: Add a focused forward-bridge E2E that onboards, starts dashboard forwarding, kills/restarts the gateway and local forward helper, verifies stale pid cleanup, and confirms host dashboard health without invoking legacy OpenShell SSH forwarding.

Dispatch hint

  • Workflow: .github/workflows/nightly-e2e.yaml
  • jobs input: cloud-onboard-e2e,sandbox-operations-e2e,inference-routing-e2e,network-policy-e2e,openclaw-onboard-security-posture-e2e,issue-4462-scope-upgrade-approval-e2e,double-onboard-e2e,hermes-e2e,skill-agent-e2e,tunnel-lifecycle-e2e,vm-driver-privileged-exec-routing-e2e

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 27, 2026

E2E Scenario Advisor Recommendation

Required scenario E2E: ubuntu-repo-cloud-openclaw, ubuntu-repo-cloud-hermes, ubuntu-repo-cloud-openclaw-repair, ubuntu-repo-cloud-openclaw-resume
Optional scenario E2E: wsl-repo-cloud-openclaw, macos-repo-cloud-openclaw, gpu-repo-local-ollama-openclaw

Dispatch required scenario E2E:

  • gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw
  • gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-hermes
  • gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-repair
  • gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-resume

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required scenario E2E

  • ubuntu-repo-cloud-openclaw: Primary Ubuntu repo-current cloud OpenClaw scenario covers the changed install/build path, OpenShell gRPC/proto adapter, onboarding, sandbox shell/connect, gateway health, inference-local routing, and credential checks on the default supported path.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw
  • ubuntu-repo-cloud-hermes: Hermes onboarding and dashboard/forwarding behavior are directly touched by installer and onboard changes, so the Hermes-specific repo-current scenario should run in addition to the OpenClaw baseline.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-hermes
  • ubuntu-repo-cloud-openclaw-repair: Sandbox process recovery, stale gateway cleanup, forward cleanup/start, and repair-oriented onboarding paths changed; this is the smallest routed scenario that exercises repair-existing-config lifecycle behavior.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-repair
  • ubuntu-repo-cloud-openclaw-resume: Onboarding/session recovery and installer changes can affect resume-after-interrupt behavior; this routed scenario targets that lifecycle without requiring the full fan-out.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-resume

Optional scenario E2E

  • wsl-repo-cloud-openclaw: Optional adjacent platform coverage for shell/connect, install, and OpenShell behavior under WSL; special-runner scenarios are not primary unless the change is WSL-only.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=wsl-repo-cloud-openclaw
  • macos-repo-cloud-openclaw: Optional adjacent platform coverage for repo install/build and CLI readiness on macOS; macOS is a special-runner platform and not the primary affected path.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=macos-repo-cloud-openclaw
  • gpu-repo-local-ollama-openclaw: Optional coverage for local Ollama inference and proxy behavior because sandbox connect/inference-route and OpenShell adapter changes may affect local-provider paths; GPU runner makes this optional.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=gpu-repo-local-ollama-openclaw

Relevant changed files

  • package-lock.json
  • package.json
  • scripts/copy-openshell-protos.mjs
  • scripts/install.sh
  • scripts/nemoclaw-start.sh
  • src/lib/actions/sandbox/connect.ts
  • src/lib/actions/sandbox/destroy.ts
  • src/lib/actions/sandbox/policy-channel.ts
  • src/lib/actions/sandbox/process-recovery.ts
  • src/lib/actions/sandbox/skill-install.ts
  • src/lib/adapters/openshell/direct-grpc.ts
  • src/lib/adapters/openshell/forward-bridge-runner.ts
  • src/lib/adapters/openshell/forward-bridge-state.ts
  • src/lib/adapters/openshell/gateway-metadata.ts
  • src/lib/adapters/openshell/grpc.ts
  • src/lib/adapters/openshell/proto/datamodel.proto
  • src/lib/adapters/openshell/proto/openshell.proto
  • src/lib/adapters/openshell/proto/sandbox.proto
  • src/lib/adapters/openshell/sync-runner.ts
  • src/lib/adapters/openshell/timeouts.ts
  • src/lib/onboard.ts
  • src/lib/onboard/agent-fixed-forward.ts
  • src/lib/onboard/dashboard-access.ts
  • src/lib/onboard/dashboard.ts
  • src/lib/onboard/forward-cleanup.ts
  • src/lib/onboard/forward-start.ts
  • src/lib/onboard/preflight-port-recovery.ts
  • src/lib/onboard/sandbox-verification-exec.ts
  • src/lib/onboard/stale-gateway-cleanup.ts
  • src/lib/sandbox/version.ts
  • src/lib/state/sandbox.ts
  • src/lib/status-command-deps.ts
  • src/lib/tunnel/services.ts
  • src/lib/verify-deployment.ts
  • vendor/openshell-sdk-placeholder/index.d.ts
  • vendor/openshell-sdk-placeholder/index.js
  • vendor/openshell-sdk-placeholder/lib.d.ts
  • vendor/openshell-sdk-placeholder/lib.mjs
  • vendor/openshell-sdk-placeholder/package.json

Comment thread src/lib/adapters/openshell/forward-bridge-state.ts Fixed
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 27, 2026

PR Review Advisor

Findings: 6 needs attention, 12 worth checking, 0 nice ideas
Since last review: 0 prior items resolved, 12 still apply, 2 new items found

Review findings

🛠️ Needs attention

  • Gateway tokens can be sent to overridden or plaintext endpoints (src/lib/adapters/openshell/gateway-metadata.ts:239): The metadata resolver accepts OPENSHELL_GATEWAY_ENDPOINT or OPENSHELL_GATEWAY_URL as the target while still deriving gatewayDir from the active or selected gateway and loading persisted OIDC or Cloudflare tokens. The raw gRPC client also uses insecure credentials for http:// endpoints while attaching bearer or Cloudflare metadata. A hostile or contaminated environment can redirect persisted gateway credentials to a different endpoint or transmit them without TLS.
    • Recommendation: When an endpoint override is present, do not attach persisted token material unless the override exactly matches the persisted gateway endpoint, or require explicit token material scoped to that override. Reject OIDC and Cloudflare token auth on plaintext HTTP except for a narrowly named test-only path. Add negative tests for override-plus-persisted-token and token-auth-over-http.
    • Evidence: resolveGatewayMetadata() selects endpointOverride from env/options, keeps gatewayDir from loaded gateway metadata, and buildConnectOptions() reads edge_token/oidc_token.json. direct-grpc.ts createCredentials() returns createInsecure() for http: and callCredentials() attaches authorization/cf-access-token/cookie metadata.
  • Fake SDK transport is enabled by ambient production environment (src/lib/adapters/openshell/sync-runner.ts:148): NEMOCLAW_SDK_TEST_TRANSPORT=1 switches the production sync runner into a fake transport that executes NEMOCLAW_SDK_TEST_FAKE_EXEC_BIN with the inherited production environment. The allowlist describes these variables as Vitest-only, but runtime enforcement is only an environment variable, so production sandbox exec paths can be redirected if the environment is contaminated.
    • Recommendation: Restrict fake transport to a verified test runtime such as VITEST_WORKER_ID or NODE_ENV=test, or move it behind test-only dependency injection. Avoid passing the full production environment to fake helpers. Add a negative test proving the env pair is ignored outside tests.
    • Evidence: sync-runner.ts checks process.env.NEMOCLAW_SDK_TEST_TRANSPORT === "1", reads NEMOCLAW_SDK_TEST_FAKE_EXEC_BIN, then spawnSyncs that helper with env: process.env.
  • Core lifecycle paths still depend on the throwing SDK placeholder (src/lib/adapters/openshell/grpc.ts:199): The PR says it migrates sandbox lifecycle operations to OpenShell gRPC, but the packaged dependency is a vendored @openshell/sdk placeholder whose OpenShellClient.connect() always throws. Lifecycle methods such as health, createSandbox, getSandbox, listSandboxes, deleteSandbox, waitReady, and waitDeleted still call the SDK directly; only exec has direct raw-gRPC fallback coverage.
    • Recommendation: Either land and pin the real OpenShell SDK before switching production lifecycle call sites, add direct-gRPC fallback coverage for lifecycle methods, or keep those call sites on the existing working transport until the SDK is available. Add a contract test that exercises lifecycle methods with the placeholder present.
    • Evidence: package.json adds "@openshell/sdk": "file:vendor/openshell-sdk-placeholder"; vendor/openshell-sdk-placeholder/index.js throws in OpenShellClient.connect(); grpc.ts lifecycle methods call await this.sdk(), while execBinary alone has direct gRPC fallback.
  • Forward state files can make NemoClaw kill unrelated processes (src/lib/adapters/openshell/forward-bridge-state.ts:145): stopForwardBridge() trusts JSON state under ~/.nemoclaw/forwards and sends SIGTERM/SIGKILL to the recorded PID without verifying a process marker, nonce, command line, start identity, or executable. A tampered or stale same-user state file can terminate an unrelated process after PID reuse, and destroy/recovery paths now call this cleanup automatically.
    • Recommendation: Write state atomically in a symlink-safe private directory, record a per-process nonce or marker plus start identity, and verify PID owner, start time, and command/marker before signaling. Add negative tests for tampered state files, symlinked paths, stale PID reuse, and foreign PIDs.
    • Evidence: readStateFile() accepts any JSON with integer pid; list/get only filter through isPidAlive(); stopForwardBridge() calls process.kill(state.pid, "SIGTERM") and then SIGKILL before deleting the state file. Current forward-bridge tests cover positive stop/release behavior, not tamper or PID-reuse cases.
  • Runtime validation and security-negative coverage are missing for the new transport boundary (src/lib/adapters/openshell/grpc-migration-guard.test.ts:54): The tests add useful fake gRPC and static migration-guard coverage, but this PR changes real sandbox infrastructure paths including gateway metadata, raw gRPC, ForwardTcp bridges, sync-runner subprocesses, installer cleanup, backup/restore, logs/watch, and lifecycle operations. Critical negative and contract paths are not covered.
    • Recommendation: Add or identify targeted runtime/integration validation for create/list/delete/wait, exec streaming and stdin, ForwardTcp lifecycle and token cleanup, logs/watch, proto packaging, installer post-check behavior, and backup/restore. Add negative tests for credential override, plaintext token endpoints, fake transport outside tests, and tampered/foreign PID forward state.
    • Evidence: grpc-migration-guard.test.ts scans source text for banned strings. direct-grpc.test.ts uses a local fake gRPC server. Gateway metadata tests cover happy paths, forward-state tests cover positive stop/release behavior, and grpc tests cover exec fallback but not lifecycle methods with the placeholder.
  • Migration adds new large sandbox lifecycle hotspots (src/lib/adapters/openshell/direct-grpc.ts:1): The PR adds large gRPC adapter modules and grows existing high-risk sandbox lifecycle files. This makes credential handling, forwarding, backup/restore, and lifecycle code harder to audit and conflicts with the repository's monolith-growth guardrails.
    • Recommendation: Split direct gRPC by concern, such as credential/client construction, exec, forwarding, logs/watch, and protobuf loading. Offset growth in state/sandbox.ts and dashboard helpers by extracting focused backup/restore/forwarding helpers before merge.
    • Evidence: Deterministic monolith analysis flagged src/lib/adapters/openshell/direct-grpc.ts at 666 new lines, src/lib/adapters/openshell/grpc.ts at 490 new lines, src/lib/state/sandbox.ts +126, src/lib/onboard/dashboard.ts +32, and src/lib/tunnel/services-sandbox.test.ts +36.

🔎 Worth checking

  • Source-of-truth review needed: SDK dependency and sandbox gRPC adapter: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: vendor/openshell-sdk-placeholder throws in OpenShellClient.connect(); grpc.ts lifecycle methods call this.sdk(); execBinary alone has direct gRPC fallback.
  • Source-of-truth review needed: Gateway metadata endpoint override: The advisor marked localized patch analysis as missing.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: resolveGatewayMetadata() selects endpointOverride and buildConnectOptions() still reads edge_token/oidc_token.json from gatewayDir.
  • Source-of-truth review needed: Forward bridge state cleanup: The advisor marked localized patch analysis as missing.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: stopForwardBridge() reads JSON state and kills state.pid without marker verification.
  • Source-of-truth review needed: Workflow residual capability tolerance: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: .github/workflows/e2e-script.yaml sets values.setdefault("NEMOCLAW_ALLOW_RESIDUAL_CAPS", "1") for all scripts.
  • Source-of-truth review needed: Process recovery gRPC exec null fallbacks: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: process-recovery.ts returns null when markerIndex is -1 and catches errors to return null.
  • Source-of-truth review needed: Policy channel durable-state cleanup fallback: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: policy-channel.ts tries executeSandboxExecCommand() and then executeSandboxCommand() for transient wrapper hiccups after both were migrated to gRPC helpers.
  • Source-of-truth review needed: Installer post-check forward restore removal: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: scripts/install.sh restore_onboard_forward_after_post_checks() removes old pid_file and comments that gRPC-only NemoClaw owns dashboard forwarding from the CLI.
  • Source-of-truth review needed: Device approval requestId retry workaround: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: scripts/nemoclaw-start.sh unsets OPENCLAW_GATEWAY_URL for devices approve, parses 'scope upgrade pending approval (requestId: ...)', and retries when output contains 'unknown requestId'.
  • Reusable E2E workflow globally enables residual-capability tolerance (.github/workflows/e2e-script.yaml:146): The workflow defaults NEMOCLAW_ALLOW_RESIDUAL_CAPS=1 for every called E2E script. That broad opt-in can mask fail-closed residual-capability regressions in sandbox hardening tests and lacks a scoped source-of-truth/removal condition.
    • Recommendation: Scope this opt-in only to scripts that require it, keep at least one default fail-closed residual-capability validation path, and document the removal condition or remove the variable when the runner capability gap is fixed.
    • Evidence: The Export script environment step runs values.setdefault("NEMOCLAW_ALLOW_RESIDUAL_CAPS", "1") for all scripts. Workflow permissions remain contents: read and checkout pins are good, but this default weakens security validation coverage.
  • Share mount is disabled but public docs and command metadata still advertise SSHFS mounts (src/lib/share-command.ts:181): The PR disables live share mount under the gRPC-only model, but public documentation, oclif metadata, public display defaults, generated skill references, and CLI selection guidance still describe SSHFS-based bidirectional mounts. Users following the documented command will hit the new runtime error.
    • Recommendation: Update public docs, generated command references, release notes, oclif summaries/descriptions, and public-display defaults to say live mounts are no longer supported, explain status/unmount for legacy mounts, and point to the replacement workflow.
    • Evidence: runShareMount() throws "Live sandbox filesystem mounts are no longer supported." Grep still finds SSHFS/live mount text in src/commands/sandbox/share.ts, src/commands/sandbox/share/mount.ts, src/lib/cli/public-display-defaults.ts, docs/reference/commands.mdx, docs/reference/cli-selection-guide.mdx, and generated skill references.
  • Production gRPC transport dependencies use floating ranges (package.json:51): The new production gRPC/protobuf dependencies use caret ranges even though they sit on the security-sensitive OpenShell transport boundary. Range drift can change parsing, TLS, or gRPC behavior without explicit source review.
    • Recommendation: Pin production transport dependencies to exact reviewed versions, or document why ranges are acceptable and ensure dependency update review explicitly covers this boundary.
    • Evidence: package.json adds "@grpc/grpc-js": "^1.14.4" and "@grpc/proto-loader": "^0.8.1" under dependencies.
  • gRPC exec recovery fallbacks collapse failures without caller contract coverage (src/lib/actions/sandbox/process-recovery.ts:167): Several recovery helpers return null when the gRPC exec marker is missing or when exec throws. That may be appropriate for status-only probes, but the source-of-truth rationale, removal condition, and caller contracts are not established across recovery, channel cleanup, installer cleanup, and startup workaround paths.
    • Recommendation: Document the invalid state each fallback handles, why the upstream/source boundary cannot be fixed in this PR, and when each workaround can be removed. Add caller/callee contract tests proving null/error outcomes do not produce false healthy status or silently skip required cleanup.
    • Evidence: executeSandboxExecCommand() returns null when markerIndex is -1 and catches errors to return null. policy-channel.ts tries executeSandboxExecCommand() and then executeSandboxCommand() for transient hiccups even though both now use gRPC helpers. scripts/nemoclaw-start.sh adds output-parsing retry behavior for device approval without a clear removal condition.

🌱 Nice ideas

  • None.
Since last review details

Current findings:

  • Source-of-truth review needed: SDK dependency and sandbox gRPC adapter: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: vendor/openshell-sdk-placeholder throws in OpenShellClient.connect(); grpc.ts lifecycle methods call this.sdk(); execBinary alone has direct gRPC fallback.
  • Source-of-truth review needed: Gateway metadata endpoint override: The advisor marked localized patch analysis as missing.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: resolveGatewayMetadata() selects endpointOverride and buildConnectOptions() still reads edge_token/oidc_token.json from gatewayDir.
  • Source-of-truth review needed: Forward bridge state cleanup: The advisor marked localized patch analysis as missing.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: stopForwardBridge() reads JSON state and kills state.pid without marker verification.
  • Source-of-truth review needed: Workflow residual capability tolerance: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: .github/workflows/e2e-script.yaml sets values.setdefault("NEMOCLAW_ALLOW_RESIDUAL_CAPS", "1") for all scripts.
  • Source-of-truth review needed: Process recovery gRPC exec null fallbacks: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: process-recovery.ts returns null when markerIndex is -1 and catches errors to return null.
  • Source-of-truth review needed: Policy channel durable-state cleanup fallback: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: policy-channel.ts tries executeSandboxExecCommand() and then executeSandboxCommand() for transient wrapper hiccups after both were migrated to gRPC helpers.
  • Source-of-truth review needed: Installer post-check forward restore removal: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: scripts/install.sh restore_onboard_forward_after_post_checks() removes old pid_file and comments that gRPC-only NemoClaw owns dashboard forwarding from the CLI.
  • Source-of-truth review needed: Device approval requestId retry workaround: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: scripts/nemoclaw-start.sh unsets OPENCLAW_GATEWAY_URL for devices approve, parses 'scope upgrade pending approval (requestId: ...)', and retries when output contains 'unknown requestId'.
  • Gateway tokens can be sent to overridden or plaintext endpoints (src/lib/adapters/openshell/gateway-metadata.ts:239): The metadata resolver accepts OPENSHELL_GATEWAY_ENDPOINT or OPENSHELL_GATEWAY_URL as the target while still deriving gatewayDir from the active or selected gateway and loading persisted OIDC or Cloudflare tokens. The raw gRPC client also uses insecure credentials for http:// endpoints while attaching bearer or Cloudflare metadata. A hostile or contaminated environment can redirect persisted gateway credentials to a different endpoint or transmit them without TLS.
    • Recommendation: When an endpoint override is present, do not attach persisted token material unless the override exactly matches the persisted gateway endpoint, or require explicit token material scoped to that override. Reject OIDC and Cloudflare token auth on plaintext HTTP except for a narrowly named test-only path. Add negative tests for override-plus-persisted-token and token-auth-over-http.
    • Evidence: resolveGatewayMetadata() selects endpointOverride from env/options, keeps gatewayDir from loaded gateway metadata, and buildConnectOptions() reads edge_token/oidc_token.json. direct-grpc.ts createCredentials() returns createInsecure() for http: and callCredentials() attaches authorization/cf-access-token/cookie metadata.
  • Fake SDK transport is enabled by ambient production environment (src/lib/adapters/openshell/sync-runner.ts:148): NEMOCLAW_SDK_TEST_TRANSPORT=1 switches the production sync runner into a fake transport that executes NEMOCLAW_SDK_TEST_FAKE_EXEC_BIN with the inherited production environment. The allowlist describes these variables as Vitest-only, but runtime enforcement is only an environment variable, so production sandbox exec paths can be redirected if the environment is contaminated.
    • Recommendation: Restrict fake transport to a verified test runtime such as VITEST_WORKER_ID or NODE_ENV=test, or move it behind test-only dependency injection. Avoid passing the full production environment to fake helpers. Add a negative test proving the env pair is ignored outside tests.
    • Evidence: sync-runner.ts checks process.env.NEMOCLAW_SDK_TEST_TRANSPORT === "1", reads NEMOCLAW_SDK_TEST_FAKE_EXEC_BIN, then spawnSyncs that helper with env: process.env.
  • Core lifecycle paths still depend on the throwing SDK placeholder (src/lib/adapters/openshell/grpc.ts:199): The PR says it migrates sandbox lifecycle operations to OpenShell gRPC, but the packaged dependency is a vendored @openshell/sdk placeholder whose OpenShellClient.connect() always throws. Lifecycle methods such as health, createSandbox, getSandbox, listSandboxes, deleteSandbox, waitReady, and waitDeleted still call the SDK directly; only exec has direct raw-gRPC fallback coverage.
    • Recommendation: Either land and pin the real OpenShell SDK before switching production lifecycle call sites, add direct-gRPC fallback coverage for lifecycle methods, or keep those call sites on the existing working transport until the SDK is available. Add a contract test that exercises lifecycle methods with the placeholder present.
    • Evidence: package.json adds "@openshell/sdk": "file:vendor/openshell-sdk-placeholder"; vendor/openshell-sdk-placeholder/index.js throws in OpenShellClient.connect(); grpc.ts lifecycle methods call await this.sdk(), while execBinary alone has direct gRPC fallback.
  • Forward state files can make NemoClaw kill unrelated processes (src/lib/adapters/openshell/forward-bridge-state.ts:145): stopForwardBridge() trusts JSON state under ~/.nemoclaw/forwards and sends SIGTERM/SIGKILL to the recorded PID without verifying a process marker, nonce, command line, start identity, or executable. A tampered or stale same-user state file can terminate an unrelated process after PID reuse, and destroy/recovery paths now call this cleanup automatically.
    • Recommendation: Write state atomically in a symlink-safe private directory, record a per-process nonce or marker plus start identity, and verify PID owner, start time, and command/marker before signaling. Add negative tests for tampered state files, symlinked paths, stale PID reuse, and foreign PIDs.
    • Evidence: readStateFile() accepts any JSON with integer pid; list/get only filter through isPidAlive(); stopForwardBridge() calls process.kill(state.pid, "SIGTERM") and then SIGKILL before deleting the state file. Current forward-bridge tests cover positive stop/release behavior, not tamper or PID-reuse cases.
  • Runtime validation and security-negative coverage are missing for the new transport boundary (src/lib/adapters/openshell/grpc-migration-guard.test.ts:54): The tests add useful fake gRPC and static migration-guard coverage, but this PR changes real sandbox infrastructure paths including gateway metadata, raw gRPC, ForwardTcp bridges, sync-runner subprocesses, installer cleanup, backup/restore, logs/watch, and lifecycle operations. Critical negative and contract paths are not covered.
    • Recommendation: Add or identify targeted runtime/integration validation for create/list/delete/wait, exec streaming and stdin, ForwardTcp lifecycle and token cleanup, logs/watch, proto packaging, installer post-check behavior, and backup/restore. Add negative tests for credential override, plaintext token endpoints, fake transport outside tests, and tampered/foreign PID forward state.
    • Evidence: grpc-migration-guard.test.ts scans source text for banned strings. direct-grpc.test.ts uses a local fake gRPC server. Gateway metadata tests cover happy paths, forward-state tests cover positive stop/release behavior, and grpc tests cover exec fallback but not lifecycle methods with the placeholder.
  • Migration adds new large sandbox lifecycle hotspots (src/lib/adapters/openshell/direct-grpc.ts:1): The PR adds large gRPC adapter modules and grows existing high-risk sandbox lifecycle files. This makes credential handling, forwarding, backup/restore, and lifecycle code harder to audit and conflicts with the repository's monolith-growth guardrails.
    • Recommendation: Split direct gRPC by concern, such as credential/client construction, exec, forwarding, logs/watch, and protobuf loading. Offset growth in state/sandbox.ts and dashboard helpers by extracting focused backup/restore/forwarding helpers before merge.
    • Evidence: Deterministic monolith analysis flagged src/lib/adapters/openshell/direct-grpc.ts at 666 new lines, src/lib/adapters/openshell/grpc.ts at 490 new lines, src/lib/state/sandbox.ts +126, src/lib/onboard/dashboard.ts +32, and src/lib/tunnel/services-sandbox.test.ts +36.
  • Reusable E2E workflow globally enables residual-capability tolerance (.github/workflows/e2e-script.yaml:146): The workflow defaults NEMOCLAW_ALLOW_RESIDUAL_CAPS=1 for every called E2E script. That broad opt-in can mask fail-closed residual-capability regressions in sandbox hardening tests and lacks a scoped source-of-truth/removal condition.
    • Recommendation: Scope this opt-in only to scripts that require it, keep at least one default fail-closed residual-capability validation path, and document the removal condition or remove the variable when the runner capability gap is fixed.
    • Evidence: The Export script environment step runs values.setdefault("NEMOCLAW_ALLOW_RESIDUAL_CAPS", "1") for all scripts. Workflow permissions remain contents: read and checkout pins are good, but this default weakens security validation coverage.
  • Share mount is disabled but public docs and command metadata still advertise SSHFS mounts (src/lib/share-command.ts:181): The PR disables live share mount under the gRPC-only model, but public documentation, oclif metadata, public display defaults, generated skill references, and CLI selection guidance still describe SSHFS-based bidirectional mounts. Users following the documented command will hit the new runtime error.
    • Recommendation: Update public docs, generated command references, release notes, oclif summaries/descriptions, and public-display defaults to say live mounts are no longer supported, explain status/unmount for legacy mounts, and point to the replacement workflow.
    • Evidence: runShareMount() throws "Live sandbox filesystem mounts are no longer supported." Grep still finds SSHFS/live mount text in src/commands/sandbox/share.ts, src/commands/sandbox/share/mount.ts, src/lib/cli/public-display-defaults.ts, docs/reference/commands.mdx, docs/reference/cli-selection-guide.mdx, and generated skill references.
  • Production gRPC transport dependencies use floating ranges (package.json:51): The new production gRPC/protobuf dependencies use caret ranges even though they sit on the security-sensitive OpenShell transport boundary. Range drift can change parsing, TLS, or gRPC behavior without explicit source review.
    • Recommendation: Pin production transport dependencies to exact reviewed versions, or document why ranges are acceptable and ensure dependency update review explicitly covers this boundary.
    • Evidence: package.json adds "@grpc/grpc-js": "^1.14.4" and "@grpc/proto-loader": "^0.8.1" under dependencies.
  • gRPC exec recovery fallbacks collapse failures without caller contract coverage (src/lib/actions/sandbox/process-recovery.ts:167): Several recovery helpers return null when the gRPC exec marker is missing or when exec throws. That may be appropriate for status-only probes, but the source-of-truth rationale, removal condition, and caller contracts are not established across recovery, channel cleanup, installer cleanup, and startup workaround paths.
    • Recommendation: Document the invalid state each fallback handles, why the upstream/source boundary cannot be fixed in this PR, and when each workaround can be removed. Add caller/callee contract tests proving null/error outcomes do not produce false healthy status or silently skip required cleanup.
    • Evidence: executeSandboxExecCommand() returns null when markerIndex is -1 and catches errors to return null. policy-channel.ts tries executeSandboxExecCommand() and then executeSandboxCommand() for transient hiccups even though both now use gRPC helpers. scripts/nemoclaw-start.sh adds output-parsing retry behavior for device approval without a clear removal condition.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26523535919
Target ref: 82c16b60c3f080041d56f8734c76e64ed555ad93
Workflow ref: main
Requested jobs: all (no filter)
Summary: 23 passed, 29 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ❌ failure
brave-search-e2e ✅ success
channels-add-remove-e2e ❌ failure
channels-stop-start-e2e ❌ failure
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ❌ failure
credential-sanitization-e2e ✅ success
device-auth-health-e2e ❌ failure
diagnostics-e2e ❌ failure
docs-validation-e2e ✅ success
double-onboard-e2e ❌ failure
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-discord-e2e ❌ failure
hermes-e2e ❌ failure
hermes-inference-switch-e2e ❌ failure
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ❌ failure
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ❌ failure
issue-3600-gpu-proof-optional-e2e ✅ success
kimi-inference-compat-e2e ❌ failure
launchable-smoke-e2e ✅ success
messaging-compatible-endpoint-e2e ❌ failure
messaging-providers-e2e ❌ failure
network-policy-e2e ❌ failure
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ❌ failure
onboard-resume-e2e ✅ success
openclaw-discord-pairing-e2e ❌ failure
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ✅ success
openclaw-slack-pairing-e2e ❌ failure
openclaw-tui-chat-correlation-e2e ❌ failure
openshell-gateway-upgrade-e2e ❌ failure
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ❌ failure
rebuild-hermes-stale-base-e2e ❌ failure
rebuild-openclaw-e2e ❌ failure
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ✅ success
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ❌ failure
snapshot-commands-e2e ❌ failure
state-backup-restore-e2e ✅ success
telegram-injection-e2e ✅ success
token-rotation-e2e ❌ failure
tunnel-lifecycle-e2e ❌ failure
upgrade-stale-sandbox-e2e ❌ failure
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: bedrock-runtime-compatible-anthropic-e2e, channels-add-remove-e2e, channels-stop-start-e2e, credential-migration-e2e, device-auth-health-e2e, diagnostics-e2e, double-onboard-e2e, hermes-discord-e2e, hermes-e2e, hermes-inference-switch-e2e, hermes-slack-e2e, issue-2478-crash-loop-recovery-e2e, kimi-inference-compat-e2e, messaging-compatible-endpoint-e2e, messaging-providers-e2e, network-policy-e2e, onboard-repair-e2e, openclaw-discord-pairing-e2e, openclaw-slack-pairing-e2e, openclaw-tui-chat-correlation-e2e, openshell-gateway-upgrade-e2e, rebuild-hermes-e2e, rebuild-hermes-stale-base-e2e, rebuild-openclaw-e2e, skill-agent-e2e, snapshot-commands-e2e, token-rotation-e2e, tunnel-lifecycle-e2e, upgrade-stale-sandbox-e2e. Check run artifacts for logs.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26527630225
Target ref: e8da94ee088c20c684ca8ef074c4a25218dd4a65
Workflow ref: feat/openshell-grpc-sandbox-ops
Requested jobs: all (no filter)
Summary: 11 passed, 6 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ❌ failure
brave-search-e2e ✅ success
channels-add-remove-e2e ⚠️ cancelled
channels-stop-start-e2e ⚠️ cancelled
cloud-e2e ⚠️ cancelled
cloud-inference-e2e ⚠️ cancelled
cloud-onboard-e2e ✅ success
credential-migration-e2e ⚠️ cancelled
credential-sanitization-e2e ⚠️ cancelled
device-auth-health-e2e ⚠️ cancelled
diagnostics-e2e ⚠️ cancelled
docs-validation-e2e ⚠️ cancelled
double-onboard-e2e ⚠️ cancelled
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ❌ failure
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ⚠️ cancelled
issue-2478-crash-loop-recovery-e2e ⚠️ cancelled
issue-3600-gpu-proof-optional-e2e ✅ success
kimi-inference-compat-e2e ⚠️ cancelled
launchable-smoke-e2e ⚠️ cancelled
messaging-compatible-endpoint-e2e ⚠️ cancelled
messaging-providers-e2e ⚠️ cancelled
network-policy-e2e ⚠️ cancelled
onboard-negative-paths-e2e ⚠️ cancelled
onboard-repair-e2e ⚠️ cancelled
onboard-resume-e2e ⚠️ cancelled
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ⚠️ cancelled
openclaw-onboard-security-posture-e2e ⚠️ cancelled
openclaw-slack-pairing-e2e ⚠️ cancelled
openclaw-tui-chat-correlation-e2e ⚠️ cancelled
openshell-gateway-upgrade-e2e ⚠️ cancelled
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ❌ failure
rebuild-hermes-stale-base-e2e ❌ failure
rebuild-openclaw-e2e ⚠️ cancelled
runtime-overrides-e2e ⚠️ cancelled
sandbox-operations-e2e ❌ failure
sandbox-survival-e2e ⚠️ cancelled
shields-config-e2e ❌ failure
skill-agent-e2e ⚠️ cancelled
snapshot-commands-e2e ⚠️ cancelled
state-backup-restore-e2e ⚠️ cancelled
telegram-injection-e2e ⚠️ cancelled
token-rotation-e2e ⚠️ cancelled
tunnel-lifecycle-e2e ⚠️ cancelled
upgrade-stale-sandbox-e2e ⚠️ cancelled
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: bedrock-runtime-compatible-anthropic-e2e, hermes-onboard-security-posture-e2e, rebuild-hermes-e2e, rebuild-hermes-stale-base-e2e, sandbox-operations-e2e, shields-config-e2e. Check run artifacts for logs.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26527923535
Target ref: cafb69e78131174df78270e8ba65415bc3c66003
Workflow ref: feat/openshell-grpc-sandbox-ops
Requested jobs: all (no filter)
Summary: 26 passed, 23 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ❌ failure
brave-search-e2e ✅ success
channels-add-remove-e2e ❌ failure
channels-stop-start-e2e ❌ failure
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ❌ failure
credential-sanitization-e2e ✅ success
device-auth-health-e2e ✅ success
diagnostics-e2e ❌ failure
docs-validation-e2e ✅ success
double-onboard-e2e ❌ failure
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-discord-e2e ✅ success
hermes-e2e ❌ failure
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ❌ failure
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ❌ failure
issue-3600-gpu-proof-optional-e2e ✅ success
kimi-inference-compat-e2e ❌ failure
launchable-smoke-e2e ❌ failure
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ❌ failure
network-policy-e2e ❌ failure
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ⚠️ cancelled
onboard-resume-e2e ✅ success
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ❌ failure
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ❌ failure
openshell-gateway-upgrade-e2e ⚠️ cancelled
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ❌ failure
rebuild-hermes-stale-base-e2e ❌ failure
rebuild-openclaw-e2e ❌ failure
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ⚠️ cancelled
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ❌ failure
snapshot-commands-e2e ❌ failure
state-backup-restore-e2e ✅ success
telegram-injection-e2e ✅ success
token-rotation-e2e ❌ failure
tunnel-lifecycle-e2e ❌ failure
upgrade-stale-sandbox-e2e ❌ failure
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: bedrock-runtime-compatible-anthropic-e2e, channels-add-remove-e2e, channels-stop-start-e2e, credential-migration-e2e, diagnostics-e2e, double-onboard-e2e, hermes-e2e, hermes-onboard-security-posture-e2e, issue-2478-crash-loop-recovery-e2e, kimi-inference-compat-e2e, launchable-smoke-e2e, messaging-providers-e2e, network-policy-e2e, openclaw-onboard-security-posture-e2e, openclaw-tui-chat-correlation-e2e, rebuild-hermes-e2e, rebuild-hermes-stale-base-e2e, rebuild-openclaw-e2e, skill-agent-e2e, snapshot-commands-e2e, token-rotation-e2e, tunnel-lifecycle-e2e, upgrade-stale-sandbox-e2e. Check run artifacts for logs.

Comment thread src/lib/actions/sandbox/process-recovery.ts Fixed
@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26528706357
Target ref: 7ea65a1a7da1d215954dc8cc1ce61b5af8de76b9
Workflow ref: feat/openshell-grpc-sandbox-ops
Requested jobs: all (no filter)
Summary: 20 passed, 16 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ❌ failure
brave-search-e2e ✅ success
channels-add-remove-e2e ❌ failure
channels-stop-start-e2e ❌ failure
cloud-e2e ❌ failure
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ❌ failure
credential-sanitization-e2e ✅ success
device-auth-health-e2e ✅ success
diagnostics-e2e ❌ failure
docs-validation-e2e ⚠️ cancelled
double-onboard-e2e ⚠️ cancelled
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ❌ failure
issue-3600-gpu-proof-optional-e2e ✅ success
kimi-inference-compat-e2e ❌ failure
launchable-smoke-e2e ❌ failure
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ❌ failure
network-policy-e2e ⚠️ cancelled
onboard-negative-paths-e2e ⚠️ cancelled
onboard-repair-e2e ⚠️ cancelled
onboard-resume-e2e ⚠️ cancelled
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ⚠️ cancelled
openclaw-onboard-security-posture-e2e ❌ failure
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ⚠️ cancelled
openshell-gateway-upgrade-e2e ⚠️ cancelled
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ❌ failure
rebuild-hermes-stale-base-e2e ❌ failure
rebuild-openclaw-e2e ⚠️ cancelled
runtime-overrides-e2e ⚠️ cancelled
sandbox-operations-e2e ⚠️ cancelled
sandbox-survival-e2e ⚠️ cancelled
shields-config-e2e ✅ success
skill-agent-e2e ⚠️ cancelled
snapshot-commands-e2e ❌ failure
state-backup-restore-e2e ⚠️ cancelled
telegram-injection-e2e ✅ success
token-rotation-e2e ⚠️ cancelled
tunnel-lifecycle-e2e ❌ failure
upgrade-stale-sandbox-e2e ❌ failure
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: bedrock-runtime-compatible-anthropic-e2e, channels-add-remove-e2e, channels-stop-start-e2e, cloud-e2e, credential-migration-e2e, diagnostics-e2e, issue-2478-crash-loop-recovery-e2e, kimi-inference-compat-e2e, launchable-smoke-e2e, messaging-providers-e2e, openclaw-onboard-security-posture-e2e, rebuild-hermes-e2e, rebuild-hermes-stale-base-e2e, snapshot-commands-e2e, tunnel-lifecycle-e2e, upgrade-stale-sandbox-e2e. Check run artifacts for logs.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26531854119
Target ref: e17f7905e9254bea3fdeb4de0972e3173182d000
Workflow ref: feat/openshell-grpc-sandbox-ops
Requested jobs: all (no filter)
Summary: 30 passed, 22 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ❌ failure
brave-search-e2e ✅ success
channels-add-remove-e2e ❌ failure
channels-stop-start-e2e ❌ failure
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ❌ failure
credential-sanitization-e2e ✅ success
device-auth-health-e2e ❌ failure
diagnostics-e2e ❌ failure
docs-validation-e2e ✅ success
double-onboard-e2e ❌ failure
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ❌ failure
issue-3600-gpu-proof-optional-e2e ✅ success
kimi-inference-compat-e2e ❌ failure
launchable-smoke-e2e ✅ success
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ❌ failure
network-policy-e2e ❌ failure
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ❌ failure
onboard-resume-e2e ✅ success
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ❌ failure
openshell-gateway-upgrade-e2e ❌ failure
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ❌ failure
rebuild-hermes-stale-base-e2e ❌ failure
rebuild-openclaw-e2e ❌ failure
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ✅ success
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ❌ failure
snapshot-commands-e2e ❌ failure
state-backup-restore-e2e ✅ success
telegram-injection-e2e ✅ success
token-rotation-e2e ❌ failure
tunnel-lifecycle-e2e ❌ failure
upgrade-stale-sandbox-e2e ❌ failure
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: bedrock-runtime-compatible-anthropic-e2e, channels-add-remove-e2e, channels-stop-start-e2e, credential-migration-e2e, device-auth-health-e2e, diagnostics-e2e, double-onboard-e2e, issue-2478-crash-loop-recovery-e2e, kimi-inference-compat-e2e, messaging-providers-e2e, network-policy-e2e, onboard-repair-e2e, openclaw-tui-chat-correlation-e2e, openshell-gateway-upgrade-e2e, rebuild-hermes-e2e, rebuild-hermes-stale-base-e2e, rebuild-openclaw-e2e, skill-agent-e2e, snapshot-commands-e2e, token-rotation-e2e, tunnel-lifecycle-e2e, upgrade-stale-sandbox-e2e. Check run artifacts for logs.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26537255180
Target ref: ffc3fdcd6e2f8f32d2a868053f5c350a0c1569ac
Workflow ref: feat/openshell-grpc-sandbox-ops
Requested jobs: all (no filter)
Summary: 36 passed, 16 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ✅ success
brave-search-e2e ✅ success
channels-add-remove-e2e ❌ failure
channels-stop-start-e2e ❌ failure
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ❌ failure
credential-sanitization-e2e ✅ success
device-auth-health-e2e ✅ success
diagnostics-e2e ❌ failure
docs-validation-e2e ✅ success
double-onboard-e2e ❌ failure
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ❌ failure
issue-3600-gpu-proof-optional-e2e ✅ success
kimi-inference-compat-e2e ✅ success
launchable-smoke-e2e ✅ success
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ❌ failure
network-policy-e2e ✅ success
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ❌ failure
onboard-resume-e2e ✅ success
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ✅ success
openshell-gateway-upgrade-e2e ❌ failure
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ❌ failure
rebuild-hermes-stale-base-e2e ❌ failure
rebuild-openclaw-e2e ❌ failure
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ✅ success
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ✅ success
snapshot-commands-e2e ❌ failure
state-backup-restore-e2e ✅ success
telegram-injection-e2e ✅ success
token-rotation-e2e ❌ failure
tunnel-lifecycle-e2e ❌ failure
upgrade-stale-sandbox-e2e ❌ failure
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: channels-add-remove-e2e, channels-stop-start-e2e, credential-migration-e2e, diagnostics-e2e, double-onboard-e2e, issue-2478-crash-loop-recovery-e2e, messaging-providers-e2e, onboard-repair-e2e, openshell-gateway-upgrade-e2e, rebuild-hermes-e2e, rebuild-hermes-stale-base-e2e, rebuild-openclaw-e2e, snapshot-commands-e2e, token-rotation-e2e, tunnel-lifecycle-e2e, upgrade-stale-sandbox-e2e. Check run artifacts for logs.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26541403310
Target ref: 347c995b1423d4e67c9f92a91fc9a2cec1016dd8
Workflow ref: main
Requested jobs: all (no filter)
Summary: 36 passed, 14 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ✅ success
brave-search-e2e ✅ success
channels-add-remove-e2e ❌ failure
channels-stop-start-e2e ❌ failure
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ❌ failure
credential-sanitization-e2e ✅ success
device-auth-health-e2e ✅ success
diagnostics-e2e ❌ failure
docs-validation-e2e ✅ success
double-onboard-e2e ❌ failure
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ❌ failure
issue-3600-gpu-proof-optional-e2e ✅ success
kimi-inference-compat-e2e ✅ success
launchable-smoke-e2e ✅ success
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ❌ failure
network-policy-e2e ✅ success
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ❌ failure
onboard-resume-e2e ✅ success
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ✅ success
openshell-gateway-upgrade-e2e ⚠️ cancelled
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ❌ failure
rebuild-hermes-stale-base-e2e ❌ failure
rebuild-openclaw-e2e ⚠️ cancelled
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ✅ success
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ✅ success
snapshot-commands-e2e ❌ failure
state-backup-restore-e2e ✅ success
telegram-injection-e2e ✅ success
token-rotation-e2e ❌ failure
tunnel-lifecycle-e2e ❌ failure
upgrade-stale-sandbox-e2e ❌ failure
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: channels-add-remove-e2e, channels-stop-start-e2e, credential-migration-e2e, diagnostics-e2e, double-onboard-e2e, issue-2478-crash-loop-recovery-e2e, messaging-providers-e2e, onboard-repair-e2e, rebuild-hermes-e2e, rebuild-hermes-stale-base-e2e, snapshot-commands-e2e, token-rotation-e2e, tunnel-lifecycle-e2e, upgrade-stale-sandbox-e2e. Check run artifacts for logs.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26545970853
Target ref: feat/openshell-grpc-sandbox-ops
Workflow ref: main
Requested jobs: all (no filter)
Summary: 51 passed, 1 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ✅ success
brave-search-e2e ✅ success
channels-add-remove-e2e ✅ success
channels-stop-start-e2e ✅ success
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ✅ success
credential-sanitization-e2e ✅ success
device-auth-health-e2e ✅ success
diagnostics-e2e ✅ success
docs-validation-e2e ✅ success
double-onboard-e2e ❌ failure
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ✅ success
issue-3600-gpu-proof-optional-e2e ✅ success
kimi-inference-compat-e2e ✅ success
launchable-smoke-e2e ✅ success
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ✅ success
network-policy-e2e ✅ success
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ✅ success
onboard-resume-e2e ✅ success
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ✅ success
openshell-gateway-upgrade-e2e ✅ success
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ✅ success
rebuild-hermes-stale-base-e2e ✅ success
rebuild-openclaw-e2e ✅ success
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ✅ success
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ✅ success
snapshot-commands-e2e ✅ success
state-backup-restore-e2e ✅ success
telegram-injection-e2e ✅ success
token-rotation-e2e ✅ success
tunnel-lifecycle-e2e ✅ success
upgrade-stale-sandbox-e2e ✅ success
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: double-onboard-e2e. Check run artifacts for logs.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26547205887
Target ref: feat/openshell-grpc-sandbox-ops
Workflow ref: main
Requested jobs: double-onboard-e2e
Summary: 0 passed, 0 failed, 0 skipped

Job Result
double-onboard-e2e ⚠️ cancelled

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26547328935
Target ref: feat/openshell-grpc-sandbox-ops
Workflow ref: main
Requested jobs: double-onboard-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job Result
double-onboard-e2e ✅ success

Comment thread src/lib/adapters/openshell/grpc.ts Fixed
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/lib/state/sandbox.ts (1)

865-875: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Don’t let cat mask a failed SQLite backup.

Line 873 runs the SQLite backup and then unconditionally cats the temp file. Because this command list is joined with ;, a Python failure can still end with cat -- "$tmp" returning 0, which makes the snapshot look successful while persisting an empty/corrupt state-file backup.

🐛 Proposed fix
   return [
     `src=${quotedRemotePath}`,
     "[ ! -e \"$src\" ] && exit 2",
     '[ -f "$src" ] && [ ! -L "$src" ] || { echo "unsafe sqlite state file: $src" >&2; exit 10; }',
     'hardlink_count="$(find "$src" -maxdepth 0 -type f -links +1 -print 2>/dev/null | wc -l | tr -d " ")"',
     '[ "${hardlink_count:-0}" = "0" ] || { echo "hard-linked sqlite state file rejected: $src" >&2; exit 11; }',
     'tmp="$(mktemp /tmp/nemoclaw-sqlite-backup.XXXXXX)"',
     'trap \'rm -f "$tmp"\' EXIT',
-    `${pythonCommand(SQLITE_BACKUP_PY)} "$src" "$tmp"`,
-    'cat -- "$tmp"',
+    `${pythonCommand(SQLITE_BACKUP_PY)} "$src" "$tmp" && cat -- "$tmp"`,
   ].join("; ");
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/state/sandbox.ts` around lines 865 - 875, The current command array
in the function that builds the shell snippet runs
`${pythonCommand(SQLITE_BACKUP_PY)} "$src" "$tmp"` followed by an unconditional
'cat -- "$tmp"', so a failing Python backup can be masked; change the sequence
so the cat only runs on successful backup (for example, chain the backup and the
cat with a conditional AND or explicitly check the backup exit status and exit
on failure) while keeping the existing safety checks (the quotedRemotePath
check, hardlink check, mktemp/trap). Ensure you modify the element containing
`${pythonCommand(SQLITE_BACKUP_PY)} "$src" "$tmp"` and the 'cat -- "$tmp"'
element so that 'cat' is executed only when the backup command succeeded.
🧹 Nitpick comments (1)
src/lib/state/sandbox.ts (1)

1005-1641: Please run the state lifecycle E2Es on this migration path.

This file now routes backup/restore through gRPC exec, tar streaming, and marker parsing, so it is worth exercising state-backup-restore-e2e, snapshot-commands-e2e, and rebuild-openclaw-e2e before merge.

As per coding guidelines, src/lib/state/sandbox.ts: "This file manages sandbox state (backup, restore, rebuild, snapshot). Changes affect data persistence across sandbox lifecycle operations."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/state/sandbox.ts` around lines 1005 - 1641, The review asks you to
execute the full state lifecycle end-to-end tests against the new
gRPC/tar/marker backup and restore flow: run state-backup-restore-e2e,
snapshot-commands-e2e, and rebuild-openclaw-e2e and verify backupSandboxState
and restoreSandboxState behavior (including marker parsing, tar streaming,
pre-backup audit, sanitizeBackupDirectory, and workspace-* discovery) on the
intended migration path; if failures appear, iterate on backupSandboxState,
restoreSandboxState, safeTarExtract, statusFromMarker,
execBinaryStreamSync/execTextSync wrappers, and related helper functions until
the E2E suites pass.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/adapters/openshell/forward-bridge-state.ts`:
- Around line 39-41: The statePath function is vulnerable because String(port)
can contain path separators or traversal sequences; update statePath to sanitize
the port similarly to safeSandbox (or validate as a numeric port) before
composing the filename: convert port to string, strip or replace any characters
that are not digits/letters/._- (e.g. replace /[^A-Za-z0-9._-]/g with "_") or
enforce a numeric check (parseInt/Number and throw on NaN) so the returned path
(stateDir() + `${safeSandbox}-${safePort}.json`) cannot escape the intended
directory; modify the function statePath to produce safePort and use it in the
path join.

In `@src/lib/onboard.ts`:
- Around line 2795-2796: forwardStatesAsListOutput() only returns
NemoClaw-managed forwards so findAvailableDashboardPort(...) can pick a port
already taken by legacy/manual forwards; update the pre-resolution to include
legacy/live forwards or probe the candidate port before finalizing
effectivePort: call or implement a legacy forward listing (e.g. merge results of
forwardStatesAsListOutput() with a new listLegacyForwards() or
openshellForwardProbe()) and pass the combined set into
findAvailableDashboardPort(sandboxName, preferredPort, combinedForwards), or
after findAvailableDashboardPort returns, perform an explicit port probe on
effectivePort and, if occupied, retry selection; ensure this change happens
before ensureDashboardForward() (so CHAT_UI_URL / NEMOCLAW_DASHBOARD_PORT baked
into the staged Dockerfile/sandbox env reflect a truly available port).

In `@test/helpers/grpc-fake-ssh.cjs`:
- Around line 37-39: The current hermetic binary check uses
path.resolve(openshell) which doesn't resolve symlinks; replace that check by
calling fs.realpathSync or fs.promises.realpath on both openshell and home
(e.g., realShell = fs.realpathSync(openshell); realHome = fs.realpathSync(home))
and then verify realShell.startsWith(`${realHome}${path.sep}`); also wrap
realpath in a try/catch so failures log the same stderr message and call
process.exit(127) as before, keeping the existing variables (openshell, home)
and the failure branch that writes the error and exits.

---

Outside diff comments:
In `@src/lib/state/sandbox.ts`:
- Around line 865-875: The current command array in the function that builds the
shell snippet runs `${pythonCommand(SQLITE_BACKUP_PY)} "$src" "$tmp"` followed
by an unconditional 'cat -- "$tmp"', so a failing Python backup can be masked;
change the sequence so the cat only runs on successful backup (for example,
chain the backup and the cat with a conditional AND or explicitly check the
backup exit status and exit on failure) while keeping the existing safety checks
(the quotedRemotePath check, hardlink check, mktemp/trap). Ensure you modify the
element containing `${pythonCommand(SQLITE_BACKUP_PY)} "$src" "$tmp"` and the
'cat -- "$tmp"' element so that 'cat' is executed only when the backup command
succeeded.

---

Nitpick comments:
In `@src/lib/state/sandbox.ts`:
- Around line 1005-1641: The review asks you to execute the full state lifecycle
end-to-end tests against the new gRPC/tar/marker backup and restore flow: run
state-backup-restore-e2e, snapshot-commands-e2e, and rebuild-openclaw-e2e and
verify backupSandboxState and restoreSandboxState behavior (including marker
parsing, tar streaming, pre-backup audit, sanitizeBackupDirectory, and
workspace-* discovery) on the intended migration path; if failures appear,
iterate on backupSandboxState, restoreSandboxState, safeTarExtract,
statusFromMarker, execBinaryStreamSync/execTextSync wrappers, and related helper
functions until the E2E suites pass.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 15ce826e-3c02-479c-b326-9b51e2257183

📥 Commits

Reviewing files that changed from the base of the PR and between 85580ae and 19a0519.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (65)
  • .github/workflows/e2e-script.yaml
  • ci/env-var-doc-allowlist.json
  • package.json
  • scripts/copy-openshell-protos.mjs
  • scripts/install.sh
  • src/lib/actions/sandbox/connect.ts
  • src/lib/actions/sandbox/destroy.ts
  • src/lib/actions/sandbox/policy-channel.ts
  • src/lib/actions/sandbox/process-recovery.ts
  • src/lib/actions/sandbox/skill-install.ts
  • src/lib/adapters/openshell/forward-bridge-runner.ts
  • src/lib/adapters/openshell/forward-bridge-state.test.ts
  • src/lib/adapters/openshell/forward-bridge-state.ts
  • src/lib/adapters/openshell/gateway-metadata.test.ts
  • src/lib/adapters/openshell/gateway-metadata.ts
  • src/lib/adapters/openshell/grpc-migration-guard.test.ts
  • src/lib/adapters/openshell/grpc.test.ts
  • src/lib/adapters/openshell/grpc.ts
  • src/lib/adapters/openshell/proto/datamodel.proto
  • src/lib/adapters/openshell/proto/openshell.proto
  • src/lib/adapters/openshell/sync-runner.ts
  • src/lib/adapters/openshell/timeouts.ts
  • src/lib/onboard.ts
  • src/lib/onboard/dashboard-access.test.ts
  • src/lib/onboard/dashboard-access.ts
  • src/lib/onboard/dashboard.ts
  • src/lib/onboard/forward-cleanup.test.ts
  • src/lib/onboard/forward-cleanup.ts
  • src/lib/onboard/forward-start.test.ts
  • src/lib/onboard/forward-start.ts
  • src/lib/onboard/preflight-port-recovery.test.ts
  • src/lib/onboard/preflight-port-recovery.ts
  • src/lib/onboard/sandbox-verification-exec.ts
  • src/lib/onboard/stale-gateway-cleanup.test.ts
  • src/lib/onboard/stale-gateway-cleanup.ts
  • src/lib/sandbox/version.test.ts
  • src/lib/sandbox/version.ts
  • src/lib/share-command-deps.ts
  • src/lib/share-command.test.ts
  • src/lib/share-command.ts
  • src/lib/skill-install.ts
  • src/lib/state/sandbox.ts
  • src/lib/status-command-deps.test.ts
  • src/lib/status-command-deps.ts
  • src/lib/tunnel/services-sandbox.test.ts
  • src/lib/tunnel/services.ts
  • src/lib/verify-deployment.ts
  • test/cli.test.ts
  • test/destroy-cleanup-sandbox-services.test.ts
  • test/e2e/test-double-onboard.sh
  • test/helpers/grpc-fake-ssh.cjs
  • test/install-preflight.test.ts
  • test/nemoclaw-cli-recovery.test.ts
  • test/onboard-dashboard.test.ts
  • test/onboard.test.ts
  • test/rebuild-credential-preflight.test.ts
  • test/rebuild-shields-auto-unlock.test.ts
  • test/recover-port-forward.test.ts
  • test/repro-2201.test.ts
  • test/sandbox-connect-inference.test.ts
  • test/share-command-deps-probe-argv.test.ts
  • test/share-command-remote-path.test.ts
  • test/shellquote-sandbox.test.ts
  • test/snapshot-gateway-guard.test.ts
  • test/snapshot.test.ts
💤 Files with no reviewable changes (6)
  • test/share-command-remote-path.test.ts
  • src/lib/onboard/forward-start.test.ts
  • src/lib/onboard/dashboard-access.test.ts
  • test/e2e/test-double-onboard.sh
  • src/lib/onboard/forward-start.ts
  • src/lib/onboard/dashboard-access.ts

Comment on lines +39 to +41
function statePath(sandboxName: string, port: number | string): string {
const safeSandbox = sandboxName.replace(/[^A-Za-z0-9._-]/g, "_");
return path.join(stateDir(), `${safeSandbox}-${String(port)}.json`);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Sanitize port before composing the state-file path.

At Line 41, String(port) is injected directly into a path segment. A string with / or .. can escape the intended ~/.nemoclaw/forwards scope for read/write/unlink operations.

Suggested fix
+function normalizePortSegment(port: number | string): string {
+  const parsed =
+    typeof port === "number" ? port : Number.parseInt(String(port).trim(), 10);
+  if (!Number.isInteger(parsed) || parsed < 1 || parsed > 65_535) {
+    throw new Error(`Invalid forward port '${String(port)}'`);
+  }
+  return String(parsed);
+}
+
 function statePath(sandboxName: string, port: number | string): string {
   const safeSandbox = sandboxName.replace(/[^A-Za-z0-9._-]/g, "_");
-  return path.join(stateDir(), `${safeSandbox}-${String(port)}.json`);
+  return path.join(stateDir(), `${safeSandbox}-${normalizePortSegment(port)}.json`);
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
function statePath(sandboxName: string, port: number | string): string {
const safeSandbox = sandboxName.replace(/[^A-Za-z0-9._-]/g, "_");
return path.join(stateDir(), `${safeSandbox}-${String(port)}.json`);
function normalizePortSegment(port: number | string): string {
const parsed =
typeof port === "number" ? port : Number.parseInt(String(port).trim(), 10);
if (!Number.isInteger(parsed) || parsed < 1 || parsed > 65_535) {
throw new Error(`Invalid forward port '${String(port)}'`);
}
return String(parsed);
}
function statePath(sandboxName: string, port: number | string): string {
const safeSandbox = sandboxName.replace(/[^A-Za-z0-9._-]/g, "_");
return path.join(stateDir(), `${safeSandbox}-${normalizePortSegment(port)}.json`);
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/adapters/openshell/forward-bridge-state.ts` around lines 39 - 41, The
statePath function is vulnerable because String(port) can contain path
separators or traversal sequences; update statePath to sanitize the port
similarly to safeSandbox (or validate as a numeric port) before composing the
filename: convert port to string, strip or replace any characters that are not
digits/letters/._- (e.g. replace /[^A-Za-z0-9._-]/g with "_") or enforce a
numeric check (parseInt/Number and throw on NaN) so the returned path
(stateDir() + `${safeSandbox}-${safePort}.json`) cannot escape the intended
directory; modify the function statePath to produce safePort and use it in the
path join.

Comment on lines +145 to +161
export function stopForwardBridge(sandboxName: string, port: number | string): boolean {
const state = readStateFile(statePath(sandboxName, port));
if (state && isPidAlive(state.pid) && !isTestForwardPid(state.pid)) {
try {
process.kill(state.pid, "SIGTERM");
} catch {
/* ignore */
}
if (!waitForPidExit(state.pid)) {
try {
process.kill(state.pid, "SIGKILL");
} catch {
/* ignore */
}
waitForPidExit(state.pid, 1_000);
}
waitForPortRelease(state.bind, state.port);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Avoid PID-only termination for forward bridge cleanup.

At Lines 147-156, termination is driven by PID liveness only. With stale state + PID reuse, this can kill an unrelated process.

Suggested hardening
 export function stopForwardBridge(sandboxName: string, port: number | string): boolean {
   const state = readStateFile(statePath(sandboxName, port));
-  if (state && isPidAlive(state.pid) && !isTestForwardPid(state.pid)) {
+  if (
+    state &&
+    isPidAlive(state.pid) &&
+    !isTestForwardPid(state.pid) &&
+    !canBindForwardPort(state.bind, state.port)
+  ) {
     try {
       process.kill(state.pid, "SIGTERM");
     } catch {
       /* ignore */
     }

Comment thread src/lib/onboard.ts
Comment on lines +2795 to 2796
const earlyForwards = forwardStatesAsListOutput();
const effectivePort = findAvailableDashboardPort(sandboxName, preferredPort, earlyForwards);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Keep legacy/manual forwards visible to the early port allocator.

forwardStatesAsListOutput() only reflects NemoClaw-managed forward-bridge state. That drops any still-live legacy openshell forward entry or manually-started forward, so effectivePort can be chosen against a port that's already occupied. ensureDashboardForward() can recover later, but by then CHAT_UI_URL / NEMOCLAW_DASHBOARD_PORT have already been baked into the staged Dockerfile and sandbox env, which defeats the purpose of this pre-resolution block. Please merge legacy/live forward data into this input, or probe the candidate port directly before finalizing effectivePort.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard.ts` around lines 2795 - 2796, forwardStatesAsListOutput()
only returns NemoClaw-managed forwards so findAvailableDashboardPort(...) can
pick a port already taken by legacy/manual forwards; update the pre-resolution
to include legacy/live forwards or probe the candidate port before finalizing
effectivePort: call or implement a legacy forward listing (e.g. merge results of
forwardStatesAsListOutput() with a new listLegacyForwards() or
openshellForwardProbe()) and pass the combined set into
findAvailableDashboardPort(sandboxName, preferredPort, combinedForwards), or
after findAvailableDashboardPort returns, perform an explicit port probe on
effectivePort and, if occupied, retry selection; ensure this change happens
before ensureDashboardForward() (so CHAT_UI_URL / NEMOCLAW_DASHBOARD_PORT baked
into the staged Dockerfile/sandbox env reflect a truly available port).

Comment thread test/helpers/grpc-fake-ssh.cjs Outdated
Comment on lines +37 to +39
if (!openshell || !home || !path.resolve(openshell).startsWith(`${home}${path.sep}`)) {
process.stderr.write("grpc fake transport could not find the hermetic fake openshell under HOME\n");
process.exit(127);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Harden the hermetic binary check against symlink escapes.

path.resolve() only normalizes the path string; it does not resolve symlinks. A symlink like $HOME/bin/openshell -> /usr/bin/openshell passes this check and can run a non-hermetic binary.

🔧 Proposed fix
 const openshell = findFakeOpenshell();
 const home = process.env.HOME ? path.resolve(process.env.HOME) : "";
-if (!openshell || !home || !path.resolve(openshell).startsWith(`${home}${path.sep}`)) {
+let realHome = "";
+let realOpenshell = "";
+try {
+  realHome = home ? fs.realpathSync(home) : "";
+  realOpenshell = openshell ? fs.realpathSync(openshell) : "";
+} catch {}
+if (!realOpenshell || !realHome || !realOpenshell.startsWith(`${realHome}${path.sep}`)) {
   process.stderr.write("grpc fake transport could not find the hermetic fake openshell under HOME\n");
   process.exit(127);
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/helpers/grpc-fake-ssh.cjs` around lines 37 - 39, The current hermetic
binary check uses path.resolve(openshell) which doesn't resolve symlinks;
replace that check by calling fs.realpathSync or fs.promises.realpath on both
openshell and home (e.g., realShell = fs.realpathSync(openshell); realHome =
fs.realpathSync(home)) and then verify
realShell.startsWith(`${realHome}${path.sep}`); also wrap realpath in a
try/catch so failures log the same stderr message and call process.exit(127) as
before, keeping the existing variables (openshell, home) and the failure branch
that writes the error and exits.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26721427994
Target ref: 46ec86c4d6d65b7b84a65301cf2a709de18c1ebf
Workflow ref: main
Requested jobs: all (no filter)
Summary: 5 passed, 50 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ❌ failure
brave-search-e2e ✅ success
channels-add-remove-e2e ❌ failure
channels-stop-start-e2e ❌ failure
cloud-e2e ❌ failure
cloud-inference-e2e ❌ failure
cloud-onboard-e2e ❌ failure
credential-migration-e2e ❌ failure
credential-sanitization-e2e ❌ failure
device-auth-health-e2e ❌ failure
diagnostics-e2e ❌ failure
docs-validation-e2e ❌ failure
double-onboard-e2e ❌ failure
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-dashboard-e2e ❌ failure
hermes-discord-e2e ❌ failure
hermes-e2e ❌ failure
hermes-inference-switch-e2e ❌ failure
hermes-onboard-security-posture-e2e ❌ failure
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ❌ failure
inference-routing-e2e ❌ failure
issue-2478-crash-loop-recovery-e2e ❌ failure
issue-3600-gpu-proof-optional-e2e ✅ success
issue-4462-gateway-pinned-approval-characterization-e2e ❌ failure
issue-4462-scope-upgrade-approval-e2e ❌ failure
kimi-inference-compat-e2e ❌ failure
launchable-smoke-e2e ❌ failure
messaging-compatible-endpoint-e2e ❌ failure
messaging-providers-e2e ❌ failure
network-policy-e2e ❌ failure
onboard-negative-paths-e2e ❌ failure
onboard-repair-e2e ❌ failure
onboard-resume-e2e ❌ failure
openclaw-discord-pairing-e2e ❌ failure
openclaw-inference-switch-e2e ❌ failure
openclaw-onboard-security-posture-e2e ❌ failure
openclaw-slack-pairing-e2e ❌ failure
openclaw-tui-chat-correlation-e2e ❌ failure
openshell-gateway-upgrade-e2e ❌ failure
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ❌ failure
rebuild-hermes-stale-base-e2e ❌ failure
rebuild-openclaw-e2e ❌ failure
runtime-overrides-e2e ❌ failure
sandbox-operations-e2e ❌ failure
sandbox-survival-e2e ❌ failure
shields-config-e2e ❌ failure
skill-agent-e2e ❌ failure
snapshot-commands-e2e ❌ failure
state-backup-restore-e2e ❌ failure
telegram-injection-e2e ❌ failure
token-rotation-e2e ❌ failure
tunnel-lifecycle-e2e ❌ failure
upgrade-stale-sandbox-e2e ❌ failure
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: bedrock-runtime-compatible-anthropic-e2e, channels-add-remove-e2e, channels-stop-start-e2e, cloud-e2e, cloud-inference-e2e, cloud-onboard-e2e, credential-migration-e2e, credential-sanitization-e2e, device-auth-health-e2e, diagnostics-e2e, docs-validation-e2e, double-onboard-e2e, hermes-dashboard-e2e, hermes-discord-e2e, hermes-e2e, hermes-inference-switch-e2e, hermes-onboard-security-posture-e2e, hermes-slack-e2e, inference-routing-e2e, issue-2478-crash-loop-recovery-e2e, issue-4462-gateway-pinned-approval-characterization-e2e, issue-4462-scope-upgrade-approval-e2e, kimi-inference-compat-e2e, launchable-smoke-e2e, messaging-compatible-endpoint-e2e, messaging-providers-e2e, network-policy-e2e, onboard-negative-paths-e2e, onboard-repair-e2e, onboard-resume-e2e, openclaw-discord-pairing-e2e, openclaw-inference-switch-e2e, openclaw-onboard-security-posture-e2e, openclaw-slack-pairing-e2e, openclaw-tui-chat-correlation-e2e, openshell-gateway-upgrade-e2e, rebuild-hermes-e2e, rebuild-hermes-stale-base-e2e, rebuild-openclaw-e2e, runtime-overrides-e2e, sandbox-operations-e2e, sandbox-survival-e2e, shields-config-e2e, skill-agent-e2e, snapshot-commands-e2e, state-backup-restore-e2e, telegram-injection-e2e, token-rotation-e2e, tunnel-lifecycle-e2e, upgrade-stale-sandbox-e2e. Check run artifacts for logs.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26724160646
Target ref: 46ec86c4d6d65b7b84a65301cf2a709de18c1ebf
Workflow ref: main
Requested jobs: cloud-e2e,cloud-onboard-e2e,double-onboard-e2e,sandbox-survival-e2e,issue-2478-crash-loop-recovery-e2e,sandbox-operations-e2e,inference-routing-e2e,network-policy-e2e,channels-add-remove-e2e,skill-agent-e2e,tunnel-lifecycle-e2e,rebuild-openclaw-e2e
Summary: 0 passed, 12 failed, 0 skipped

Job Result
channels-add-remove-e2e ❌ failure
cloud-e2e ❌ failure
cloud-onboard-e2e ❌ failure
double-onboard-e2e ❌ failure
inference-routing-e2e ❌ failure
issue-2478-crash-loop-recovery-e2e ❌ failure
network-policy-e2e ❌ failure
rebuild-openclaw-e2e ❌ failure
sandbox-operations-e2e ❌ failure
sandbox-survival-e2e ❌ failure
skill-agent-e2e ❌ failure
tunnel-lifecycle-e2e ❌ failure

Failed jobs: channels-add-remove-e2e, cloud-e2e, cloud-onboard-e2e, double-onboard-e2e, inference-routing-e2e, issue-2478-crash-loop-recovery-e2e, network-policy-e2e, rebuild-openclaw-e2e, sandbox-operations-e2e, sandbox-survival-e2e, skill-agent-e2e, tunnel-lifecycle-e2e. Check run artifacts for logs.

ericksoa added 2 commits May 31, 2026 14:23
Use raw OpenShell gRPC for ForwardTcp and log/watch endpoints that are not exposed by the current TypeScript SDK binding. Add a detached dashboard forward runner and raw ExecSandbox fallback so SDK-preview builds can validate runtime flows without the placeholder implementation.

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26724916520
Target ref: 1f6618924b9acce814395e7ead8720d2591aea79
Workflow ref: main
Requested jobs: all (no filter)
Summary: 44 passed, 2 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ✅ success
brave-search-e2e ✅ success
channels-add-remove-e2e ⚠️ cancelled
channels-stop-start-e2e ⚠️ cancelled
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ✅ success
credential-sanitization-e2e ✅ success
device-auth-health-e2e ✅ success
diagnostics-e2e ✅ success
docs-validation-e2e ✅ success
double-onboard-e2e ⚠️ cancelled
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-dashboard-e2e ❌ failure
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ✅ success
issue-3600-gpu-proof-optional-e2e ✅ success
issue-4462-gateway-pinned-approval-characterization-e2e ✅ success
issue-4462-scope-upgrade-approval-e2e ❌ failure
kimi-inference-compat-e2e ✅ success
launchable-smoke-e2e ✅ success
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ⚠️ cancelled
network-policy-e2e ✅ success
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ✅ success
onboard-resume-e2e ✅ success
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ✅ success
openshell-gateway-upgrade-e2e ⚠️ cancelled
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ✅ success
rebuild-hermes-stale-base-e2e ✅ success
rebuild-openclaw-e2e ⚠️ cancelled
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ⚠️ cancelled
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ⚠️ cancelled
snapshot-commands-e2e ✅ success
state-backup-restore-e2e ✅ success
telegram-injection-e2e ✅ success
token-rotation-e2e ⚠️ cancelled
tunnel-lifecycle-e2e ✅ success
upgrade-stale-sandbox-e2e ✅ success
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: hermes-dashboard-e2e, issue-4462-scope-upgrade-approval-e2e. Check run artifacts for logs.

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26725221329
Target ref: 2a94b6dee669617984e96ad688cc6c79bddadfea
Workflow ref: main
Requested jobs: all (no filter)
Summary: 32 passed, 2 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ✅ success
brave-search-e2e ✅ success
channels-add-remove-e2e ⚠️ cancelled
channels-stop-start-e2e ⚠️ cancelled
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ⚠️ cancelled
credential-sanitization-e2e ✅ success
device-auth-health-e2e ✅ success
diagnostics-e2e ⚠️ cancelled
docs-validation-e2e ✅ success
double-onboard-e2e ⚠️ cancelled
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-dashboard-e2e ✅ success
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ⚠️ cancelled
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ⚠️ cancelled
issue-3600-gpu-proof-optional-e2e ✅ success
issue-4462-gateway-pinned-approval-characterization-e2e ❌ failure
issue-4462-scope-upgrade-approval-e2e ❌ failure
kimi-inference-compat-e2e ✅ success
launchable-smoke-e2e ✅ success
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ⚠️ cancelled
network-policy-e2e ⚠️ cancelled
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ⚠️ cancelled
onboard-resume-e2e ⚠️ cancelled
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ⚠️ cancelled
openshell-gateway-upgrade-e2e ⚠️ cancelled
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ⚠️ cancelled
rebuild-hermes-stale-base-e2e ⚠️ cancelled
rebuild-openclaw-e2e ⚠️ cancelled
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ⚠️ cancelled
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ⚠️ cancelled
snapshot-commands-e2e ✅ success
state-backup-restore-e2e ⚠️ cancelled
telegram-injection-e2e ✅ success
token-rotation-e2e ⚠️ cancelled
tunnel-lifecycle-e2e ✅ success
upgrade-stale-sandbox-e2e ⚠️ cancelled
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: issue-4462-gateway-pinned-approval-characterization-e2e, issue-4462-scope-upgrade-approval-e2e. Check run artifacts for logs.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26725411482
Target ref: d6029961013fe1168dfef9bc6b0587a7ba6a6e6d
Workflow ref: main
Requested jobs: all (no filter)
Summary: 54 passed, 1 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ✅ success
brave-search-e2e ✅ success
channels-add-remove-e2e ✅ success
channels-stop-start-e2e ✅ success
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ✅ success
credential-sanitization-e2e ✅ success
device-auth-health-e2e ✅ success
diagnostics-e2e ✅ success
docs-validation-e2e ✅ success
double-onboard-e2e ✅ success
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-dashboard-e2e ✅ success
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ✅ success
issue-3600-gpu-proof-optional-e2e ✅ success
issue-4462-gateway-pinned-approval-characterization-e2e ✅ success
issue-4462-scope-upgrade-approval-e2e ✅ success
kimi-inference-compat-e2e ✅ success
launchable-smoke-e2e ✅ success
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ✅ success
network-policy-e2e ✅ success
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ✅ success
onboard-resume-e2e ✅ success
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ✅ success
openshell-gateway-upgrade-e2e ✅ success
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ✅ success
rebuild-hermes-stale-base-e2e ✅ success
rebuild-openclaw-e2e ✅ success
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ✅ success
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ✅ success
snapshot-commands-e2e ✅ success
state-backup-restore-e2e ✅ success
telegram-injection-e2e ❌ failure
token-rotation-e2e ✅ success
tunnel-lifecycle-e2e ✅ success
upgrade-stale-sandbox-e2e ✅ success
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: telegram-injection-e2e. Check run artifacts for logs.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26725411482
Target ref: d6029961013fe1168dfef9bc6b0587a7ba6a6e6d
Workflow ref: main
Requested jobs: all (no filter)
Summary: 55 passed, 0 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ✅ success
brave-search-e2e ✅ success
channels-add-remove-e2e ✅ success
channels-stop-start-e2e ✅ success
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ✅ success
credential-sanitization-e2e ✅ success
device-auth-health-e2e ✅ success
diagnostics-e2e ✅ success
docs-validation-e2e ✅ success
double-onboard-e2e ✅ success
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-dashboard-e2e ✅ success
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ✅ success
issue-3600-gpu-proof-optional-e2e ✅ success
issue-4462-gateway-pinned-approval-characterization-e2e ✅ success
issue-4462-scope-upgrade-approval-e2e ✅ success
kimi-inference-compat-e2e ✅ success
launchable-smoke-e2e ✅ success
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ✅ success
network-policy-e2e ✅ success
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ✅ success
onboard-resume-e2e ✅ success
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ✅ success
openshell-gateway-upgrade-e2e ✅ success
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ✅ success
rebuild-hermes-stale-base-e2e ✅ success
rebuild-openclaw-e2e ✅ success
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ✅ success
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ✅ success
snapshot-commands-e2e ✅ success
state-backup-restore-e2e ✅ success
telegram-injection-e2e ✅ success
token-rotation-e2e ✅ success
tunnel-lifecycle-e2e ✅ success
upgrade-stale-sandbox-e2e ✅ success
vm-driver-privileged-exec-routing-e2e ✅ success

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26726398782
Target ref: 1999ea448d34ae95764ffb993fd8e9b190f03d13
Workflow ref: main
Requested jobs: all (no filter)
Summary: 54 passed, 1 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ✅ success
brave-search-e2e ✅ success
channels-add-remove-e2e ✅ success
channels-stop-start-e2e ❌ failure
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ✅ success
credential-sanitization-e2e ✅ success
device-auth-health-e2e ✅ success
diagnostics-e2e ✅ success
docs-validation-e2e ✅ success
double-onboard-e2e ✅ success
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-dashboard-e2e ✅ success
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ✅ success
issue-3600-gpu-proof-optional-e2e ✅ success
issue-4462-gateway-pinned-approval-characterization-e2e ✅ success
issue-4462-scope-upgrade-approval-e2e ✅ success
kimi-inference-compat-e2e ✅ success
launchable-smoke-e2e ✅ success
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ✅ success
network-policy-e2e ✅ success
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ✅ success
onboard-resume-e2e ✅ success
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ✅ success
openshell-gateway-upgrade-e2e ✅ success
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ✅ success
rebuild-hermes-stale-base-e2e ✅ success
rebuild-openclaw-e2e ✅ success
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ✅ success
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ✅ success
snapshot-commands-e2e ✅ success
state-backup-restore-e2e ✅ success
telegram-injection-e2e ✅ success
token-rotation-e2e ✅ success
tunnel-lifecycle-e2e ✅ success
upgrade-stale-sandbox-e2e ✅ success
vm-driver-privileged-exec-routing-e2e ✅ success

Failed jobs: channels-stop-start-e2e. Check run artifacts for logs.

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26726398782
Target ref: 1999ea448d34ae95764ffb993fd8e9b190f03d13
Workflow ref: main
Requested jobs: all (no filter)
Summary: 55 passed, 0 failed, 2 skipped

Job Result
bedrock-runtime-compatible-anthropic-e2e ✅ success
brave-search-e2e ✅ success
channels-add-remove-e2e ✅ success
channels-stop-start-e2e ✅ success
cloud-e2e ✅ success
cloud-inference-e2e ✅ success
cloud-onboard-e2e ✅ success
credential-migration-e2e ✅ success
credential-sanitization-e2e ✅ success
device-auth-health-e2e ✅ success
diagnostics-e2e ✅ success
docs-validation-e2e ✅ success
double-onboard-e2e ✅ success
gpu-double-onboard-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-dashboard-e2e ✅ success
hermes-discord-e2e ✅ success
hermes-e2e ✅ success
hermes-inference-switch-e2e ✅ success
hermes-onboard-security-posture-e2e ✅ success
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-slack-e2e ✅ success
inference-routing-e2e ✅ success
issue-2478-crash-loop-recovery-e2e ✅ success
issue-3600-gpu-proof-optional-e2e ✅ success
issue-4462-gateway-pinned-approval-characterization-e2e ✅ success
issue-4462-scope-upgrade-approval-e2e ✅ success
kimi-inference-compat-e2e ✅ success
launchable-smoke-e2e ✅ success
messaging-compatible-endpoint-e2e ✅ success
messaging-providers-e2e ✅ success
network-policy-e2e ✅ success
onboard-negative-paths-e2e ✅ success
onboard-repair-e2e ✅ success
onboard-resume-e2e ✅ success
openclaw-discord-pairing-e2e ✅ success
openclaw-inference-switch-e2e ✅ success
openclaw-onboard-security-posture-e2e ✅ success
openclaw-slack-pairing-e2e ✅ success
openclaw-tui-chat-correlation-e2e ✅ success
openshell-gateway-upgrade-e2e ✅ success
overlayfs-autofix-e2e ✅ success
rebuild-hermes-e2e ✅ success
rebuild-hermes-stale-base-e2e ✅ success
rebuild-openclaw-e2e ✅ success
runtime-overrides-e2e ✅ success
sandbox-operations-e2e ✅ success
sandbox-survival-e2e ✅ success
shields-config-e2e ✅ success
skill-agent-e2e ✅ success
snapshot-commands-e2e ✅ success
state-backup-restore-e2e ✅ success
telegram-injection-e2e ✅ success
token-rotation-e2e ✅ success
tunnel-lifecycle-e2e ✅ success
upgrade-stale-sandbox-e2e ✅ success
vm-driver-privileged-exec-routing-e2e ✅ success

@wscurran wscurran added area: cli Command line interface, flags, terminal UX, or output area: integrations Third-party service integration behavior area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery feature PR adds or expands user-visible functionality and removed NemoClaw CLI labels Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cli Command line interface, flags, terminal UX, or output area: integrations Third-party service integration behavior area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery feature PR adds or expands user-visible functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants