Skip to content

test(scenario-92): prove dynamic apply→CREATE→commit-back + ExecEnv (infra-admin P2/P3)#74

Merged
intel352 merged 2 commits into
mainfrom
feat/scenario92-p2-p3
Jun 3, 2026
Merged

test(scenario-92): prove dynamic apply→CREATE→commit-back + ExecEnv (infra-admin P2/P3)#74
intel352 merged 2 commits into
mainfrom
feat/scenario92-p2-p3

Conversation

@intel352
Copy link
Copy Markdown
Contributor

@intel352 intel352 commented Jun 3, 2026

What

PR11 (final tail) of the infra-admin Phase 2/3 cascade. Extends scenario 92 to prove the headline dynamic-apply flow against the real released stack: workflow engine v0.74.0 (ResourceDriver wired end-to-end) + workflow-plugin-infra v1.2.0.

Headline proof (was a SKIP, now a hard PASS)

Operator POSTs operator-edited specs (carrying a secret:// ref) to /plan then /apply:

  • step.iac_provider_apply genuinely CREATEs resources via the stub provider's ResourceDriver.Createapply_result.resources non-empty (a real provider_id assigned), zero per-action errors. The assertion is hardened to fail on null/empty apply_result or any per-action error (proven against synthetic inputs: null→fail, []→fail, error→fail, real-create→pass).
  • step.iac_commit_back pushes a GitOps branch (gitops/infra-apply-demo) to the bare repo whose committed resources.yaml carries the literal secret://scenario/stub_api_key ref UN-resolved (secrets never committed in cleartext).

Also exercised

  • Reachability 409: specs with secret:// + exec_env=remote/apply 409 (host-local secrets unreachable from a remote exec_env — ADR 0017).
  • ExecEnv: local + remote (sandbox-runner agent, profile-clamped); Argo path env-gated on SCENARIO_92_ARGO=1.

Result

42 pass / 0 fail / 1 skip (Argo env-gated), curl + Playwright, against seed.sh (real v0.74.0 engine + agent built from the same pin) → run.sh.

Fixes found during finalization

  • bare-repo mount :ro:rw (a git push writes objects+refs into the bare repo).
  • collapsed a double-/apply that collided on the static commit-back branch name (state_diverged).

Pin

go.mod: workflow v0.72.0v0.74.0. seed.sh builds the engine and cmd/workflow-sandbox-runner from the same released pin.

🤖 Generated with Claude Code

…P2/P3)

Extend scenario 92 to prove the infra-admin Phase 2/3 headline against the
real released stack (workflow v0.74.0 — ResourceDriver wired — +
workflow-plugin-infra v1.2.0):

- HEADLINE: operator POSTs edited specs (carrying a `secret://` ref) to
  /plan then /apply → step.iac_provider_apply genuinely CREATEs resources
  via the stub provider's ResourceDriver.Create (apply_result.resources
  non-empty, zero per-action errors) → step.iac_commit_back pushes a GitOps
  branch to the bare repo whose committed resources.yaml carries the literal
  `secret://scenario/stub_api_key` ref UN-resolved. Hardened so the assertion
  fails on null/empty apply_result or any per-action error (not a vacuous
  green).
- Reachability 409: specs with `secret://` + exec_env=remote → /apply 409
  (host-local secrets unreachable from a remote exec_env, ADR 0017).
- ExecEnv: local + remote (sandbox-runner agent, profile-clamped) exercised;
  Argo path env-gated on SCENARIO_92_ARGO=1.
- Stub provider advertises workflow.plugin.external.iac.ResourceDriver in
  plugin.json iacServices + implements Create.
- Pin workflow v0.72.0 → v0.74.0; seed.sh builds engine + workflow-sandbox-
  runner agent from the same released pin.
- Fixes found during finalization: bare-repo mount :ro → :rw (a push writes
  objects); collapse a double-/apply that collided on the static commit-back
  branch name.

Result: 42 pass / 0 fail / 1 skip (Argo env-gated), curl + Playwright.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings June 3, 2026 09:10
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends Scenario 92 (“infra-admin”) to validate Phase 2/3 behavior end-to-end against the pinned released workflow engine (v0.74.0), including dynamic specs posted to /plan/apply, ResourceDriver-backed CREATE results, git commit-back into a bare repo, secret reachability gating for remote exec env, and a remote sandbox-runner agent path.

Changes:

  • Updates Scenario 92 pipelines/config to accept dynamic specs from request bodies, enforce secret reachability (409 on remote), integrate commit-back into /apply, and add reconcile + exec-envs + sandbox-demo routes.
  • Enhances seeding and test runner scripts to build the pinned engine + sandbox-runner, initialize git fixtures (bare repo + workclone), and assert CREATE+commit-back outcomes (including secret:// refs preserved).
  • Extends the stub IaC provider and Playwright coverage to exercise ResourceDriver Create, commit-back integration, reachability failures, reconcile response shape, exec-env discovery, and remote runner execution.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
scenarios/92-infra-admin-demo/test/run.sh Expands curl-based assertions for dynamic plan/apply, commit-back branch+YAML checks, reachability 409, reconcile shape, exec-envs, and remote runner marker.
scenarios/92-infra-admin-demo/seed/seed.sh Builds pinned engine + sandbox-runner, prepares git bare repo/workclone fixtures, switches engine image to include git, and brings up the extended compose stack.
scenarios/92-infra-admin-demo/scenario.yaml Updates scenario metadata/docs to Phase 2/3 scope, routes, and assertions.
scenarios/92-infra-admin-demo/README.md Documents Phase 2/3 architecture, routes, dynamic apply→commit-back flow, and remote agent.
scenarios/92-infra-admin-demo/fixtures/stub-iac-provider/internal/provider.go Adds ResourceDriver server implementation (Create) to support real apply CREATE paths.
scenarios/92-infra-admin-demo/fixtures/stub-iac-provider/cmd/stub-iac-provider/plugin.json Advertises ResourceDriver service capability for the stub provider.
scenarios/92-infra-admin-demo/docker-compose.yml Adds sandbox-runner + docker-dind services and mounts git bare repo/workclone into app for commit-back/reconcile.
scenarios/92-infra-admin-demo/config/app.yaml Rewires infra pipelines for dynamic specs, reachability gating, commit-back integration, reconcile, exec-envs, and sandbox-demo routes.
go.mod Pins github.com/GoCodeAlone/workflow to v0.74.0.
go.sum Updates checksums for the workflow v0.74.0 pin.
e2e/tests/scenario-92-infra-admin.spec.ts Updates Playwright tests for dynamic specs, commit-back integration (no standalone /commit), reachability 409, reconcile, exec-envs, and sandbox-demo.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread scenarios/92-infra-admin-demo/config/app.yaml
Comment thread scenarios/92-infra-admin-demo/docker-compose.yml Outdated
Comment thread scenarios/92-infra-admin-demo/seed/seed.sh
Comment thread e2e/tests/scenario-92-infra-admin.spec.ts Outdated
Comment thread scenarios/92-infra-admin-demo/config/app.yaml
Comment thread scenarios/92-infra-admin-demo/seed/seed.sh Outdated
…afe.directory

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@intel352
Copy link
Copy Markdown
Contributor Author

intel352 commented Jun 3, 2026

Addressed in 6dba99f2:

  1. app.yaml — bare-repo mount comment corrected :ro:rw (commit-back writes objects/refs into the bare repo).
  2. docker-compose.yml — sandbox-runner comment now matches reality: talks to a separate docker-dind service over DOCKER_HOST=tcp://docker-dind:2375, no host-socket mount.
  3. seed.sh — safe.directory = * narrowed to explicit /gitops/workclone + /gitops/bare.git; re-run confirmed those two are the only paths git needs (no *, zero dubious-ownership rejections, commit-back succeeds).
  4. e2e spec header — v0.72.0v0.74.0 (the only remaining stale reference).
  5. app.yaml — RBAC comment dropped the stale standalone commit action; documents the real mutation endpoints (commit-back runs inside /apply). Config unchanged (no dangling route).
  6. seed.sh — git-identity comment corrected to the build-time config file /home/nonroot/.config/git/config.

Re-ran the full stack: 42 pass / 0 fail / 1 skip (Argo env-gated). Headline apply→CREATE→commit-back still a hard PASS.

@intel352 intel352 merged commit bcba0a9 into main Jun 3, 2026
10 checks passed
@intel352 intel352 deleted the feat/scenario92-p2-p3 branch June 3, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants