test(scenario-92): prove dynamic apply→CREATE→commit-back + ExecEnv (infra-admin P2/P3)#74
Merged
Conversation
…P2/P3) Extend scenario 92 to prove the infra-admin Phase 2/3 headline against the real released stack (workflow v0.74.0 — ResourceDriver wired — + workflow-plugin-infra v1.2.0): - HEADLINE: operator POSTs edited specs (carrying a `secret://` ref) to /plan then /apply → step.iac_provider_apply genuinely CREATEs resources via the stub provider's ResourceDriver.Create (apply_result.resources non-empty, zero per-action errors) → step.iac_commit_back pushes a GitOps branch to the bare repo whose committed resources.yaml carries the literal `secret://scenario/stub_api_key` ref UN-resolved. Hardened so the assertion fails on null/empty apply_result or any per-action error (not a vacuous green). - Reachability 409: specs with `secret://` + exec_env=remote → /apply 409 (host-local secrets unreachable from a remote exec_env, ADR 0017). - ExecEnv: local + remote (sandbox-runner agent, profile-clamped) exercised; Argo path env-gated on SCENARIO_92_ARGO=1. - Stub provider advertises workflow.plugin.external.iac.ResourceDriver in plugin.json iacServices + implements Create. - Pin workflow v0.72.0 → v0.74.0; seed.sh builds engine + workflow-sandbox- runner agent from the same released pin. - Fixes found during finalization: bare-repo mount :ro → :rw (a push writes objects); collapse a double-/apply that collided on the static commit-back branch name. Result: 42 pass / 0 fail / 1 skip (Argo env-gated), curl + Playwright. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Extends Scenario 92 (“infra-admin”) to validate Phase 2/3 behavior end-to-end against the pinned released workflow engine (v0.74.0), including dynamic specs posted to /plan → /apply, ResourceDriver-backed CREATE results, git commit-back into a bare repo, secret reachability gating for remote exec env, and a remote sandbox-runner agent path.
Changes:
- Updates Scenario 92 pipelines/config to accept dynamic specs from request bodies, enforce secret reachability (409 on remote), integrate commit-back into
/apply, and add reconcile + exec-envs + sandbox-demo routes. - Enhances seeding and test runner scripts to build the pinned engine + sandbox-runner, initialize git fixtures (bare repo + workclone), and assert CREATE+commit-back outcomes (including secret:// refs preserved).
- Extends the stub IaC provider and Playwright coverage to exercise ResourceDriver Create, commit-back integration, reachability failures, reconcile response shape, exec-env discovery, and remote runner execution.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| scenarios/92-infra-admin-demo/test/run.sh | Expands curl-based assertions for dynamic plan/apply, commit-back branch+YAML checks, reachability 409, reconcile shape, exec-envs, and remote runner marker. |
| scenarios/92-infra-admin-demo/seed/seed.sh | Builds pinned engine + sandbox-runner, prepares git bare repo/workclone fixtures, switches engine image to include git, and brings up the extended compose stack. |
| scenarios/92-infra-admin-demo/scenario.yaml | Updates scenario metadata/docs to Phase 2/3 scope, routes, and assertions. |
| scenarios/92-infra-admin-demo/README.md | Documents Phase 2/3 architecture, routes, dynamic apply→commit-back flow, and remote agent. |
| scenarios/92-infra-admin-demo/fixtures/stub-iac-provider/internal/provider.go | Adds ResourceDriver server implementation (Create) to support real apply CREATE paths. |
| scenarios/92-infra-admin-demo/fixtures/stub-iac-provider/cmd/stub-iac-provider/plugin.json | Advertises ResourceDriver service capability for the stub provider. |
| scenarios/92-infra-admin-demo/docker-compose.yml | Adds sandbox-runner + docker-dind services and mounts git bare repo/workclone into app for commit-back/reconcile. |
| scenarios/92-infra-admin-demo/config/app.yaml | Rewires infra pipelines for dynamic specs, reachability gating, commit-back integration, reconcile, exec-envs, and sandbox-demo routes. |
| go.mod | Pins github.com/GoCodeAlone/workflow to v0.74.0. |
| go.sum | Updates checksums for the workflow v0.74.0 pin. |
| e2e/tests/scenario-92-infra-admin.spec.ts | Updates Playwright tests for dynamic specs, commit-back integration (no standalone /commit), reachability 409, reconcile, exec-envs, and sandbox-demo. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…afe.directory Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Addressed in
Re-ran the full stack: 42 pass / 0 fail / 1 skip (Argo env-gated). Headline apply→CREATE→commit-back still a hard PASS. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
PR11 (final tail) of the infra-admin Phase 2/3 cascade. Extends scenario 92 to prove the headline dynamic-apply flow against the real released stack: workflow engine v0.74.0 (ResourceDriver wired end-to-end) + workflow-plugin-infra v1.2.0.
Headline proof (was a SKIP, now a hard PASS)
Operator POSTs operator-edited specs (carrying a
secret://ref) to/planthen/apply:step.iac_provider_applygenuinely CREATEs resources via the stub provider'sResourceDriver.Create—apply_result.resourcesnon-empty (a realprovider_idassigned), zero per-action errors. The assertion is hardened to fail on null/emptyapply_resultor any per-action error (proven against synthetic inputs:null→fail,[]→fail, error→fail, real-create→pass).step.iac_commit_backpushes a GitOps branch (gitops/infra-apply-demo) to the bare repo whose committedresources.yamlcarries the literalsecret://scenario/stub_api_keyref UN-resolved (secrets never committed in cleartext).Also exercised
secret://+exec_env=remote→/apply409 (host-local secrets unreachable from a remote exec_env — ADR 0017).SCENARIO_92_ARGO=1.Result
42 pass / 0 fail / 1 skip (Argo env-gated), curl + Playwright, against
seed.sh(real v0.74.0 engine + agent built from the same pin) →run.sh.Fixes found during finalization
:ro→:rw(agit pushwrites objects+refs into the bare repo)./applythat collided on the static commit-back branch name (state_diverged).Pin
go.mod: workflowv0.72.0→v0.74.0.seed.shbuilds the engine andcmd/workflow-sandbox-runnerfrom the same released pin.🤖 Generated with Claude Code