Skip to content

test: capture EVRUN-full smoke evidence #79

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aryeko merged 1 commit into from
Jul 6, 2026
Merged

test: capture EVRUN-full smoke evidence #79

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,10 @@ the success-path real Forge `open-pr` smoke has been rerun on the current checko
held-merge and commit-status/comment block-surfacing legs remain unit-proven with real-effect
evidence still owed.
Strong no-phone-home evidence remains future work. EVRUN-partial is recorded; P11 also records a
blocked EVRUN-full capture attempt where the narrow Codex app-server and real-host smokes passed
outside the Codex sandbox, but the combined real GitHub path could not run without sandbox
credentials and an integrity key. EVRUN-full remains future work.
blocked EVRUN-full capture attempt plus a later successful combined EVRUN-full smoke: real GitHub
Issues, real Codex app-server, real host attestation, real GitHub Forge `open-pr`, and records
integrity on the disposable target repo. Strong no-phone-home, held-merge/idempotency, hosted, and
Windows evidence remain future work.

## Commands

Expand Down
65 changes: 33 additions & 32 deletions docs/delivery/target-state-implementation/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ status: in progress

# Target-state implementation — delivery track

**Status: in progress.** P01–P12 are merged (see the [phase table](#phase-table)): P11 merged with a
blocked EVRUN-full evidence record, so the full-capture demonstration is still owed. P13 is
currently blocked pending the contract-owner freeze decision and the remaining evidence/remediation
closeout; P14 is planned. Phase statuses live in the phase table below and in each phase doc's
frontmatter.
**Status: in progress.** P01–P12 are merged (see the [phase table](#phase-table)): P11 first
merged with a blocked EVRUN-full evidence record, and a later follow-up captured the combined
real Codex / real GitHub `open-pr` smoke. P13 is currently blocked pending the contract-owner
freeze decision and the remaining no-phone-home/idempotency evidence or explicit deferral; P14 is
planned. Phase statuses live in the phase table below and in each phase doc's frontmatter.

## Overview

Expand Down Expand Up @@ -59,11 +59,11 @@ Verified against the repo after Phase 02 workspace split:
driver) and Work source (GitHub Issues importer) are **real and usable**. The Agent seam now
includes the private production Codex app-server transport: `bootstrap.ts` composes
`createProductionCodexAgentSession`, and driver selection supports `agent: 'codex'`. The
remaining limitation is evidence, not absence of implementation: P11 still records only a
blocked EVRUN-full attempt, so the full real-effect proof for the combined Codex/GitHub path is
still owed. The real Execution host is now selectable on macOS and exercises a local
`process-group` confinement probe that reports an honest proven `weak` posture at compose time;
stronger no-phone-home evidence remains open for P11.
remaining limitation is evidence, not absence of implementation: P11 now records the combined
real Codex / real GitHub `open-pr` smoke, while stronger no-phone-home and repeated-effect
idempotency evidence remain open. The real Execution host is now selectable on macOS and
exercises a local `process-group` confinement probe that reports an honest proven `weak` posture
at compose time.
- **Conformance and controlled doubles** now live in `packages/jig-testkit`
(`packages/jig-testkit/src/provider-conformance.ts`, `packages/jig-testkit/tests/conformance/`),
outside the production SDK dependency graph.
Expand All @@ -77,10 +77,10 @@ Verified against the repo after Phase 02 workspace split:
MCP adapter.
- **Evidence**: EVRUN-partial is committed
([evidence index](../../design/evidence/README.md)) — one real
work-source → forge → records-integrity run with a **scripted** agent leg. P11 captured a
blocked EVRUN-full attempt: narrow Codex app-server and real-host smokes pass outside the
sandbox, but the combined real GitHub path could not run without sandbox credentials and an
integrity key. EVRUN-full remains open.
work-source → forge → records-integrity run with a **scripted** agent leg. P11 also records the
earlier blocked EVRUN-full attempt and the later combined real Codex / real-host / real GitHub
`open-pr` smoke. Strong no-phone-home, held-merge/idempotency, hosted, and Windows evidence
remain open.

## Desired target state

Expand All @@ -103,8 +103,9 @@ Verified against the repo after Phase 02 workspace split:
- **Observability** ([guarantee 5](../../product/guarantees.md#5-full-observability)): watch,
notices with acknowledge/snooze, ask-why, and write-once redacted export, all answered from
the run's own records.
- **Evidence**: EVRUN-full committed; contract-freeze readiness prepared for the contract
owner's T14 decision.
- **Evidence**: EVRUN-full combined smoke committed; remaining no-phone-home/idempotency evidence
captured or explicitly deferred; contract-freeze readiness prepared for the contract owner's T14
decision.
- **Docs**: README, AGENTS.md, and docs indexes state the shipped surface truthfully at every
phase boundary.

Expand Down Expand Up @@ -285,22 +286,22 @@ release posture`")

## Phase table

| ID | Phase | Status | Hard dependencies | Parallelization |
| --- | ------------------------------------------------------------------------------------------ | ------------------------------ | ----------------- | ---------------------------------------------------------------------------- |
| P01 | [SDK boundary and operator-control surface](./phases/01-sdk-boundary-and-operator-port.md) | merged (#55) | — | First; everything else keys off this surface. |
| P02 | [Package split: jig-sdk, jig-cli, jig-testkit](./phases/02-package-split-workspace.md) | merged (#56, #58) | P01 | Sole occupant of its slot — it moves every source file. |
| P03 | [Codex app-server transport](./phases/03-codex-app-server-transport.md) | merged (#57, #58, #59, #60) | P01 | Parallel with P04–P10 after P02 (soft). |
| P04 | [Execution-host containment and substrate](./phases/04-execution-host-containment.md) | merged (#63) | — | Parallel with P03, P05–P10 after P02 (soft). |
| P05 | [Forge and work-source completion](./phases/05-forge-and-work-source-completion.md) | merged (#64) | — | Parallel with P03, P04, P06–P10 after P02 (soft). |
| P06 | [Owner configuration model](./phases/06-owner-configuration-model.md) | merged (#65) | — | Parallel with P03–P05, P08–P10 after P02 (soft). |
| P07 | [Guided setup](./phases/07-guided-setup.md) | merged (#66) | P06 | Parallel with anything not touching config templates. |
| P08 | [Watch, notices, ask-why](./phases/08-observation-surfaces.md) | merged (#67) | P01 | Parallel with P03–P06, P09, P10; coordinate record vocabulary with P09. |
| P09 | [Decide and stop](./phases/09-owner-decision-and-run-control.md) | merged (#68) | P01 | Parallel with P03–P06, P08, P10; coordinate record vocabulary with P08. |
| P10 | [Export: write-once audit record](./phases/10-export-audit-record.md) | merged (#69) | P01 | Parallel with P03–P09. |
| P11 | [EVRUN-full evidence](./phases/11-evrun-full-evidence.md) | merged (#70; blocked evidence) | P03, P04 | Sequential after both provider phases; benefits from P05. |
| P12 | [MCP driving adapter](./phases/12-mcp-adapter.md) | merged (#71) | P02 | Parallel with P11; soft dependency on P08/P09 for verb coverage. |
| P13 | [Contract v0 freeze readiness](./phases/13-contract-freeze-readiness.md) | blocked | P05, P07–P12 | Requires evidence/remediation closeout and a contract-owner freeze decision. |
| P14 | [Target-state audit, docs, release posture](./phases/14-docs-and-release-readiness.md) | planned | All other phases | Last; closes the track. |
| ID | Phase | Status | Hard dependencies | Parallelization |
| --- | ------------------------------------------------------------------------------------------ | ----------------------------------------------------------- | ----------------- | ---------------------------------------------------------------------------- |
| P01 | [SDK boundary and operator-control surface](./phases/01-sdk-boundary-and-operator-port.md) | merged (#55) | — | First; everything else keys off this surface. |
| P02 | [Package split: jig-sdk, jig-cli, jig-testkit](./phases/02-package-split-workspace.md) | merged (#56, #58) | P01 | Sole occupant of its slot — it moves every source file. |
| P03 | [Codex app-server transport](./phases/03-codex-app-server-transport.md) | merged (#57, #58, #59, #60) | P01 | Parallel with P04–P10 after P02 (soft). |
| P04 | [Execution-host containment and substrate](./phases/04-execution-host-containment.md) | merged (#63) | — | Parallel with P03, P05–P10 after P02 (soft). |
| P05 | [Forge and work-source completion](./phases/05-forge-and-work-source-completion.md) | merged (#64) | — | Parallel with P03, P04, P06–P10 after P02 (soft). |
| P06 | [Owner configuration model](./phases/06-owner-configuration-model.md) | merged (#65) | — | Parallel with P03–P05, P08–P10 after P02 (soft). |
| P07 | [Guided setup](./phases/07-guided-setup.md) | merged (#66) | P06 | Parallel with anything not touching config templates. |
| P08 | [Watch, notices, ask-why](./phases/08-observation-surfaces.md) | merged (#67) | P01 | Parallel with P03–P06, P09, P10; coordinate record vocabulary with P09. |
| P09 | [Decide and stop](./phases/09-owner-decision-and-run-control.md) | merged (#68) | P01 | Parallel with P03–P06, P08, P10; coordinate record vocabulary with P08. |
| P10 | [Export: write-once audit record](./phases/10-export-audit-record.md) | merged (#69) | P01 | Parallel with P03–P09. |
| P11 | [EVRUN-full evidence](./phases/11-evrun-full-evidence.md) | merged (#70; combined smoke captured, stronger probes open) | P03, P04 | Sequential after both provider phases; benefits from P05. |
| P12 | [MCP driving adapter](./phases/12-mcp-adapter.md) | merged (#71) | P02 | Parallel with P11; soft dependency on P08/P09 for verb coverage. |
| P13 | [Contract v0 freeze readiness](./phases/13-contract-freeze-readiness.md) | blocked | P05, P07–P12 | Requires evidence/remediation closeout and a contract-owner freeze decision. |
| P14 | [Target-state audit, docs, release posture](./phases/14-docs-and-release-readiness.md) | planned | All other phases | Last; closes the track. |

## What can run in parallel

Expand Down
38 changes: 21 additions & 17 deletions docs/delivery/target-state-implementation/phases/11-evrun-full-evidence.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Phase 11 - EVRUN-full evidence"
status: "merged (#70; blocked evidence)"
status: "merged (#70; combined smoke captured, stronger probes open)"
---

# Phase 11 - EVRUN-full evidence
Expand All @@ -10,10 +10,10 @@ status: "merged (#70; blocked evidence)"
Run and commit the EVRUN-full evidence: a real, end-to-end Codex-driven delivery — GitHub
Issues work source, real Codex agent over the app-server transport, real execution-host
containment, real GitHub Forge landing, integrity and redaction active — plus the adversarial
probes EVRUN-partial could not claim (no-phone-home, multi-run idempotency). The implemented P11
record is an honest blocked capture attempt: the narrow Codex app-server and real-host smoke probes
were refreshed, but the combined EVRUN-full path could not run because the sandbox GitHub and
integrity prerequisites were absent. Product code changes are not this phase's purpose.
probes EVRUN-partial could not claim (no-phone-home, multi-run idempotency). P11 first landed an
honest blocked capture attempt; a later follow-up captured the combined real GitHub / real Codex /
real-host `open-pr` smoke. Strong no-phone-home, held-merge replay, and multi-run idempotency
remain open. Product code changes are not this phase's purpose.

## Background

Expand Down Expand Up @@ -54,8 +54,9 @@ gated on this evidence path. This phase is the proof step: it converts "implemen
in [`product/jig.md`](../../../product/jig.md#what-jig-isnt-yet).
- A successful capture would prove `SEC-2` (no-phone-home proven, not asserted), `RESUME-3` (no
double effect against real systems), and `MERGE-2`/`FENCE-3` observations on a real path.
- The implemented blocked capture attempt does not unlock P13 or authorize P14 status claims;
those remain gated on either EVRUN-full evidence or an explicit owner decision to defer the gate.
- The later combined smoke removes the original "real Codex plus real GitHub path unproven"
blocker. P13/P14 status claims still need either stronger no-phone-home/idempotency evidence or
an explicit owner decision to defer those gates.

## Technical Requirements

Expand Down Expand Up @@ -86,28 +87,31 @@ gated on this evidence path. This phase is the proof step: it converts "implemen
- **Requires:** P03 and P04 (hard).
- **Benefits from:** P05 (block-surfacing and held-merge paths worth capturing in the same
sweep).
- **Would unlock:** P13 (hard) and P14 status claims only after EVRUN-full evidence or an explicit
owner deferral decision.
- **Would unlock:** P13 (hard) and P14 status claims only after the remaining no-phone-home /
idempotency evidence is captured or explicitly deferred by the owner.
- **Parallel:** P07–P10, P12 may proceed concurrently.

## Acceptance Criteria

These criteria remain the bar for closing this phase. The per-AC disposition markers below
record honest current state as of the #70 merge; they do not lower the bar.
record honest current state after the #70 merge and the later 2026-07-06 follow-up capture; they
do not lower the bar for the stronger probes.

1. A committed, dated EVRUN-full evidence record (or record set) exists under
`docs/design/evidence/`, indexed, convention-complete (versions, hashes, Limitations,
redaction statement, ID citations).
**Disposition at #70 merge: not met.** What was delivered instead is the blocked-attempt
record `docs/design/evidence/2026-07-06-evrun-full-capture-attempt.md`. A full capture
re-attempt is still owed before this criterion is met.
**Current disposition: met for the combined smoke record.**
`docs/design/evidence/2026-07-06-evrun-full-smoke.md` records the successful combined path.
`docs/design/evidence/2026-07-06-evrun-full-capture-attempt.md` remains as provenance for the
earlier blocked attempt.
2. The record demonstrates: real Codex editing through the owned app-server transport; real,
exercised confinement with honest strength; an adversarial no-phone-home observation; and
multi-run idempotency against a real landed effect.
**Disposition at #70 merge: not met.** The combined EVRUN-full path could not run because
the sandbox GitHub and integrity prerequisites were absent; the blocked-attempt record
`docs/design/evidence/2026-07-06-evrun-full-capture-attempt.md` documents what was observed
instead. A full capture re-attempt remains required before this criterion is met.
**Current disposition: partially met.** The combined smoke demonstrates real Codex editing
through the owned app-server transport, real macOS execution-host attestation with honest
`weak` strength, real GitHub Issues / Forge effects, records integrity, and redaction. The
adversarial no-phone-home observation remains `ambiguous`, and multi-run idempotency against a
repeated real landed effect remains unproven.
3. Hosted/remote operation and Windows behavior are explicitly restated as out of scope in the
Limitations section.
4. README/AGENTS/product status lines about EVRUN reflect the new boundary — no claim exceeds
Expand Down
Loading