From f453ab136e653009f3d1f49b5fc6ff706a2f4d84 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Fri, 29 May 2026 17:08:02 -0400 Subject: [PATCH 1/5] feat(skill): add pre-flight and post-flight release procedures The cutting-releases skill now covers the full release lifecycle: Pre-flight: diff reusable workflows, scaffold templates, and CLI changes against the v0 tag to detect breaking changes before release. Process: added v0 tag move (step 8) and workflow wait (step 9), renumbered remaining steps. Post-flight: verify downstream orgs resolve @v0 correctly, retrigger failed runs for confirmation, present summary table. All variable-dependent commands prompt the user via AskUserQuestion rather than assuming values are in the environment. Signed-off-by: Wayne Sun --- skills/cutting-releases/SKILL.md | 156 ++++++++++++++++++++++++++++++- 1 file changed, 152 insertions(+), 4 deletions(-) diff --git a/skills/cutting-releases/SKILL.md b/skills/cutting-releases/SKILL.md index 2d9cdd30b..bc65a9f22 100644 --- a/skills/cutting-releases/SKILL.md +++ b/skills/cutting-releases/SKILL.md @@ -4,7 +4,7 @@ description: > Use when the user wants to tag a release, cut a release candidate, or ship a new version. Also use when asking about release process, versioning, or how GoReleaser is configured. -allowed-tools: Read, Grep, Glob, AskUserQuestion, Bash(git tag:*), Bash(git log:*), Bash(git diff:*), Bash(git pull:*), Bash(git push:*), Bash(gh release:*), Bash(gh run:*), Bash(git checkout:*), Bash(bash skills/cutting-releases/scripts/install-binary.sh:*) +allowed-tools: Read, Grep, Glob, AskUserQuestion, Bash(git tag:*), Bash(git log:*), Bash(git diff:*), Bash(git pull:*), Bash(git push:*), Bash(gh release:*), Bash(gh run:*), Bash(git checkout:*), Bash(git fetch:*), Bash(make lint:*), Bash(bash skills/cutting-releases/scripts/install-binary.sh:*) --- # Cutting Releases @@ -14,6 +14,70 @@ the `.github/workflows/release.yml` workflow runs GoReleaser, which builds binaries, generates a changelog, and creates the GitHub release. The release title comes from the tag annotation via `name_template` in `.goreleaser.yml`. +## Pre-Flight Release Check + +Run this audit **before** tagging. The goal is to verify that moving +the `v0` reusable-workflow tag and publishing new scaffold templates +will not break downstream orgs. + +### A. Diff reusable workflows + +Compare the current `v0` tag to `main` for all reusable workflows: + +``` +git diff v0..origin/main -- .github/workflows/reusable-*.yml +``` + +For each changed workflow, classify: +- **Additive** (new optional inputs, new env vars) — safe. +- **Default change** (different default values) — check downstream callers. +- **Breaking** (removed inputs, renamed jobs, changed required outputs) — block. + +### B. Diff scaffold templates + +``` +git diff v0..origin/main -- internal/scaffold/fullsend-repo/ +``` + +Scaffold files are deployed at `github setup` time, not consumed live. +Changes here affect **new installs and re-scaffolds only**, not running +workflows. Still note anything that alters agent behavior (skill files, +harness configs, hook scripts). + +### C. Review CLI changes + +``` +git log --oneline v0..origin/main -- cmd/ internal/cli/ +``` + +Check for: +- Renamed flags or sub-commands — deprecated aliases must be preserved. +- Changed defaults (pool names, regions, project IDs) — document migration. +- Removed functionality — block or add deprecation notice. + +### D. Check CI on main + +``` +gh run list --branch=main --limit=5 +``` + +All recent runs should be passing. If E2E tests are failing, investigate +before releasing. + +### E. Present summary + +Summarize findings to the user in a table: + +| Area | Changes | Breaking? | +|------|---------|-----------| +| Reusable workflows | ... | No/Yes | +| Scaffold templates | ... | No/Yes | +| CLI | ... | No/Yes | + +Give a **GO / NO-GO** verdict. Do not proceed until the user confirms. + +--- + ## Process Follow these steps in order. @@ -104,9 +168,35 @@ GoReleaser takes over from here. Verify the workflow starts: gh run list --workflow=release.yml --limit=1 ``` -### 8. Verify the release +### 8. Move the `v0` tag + +Downstream orgs reference reusable workflows via `@v0`. After the +version tag is pushed, move `v0` to the same commit: + +``` +git tag -f v0 +git push origin v0 --force +``` + +This updates all `@v0` workflow references immediately. The Sandbox +Images workflow (triggered by tag push) will also run. + +### 9. Wait for workflows + +Wait for both the Release workflow (triggered by the `v*` tag) and +the Sandbox Images workflow (triggered by the `v0` tag move) to +complete: + +``` +gh run list --workflow=release.yml --limit=1 +gh run list --workflow=sandbox-images.yml --limit=1 +``` + +Both must pass before proceeding. + +### 10. Verify the release -Once the workflow completes, confirm the release was created: +Once the workflows complete, confirm the release was created: ``` gh release view @@ -114,7 +204,7 @@ gh release view Check that the title, changelog, and binary assets look correct. -### 9. Install the binary locally +### 11. Install the binary locally Ask the user where to install (default: `~/.local/bin/`), then run the install script from this skill's base directory: @@ -127,6 +217,62 @@ The script downloads the release archive, verifies its SHA-256 checksum against the release's `checksums.txt`, and installs the binary as `fullsend-` so multiple versions can coexist. +## Post-Flight Verification + +After the release and sandbox-images workflows pass, verify that +downstream orgs can resolve the new `@v0` tag. + +### A. Check downstream orgs + +Use `AskUserQuestion` to ask which downstream orgs and repos to verify: + +> Which orgs/repos should I check for post-release verification? +> (e.g. "fullsend-ai, konflux-ci/integration-service, redhat-developer/rhdh-agentic") + +For each org or repo provided, check recent workflow runs: + +``` +gh run list --repo /.fullsend --limit=3 +gh run list --repo / --limit=3 +``` + +Look for runs that started **after** the `v0` tag move. Confirm they +completed without workflow-resolution errors (e.g. "could not find +reusable workflow"). + +### B. Retrigger a verification run + +If no natural workflow runs occurred after the tag move, find a recent +failed or cancelled run and present it to the user for confirmation +before retriggering: + +> I found run `` (failed) in `/`. Retrigger it +> to verify `@v0` resolves? + +Once confirmed: + +``` +gh run rerun --failed --repo / +``` + +Watch for the run to complete. A successful run confirms the new `v0` +tag is working. + +### C. Present post-flight summary + +Summarize results to the user: + +| Org/Repo | `@v0` Refs | Status | +|----------|-----------|--------| +| org/.fullsend | Confirmed | Passing | +| org/target-repo | Confirmed | Passing | + +Note any failures that are **unrelated** to the tag move (e.g. agent +runtime errors, external API issues) vs. failures caused by the +release. + +--- + ## Notes - **Pre-releases:** Tags with `-rc.N`, `-alpha.N`, or `-beta.N` suffixes are @@ -134,3 +280,5 @@ against the release's `checksums.txt`, and installs the binary as - **Never delete a published tag.** If a release is bad, cut a new patch or RC. - **The changelog** is auto-generated from commit messages. Conventional commit prefixes (`feat:`, `fix:`, etc.) produce clean changelogs. +- **The `v0` tag** is a moving tag consumed by downstream orgs for reusable + workflows. Always move it as part of the release process (step 8). From 65c9dfb0dfeeb7d9013deaf89b026b9987bf0a91 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Fri, 29 May 2026 17:09:59 -0400 Subject: [PATCH 2/5] fix(skill): add git fetch before pre-flight diffs, drop unused lint tool Pre-flight steps use origin/main refs but ran before the pull step. Add explicit git fetch at the top of pre-flight to ensure fresh state. Remove Bash(make lint:*) from allowed-tools since it is not referenced in any skill step. Signed-off-by: Wayne Sun --- skills/cutting-releases/SKILL.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/skills/cutting-releases/SKILL.md b/skills/cutting-releases/SKILL.md index bc65a9f22..8ba812ff1 100644 --- a/skills/cutting-releases/SKILL.md +++ b/skills/cutting-releases/SKILL.md @@ -4,7 +4,7 @@ description: > Use when the user wants to tag a release, cut a release candidate, or ship a new version. Also use when asking about release process, versioning, or how GoReleaser is configured. -allowed-tools: Read, Grep, Glob, AskUserQuestion, Bash(git tag:*), Bash(git log:*), Bash(git diff:*), Bash(git pull:*), Bash(git push:*), Bash(gh release:*), Bash(gh run:*), Bash(git checkout:*), Bash(git fetch:*), Bash(make lint:*), Bash(bash skills/cutting-releases/scripts/install-binary.sh:*) +allowed-tools: Read, Grep, Glob, AskUserQuestion, Bash(git tag:*), Bash(git log:*), Bash(git diff:*), Bash(git pull:*), Bash(git push:*), Bash(gh release:*), Bash(gh run:*), Bash(git checkout:*), Bash(git fetch:*), Bash(bash skills/cutting-releases/scripts/install-binary.sh:*) --- # Cutting Releases @@ -20,6 +20,12 @@ Run this audit **before** tagging. The goal is to verify that moving the `v0` reusable-workflow tag and publishing new scaffold templates will not break downstream orgs. +Start by fetching the latest remote state: + +``` +git fetch origin +``` + ### A. Diff reusable workflows Compare the current `v0` tag to `main` for all reusable workflows: From 1b87ea9380cda11612b4cabddf8749bcdf9a2079 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Fri, 29 May 2026 17:16:01 -0400 Subject: [PATCH 3/5] refactor(skill): deepen pre-flight analysis and scope post-flight checks Pre-flight now audits workflow inputs/outputs/secrets/permissions, reads CLI function diffs for behavioral changes, and identifies post-flight check areas to focus verification. Post-flight now waits for CI workflows first, checks fullsend-ai repos by default (always accessible to repo admins), and only asks about additional downstream repos the user has access to. Removed duplicate wait/verify steps from the process section. Signed-off-by: Wayne Sun --- skills/cutting-releases/SKILL.md | 208 +++++++++++++++++++++---------- 1 file changed, 139 insertions(+), 69 deletions(-) diff --git a/skills/cutting-releases/SKILL.md b/skills/cutting-releases/SKILL.md index 8ba812ff1..4ec7a2182 100644 --- a/skills/cutting-releases/SKILL.md +++ b/skills/cutting-releases/SKILL.md @@ -17,8 +17,8 @@ title comes from the tag annotation via `name_template` in `.goreleaser.yml`. ## Pre-Flight Release Check Run this audit **before** tagging. The goal is to verify that moving -the `v0` reusable-workflow tag and publishing new scaffold templates -will not break downstream orgs. +the `v0` reusable-workflow tag will not break downstream consumers, +and to identify what needs post-flight verification. Start by fetching the latest remote state: @@ -26,40 +26,77 @@ Start by fetching the latest remote state: git fetch origin ``` -### A. Diff reusable workflows - -Compare the current `v0` tag to `main` for all reusable workflows: +### A. Audit reusable workflow changes ``` git diff v0..origin/main -- .github/workflows/reusable-*.yml ``` -For each changed workflow, classify: +For each changed workflow, read the full diff and check: + +- **Inputs:** Were any inputs removed or renamed? Were required inputs + added without defaults? These are breaking — callers will fail. +- **Outputs:** Were any job outputs removed or renamed? Callers that + reference them will break. +- **Secrets:** Were new secrets added to `secrets:` blocks? Callers + must already have those secrets or the workflow will fail silently. +- **Environment variables:** New env vars passed to steps are additive + and safe. Changed env var names used in conditionals may alter + behavior. +- **Job/step IDs:** Renamed job IDs break `needs:` references in + caller workflows. +- **Permissions:** Changes to `permissions:` blocks may fail if the + calling workflow's token doesn't grant the new scopes. + +Classify each change as: - **Additive** (new optional inputs, new env vars) — safe. -- **Default change** (different default values) — check downstream callers. -- **Breaking** (removed inputs, renamed jobs, changed required outputs) — block. +- **Default change** (different default values) — note for migration. +- **Breaking** (removed/renamed inputs, outputs, jobs, new required + secrets) — block the release until resolved. -### B. Diff scaffold templates +### B. Audit scaffold and template changes ``` git diff v0..origin/main -- internal/scaffold/fullsend-repo/ ``` -Scaffold files are deployed at `github setup` time, not consumed live. -Changes here affect **new installs and re-scaffolds only**, not running -workflows. Still note anything that alters agent behavior (skill files, -harness configs, hook scripts). +Scaffold files are deployed at `github setup` time, not consumed live +via `@v0`. Changes here affect **new installs and re-scaffolds only**. +Review for: -### C. Review CLI changes +- **Agent definitions** (`agents/`): Changed models, tools, or + instructions alter agent behavior on next scaffold. +- **Harness configs** (`harness/`): Changed resource limits, allowed + tools, or validation rules. +- **Hook scripts** (`scripts/`): Changed pre/post hooks run inside + agent sandboxes. +- **Skill files** (`skills/`): New or changed agent skills. +- **Workflow templates** (`.github/workflows/`): Templates that get + copied into target repos at scaffold time. + +These do not require post-flight verification against running systems, +but note significant behavior changes for the release summary. + +### C. Audit CLI and function changes ``` -git log --oneline v0..origin/main -- cmd/ internal/cli/ +git log --oneline v0..origin/main -- cmd/ internal/ ``` -Check for: -- Renamed flags or sub-commands — deprecated aliases must be preserved. -- Changed defaults (pool names, regions, project IDs) — document migration. -- Removed functionality — block or add deprecation notice. +For commits touching `cmd/` or `internal/cli/`, read the diffs and +check: + +- **Renamed flags or sub-commands:** Deprecated aliases must be + preserved via `MarkDeprecated` + `MarkHidden`. If a flag was + removed without an alias, this is breaking. +- **Changed defaults:** Pool names, regions, WIF provider names, or + project ID defaults that differ from the previous release require + a migration note in the release summary. +- **New sub-commands or flags:** Additive, safe. Note for changelog. +- **Behavioral changes in `internal/`:** Read the changed functions + to understand if existing workflows (mint enroll/unenroll, inference + provision, app setup) behave differently. Check backward compat by + verifying the old invocation still works. ### D. Check CI on main @@ -70,7 +107,21 @@ gh run list --branch=main --limit=5 All recent runs should be passing. If E2E tests are failing, investigate before releasing. -### E. Present summary +### E. Identify post-flight check areas + +Based on the changes found in steps A–C, determine what needs +post-flight verification after the `v0` tag moves: + +- **Reusable workflow changes** → verify workflow runs in fullsend-ai + repos resolve `@v0` correctly and pass. +- **New secrets or permissions** → verify affected workflows don't + fail on missing secrets. +- **CLI default changes** → note migration steps for existing + installs in the release summary. +- **No reusable workflow changes** → post-flight can be limited to + confirming the release artifacts built correctly. + +### F. Present summary Summarize findings to the user in a table: @@ -78,7 +129,9 @@ Summarize findings to the user in a table: |------|---------|-----------| | Reusable workflows | ... | No/Yes | | Scaffold templates | ... | No/Yes | -| CLI | ... | No/Yes | +| CLI / internal | ... | No/Yes | + +List the post-flight check areas identified in step E. Give a **GO / NO-GO** verdict. Do not proceed until the user confirms. @@ -187,30 +240,13 @@ git push origin v0 --force This updates all `@v0` workflow references immediately. The Sandbox Images workflow (triggered by tag push) will also run. -### 9. Wait for workflows - -Wait for both the Release workflow (triggered by the `v*` tag) and -the Sandbox Images workflow (triggered by the `v0` tag move) to -complete: +### 9. Run post-flight verification -``` -gh run list --workflow=release.yml --limit=1 -gh run list --workflow=sandbox-images.yml --limit=1 -``` - -Both must pass before proceeding. - -### 10. Verify the release +Proceed to the **Post-Flight Verification** section below. This +waits for CI workflows, verifies release artifacts, and checks +downstream `@v0` resolution. -Once the workflows complete, confirm the release was created: - -``` -gh release view -``` - -Check that the title, changelog, and binary assets look correct. - -### 11. Install the binary locally +### 10. Install the binary locally Ask the user where to install (default: `~/.local/bin/`), then run the install script from this skill's base directory: @@ -225,57 +261,91 @@ against the release's `checksums.txt`, and installs the binary as ## Post-Flight Verification -After the release and sandbox-images workflows pass, verify that -downstream orgs can resolve the new `@v0` tag. +Run after the version tag is pushed, the `v0` tag is moved, and the +CI workflows complete. Focus on the areas identified during pre-flight +step E. -### A. Check downstream orgs +### A. Wait for CI workflows -Use `AskUserQuestion` to ask which downstream orgs and repos to verify: +Wait for the Release workflow (triggered by the `v*` tag) and the +Sandbox Images workflow (triggered by the `v0` tag move) to complete: -> Which orgs/repos should I check for post-release verification? -> (e.g. "fullsend-ai, konflux-ci/integration-service, redhat-developer/rhdh-agentic") +``` +gh run list --workflow=release.yml --limit=1 +gh run list --workflow=sandbox-images.yml --limit=1 +``` -For each org or repo provided, check recent workflow runs: +Both must pass before proceeding. If either fails, investigate and +resolve before continuing — a broken release or sandbox image affects +all downstream consumers. + +### B. Verify the release artifacts ``` -gh run list --repo /.fullsend --limit=3 -gh run list --repo / --limit=3 +gh release view +``` + +Check that the title, changelog, and binary assets look correct. +Verify the release is not marked as a draft. + +### C. Check fullsend-ai repos + +The skill user is a fullsend repo admin, so fullsend-ai org repos +are always accessible. Check recent workflow runs in the org's repos +that consume `@v0` reusable workflows: + +``` +gh run list --repo fullsend-ai/fullsend --limit=3 +gh run list --repo fullsend-ai/.fullsend --limit=3 ``` Look for runs that started **after** the `v0` tag move. Confirm they completed without workflow-resolution errors (e.g. "could not find -reusable workflow"). +reusable workflow"). If no runs occurred naturally, check for any +recent failed or cancelled runs that can be retriggered: -### B. Retrigger a verification run +``` +gh run list --repo fullsend-ai/.fullsend --status=failure --limit=3 +``` -If no natural workflow runs occurred after the tag move, find a recent -failed or cancelled run and present it to the user for confirmation -before retriggering: +Present any candidate to the user for confirmation before retriggering: -> I found run `` (failed) in `/`. Retrigger it -> to verify `@v0` resolves? +> I found run `` (failed) in `fullsend-ai/.fullsend`. +> Retrigger it to verify `@v0` resolves? Once confirmed: ``` -gh run rerun --failed --repo / +gh run rerun --failed --repo fullsend-ai/.fullsend ``` -Watch for the run to complete. A successful run confirms the new `v0` -tag is working. +### D. Check additional downstream repos (optional) + +Use `AskUserQuestion` to ask if the user has access to additional +downstream orgs: + +> Do you have access to any other downstream orgs/repos to verify? +> (e.g. "konflux-ci, redhat-developer/rhdh-agentic") +> Leave blank to skip. -### C. Present post-flight summary +If the user provides repos, repeat the same checks from step C for +each one. If blank, skip this step — not all admins have access to +every enrolled org. + +### E. Present post-flight summary Summarize results to the user: | Org/Repo | `@v0` Refs | Status | |----------|-----------|--------| -| org/.fullsend | Confirmed | Passing | -| org/target-repo | Confirmed | Passing | - -Note any failures that are **unrelated** to the tag move (e.g. agent -runtime errors, external API issues) vs. failures caused by the -release. +| fullsend-ai/.fullsend | Confirmed | Passing | +| ... | ... | ... | + +Distinguish between: +- **Release-related failures** — workflow resolution errors, missing + secrets, or permission failures caused by the tag move. +- **Unrelated failures** — agent runtime errors, external API issues, + or pre-existing test failures. --- From bf67447367071fd2299ebffc05c7e1393959ad56 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Fri, 29 May 2026 17:22:48 -0400 Subject: [PATCH 4/5] refactor(skill): split cutting-releases into phase files MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Extract pre-flight and post-flight procedures into separate files for progressive disclosure. SKILL.md routes to each phase file with Read instructions. - SKILL.md: 152 lines (overview + process steps 1-10 + notes) - pre-flight.md: 122 lines (audit steps A-F) - post-flight.md: 89 lines (verification steps A-E) Each file stays under 150 lines. No content loss — just reorganization. Signed-off-by: Wayne Sun --- skills/cutting-releases/SKILL.md | 224 +------------------------ skills/cutting-releases/post-flight.md | 89 ++++++++++ skills/cutting-releases/pre-flight.md | 122 ++++++++++++++ 3 files changed, 219 insertions(+), 216 deletions(-) create mode 100644 skills/cutting-releases/post-flight.md create mode 100644 skills/cutting-releases/pre-flight.md diff --git a/skills/cutting-releases/SKILL.md b/skills/cutting-releases/SKILL.md index 4ec7a2182..39f9de4c7 100644 --- a/skills/cutting-releases/SKILL.md +++ b/skills/cutting-releases/SKILL.md @@ -14,131 +14,12 @@ the `.github/workflows/release.yml` workflow runs GoReleaser, which builds binaries, generates a changelog, and creates the GitHub release. The release title comes from the tag annotation via `name_template` in `.goreleaser.yml`. -## Pre-Flight Release Check - -Run this audit **before** tagging. The goal is to verify that moving -the `v0` reusable-workflow tag will not break downstream consumers, -and to identify what needs post-flight verification. - -Start by fetching the latest remote state: - -``` -git fetch origin -``` - -### A. Audit reusable workflow changes - -``` -git diff v0..origin/main -- .github/workflows/reusable-*.yml -``` - -For each changed workflow, read the full diff and check: - -- **Inputs:** Were any inputs removed or renamed? Were required inputs - added without defaults? These are breaking — callers will fail. -- **Outputs:** Were any job outputs removed or renamed? Callers that - reference them will break. -- **Secrets:** Were new secrets added to `secrets:` blocks? Callers - must already have those secrets or the workflow will fail silently. -- **Environment variables:** New env vars passed to steps are additive - and safe. Changed env var names used in conditionals may alter - behavior. -- **Job/step IDs:** Renamed job IDs break `needs:` references in - caller workflows. -- **Permissions:** Changes to `permissions:` blocks may fail if the - calling workflow's token doesn't grant the new scopes. - -Classify each change as: -- **Additive** (new optional inputs, new env vars) — safe. -- **Default change** (different default values) — note for migration. -- **Breaking** (removed/renamed inputs, outputs, jobs, new required - secrets) — block the release until resolved. - -### B. Audit scaffold and template changes - -``` -git diff v0..origin/main -- internal/scaffold/fullsend-repo/ -``` - -Scaffold files are deployed at `github setup` time, not consumed live -via `@v0`. Changes here affect **new installs and re-scaffolds only**. -Review for: - -- **Agent definitions** (`agents/`): Changed models, tools, or - instructions alter agent behavior on next scaffold. -- **Harness configs** (`harness/`): Changed resource limits, allowed - tools, or validation rules. -- **Hook scripts** (`scripts/`): Changed pre/post hooks run inside - agent sandboxes. -- **Skill files** (`skills/`): New or changed agent skills. -- **Workflow templates** (`.github/workflows/`): Templates that get - copied into target repos at scaffold time. - -These do not require post-flight verification against running systems, -but note significant behavior changes for the release summary. - -### C. Audit CLI and function changes - -``` -git log --oneline v0..origin/main -- cmd/ internal/ -``` - -For commits touching `cmd/` or `internal/cli/`, read the diffs and -check: - -- **Renamed flags or sub-commands:** Deprecated aliases must be - preserved via `MarkDeprecated` + `MarkHidden`. If a flag was - removed without an alias, this is breaking. -- **Changed defaults:** Pool names, regions, WIF provider names, or - project ID defaults that differ from the previous release require - a migration note in the release summary. -- **New sub-commands or flags:** Additive, safe. Note for changelog. -- **Behavioral changes in `internal/`:** Read the changed functions - to understand if existing workflows (mint enroll/unenroll, inference - provision, app setup) behave differently. Check backward compat by - verifying the old invocation still works. - -### D. Check CI on main - -``` -gh run list --branch=main --limit=5 -``` - -All recent runs should be passing. If E2E tests are failing, investigate -before releasing. - -### E. Identify post-flight check areas - -Based on the changes found in steps A–C, determine what needs -post-flight verification after the `v0` tag moves: - -- **Reusable workflow changes** → verify workflow runs in fullsend-ai - repos resolve `@v0` correctly and pass. -- **New secrets or permissions** → verify affected workflows don't - fail on missing secrets. -- **CLI default changes** → note migration steps for existing - installs in the release summary. -- **No reusable workflow changes** → post-flight can be limited to - confirming the release artifacts built correctly. - -### F. Present summary - -Summarize findings to the user in a table: - -| Area | Changes | Breaking? | -|------|---------|-----------| -| Reusable workflows | ... | No/Yes | -| Scaffold templates | ... | No/Yes | -| CLI / internal | ... | No/Yes | - -List the post-flight check areas identified in step E. - -Give a **GO / NO-GO** verdict. Do not proceed until the user confirms. - ---- - ## Process +Before starting step 1, read +[pre-flight.md](pre-flight.md) in this skill's directory and complete +the pre-flight audit. Do not proceed until the user confirms GO. + Follow these steps in order. ### 1. Confirm the branch @@ -242,9 +123,10 @@ Images workflow (triggered by tag push) will also run. ### 9. Run post-flight verification -Proceed to the **Post-Flight Verification** section below. This -waits for CI workflows, verifies release artifacts, and checks -downstream `@v0` resolution. +Read [post-flight.md](post-flight.md) in this skill's directory and +follow the post-flight verification procedure. This waits for CI +workflows, verifies release artifacts, and checks downstream `@v0` +resolution. ### 10. Install the binary locally @@ -259,96 +141,6 @@ The script downloads the release archive, verifies its SHA-256 checksum against the release's `checksums.txt`, and installs the binary as `fullsend-` so multiple versions can coexist. -## Post-Flight Verification - -Run after the version tag is pushed, the `v0` tag is moved, and the -CI workflows complete. Focus on the areas identified during pre-flight -step E. - -### A. Wait for CI workflows - -Wait for the Release workflow (triggered by the `v*` tag) and the -Sandbox Images workflow (triggered by the `v0` tag move) to complete: - -``` -gh run list --workflow=release.yml --limit=1 -gh run list --workflow=sandbox-images.yml --limit=1 -``` - -Both must pass before proceeding. If either fails, investigate and -resolve before continuing — a broken release or sandbox image affects -all downstream consumers. - -### B. Verify the release artifacts - -``` -gh release view -``` - -Check that the title, changelog, and binary assets look correct. -Verify the release is not marked as a draft. - -### C. Check fullsend-ai repos - -The skill user is a fullsend repo admin, so fullsend-ai org repos -are always accessible. Check recent workflow runs in the org's repos -that consume `@v0` reusable workflows: - -``` -gh run list --repo fullsend-ai/fullsend --limit=3 -gh run list --repo fullsend-ai/.fullsend --limit=3 -``` - -Look for runs that started **after** the `v0` tag move. Confirm they -completed without workflow-resolution errors (e.g. "could not find -reusable workflow"). If no runs occurred naturally, check for any -recent failed or cancelled runs that can be retriggered: - -``` -gh run list --repo fullsend-ai/.fullsend --status=failure --limit=3 -``` - -Present any candidate to the user for confirmation before retriggering: - -> I found run `` (failed) in `fullsend-ai/.fullsend`. -> Retrigger it to verify `@v0` resolves? - -Once confirmed: - -``` -gh run rerun --failed --repo fullsend-ai/.fullsend -``` - -### D. Check additional downstream repos (optional) - -Use `AskUserQuestion` to ask if the user has access to additional -downstream orgs: - -> Do you have access to any other downstream orgs/repos to verify? -> (e.g. "konflux-ci, redhat-developer/rhdh-agentic") -> Leave blank to skip. - -If the user provides repos, repeat the same checks from step C for -each one. If blank, skip this step — not all admins have access to -every enrolled org. - -### E. Present post-flight summary - -Summarize results to the user: - -| Org/Repo | `@v0` Refs | Status | -|----------|-----------|--------| -| fullsend-ai/.fullsend | Confirmed | Passing | -| ... | ... | ... | - -Distinguish between: -- **Release-related failures** — workflow resolution errors, missing - secrets, or permission failures caused by the tag move. -- **Unrelated failures** — agent runtime errors, external API issues, - or pre-existing test failures. - ---- - ## Notes - **Pre-releases:** Tags with `-rc.N`, `-alpha.N`, or `-beta.N` suffixes are diff --git a/skills/cutting-releases/post-flight.md b/skills/cutting-releases/post-flight.md new file mode 100644 index 000000000..e565f86ca --- /dev/null +++ b/skills/cutting-releases/post-flight.md @@ -0,0 +1,89 @@ +# Post-Flight Verification + +Part of the [cutting-releases](SKILL.md) skill. + +Run after the version tag is pushed, the `v0` tag is moved, and the +CI workflows complete. Focus on the areas identified during pre-flight +step E. + +## A. Wait for CI workflows + +Wait for the Release workflow (triggered by the `v*` tag) and the +Sandbox Images workflow (triggered by the `v0` tag move) to complete: + +``` +gh run list --workflow=release.yml --limit=1 +gh run list --workflow=sandbox-images.yml --limit=1 +``` + +Both must pass before proceeding. If either fails, investigate and +resolve before continuing — a broken release or sandbox image affects +all downstream consumers. + +## B. Verify the release artifacts + +``` +gh release view +``` + +Check that the title, changelog, and binary assets look correct. +Verify the release is not marked as a draft. + +## C. Check fullsend-ai repos + +The skill user is a fullsend repo admin, so fullsend-ai org repos +are always accessible. Check recent workflow runs in the org's repos +that consume `@v0` reusable workflows: + +``` +gh run list --repo fullsend-ai/fullsend --limit=3 +gh run list --repo fullsend-ai/.fullsend --limit=3 +``` + +Look for runs that started **after** the `v0` tag move. Confirm they +completed without workflow-resolution errors (e.g. "could not find +reusable workflow"). If no runs occurred naturally, check for any +recent failed or cancelled runs that can be retriggered: + +``` +gh run list --repo fullsend-ai/.fullsend --status=failure --limit=3 +``` + +Present any candidate to the user for confirmation before retriggering: + +> I found run `` (failed) in `fullsend-ai/.fullsend`. +> Retrigger it to verify `@v0` resolves? + +Once confirmed: + +``` +gh run rerun --failed --repo fullsend-ai/.fullsend +``` + +## D. Check additional downstream repos (optional) + +Use `AskUserQuestion` to ask if the user has access to additional +downstream orgs: + +> Do you have access to any other downstream orgs/repos to verify? +> (e.g. "konflux-ci, redhat-developer/rhdh-agentic") +> Leave blank to skip. + +If the user provides repos, repeat the same checks from step C for +each one. If blank, skip this step — not all admins have access to +every enrolled org. + +## E. Present post-flight summary + +Summarize results to the user: + +| Org/Repo | `@v0` Refs | Status | +|----------|-----------|--------| +| fullsend-ai/.fullsend | Confirmed | Passing | +| ... | ... | ... | + +Distinguish between: +- **Release-related failures** — workflow resolution errors, missing + secrets, or permission failures caused by the tag move. +- **Unrelated failures** — agent runtime errors, external API issues, + or pre-existing test failures. diff --git a/skills/cutting-releases/pre-flight.md b/skills/cutting-releases/pre-flight.md new file mode 100644 index 000000000..7616d6113 --- /dev/null +++ b/skills/cutting-releases/pre-flight.md @@ -0,0 +1,122 @@ +# Pre-Flight Release Check + +Part of the [cutting-releases](SKILL.md) skill. + +Run this audit **before** tagging. The goal is to verify that moving +the `v0` reusable-workflow tag will not break downstream consumers, +and to identify what needs post-flight verification. + +Start by fetching the latest remote state: + +``` +git fetch origin +``` + +## A. Audit reusable workflow changes + +``` +git diff v0..origin/main -- .github/workflows/reusable-*.yml +``` + +For each changed workflow, read the full diff and check: + +- **Inputs:** Were any inputs removed or renamed? Were required inputs + added without defaults? These are breaking — callers will fail. +- **Outputs:** Were any job outputs removed or renamed? Callers that + reference them will break. +- **Secrets:** Were new secrets added to `secrets:` blocks? Callers + must already have those secrets or the workflow will fail silently. +- **Environment variables:** New env vars passed to steps are additive + and safe. Changed env var names used in conditionals may alter + behavior. +- **Job/step IDs:** Renamed job IDs break `needs:` references in + caller workflows. +- **Permissions:** Changes to `permissions:` blocks may fail if the + calling workflow's token doesn't grant the new scopes. + +Classify each change as: +- **Additive** (new optional inputs, new env vars) — safe. +- **Default change** (different default values) — note for migration. +- **Breaking** (removed/renamed inputs, outputs, jobs, new required + secrets) — block the release until resolved. + +## B. Audit scaffold and template changes + +``` +git diff v0..origin/main -- internal/scaffold/fullsend-repo/ +``` + +Scaffold files are deployed at `github setup` time, not consumed live +via `@v0`. Changes here affect **new installs and re-scaffolds only**. +Review for: + +- **Agent definitions** (`agents/`): Changed models, tools, or + instructions alter agent behavior on next scaffold. +- **Harness configs** (`harness/`): Changed resource limits, allowed + tools, or validation rules. +- **Hook scripts** (`scripts/`): Changed pre/post hooks run inside + agent sandboxes. +- **Skill files** (`skills/`): New or changed agent skills. +- **Workflow templates** (`.github/workflows/`): Templates that get + copied into target repos at scaffold time. + +These do not require post-flight verification against running systems, +but note significant behavior changes for the release summary. + +## C. Audit CLI and function changes + +``` +git log --oneline v0..origin/main -- cmd/ internal/ +``` + +For commits touching `cmd/` or `internal/cli/`, read the diffs and +check: + +- **Renamed flags or sub-commands:** Deprecated aliases must be + preserved via `MarkDeprecated` + `MarkHidden`. If a flag was + removed without an alias, this is breaking. +- **Changed defaults:** Pool names, regions, WIF provider names, or + project ID defaults that differ from the previous release require + a migration note in the release summary. +- **New sub-commands or flags:** Additive, safe. Note for changelog. +- **Behavioral changes in `internal/`:** Read the changed functions + to understand if existing workflows (mint enroll/unenroll, inference + provision, app setup) behave differently. Check backward compat by + verifying the old invocation still works. + +## D. Check CI on main + +``` +gh run list --branch=main --limit=5 +``` + +All recent runs should be passing. If E2E tests are failing, investigate +before releasing. + +## E. Identify post-flight check areas + +Based on the changes found in steps A–C, determine what needs +post-flight verification after the `v0` tag moves: + +- **Reusable workflow changes** → verify workflow runs in fullsend-ai + repos resolve `@v0` correctly and pass. +- **New secrets or permissions** → verify affected workflows don't + fail on missing secrets. +- **CLI default changes** → note migration steps for existing + installs in the release summary. +- **No reusable workflow changes** → post-flight can be limited to + confirming the release artifacts built correctly. + +## F. Present summary + +Summarize findings to the user in a table: + +| Area | Changes | Breaking? | +|------|---------|-----------| +| Reusable workflows | ... | No/Yes | +| Scaffold templates | ... | No/Yes | +| CLI / internal | ... | No/Yes | + +List the post-flight check areas identified in step E. + +Give a **GO / NO-GO** verdict. Do not proceed until the user confirms. From 2812a346113e986c9abd8559ca1118c9cb66c820 Mon Sep 17 00:00:00 2001 From: Wayne Sun Date: Fri, 29 May 2026 17:29:52 -0400 Subject: [PATCH 5/5] =?UTF-8?q?fix(skill):=20address=20review=20findings?= =?UTF-8?q?=20=E2=80=94=20confirm=20force-push,=20trim=20to=20150=20lines,?= =?UTF-8?q?=20add=20grep=20backstop?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add AskUserQuestion gate before force-pushing v0 tag (step 8) - Name AskUserQuestion explicitly in step 10 - Trim SKILL.md to 149 lines (was 156) by condensing verbose sections - Add grep backstop in pre-flight step A for mechanical verification of removed/renamed workflow identifiers Signed-off-by: Wayne Sun --- skills/cutting-releases/SKILL.md | 47 +++++++++++++-------------- skills/cutting-releases/pre-flight.md | 9 +++++ 2 files changed, 31 insertions(+), 25 deletions(-) diff --git a/skills/cutting-releases/SKILL.md b/skills/cutting-releases/SKILL.md index 39f9de4c7..8153d639e 100644 --- a/skills/cutting-releases/SKILL.md +++ b/skills/cutting-releases/SKILL.md @@ -10,9 +10,8 @@ allowed-tools: Read, Grep, Glob, AskUserQuestion, Bash(git tag:*), Bash(git log: # Cutting Releases Releases are driven by annotated git tags. When a tag matching `v*` is pushed, -the `.github/workflows/release.yml` workflow runs GoReleaser, which builds -binaries, generates a changelog, and creates the GitHub release. The release -title comes from the tag annotation via `name_template` in `.goreleaser.yml`. +`.github/workflows/release.yml` runs GoReleaser to build binaries, generate a +changelog, and create the GitHub release. ## Process @@ -65,9 +64,8 @@ Use `AskUserQuestion` to ask: > Any special title for this release? (e.g. "MVP Release Candidate 1") > Leave blank to use just the version tag. -The answer becomes the tag subject line. If blank, do **not** use the version -as the subject — leave the subject empty so that GoReleaser's `name_template` -renders just the tag without duplication. +The answer becomes the tag subject line. If blank, leave the subject empty +so GoReleaser's `name_template` renders just the tag without duplication. ### 5. Gather changes since last tag @@ -75,9 +73,8 @@ renders just the tag without duplication. git log --oneline ..HEAD ``` -Summarize the changes into categories (features, fixes, refactors). Exclude -commits that start with `docs:`, `test:`, or `chore:` — GoReleaser filters -these from the changelog anyway. +Summarize changes into categories (features, fixes, refactors). Exclude +`docs:`, `test:`, `chore:` commits — GoReleaser filters these anyway. ### 6. Create the annotated tag @@ -93,8 +90,8 @@ Build the tag message: git tag -a v0.X.0 -m "" ``` -The first line of the annotation flows into the GitHub release title via -GoReleaser's `name_template: "{{ .Tag }}{{ if and .TagSubject (ne .TagSubject .Tag) }}: {{ .TagSubject }}{{ end }}"`. +The first line of the annotation becomes the release title suffix via +GoReleaser's `name_template` (see `.goreleaser.yml`). ### 7. Push the tag @@ -110,43 +107,43 @@ gh run list --workflow=release.yml --limit=1 ### 8. Move the `v0` tag -Downstream orgs reference reusable workflows via `@v0`. After the -version tag is pushed, move `v0` to the same commit: +Downstream orgs reference reusable workflows via `@v0`. Use +`AskUserQuestion` to confirm before force-pushing: + +> About to force-push `v0` to ``. This immediately changes what +> all downstream `@v0` consumers resolve. Proceed? + +Once confirmed: ``` git tag -f v0 git push origin v0 --force ``` -This updates all `@v0` workflow references immediately. The Sandbox -Images workflow (triggered by tag push) will also run. +The Sandbox Images workflow (triggered by tag push) will also run. ### 9. Run post-flight verification Read [post-flight.md](post-flight.md) in this skill's directory and -follow the post-flight verification procedure. This waits for CI -workflows, verifies release artifacts, and checks downstream `@v0` -resolution. +follow the post-flight verification procedure. ### 10. Install the binary locally -Ask the user where to install (default: `~/.local/bin/`), then run -the install script from this skill's base directory: +Use `AskUserQuestion` to ask where to install (default: `~/.local/bin/`), +then run the install script from this skill's base directory: ```bash bash /scripts/install-binary.sh [install-dir] ``` -The script downloads the release archive, verifies its SHA-256 checksum -against the release's `checksums.txt`, and installs the binary as -`fullsend-` so multiple versions can coexist. +The script downloads the archive, verifies its SHA-256 checksum, and +installs the binary as `fullsend-` so multiple versions can coexist. ## Notes - **Pre-releases:** Tags with `-rc.N`, `-alpha.N`, or `-beta.N` suffixes are automatically marked as pre-releases by GoReleaser. - **Never delete a published tag.** If a release is bad, cut a new patch or RC. -- **The changelog** is auto-generated from commit messages. Conventional commit - prefixes (`feat:`, `fix:`, etc.) produce clean changelogs. +- **The changelog** is auto-generated from conventional commit prefixes. - **The `v0` tag** is a moving tag consumed by downstream orgs for reusable workflows. Always move it as part of the release process (step 8). diff --git a/skills/cutting-releases/pre-flight.md b/skills/cutting-releases/pre-flight.md index 7616d6113..2d02de65f 100644 --- a/skills/cutting-releases/pre-flight.md +++ b/skills/cutting-releases/pre-flight.md @@ -34,6 +34,15 @@ For each changed workflow, read the full diff and check: - **Permissions:** Changes to `permissions:` blocks may fail if the calling workflow's token doesn't grant the new scopes. +As a mechanical backstop, grep for removed or renamed identifiers: + +``` +git diff v0..origin/main -- .github/workflows/reusable-*.yml | grep -E '^\-\s+(\w+:)' | grep -v '^\-\s*#' +``` + +Cross-reference any removed lines against caller workflows to confirm +they are unused before classifying as safe. + Classify each change as: - **Additive** (new optional inputs, new env vars) — safe. - **Default change** (different default values) — note for migration.