Skip to content

Isolate E2E matrix entries on per-entry branches and reset failure state per run#2876

Draft
Copilot wants to merge 2 commits into
mainfrom
copilot/e2e-status-report-fix
Draft

Isolate E2E matrix entries on per-entry branches and reset failure state per run#2876
Copilot wants to merge 2 commits into
mainfrom
copilot/e2e-status-report-fix

Conversation

Copilot AI commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

The nightly gh-aw matrix reported all entries as failed (52 failed test runs), but the failures were dominated by harness/reporting behavior rather than a single workflow defect. Matrix entries were sharing branch/failure state, causing noisy and misleading status reports.

  • Problem summary

    • Matrix entries were running in ways that could inherit stale fails.txt state and depended on pushing compiled workflows through shared branch behavior.
    • This inflated per-entry failure reporting (including test-copilot-call-workflow) and made status output less actionable.
  • Workflow execution isolation

    • Updated .github/workflows/e2e.yml to run each matrix entry on a dedicated branch derived from ref × samples:
      • e2e_branch="e2e-matrix-${slug}"
    • Passed --branch "$e2e_branch" to e2e.sh in both source and extension modes so compile/push/dispatch are scoped to that entry.
  • Per-entry failure-state reset

    • Removed carry-over failure state at the start of each matrix entry:
      • rm -f fails.txt
    • Ensures per-entry report artifacts reflect only failures produced by that entry.
  • Documentation/comments alignment

    • Updated in-file workflow comments and status-report narrative to describe dedicated-branch dispatch behavior instead of “push/dispatch from main”.
slug=$(echo "${E2E_REF}-samples-${E2E_USE_SAMPLES}" | tr '/' '_' | tr -cd 'A-Za-z0-9._-')
e2e_branch="e2e-matrix-${slug}"
rm -f fails.txt

./e2e.sh --gh-aw-ref "$E2E_REF" --branch "$e2e_branch" --workflow-dispatch-only --verbose $SAMPLES_FLAG
# extension mode:
./e2e.sh --branch "$e2e_branch" --workflow-dispatch-only --verbose $SAMPLES_FLAG

Copilot AI changed the title [WIP] Fix E2E failures in gh-aw matrix report Isolate E2E matrix entries on per-entry branches and reset failure state per run Jun 19, 2026
Copilot AI requested a review from pelikhan June 19, 2026 05:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants