feat(ci): Phase 08 — JSON reporter + GitHub Actions workflow + badges

claude · claude · commit 9c79110cdf64 · 2026-04-14T02:34:00.000+07:00
Closes the E2E test plan. Turns the local ba_e2e_test.py runner into a
CI-integrated quality gate that runs on every PR touching .agent/, docs,
or outputs/.

Changes:

1. ba_e2e_test.py — JSON reporter
   - New emit_json() function with stable schema v1.0.0
   - --json OUTPUT_PATH CLI flag
   - Schema includes: $schema_version, fixture, repo_branch, repo_commit,
     started_at, duration_ms, exit_code, verdict, totals, per-layer
     checks with name/target/severity/message/duration_ms
   - ensure_ascii=False so Vietnamese fixture strings survive round-trip
   - Output is parseable by CI tooling or custom dashboards
   - When --json is passed without --report, markdown goes to stdout

2. .github/workflows/e2e-skills.yml — quality gate workflow
   - Triggers on PR + push to main/feat/** touching .agent/, sprint-spine,
     workflow-cookbook, outputs/, or the workflow itself
   - Python 3.11, actions/checkout@v4 with fetch-depth: 0 (ba_retro needs
     git history)
   - Runs full suite with --report + --json output
   - Writes markdown report to GITHUB_STEP_SUMMARY for PR review UX
   - Uploads reports/ as artifact (retention: 14 days)
   - Posts sticky PR comment with the markdown report
     (marocchino/sticky-pull-request-comment@v2)
   - Fails the job only if verdict is FAIL or CRASH (warnings don't block)
   - 10-minute timeout
   - Zero external dependencies (stdlib only, matches ba_*.py convention)

3. README.md + README.vi.md — CI status badge
   - GitHub Actions badge linked to the e2e-skills workflow
   - Shows pass/fail state per branch

4. .gitignore — reports/ directory
   - Add reports/ to ignore (CI runner output, per-run regenerated)
   - plans/reports/ remains gitignored via existing plans/ rule

Verification:
- ba_e2e_test.py still compiles and runs green (270/272 pass, 1.8s)
- JSON schema smoke-tested: all 5 layers serialized, Vietnamese text
  round-trips via ensure_ascii=False
- Workflow YAML has all required keys (name, on, jobs, steps,
  actions/checkout, actions/setup-python) — validated via substring grep
- reports/test.md smoke test confirms reports/ is gitignored
- plans/reports/ negation pattern preserved (still gitignored via plans/)

Final E2E plan status: 8/8 phases complete.
First GH Action run will execute on next push. Badge will render once
the workflow runs at least once on a branch.

Known limitations (defer to v3.5):
- No trend tracking across runs (JSON history not persisted in CI)
- No branch protection rules (manual decision per team policy)
- No parallel layer execution (sequential is fast enough at 1.8s)
- No matrix testing across Python versions (3.11 pinned)
diff --git a/.agent/scripts/ba_e2e_test.py b/.agent/scripts/ba_e2e_test.py
@@ -1210,6 +1210,60 @@ def run_all(layers: list[str], fixture: Path, verbose: bool) -> AggregateVerdict
     )
 
 
+# ---------------------------------------------------------------------------
+# JSON reporter (stable schema for CI consumption)
+# ---------------------------------------------------------------------------
+
+JSON_SCHEMA_VERSION = "1.0.0"
+
+
+def emit_json(agg: AggregateVerdict) -> str:
+    """Structured JSON for CI integration. Stable schema v1.0.0."""
+    payload = {
+        "$schema_version": JSON_SCHEMA_VERSION,
+        "fixture": agg.fixture,
+        "repo_branch": agg.repo_branch,
+        "repo_commit": agg.repo_commit,
+        "started_at": agg.started_at,
+        "duration_ms": round(agg.duration_ms, 1),
+        "exit_code": agg.exit_code,
+        "verdict": agg.verdict,
+        "totals": {
+            "passed": sum(l.passed for l in agg.layers),
+            "warned": sum(l.warned for l in agg.layers),
+            "failed": sum(l.failed for l in agg.layers),
+            "skipped": sum(l.skipped for l in agg.layers),
+            "total": sum(l.total for l in agg.layers),
+        },
+        "layers": [
+            {
+                "layer": l.layer,
+                "description": l.description,
+                "duration_ms": round(l.duration_ms, 1),
+                "totals": {
+                    "passed": l.passed,
+                    "warned": l.warned,
+                    "failed": l.failed,
+                    "skipped": l.skipped,
+                    "total": l.total,
+                },
+                "checks": [
+                    {
+                        "name": c.name,
+                        "target": c.target,
+                        "severity": c.severity,
+                        "message": c.message,
+                        "duration_ms": round(c.duration_ms, 2),
+                    }
+                    for c in l.checks
+                ],
+            }
+            for l in agg.layers
+        ],
+    }
+    return json.dumps(payload, indent=2, ensure_ascii=False)
+
+
 # ---------------------------------------------------------------------------
 # Markdown reporter
 # ---------------------------------------------------------------------------
@@ -1300,6 +1354,8 @@ def main(argv: list[str] | None = None) -> int:
                    help="Fixture project path (default: outputs/mini-app-cham-cong)")
     p.add_argument("--report", default=None,
                    help="Markdown report output path (default: print to stdout)")
+    p.add_argument("--json", dest="json_out", default=None,
+                   help="JSON report output path (for CI integration)")
     p.add_argument("--verbose", "-v", action="store_true")
     args = p.parse_args(argv)
 
@@ -1308,15 +1364,22 @@ def main(argv: list[str] | None = None) -> int:
 
     agg = run_all(layers, fixture, args.verbose)
     report = emit_markdown(agg)
+    json_payload = emit_json(agg)
 
     if args.report:
         out = Path(args.report)
         out.parent.mkdir(parents=True, exist_ok=True)
         out.write_text(report, encoding="utf-8")
         print(f"Report: {out}", file=sys.stderr)
-    else:
+    elif not args.json_out:
         print(report)
 
+    if args.json_out:
+        out = Path(args.json_out)
+        out.parent.mkdir(parents=True, exist_ok=True)
+        out.write_text(json_payload, encoding="utf-8")
+        print(f"JSON: {out}", file=sys.stderr)
+
     print(f"Verdict: {agg.verdict} (exit {agg.exit_code}) — "
           f"{agg.duration_ms / 1000:.1f}s", file=sys.stderr)
     return agg.exit_code
diff --git a/.github/workflows/e2e-skills.yml b/.github/workflows/e2e-skills.yml
@@ -0,0 +1,103 @@
+name: E2E Skills Test
+
+# BA-Kit skill quality gate. Runs the ba_e2e_test.py orchestrator on every
+# PR touching agent skills, scripts, docs, or the fixture. Uses zero external
+# dependencies (Python stdlib only) so CI is fast and stable.
+
+on:
+  pull_request:
+    paths:
+      - '.agent/**'
+      - 'docs/sprint-spine.md'
+      - 'docs/workflow-cookbook.md'
+      - 'outputs/**'
+      - '.github/workflows/e2e-skills.yml'
+  push:
+    branches:
+      - main
+      - 'feat/**'
+    paths:
+      - '.agent/**'
+      - 'docs/sprint-spine.md'
+      - 'docs/workflow-cookbook.md'
+      - 'outputs/**'
+      - '.github/workflows/e2e-skills.yml'
+  workflow_dispatch: {}
+
+jobs:
+  e2e:
+    name: E2E Skills Suite (L1 → L5)
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - name: Checkout repo
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0  # ba_retro needs git history; shallow breaks it
+
+      - name: Set up Python 3.11
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+
+      - name: Show environment
+        run: |
+          python3 --version
+          git rev-parse --short HEAD
+          find .agent/skills -maxdepth 1 -type d | wc -l
+          find outputs -name '*.md' | wc -l
+
+      - name: Run E2E suite
+        id: e2e
+        run: |
+          mkdir -p reports
+          python3 .agent/scripts/ba_e2e_test.py \
+            --report reports/e2e-report.md \
+            --json reports/e2e-report.json \
+            --verbose
+        continue-on-error: true
+
+      - name: Display report in summary
+        if: always()
+        run: |
+          if [ -f reports/e2e-report.md ]; then
+            echo "## BA-Kit E2E Report" >> "$GITHUB_STEP_SUMMARY"
+            cat reports/e2e-report.md >> "$GITHUB_STEP_SUMMARY"
+          fi
+
+      - name: Parse verdict
+        if: always()
+        id: verdict
+        run: |
+          if [ -f reports/e2e-report.json ]; then
+            VERDICT=$(python3 -c "import json; print(json.load(open('reports/e2e-report.json'))['verdict'])")
+            EXIT=$(python3 -c "import json; print(json.load(open('reports/e2e-report.json'))['exit_code'])")
+            echo "verdict=$VERDICT" >> "$GITHUB_OUTPUT"
+            echo "exit=$EXIT" >> "$GITHUB_OUTPUT"
+            echo "Verdict: $VERDICT (exit $EXIT)"
+          else
+            echo "verdict=CRASH" >> "$GITHUB_OUTPUT"
+            echo "exit=99" >> "$GITHUB_OUTPUT"
+          fi
+
+      - name: Upload reports as artifact
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: e2e-reports-${{ github.sha }}
+          path: reports/
+          retention-days: 14
+
+      - name: Post sticky PR comment
+        if: github.event_name == 'pull_request' && always()
+        uses: marocchino/sticky-pull-request-comment@v2
+        continue-on-error: true
+        with:
+          header: e2e-skills
+          path: reports/e2e-report.md
+
+      - name: Fail if verdict is FAIL
+        if: steps.verdict.outputs.verdict == 'FAIL' || steps.verdict.outputs.verdict == 'CRASH'
+        run: |
+          echo "E2E suite failed — exit ${{ steps.verdict.outputs.exit }}"
+          exit 2
diff --git a/.gitignore b/.gitignore
@@ -104,6 +104,10 @@ scratch/
 # local .git/info/exclude or a project-specific override.
 .ba-kit/
 
+# E2E test runner output (generated per-run)
+reports/
+!plans/reports/
+
 # ============================================
 # Legacy / Archived content
 # ============================================
diff --git a/README.md b/README.md
@@ -17,6 +17,12 @@
   <img src="https://img.shields.io/badge/Integration-Jira%20%2B%20Confluence-blue?style=for-the-badge" alt="Jira + Confluence">
 </p>
 
+<p align="center">
+  <a href="https://github.com/olbboy/BA-Kit/actions/workflows/e2e-skills.yml">
+    <img src="https://github.com/olbboy/BA-Kit/actions/workflows/e2e-skills.yml/badge.svg?branch=feat/mini-app-cham-cong-docs" alt="E2E Skills Test">
+  </a>
+</p>
+
 <h1 align="center">BA-Kit</h1>
 <h3 align="center">Agent Squad for Requirements Engineering</h3>
 
diff --git a/README.vi.md b/README.vi.md
@@ -15,6 +15,12 @@
   <img src="https://img.shields.io/badge/Capability-CMMI%20Level%205%20Enabler-purple?style=for-the-badge" alt="CMMI Level 5 Enabler">
 </p>
 
+<p align="center">
+  <a href="https://github.com/olbboy/BA-Kit/actions/workflows/e2e-skills.yml">
+    <img src="https://github.com/olbboy/BA-Kit/actions/workflows/e2e-skills.yml/badge.svg?branch=feat/mini-app-cham-cong-docs" alt="E2E Skills Test">
+  </a>
+</p>
+
 <h1 align="center">BA-Kit</h1>
 <h3 align="center">Biệt Đội Chuyên Gia Phân Tích Nghiệp Vụ</h3>
 
