Skip to content

Commit 4f2b528

Browse files
committed
feat: add path-filtered CLI UX review and scope existing reviews by change type
## Summary - Add new `claude-ux-review` job that evaluates CLI user experience: help text, error messages, log/JSON output consistency, and behavioral consistency with existing commands - Path-filter all three Claude review jobs so they only run when relevant files change: - **Standard code review** (`code`): `src/**`, `integration/**`, `deno.json` - **Adversarial review** (`core`): `src/domain/**`, `src/infrastructure/**`, `src/libswamp/**` - **UX review** (`ux`): `src/cli/commands/**`, `src/presentation/**`, `src/domain/errors.ts`, `src/libswamp/**` - PRs that only touch docs, skills, workflows, or scripts skip all Claude reviews and auto-merge faster - UX review uses Sonnet for speed; standard and adversarial reviews remain on Opus ## Test plan - [ ] Open a PR that only changes README.md — verify all three Claude reviews are skipped and auto-merge proceeds - [ ] Open a PR that changes a file in `src/presentation/` — verify standard code review and UX review run, adversarial review is skipped - [ ] Open a PR that changes a file in `src/domain/` — verify all three reviews run - [ ] Verify UX review approves quickly on a PR that touches `src/libswamp/` internals without changing user-facing output shapes - [ ] Verify UX review catches a missing `--json` output field or inconsistent flag name
1 parent 184d3e8 commit 4f2b528

1 file changed

Lines changed: 135 additions & 4 deletions

File tree

.github/workflows/ci.yml

Lines changed: 135 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,11 +17,14 @@ jobs:
1717
runs-on: ubuntu-latest
1818
outputs:
1919
skills: ${{ steps.filter.outputs.skills }}
20+
code: ${{ steps.filter.outputs.code }}
21+
core: ${{ steps.filter.outputs.core }}
22+
ux: ${{ steps.filter.outputs.ux }}
2023
steps:
2124
- name: Checkout code
2225
uses: actions/checkout@v4
2326

24-
- name: Check for skill changes
27+
- name: Check for changes
2528
uses: dorny/paths-filter@v3
2629
id: filter
2730
with:
@@ -31,6 +34,19 @@ jobs:
3134
- 'CLAUDE.md'
3235
- 'scripts/review_skills.ts'
3336
- 'scripts/eval_skill_triggers.ts'
37+
code:
38+
- 'src/**'
39+
- 'integration/**'
40+
- 'deno.json'
41+
core:
42+
- 'src/domain/**'
43+
- 'src/infrastructure/**'
44+
- 'src/libswamp/**'
45+
ux:
46+
- 'src/cli/commands/**'
47+
- 'src/presentation/**'
48+
- 'src/domain/errors.ts'
49+
- 'src/libswamp/**'
3450
3551
test:
3652
name: Lint, Test, and Format Check
@@ -144,10 +160,11 @@ jobs:
144160
# "Require a pull request before merging" + "Require approving reviews"
145161
# Claude will use --request-changes to block or --approve to allow
146162
name: Claude Code Review
147-
needs: [test, deps-audit, skill-review]
163+
needs: [changes, test, deps-audit, skill-review]
148164
runs-on: ubuntu-latest
149165
if: |
150166
!failure() && !cancelled() &&
167+
needs.changes.outputs.code == 'true' &&
151168
github.event.pull_request.draft == false
152169
concurrency:
153170
group: claude-review-${{ github.event.pull_request.number }}
@@ -206,10 +223,11 @@ jobs:
206223
207224
claude-adversarial-review:
208225
name: Adversarial Code Review
209-
needs: [test, deps-audit, skill-review]
226+
needs: [changes, test, deps-audit, skill-review]
210227
runs-on: ubuntu-latest
211228
if: |
212229
!failure() && !cancelled() &&
230+
needs.changes.outputs.core == 'true' &&
213231
github.event.pull_request.draft == false
214232
concurrency:
215233
group: claude-adversarial-review-${{ github.event.pull_request.number }}
@@ -337,10 +355,123 @@ jobs:
337355
--model claude-opus-4-6
338356
--allowedTools Read,Glob,Grep,Bash(gh pr review:*),Bash(gh pr view:*),Bash(gh pr diff:*)
339357
358+
claude-ux-review:
359+
name: CLI UX Review
360+
needs: [changes, test]
361+
runs-on: ubuntu-latest
362+
if: |
363+
!failure() && !cancelled() &&
364+
needs.changes.outputs.ux == 'true' &&
365+
github.event.pull_request.draft == false
366+
concurrency:
367+
group: claude-ux-review-${{ github.event.pull_request.number }}
368+
cancel-in-progress: true
369+
370+
steps:
371+
- name: Checkout code
372+
uses: actions/checkout@v4
373+
374+
- name: Setup Claude environment
375+
run: |
376+
mkdir -p ~/.claude
377+
378+
- name: CLI UX Review
379+
uses: anthropics/claude-code-action@v1
380+
with:
381+
github_token: ${{ secrets.GITHUB_TOKEN }}
382+
prompt: |
383+
REPO: ${{ github.repository }}
384+
PR NUMBER: ${{ github.event.pull_request.number }}
385+
386+
You are a CLI UX reviewer. Your job is to evaluate how this PR affects the
387+
user experience of the swamp CLI tool. You are reviewing from the perspective
388+
of someone USING the CLI, not reading the code.
389+
390+
First, read CLAUDE.md to understand the project's conventions, especially:
391+
- Every command must support both "log" and "json" output modes
392+
- The CLI uses Cliffy for commands and LogTape for log-mode output
393+
394+
Then read every changed file in this PR. Focus ONLY on files that affect
395+
what users see: commands, renderers, output formatters, and error handling.
396+
397+
## Review Dimensions
398+
399+
### 1. Help Text & Discoverability
400+
- Are new flags and options documented in the command's help text?
401+
- Are flag names consistent with existing commands? (check similar commands for patterns)
402+
- Are option descriptions clear and concise?
403+
- Do flag names use the same conventions as the rest of the CLI? (e.g., --verbose, --json, --field vs --filter)
404+
405+
### 2. Error Messages
406+
- When the command fails, does the user get a clear, actionable message?
407+
- Does the error tell the user WHAT went wrong and HOW to fix it?
408+
- Are error messages consistent in tone and format with existing commands?
409+
- Read `src/domain/errors.ts` to understand the UserError pattern
410+
411+
### 3. Log-Mode Output (human-readable)
412+
- Is the output readable and scannable?
413+
- Is the information hierarchy clear? (most important info first)
414+
- Is formatting consistent with other commands? (check similar renderers for patterns)
415+
- Are colors, icons, or formatting used consistently?
416+
417+
### 4. JSON-Mode Output (machine-readable)
418+
- Does the JSON output include all the data a script would need?
419+
- Are field names consistent with other commands' JSON output?
420+
- Is the shape documented or self-evident?
421+
- Are there fields that are present in log mode but missing from JSON mode (or vice versa)?
422+
423+
### 5. Behavioral Consistency
424+
- Does the command behave consistently with similar commands in the CLI?
425+
- Are exit codes correct? (0 for success, non-zero for failure)
426+
- If the command is destructive, does it require confirmation or support --force?
427+
- Does the output change correctly between normal and --verbose modes?
428+
429+
## Review Rules
430+
431+
- Compare against EXISTING commands to check consistency. Don't just review in isolation.
432+
- Be SPECIFIC — reference the exact flag, message, or output format.
433+
- Only flag issues that affect the user experience. Ignore internal code quality.
434+
- If the PR doesn't meaningfully change UX (e.g., only refactors internals), say so and approve.
435+
436+
## Severity Classification
437+
438+
- **Blocking**: Broken help text, misleading error messages, missing JSON output for new
439+
functionality, inconsistent flag names that would confuse users. These block merge.
440+
- **Suggestion**: Minor wording improvements, optional additional output, nice-to-have
441+
consistency tweaks. These do NOT block merge.
442+
443+
After reviewing, submit your review:
444+
445+
If there are NO blocking issues:
446+
```
447+
gh pr review ${{ github.event.pull_request.number }} --approve --body "your review here"
448+
```
449+
450+
If there ARE blocking issues:
451+
```
452+
gh pr review ${{ github.event.pull_request.number }} --request-changes --body "your review here"
453+
```
454+
455+
Format your review body as:
456+
## CLI UX Review
457+
458+
### Blocking (if any)
459+
[numbered list with specific file, flag/message/output, what's wrong, suggested fix]
460+
461+
### Suggestions (if any)
462+
[numbered list]
463+
464+
### Verdict
465+
[PASS / NEEDS CHANGES with one-line summary]
466+
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
467+
claude_args: |
468+
--model claude-sonnet-4-6
469+
--allowedTools Read,Glob,Grep,Bash(gh pr review:*),Bash(gh pr view:*),Bash(gh pr diff:*)
470+
340471
auto-merge:
341472
name: Auto-merge PR
342473
needs:
343-
[test, deps-audit, skill-review, claude-review, claude-adversarial-review]
474+
[test, deps-audit, skill-review, claude-review, claude-adversarial-review, claude-ux-review]
344475
runs-on: ubuntu-latest
345476
# Only auto-merge PRs from the same repo (not forks) for security
346477
# Skip auto-merge if PR has the 'hold' label

0 commit comments

Comments
 (0)