Skip to content

Commit 7f3fe51

Browse files
NathanFlurryclaude
andcommitted
chore: add test verification and code quality rules to Ralph CLAUDE.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent ab92cc0 commit 7f3fe51

1 file changed

Lines changed: 50 additions & 0 deletions

File tree

scripts/ralph/CLAUDE.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,55 @@ You are an autonomous coding agent working on a software project.
1515
9. Update the PRD to set `passes: true` for the completed story
1616
10. Append your progress to `progress.txt`
1717

18+
## Test Verification (CRITICAL)
19+
20+
A story is **only complete** when its tests actually EXECUTE and PASS. Follow these rules strictly:
21+
22+
### Skipped Tests Are Not Passing Tests
23+
24+
- After running tests, check the vitest summary line: `Tests X passed | Y skipped | Z failed`
25+
- If any tests relevant to your story are **skipped**, the story does NOT pass
26+
- Skipped tests mean there's a blocker (missing dependency, unimplemented bridge, env issue)
27+
- If you write tests that skip due to a known blocker, set `passes: false` and document the blocker in the story's `notes` field
28+
- NEVER set `passes: true` for a story whose tests skip — this is dishonest and wastes future iterations
29+
30+
### Check for Unhandled Errors
31+
32+
- After running tests, check for the `Unhandled Errors` section in vitest output
33+
- Unhandled rejections and uncaught exceptions indicate bugs even when individual tests pass
34+
- If your story's tests produce unhandled errors, fix them before marking the story complete
35+
- Common cause: use-after-dispose (V8 context cleaned up before pending callbacks resolve)
36+
37+
### Run Targeted Tests
38+
39+
- Run ONLY the test file(s) relevant to your story, not the entire test suite
40+
- Example: `npx vitest run packages/secure-exec/tests/cli-tools/my-test.test.ts --reporter=verbose`
41+
- Check the verbose output to confirm each test case passes (not skips)
42+
- Only run the full suite if your changes could affect other tests
43+
44+
### Tests With Runtime Probes
45+
46+
Some tests use runtime probes (e.g., checking if a binary exists, if isTTY bridge works, if a module loads in the VM). If ALL tests skip because a probe fails:
47+
- The story is **blocked**, not complete
48+
- Set `passes: false` with a clear note explaining the blocker
49+
- If possible, fix the blocker as part of the story
50+
- If the blocker is out of scope, document it and move on — do NOT lie about completion
51+
52+
## Code Quality
53+
54+
### No Copy-Paste Duplication
55+
56+
- Before writing a utility class or helper function, check if it already exists in a shared module
57+
- If you need the same code in multiple test files, extract it into a shared module (e.g., `test-utils.ts`, `shared-harness.ts`)
58+
- Common offenders: command executors, stdio capture helpers, overlay VFS factories, driver classes
59+
- If you find existing duplication, refactor it into a shared module as part of your story
60+
61+
### Follow Existing Patterns
62+
63+
- Read nearby test files and follow the same patterns
64+
- Use `createTestNodeRuntime()` from `test-utils.ts` when creating test runtimes
65+
- Use existing shared helpers before creating new ones
66+
1867
## Progress Report Format
1968

2069
APPEND to progress.txt (never replace, always append):
@@ -102,3 +151,4 @@ If there are still stories with `passes: false`, end your response normally (ano
102151
- Commit frequently
103152
- Keep CI green
104153
- Read the Codebase Patterns section in progress.txt before starting
154+
- NEVER mark `passes: true` unless tests actually execute and pass (not skip)

0 commit comments

Comments
 (0)