Skip to content

Commit e9cacf7

Browse files
committed
feat(ship): make test and eval commands configurable via .gstack.json
1 parent 1717ed2 commit e9cacf7

2 files changed

Lines changed: 39 additions & 4 deletions

File tree

.gstack.example.json

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"testCommand": "bin/test-lane",
3+
"evalCommand": "bin/eval --lane test",
4+
"evalPatterns": [
5+
".claude/evals/results/**/*.md",
6+
".claude/evals/results/**/*.txt"
7+
],
8+
"reviewChecklist": ".claude/skills/review/checklist.md"
9+
}

ship/SKILL.md

Lines changed: 30 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -76,9 +76,22 @@ Running bare test migrations without INSTANCE hits an orphan DB and corrupts str
7676
Run both test suites in parallel:
7777

7878
```bash
79-
bin/test-lane 2>&1 | tee /tmp/ship_tests.txt &
80-
npm run test 2>&1 | tee /tmp/ship_vitest.txt &
81-
wait
79+
# First check if the user has defined test commands in .gstack.json
80+
if [ -f .gstack.json ]; then
81+
TEST_CMD=$(cat .gstack.json | grep '"testCommand"' | cut -d'"' -f4)
82+
else
83+
TEST_CMD=""
84+
fi
85+
86+
if [ -n "$TEST_CMD" ]; then
87+
echo "Running configured test command: $TEST_CMD"
88+
bash -c "$TEST_CMD" 2>&1 | tee /tmp/ship_tests.txt
89+
else
90+
# Fallback to default Rails + Vitest behavior
91+
bin/test-lane 2>&1 | tee /tmp/ship_tests.txt &
92+
npm run test 2>&1 | tee /tmp/ship_vitest.txt &
93+
wait
94+
fi
8295
```
8396

8497
After both complete, read the output files and check pass/fail.
@@ -130,7 +143,20 @@ Map runner → test file: `post_generation_eval_runner.rb` → `post_generation_
130143
`/ship` is a pre-merge gate, so always use full tier (Sonnet structural + Opus persona judges).
131144

132145
```bash
133-
EVAL_JUDGE_TIER=full EVAL_VERBOSE=1 bin/test-lane --eval test/evals/<suite>_eval_test.rb 2>&1 | tee /tmp/ship_evals.txt
146+
if [ -f .gstack.json ]; then
147+
EVAL_CMD=$(cat .gstack.json | grep '"evalCommand"' | cut -d'"' -f4)
148+
else
149+
EVAL_CMD=""
150+
fi
151+
152+
if [ -n "$EVAL_CMD" ]; then
153+
# Inject the suite name into the configured command
154+
# e.g., if command is "npm run eval -- <suite>", replace <suite>
155+
FINAL_CMD="${EVAL_CMD/<suite>/<suite>_eval_test.rb}"
156+
eval "$FINAL_CMD" 2>&1 | tee /tmp/ship_evals.txt
157+
else
158+
EVAL_JUDGE_TIER=full EVAL_VERBOSE=1 bin/test-lane --eval test/evals/<suite>_eval_test.rb 2>&1 | tee /tmp/ship_evals.txt
159+
fi
134160
```
135161

136162
If multiple suites need to run, run them sequentially (each needs a test lane). If the first suite fails, stop immediately — don't burn API cost on remaining suites.

0 commit comments

Comments
 (0)