Problem
scripts/benchmark.sh currently tests 22 commands but is missing ~12 testable commands that exist in RTK. This means regressions in those filters can slip through the pre-release pipeline undetected.
Missing commands (testable)
| Command |
Notes |
rtk cargo build/test/clippy |
Always available (we're a Rust project) |
rtk smart <file> |
2-line summary, easy to test |
rtk diff <f1> <f2> |
File diff, easy to test with temp files |
rtk tree |
Directory tree, easy to test |
rtk curl <url> |
HTTP fetch, test with public URL |
rtk wget <url> |
Download, test with public URL |
rtk npm run |
If package.json present |
rtk npx |
If package.json present |
rtk proxy <cmd> |
Passthrough, always available |
rtk ruff check |
Already has Python fixture, just uses wrong $RTK test wrapper |
rtk pytest |
Same — should use $RTK pytest not $RTK test pytest |
rtk pip list |
If pip available |
rtk go test/build |
Already has Go fixture, just uses wrong $RTK test wrapper |
rtk golangci-lint |
Same — should use $RTK golangci-lint not $RTK test golangci-lint |
rtk format |
Universal format checker |
Not benchmarkable (skip)
gain, discover, learn, config, init, cc-economics, hook-audit — these are analytics/setup commands, not filters.
Current bugs in benchmark.sh
The Python and Go sections use $RTK test ruff check . and $RTK test golangci-lint run instead of the dedicated $RTK ruff check . and $RTK golangci-lint run commands. This means we're benchmarking the generic test runner instead of the specialized filters.
Expected outcome
All filter commands covered in benchmark.sh so we can catch token savings regressions before release.
# Pre-release validation
cargo build --release
bash scripts/benchmark.sh # Should test ALL filters
Problem
scripts/benchmark.shcurrently tests 22 commands but is missing ~12 testable commands that exist in RTK. This means regressions in those filters can slip through the pre-release pipeline undetected.Missing commands (testable)
rtk cargo build/test/clippyrtk smart <file>rtk diff <f1> <f2>rtk treertk curl <url>rtk wget <url>rtk npm runrtk npxrtk proxy <cmd>rtk ruff check$RTK testwrapperrtk pytest$RTK pytestnot$RTK test pytestrtk pip listrtk go test/build$RTK testwrapperrtk golangci-lint$RTK golangci-lintnot$RTK test golangci-lintrtk formatNot benchmarkable (skip)
gain,discover,learn,config,init,cc-economics,hook-audit— these are analytics/setup commands, not filters.Current bugs in benchmark.sh
The Python and Go sections use
$RTK test ruff check .and$RTK test golangci-lint runinstead of the dedicated$RTK ruff check .and$RTK golangci-lint runcommands. This means we're benchmarking the generic test runner instead of the specialized filters.Expected outcome
All filter commands covered in benchmark.sh so we can catch token savings regressions before release.