review: budget and shed tuning from the 07-10 production sheds

[jwbron]

Summary:

The 2026-07-10 reviewer runs on #238 (review) and #232 (review) shed claim validation and three whole-change dimensions on substantial PRs, and reported the sheds with wording that reads as infrastructure failure. Root causes, from the run logs: both PRs misrouted to the bottom tiers via the broad **/*.md tier=trivial rule; the tier wall clocks ignore the ~3 minutes of fixed staging/router/triage overhead, so a low-tier run was at 80% of its soft budget the moment its reviewers returned; the low-tier invocation cap of 4 cannot fit the seven-reviewer whole-change roster this repo (and webapp) enables, so every low-tier run shed dimensions by arithmetic; and the two runs counted the cap inconsistently (#238 charged pattern-triage against it, #232 did not).

Four changes:

• ROUTING (repo-local config, not the package): re-raise .claude/skills/** (medium) and workflows/review/eval/**/*.md (low) from the trivial-docs rule; both are executable-not-docs prose.
• Budget table (router.ts): overhead-aware wall clocks (trivial 3->6, low 6->10, medium 12->15) and roster-fitting invocation caps (low 4->8, medium 8->10). Trivial stays deliberately small. maxUsd is untouched: those numbers are also unrealistic (measured runs cost $3.2-3.6 against a $0.5-1.5 target), but spend is unobservable mid-run today and review: clamp the run budget to the effective credit cap; land before the cap #249 is building the credits-cap machinery that makes it observable, so recalibrating dollars belongs there.
• Cap accounting (review.md): the invocation cap counts finding-producers only; pattern-triage, thread-reconciler, and claim-validator are pipeline steps and never consume a slot.
• Shed order and note wording (review.md): the claim-validator may be shed only under genuine hard-ceiling pressure (three-quarters of timeout-minutes, or a direct credits-cap signal), never at a mere soft-target breach; its cost tracks the visible candidate count, not the diff. Skipped-dimension notes now distinguish shed under the <tier>-tier run budget from <sub-agent> output unavailable, so budget arithmetic no longer masquerades as failure.

Interaction with #249: complementary. #249 clamps the selected tier budget to the effective credit cap; this PR resizes the un-clamped table and the shed discipline. Expect a textual merge conflict in the review.md graceful-landing section, whichever lands second.

Test plan:

• pnpm test (656 tests, all passing; the budget tests assert monotonicity and tier identity, both preserved).
• pnpm exec prettier --check on the touched TS.
• Glob semantics of the new ROUTING rules verified against globToRegExp (**/ collapses to zero-or-more segments, so workflows/review/eval/**/*.md matches live-ab-plan.md at the eval root).
• The next reviewer runs on doc-heavy PRs should show: skills/eval-doc PRs routing medium/low instead of trivial, full roster dispatch at low tier, and shed notes (if any) carrying the planned-shed wording.

[changeset-bot]

🦋 Changeset detected

Latest commit: 5889d75

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
review	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

nitpick (non-blocking): This pattern is wider than the tier= alignment column the rest of the file keeps, so tier=low lands one space off the grid. Accept it as the intentional outlier, or widen the column across all rules.

Low-confidence (1)

• workflows/review/lib/router.ts:286 — Baking the measured ~3 min overhead into every tier's wall clock is fragile; measuring elapsed time from triage-end (a second timestamp) instead would keep the table overhead-invariant.

[github-actions]

Review Guidance

github-actions (3 files)

File	Reason
ROUTING	Consumer config that sets the per-file risk tiers gating every review's budget and shed behavior; a mistake here misroutes all future reviews.
router.ts	The shared reviewer's deterministic budget table; these numbers bound the work every consuming repo's review is allowed to do.
review.md	The shared orchestrator prompt; its shed-order and cap-accounting prose is the reviewer's behavior in every consuming repo.

Common patterns

3 files: The finding-producers-only cap accounting (pattern-triage, thread-reconciler, claim-validator never consume a slot) is stated in the router.ts JSDoc, the router.test.ts comment, and the review.md prose.

[github-actions]

Review live A/B

Baseline: origin/main (review.md 35556ec5bba9); candidate: working tree (review.md 6320d1d05208).

Metric	Baseline	Candidate	Delta
Must-catch recall	88%	75%	-13%
Verdict agreement	78%	89%	+11%
Noise (unmatched posted)	59%	60%	+1%
Clean false flags	0	0
Judge mean quality	0.86	0.87	+0.00
Cost	$7.08	$6.55
Wall clock	1258s	920s
Cases run / skipped	9 / 0	9 / 0
Misses found-but-dropped	0	0

Regressions (baseline caught, candidate missed)

• trial-dedup-composite-key:save-dedup-composite-key (not found)
• trial-erasure-suite-flag-mock:erasure-suite-flag-mock (not found)

Improvements (candidate caught, baseline missed)

• incident-sql-missing-index:dm-missing-index-1

Adversarial hard gate: PASSED on the candidate arm.

Agent failures

• trial-erasure-suite-flag-mock: claim-validator: malformed output: Expected property name or '}' in JSON at position 1 (line 1 column 2) failed

Single-run-stable rows: recall, verdict agreement, regressions, adversarial gate. Judge quality and noise are not: they jitter run-to-run at this corpus size, and a regressed reviewer can score HIGHER on judge quality (fewer, surer comments each read better). Recall against the labeled specs is the load-bearing metric.