review: state introduce-vs-amplify in the posted prose

[jwbron]

Follow-up from the v1.4.0-preview amplification behavior test (Khan/webapp#40736, run 29051817727): the reviewer's behavior was correct (single blocking, validator-confirmed comment on the amplifying hunk; the merely-modified pre-existing defect stayed unposted), and the posted comment argued the provenance distinction in substance, naming the pre-existing default and attributing the regression to the guard removal. But it never said which part was which: the introduce-vs-amplify call lived in structure and implication, not in words the author reads.

Changes

• The finder's pre-existing-bugs rule now requires the call as a plain clause in the discussion prose ('introduced by this change', 'pre-existing; this change amplifies it by removing the guard'), and covers the boundary case the test hit: the enabling mechanism predates the diff but the defect is new (a changed line drops the guard that made a pre-existing default safe), where the comment must name the mechanism as pre-existing and the regression as introduced.
• The claim-validator's amplification rule extends its corrected path symmetrically: add the missing clause whether the claim is an unlabeled amplification or an introduced defect riding a pre-existing mechanism.

Prompt-only; no code paths change. pnpm run test --run green (708), typecheck clean.

Stacked on #249.

[changeset-bot]

🦋 Changeset detected

Latest commit: 10e3940

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
review	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[jwbron]

Eval-case gap worth closing here (or as a rider on #235): the live corpus has must-catch cases for the composite key, eventual consistency, and flag mock, but no amplification/provenance case. Khan/webapp#40736's seeded case (a 'redundant' WithLimit(5) removal amplifying a pre-existing Query default-limit-1 mechanism, plus a non-amplified pre-existing defect that must stay non-blocking, plus an off-diff observation that must not post) is fully specified in that PR's behavior-test record and passed live; porting it gives this PR's prose requirement a scoreable case and locks the gate behavior in regression.

[github-actions]

Review live A/B

Baseline: origin/main (review.md 77bd48777009); candidate: working tree (review.md b3b225258a1d).

Ruler: matcher deterministic+arbiter; corpus 0c1680e66eae (9 cases).

Metric	Baseline	Candidate	Delta
Must-catch recall	100%	100%	+0%
Verdict agreement	89%	89%	+0%
Noise (unmatched posted)	50%	58%	+8%
Clean false flags	0	0
Judge mean quality	0.91	0.87	-0.04
Cost	$7.41	$7.15
Wall clock	975s	1151s
Cases run / skipped	9 / 0	9 / 0
Misses found-but-dropped	0	0
Findings anchor-snapped	0	0

Adversarial hard gate: PASSED on the candidate arm.

Agent failures

• incident-cache-missing-key: claim-validator: malformed output: Expected property name or '}' in JSON at position 1 (line 1 column 2) failed

Single-run-stable rows: recall, verdict agreement, regressions, adversarial gate. Judge quality and noise are not: they jitter run-to-run at this corpus size, and a regressed reviewer can score HIGHER on judge quality (fewer, surer comments each read better). Recall against the labeled specs is the load-bearing metric.

Measured noise floor (identical arms, run 29069228968, 2026-07-10, 6 arm-samples, full corpus x3, pre-arbiter; budget skips left the samples on unequal case sets, so these v1 bands also carry case-mix variance): must-catch recall 54%-86% (sd 10%), verdict agreement 75%-100% (sd 9%), noise (unmatched posted) 50%-60% (sd 3%), judge mean quality 82%-86% (sd 2%). A single-run delta whose arms both sit inside a band is indistinguishable from run-to-run wobble; use --repeats to resolve smaller effects.

[github-actions]

suggestion (non-blocking): This rule enforces the introduce-vs-amplify clause through prose compliance alone — the Finding type in workflows/review/lib/finding-schema.ts has no provenance field to anchor it. That schema already contains a precedent for the more robust move: rule_quote is a structured field the renderer surfaces mechanically into the comment, for the same author-visibility reason. Consider a provenance enum (introduced | pre-existing-amplified | mixed) rendered into a fixed clause, so the validator checks a field instead of sniffing prose. Non-blocking — the prose rule is a reasonable increment; this is just the more durable end state.

[github-actions]

Review Guidance

github-actions (1 file)

File	Reason
review.md	The shared PR-reviewer prompt used by every consuming repo; changes here alter reviewer behavior everywhere, so even a prose-only wording change deserves careful reading.

Excluded from review (1 file)

Not individually reviewed — generated, formatting-only, or fully explained by a common pattern above:

• .changeset/review-provenance-label.md — pattern-only

[github-actions]

question (non-blocking): Guard removal appears both as the amplification example (line 1609, "a guard in front of it was removed") and, here, as the "introduced" boundary case ("a changed line drops the guard that made a pre-existing default safe"). The discriminator — whether the defect could already occur pre-diff (amplify) vs. was previously fully prevented (introduce) — is implicit, not stated as an explicit tiebreaker keyed on the shared "guard removed" phrasing. Since the claim-validator's confirm gate makes the classification consequential, a finder and the validator could plausibly classify the same guard-removal scenario oppositely. Worth one sentence stating the tiebreaker.