test(evals): add technical-design bad pointwise fixtures#31
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3ee7c6b721
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ## Intentional Calibration Defect | ||
|
|
||
| Delivery owns long-term delivery history. | ||
|
|
||
| This contradicts the source-visible retention boundary: active delivery execution may be separate from long-term delivery history ownership. |
There was a problem hiding this comment.
Remove answer-key text from judged candidates
For these expected-bad pointwise fixtures, this defect section is part of the Markdown that the manual pointwise judge receives as the candidate, so the provider sees both that the file is intentionally bad and what contradiction to look for. That contaminates false-pass calibration: a judge can return red from the answer-key label/explanation instead of detecting a plausible semantic defect in the design itself. Keep this metadata outside the candidate file or strip it before pointwise runs.
Useful? React with 👍 / 👎.
Summary
Verification
No provider-backed evals were run. Raw c4 evals/results bundles remain ignored/local.