docs(evals): add model-judge calibration notes

evals/model-judge-calibration-notes.md

@@ -0,0 +1,44 @@
+# Model-Judge Calibration Notes
+
+This file is the curated, human-readable calibration summary for the technical-design manual pointwise model-judge lane. Raw provider result bundles under `evals/results/**` remain local/ignored and are not committed.
+
+Deterministic eval verdicts remain authoritative. Pointwise model-judge evidence is manual, advisory, and cannot upgrade deterministic blockers.
+
+## Run Set
+
+These notes summarize the existing local provider-backed runs from July 4, 2026. No fresh provider calls were made for this curated summary.
+
+| Field             | Value                                        |
+| ----------------- | -------------------------------------------- |
+| Config            | `evals/eval-kit.model-judge.config.json`     |
+| Model             | `gpt-5.4`                                    |
+| Provider          | `openai:codex-app-server`                    |
+| Reasoning effort  | `medium`                                     |
+| Prompt version    | `pointwise-prompt-v1`                        |
+| Rubric version    | `pointwise-coverage-rubric-v1`               |
+| Candidate profile | expected-good `reference-design.md` fixtures |
+
+## Calibration Summary
+
+| Case                                    | Candidate     | Run id                                                                   | Covered | Partial | Missing | Contradicted | Unknown | Human disposition                                    |
+| --------------------------------------- | ------------- | ------------------------------------------------------------------------ | ------: | ------: | ------: | -----------: | ------: | ---------------------------------------------------- |
+| `case-aerial-delivery-shipping-v1`      | expected-good | `provider-20260704-v015-case-aerial-delivery-shipping-v1-reference`      |      16 |       0 |       0 |            0 |       0 | Pass-like coverage; no calibration concern observed. |
+| `case-cloudevents-core-contract-v1`     | expected-good | `provider-20260704-v015-case-cloudevents-core-contract-v1-reference`     |       6 |       0 |       0 |            0 |       0 | Pass-like coverage; no calibration concern observed. |
+| `case-customer-credit-order-saga-v1`    | expected-good | `provider-20260704-v015-case-customer-credit-order-saga-v1-reference`    |      11 |       0 |       0 |            0 |       0 | Pass-like coverage; no calibration concern observed. |
+| `case-fineract-loan-lifecycle-v1`       | expected-good | `provider-20260704-v015-case-fineract-loan-lifecycle-v1-reference`       |      10 |       0 |       0 |            0 |       0 | Pass-like coverage; no calibration concern observed. |
+| `case-kubernetes-sidecar-containers-v1` | expected-good | `provider-20260704-v015-case-kubernetes-sidecar-containers-v1-reference` |       9 |       0 |       0 |            0 |       0 | Pass-like coverage; no calibration concern observed. |
+| `case-openfeature-evaluation-api-v1`    | expected-good | `provider-20260704-v015-case-openfeature-evaluation-api-v1-reference`    |       8 |       0 |       0 |            0 |       0 | Pass-like coverage; no calibration concern observed. |
+| `case-tiny-laundry-pickup-v1`           | expected-good | `provider-20260704-v015-case-tiny-laundry-pickup-v1-reference`           |      12 |       0 |       0 |            0 |       0 | Pass-like coverage; no calibration concern observed. |
+
+## Human Calibration Notes
+
+- False pass risk: not yet calibrated for technical-design because this run set contains expected-good pointwise candidates only. Phase C4 is the planned follow-up for expected-bad pointwise calibration fixtures.
+- False fail risk: no false-fail evidence observed in the expected-good run set.
+- Ambiguity: no `partial` or `unknown` verdicts appeared in these runs.
+- Verbosity bias: no evidence that longer wording was required for coverage, but expected-bad fixtures are needed before drawing stronger conclusions.
+- Reference wording bias: current evidence only proves that the judge recognizes committed reference designs; it does not yet prove rejection of plausible but semantically defective designs.
+- Unknown rate: zero unknown verdicts in the summarized run set.
+
+## Decision
+
+Keep the technical-design pointwise lane manual and advisory. The current expected-good evidence is usable for smoke calibration, but false-pass detection remains untested until Phase C4 adds expected-bad pointwise calibration fixtures.