Skip to content

docs: align technical-design eval policy#23

Merged
aryeko merged 2 commits into
mainfrom
docs/eval-policy-alignment
Jul 3, 2026
Merged

docs: align technical-design eval policy#23
aryeko merged 2 commits into
mainfrom
docs/eval-policy-alignment

Conversation

@aryeko

@aryeko aryeko commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Document pnpm check as the fast offline/static/local deterministic gate for technical-design.
  • Clarify that Promptfoo/Codex/model-assisted evals stay manual/advisory, outside CI, and disabled by default.
  • Pin @agentic-workflow-kit/eval-kit to github:agentic-workflow-kit/eval-kit#v0.1.2, whose CLI enforces disabled method flags before auth/provider/artifact paths.

Policy Alignment

  • pnpm check remains limited to lint/format, root-visible static checks, adapter import/syntax validation, fixture manifest validation, local unit tests, and seeded deterministic fixture checks.
  • Root scripts do not invoke generate, judge-coverage, or judge-pairwise from check.
  • Model-assisted methods remain disabled in evals/eval-kit.config.json.
  • No Promptfoo/model-assisted smokes were run.

Verification

  • pnpm install --frozen-lockfile
  • pnpm check
  • pnpm eval:doctor
  • pnpm eval:list
  • git diff --check

Notes: the exact frozen install required network access after the package tag update and resolved eval-kit to e45db38c0e5b151942ebdf579089d0bdd284f974. The optional better-sqlite3 native postinstall fails under Node 26.4.0 and is skipped as optional; all required deterministic gates pass.

@aryeko aryeko merged commit 47f7641 into main Jul 3, 2026
1 check passed
@aryeko aryeko deleted the docs/eval-policy-alignment branch July 3, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant