Skip to content

feat: harden EvalOps PR review automation#95

Merged
haasonsaas merged 1 commit into
mainfrom
improve/pr-lens-resilience
May 18, 2026
Merged

feat: harden EvalOps PR review automation#95
haasonsaas merged 1 commit into
mainfrom
improve/pr-lens-resilience

Conversation

@haasonsaas
Copy link
Copy Markdown
Contributor

Summary

  • Make PR-lens jobs race-tolerant: target PRs are checked against the discovery snapshot, prepared with a token-aware git checkout, or converted into explicit skipped lens artifacts.
  • Classify changed files into relevant lenses so each PR only runs the review lenses it needs.
  • Teach meta-review about expected vs completed/skipped/missing lens coverage, and mark incomplete coverage separately from no findings.
  • Add a checked-in EvalOpsBot webhook relay core with signature validation, and demote the polling workflow to hourly fallback.
  • Add full workflow/template actionlint validation to codex-rails-check with the Blacksmith runner label configured.

Verification

  • ruby -c .github/scripts/evalops-pr-lens-review.rb
  • ruby -c .github/scripts/evalopsbot-webhook-relay.rb
  • ruby -Itest -e 'ARGV.each { |path| require "./#{path}" }' test/*_test.rb
  • actionlint .github/workflows/.yml .github/workflow-templates/.yml
  • ruby -e 'require "yaml"; ARGV.each { |f| YAML.load_file(f); puts "ok #{f}" }' .github/workflows/.yml .github/workflow-templates/.yml .github/actionlint.yaml
  • git diff --check
  • Live smoke: prepare-workspace against evalops/deploy#3671 emitted skip=false with expected base/head SHAs.

@haasonsaas haasonsaas merged commit 4b73122 into main May 18, 2026
3 checks passed
@haasonsaas haasonsaas deleted the improve/pr-lens-resilience branch May 18, 2026 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant