Skip to content

test: EvalOpsBot relay canary fixture#97

Closed
haasonsaas wants to merge 1 commit into
mainfrom
codex/evalopsbot-relay-canary-20260520
Closed

test: EvalOpsBot relay canary fixture#97
haasonsaas wants to merge 1 commit into
mainfrom
codex/evalopsbot-relay-canary-20260520

Conversation

@haasonsaas
Copy link
Copy Markdown
Contributor

Canary PR for the EvalOpsBot org-webhook requested-review path. This intentionally adds an unsafe workflow fixture so the deep IAM lens has a deterministic finding surface.

Test Plan:

  • Request review from EvalOpsBot.
  • Verify the org webhook dispatches evalops-pr-lens-review.
  • Verify EvalOpsBot posts or updates the PR lens review signal.

@haasonsaas haasonsaas requested a review from EvalOpsBot May 20, 2026 01:30
@EvalOpsBot
Copy link
Copy Markdown

EvalOps PR lens review

High-confidence findings only. Threshold: 0.82.
Run: https://github.com/evalops/.github/actions/runs/26135712754

  1. P0 0.98 iam-blast-radius: pull_request_target with write-all checks out untrusted PR head

    • Location: .github/workflows/evalopsbot-relay-canary-fixture.yml:4
    • Check: evalops-pr-lens/iam-blast-radius
    • The new workflow triggers on pull_request_target (which runs in the base repo context with access to secrets) and uses permissions: write-all, then checks out the PR head SHA via actions/checkout with ref: github.event.pull_request.head.sha. Any subsequent build/script step from an attacker-controlled PR would execute with a write-all GITHUB_TOKEN, allowing repo write, packages, actions, and potentially org-level escalation through downstream steps. Even though the only current step is gh pr comment, the checked-out untrusted code plus pwn-ready token is the classic pwn-request pattern. Fix: remove pull_request_target (use pull_request), or drop the head checkout entirely, scope permissions: to the minimum (e.g., pull-requests: write only), and never execute code from the PR head under pull_request_target. If a canary is needed, gate it behind a label/environment with required reviewers and avoid checking out head SHA.
  2. P0 0.95 iam-blast-radius: permissions: write-all grants excessive GITHUB_TOKEN scope

    • Location: .github/workflows/evalopsbot-relay-canary-fixture.yml:6
    • Check: evalops-pr-lens/iam-blast-radius
    • permissions: write-all grants the GITHUB_TOKEN every available scope (contents, packages, actions, id-token, etc.) for this workflow. Combined with pull_request_target this materially expands blast radius beyond what the documented purpose (posting a PR comment) requires. Fix: replace with least-privilege, e.g. permissions: pull-requests: write contents: read and remove id-token/actions/packages write implicitly granted by write-all.

Repo: evalops/.github PR: #97

@haasonsaas
Copy link
Copy Markdown
Contributor Author

EvalOpsBot org-webhook relay canary complete: webhook delivery 3820890163140100000 dispatched central run 26135712754, EvalOpsBot posted the expected deep review findings, and evalops-pr-lens/meta-review failed the canary head as expected. Closing the intentionally unsafe fixture.

@haasonsaas haasonsaas closed this May 20, 2026
@haasonsaas haasonsaas deleted the codex/evalopsbot-relay-canary-20260520 branch May 20, 2026 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants