fix: estc-pr-buildkite-detective bot allowlist, noop safe-output, and PR discovery by Copilot · Pull Request #634 · elastic/ai-github-actions

Copilot · 2026-03-09T16:21:01Z

This PR fixes three issues in the estc-pr-buildkite-detective workflow where PR association and safe output handling could fail.

What changed

Bot allowlist: Added buildkite-limited-access[bot] alongside the configurable allowed-bot-users input.
noop safe-output: Declared noop under safe-outputs so the agent can emit it when no PR can be resolved.
Status-event commit SHA extraction: For status events, switched to .commit.sha rather than .sha.
Event context extraction: Added:
- check_run: BK_PR_NUMBERS from .check_run.pull_requests[].number, BK_TARGET_URL from .check_run.details_url
- status: BK_BRANCHES from .branches[].name, BK_TARGET_URL from .target_url
jq null-safety: Added // [] fallback for .branches and .check_run.pull_requests arrays to prevent errors when these fields are null.
BK_PR_NUMBERS empty handling: Emits an empty string instead of [] when no pull requests are present, so downstream "non-empty" checks are unambiguous.
PR discovery flow (Step 1): Updated to prefer event-provided PR numbers (check_run) and otherwise fall back to gh api repos/{repo}/commits/{sha}/pulls plus branch matching (status), instead of listing all open PRs and matching only head.sha. Results from commits/{sha}/pulls are now filtered to keep only PRs with state == "open" and matching head.ref, preventing misrouting to closed or historical PRs.
Prompt context additions: The context now includes:

- **Target URL**: $\{\{ env.BK_TARGET_URL }}
- **PR Numbers**: $\{\{ env.BK_PR_NUMBERS }}
- **Branches**: $\{\{ env.BK_BRANCHES }}

Also includes the corresponding compiled lockfile updates.

The body of this PR is automatically managed by the Trigger Update PR Body workflow.

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

…afe-output - Add buildkite-limited-access[bot] to the bots allowlist - Extract PR numbers from check_run events, branches and target URL from status events - Use .commit.sha instead of .sha for status events - Update Step 1 to follow the PR Actions Detective approach for PR discovery - Add noop safe-output (prompt already referenced it) - Add Target URL, PR Numbers, and Branches to the context section Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

Per code review feedback: .sha and .commit.sha should be equivalent in GitHub status events per the docs. Keep .sha for safety. Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

v1v · 2026-03-09T16:49:00Z

.github/workflows/gh-aw-estc-pr-buildkite-detective.md

@@ -99,10 +101,16 @@ steps:
        echo "BK_EVENT_ID=$(jq -r '.id' "$GITHUB_EVENT_PATH")" >> "$GITHUB_ENV"
        echo "BK_FAILURE_STATE=$(jq -r '.state' "$GITHUB_EVENT_PATH")" >> "$GITHUB_ENV"
        echo "BK_COMMIT_SHA=$(jq -r '.sha' "$GITHUB_EVENT_PATH")" >> "$GITHUB_ENV"


This is not correct, the .sha points to the workflow reference

It should point to the branch from the PR.

For instance, see #632 (comment)

https://docs.github.com/en/actions/reference/workflows-and-actions/contexts

github.sha | string | The commit SHA that triggered the workflow. The value of this commit SHA depends on the event that triggered the workflow. For more information, see Events that trigger workflows. For example, ffac537e6cbbf934b08745a378932722df287a53. -- | -- | -- github.sha string The commit SHA that triggered the workflow. The value of this commit SHA depends on the event that triggered the workflow. For more information, see [Events that trigger workflows](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows). For example, ffac537e6cbbf934b08745a378932722df287a53.

Fixed in 24933f3. For status events, BK_COMMIT_SHA now extracts the PR branch commit from .branches[] by filtering out the default branch (read from .repository.default_branch), falling back to .sha only when no non-default branch is present.

strawgate · 2026-03-09T17:11:41Z

@copilot please address feedback

coderabbitai · 2026-03-09T17:14:27Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4088b5fd-5436-4126-9151-4e2cdb174f7b

📥 Commits

Reviewing files that changed from the base of the PR and between 9726469 and 1d5daa1.

📒 Files selected for processing (2)

.github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml
.github/workflows/gh-aw-estc-pr-buildkite-detective.md

📝 Walkthrough

Walkthrough

Adds Buildkite context variables (BK_BRANCHES, BK_PR_NUMBERS, BK_TARGET_URL) and propagates them through prompt rendering and placeholder substitution; prefers explicit PR numbers when present, otherwise resolves PRs via GitHub commit- and branch-based queries and noops with a standard message if none found. Appends buildkite-limited-access[bot] to allowed-bots, adds a noop safe-output, and expands investigation into two paths (MCP/API access vs public Buildkite page fallback) for evidence collection.

Possibly related PRs

Split out Elastic-specific workflows with estc- prefix #431: Direct edits to the same gh-aw-estc-pr-buildkite-detective workflow file and related Buildkite context propagation.
Fix PR Buildkite Detective for status/check_run failures #422: Overlaps PR-resolution logic for resolving PRs from commit SHAs and branch/context data.
Support public Buildkite pipelines in pr-buildkite-detective #427: Implements or modifies the Buildkite public-fallback vs API/MCP investigation flow and log-evidence collection.

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	PR addresses all objectives from issue `#632`: bot allowlist fix, noop safe-output declaration, event field extraction (BK_PR_NUMBERS, BK_BRANCHES, BK_TARGET_URL), and PR discovery logic improvements.
Out of Scope Changes check	✅ Passed	All changes are scoped to issue `#632` requirements: workflow configuration updates, bot allowlist, safe-output declarations, and PR discovery logic.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch copilot/fix-buildkite-config-issues

🛠️ Update Documentation: Commit on current branch
🛠️ Update Documentation: Create PR

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions

Requesting changes: PR association in Step 1 still needs candidate validation to avoid selecting unrelated historical PRs from commits/{sha}/pulls.

What is this? | From workflow: Trigger PR Review

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

github-actions · 2026-03-09T17:18:04Z

.github/workflows/gh-aw-estc-pr-buildkite-detective.md

-3. For each matching PR, keep author, branches, and fork status for downstream analysis.
+2. Find the associated pull request(s):
+   - If **PR Numbers** in the Context section above is non-empty (e.g., from `check_run` events), use those PR numbers directly with `pull_request_read` method `get`.
+   - Otherwise, use `bash` + `gh api repos/${{ github.repository }}/commits/{commit_sha}/pulls` to find PRs containing the commit SHA. If no results, also try searching open PRs whose head branch matches one of the **Branches** listed in the Context section.


gh api repos/.../commits/{commit_sha}/pulls can return closed/historical PR associations, so treating any result as the target PR may mis-route analysis to the wrong PR for status events.

In the exact failure mode this PR is fixing (status SHA not equal to current PR head), this step can still stop on an unrelated merged PR and never reach branch-based open-PR matching. Please require candidate validation (e.g., open state and/or head.ref match against BK_BRANCHES) before accepting commit-associated PRs, and fall back when none validate.

…r status events Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

strawgate · 2026-03-09T18:05:22Z

@copilot please address feedback

Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

Copilot · 2026-03-09T18:10:46Z

@copilot please address feedback

Addressed in 95a73f1. Simplified the commit SHA extraction for status events to use .commit.sha from the event payload (as @v1v identified), replacing the overcomplicated .branches[] workaround.

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/gh-aw-estc-pr-buildkite-detective.md:
- Around line 103-106: The jq expressions that set BK_BRANCHES and the later PR
extraction read arrays that may be null; update the echo lines that build
BK_BRANCHES and BK_PR_NUMBERS to defensively extract arrays using the optional
iterator and a default empty array (e.g., use .branches[]? and/or wrap with //
[]), and do the same for any `.pull_requests` or similar array extractions
reading from GITHUB_EVENT_PATH so iterations over BK_BRANCHES and BK_PR_NUMBERS
never break on null.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: faeef151-0896-4f83-9e4c-a9cc53b3a5da

📥 Commits

Reviewing files that changed from the base of the PR and between 530cc72 and 9726469.

📒 Files selected for processing (2)

.github/workflows/gh-aw-estc-pr-buildkite-detective.lock.yml
.github/workflows/gh-aw-estc-pr-buildkite-detective.md

.github/workflows/gh-aw-estc-pr-buildkite-detective.md

strawgate · 2026-03-09T19:27:01Z

/ai please address feedback

strawgate · 2026-03-09T19:27:15Z

@copilot please address feedback

Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

Copilot · 2026-03-09T19:34:33Z

@copilot please address feedback

Addressed all review feedback in 9deb89a:

jq null-safety: .branches and .pull_requests now use // [] fallback to avoid errors on null arrays
BK_PR_NUMBERS: Emits empty string instead of [] when no PRs, so the downstream "non-empty" check is unambiguous
PR candidate validation: commits/{sha}/pulls results are now filtered for state == "open" and head.ref matching BK_BRANCHES before accepting, preventing misrouting to closed/historical PRs

github-actions · 2026-03-10T00:06:49Z

.github/workflows/gh-aw-estc-pr-buildkite-detective.md

-3. For each matching PR, keep author, branches, and fork status for downstream analysis.
+2. Find the associated pull request(s):
+   - If **PR Numbers** in the Context section above is non-empty (e.g., from `check_run` events), use those PR numbers directly with `pull_request_read` method `get`.
+   - Otherwise, use `bash` + `gh api repos/${{ github.repository }}/commits/{commit_sha}/pulls` to find PRs containing the commit SHA. Filter the results to keep only PRs whose `state` is `"open"` and, when **Branches** is available, whose `head.ref` matches one of the listed branches. If no candidates remain, also try searching open PRs whose head branch matches one of the **Branches** listed in the Context section.


The branch-only fallback can mis-associate unrelated PRs when multiple open PRs share the same head.ref (common with forks or generic branch names like main/fix-ci).

In this path, candidates are matched only by branch name from BK_BRANCHES, which doesn’t include repo owner/context. That can route analysis/comments to the wrong PR.

Please add a disambiguation requirement in the fallback (for example: match head.repo.full_name and/or verify candidate head.sha/status target_url context before accepting).

Initial plan

8f7c7d1

Copilot AI assigned Copilot and strawgate Mar 9, 2026

Copilot started work on behalf of strawgate March 9, 2026 16:21 View session

Copilot AI and others added 2 commits March 9, 2026 16:31

fix: revert .commit.sha back to .sha for status events

66843be

Per code review feedback: .sha and .commit.sha should be equivalent in GitHub status events per the docs. Keep .sha for safety. Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix Buildkite configuration in gh-aw-estc-pr-buildkite-detective.lock.yml~~ fix: estc-pr-buildkite-detective bot allowlist, noop safe-output, and PR discovery Mar 9, 2026

Copilot finished work on behalf of strawgate March 9, 2026 16:41

This comment was marked as resolved.

Sign in to view

v1v self-requested a review March 9, 2026 16:48

v1v reviewed Mar 9, 2026

View reviewed changes

strawgate marked this pull request as ready for review March 9, 2026 17:11

Copilot started work on behalf of strawgate March 9, 2026 17:11 View session

github-actions bot added the medium_boom Medium PR blast radius; likely benefits from human review label Mar 9, 2026

github-actions bot requested changes Mar 9, 2026

View reviewed changes

fix: extract PR branch commit SHA from .branches[] instead of .sha fo…

24933f3

…r status events Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

Copilot finished work on behalf of strawgate March 9, 2026 17:23

Merge branch 'main' into copilot/fix-buildkite-config-issues

530cc72

This comment was marked as resolved.

Sign in to view

Copilot started work on behalf of strawgate March 9, 2026 18:05 View session

fix: use .commit.sha for status events instead of .branches[] workaround

95a73f1

Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

Copilot finished work on behalf of strawgate March 9, 2026 18:14

Merge branch 'main' into copilot/fix-buildkite-config-issues

9726469

coderabbitai bot reviewed Mar 9, 2026

View reviewed changes

.github/workflows/gh-aw-estc-pr-buildkite-detective.md Show resolved Hide resolved

Merge branch 'main' into copilot/fix-buildkite-config-issues

6caa1f0

Copilot started work on behalf of strawgate March 9, 2026 19:27 View session

fix: jq null-safety, empty BK_PR_NUMBERS, and PR candidate validation

9deb89a

Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

Copilot finished work on behalf of strawgate March 9, 2026 19:35

Merge branch 'main' into copilot/fix-buildkite-config-issues

1d5daa1

github-actions bot reviewed Mar 10, 2026

View reviewed changes

github-actions bot mentioned this pull request Mar 10, 2026

https://github.com/elastic/integrations/pull/17730 contains multiple comments for the PR Buildkite Detective #647

Open

Conversation

Copilot AI commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

Uh oh!

This comment was marked as resolved.

Uh oh!

v1v Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

v1v Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

strawgate commented Mar 9, 2026

Uh oh!

coderabbitai bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Possibly related PRs

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

strawgate commented Mar 9, 2026

Uh oh!

Copilot AI commented Mar 9, 2026

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

strawgate commented Mar 9, 2026

Uh oh!

strawgate commented Mar 9, 2026

Uh oh!

Copilot AI commented Mar 9, 2026

Uh oh!

github-actions bot Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Mar 9, 2026 •

edited

Loading

coderabbitai bot commented Mar 9, 2026 •

edited

Loading