Skip to content

DO NOT MERGE: e2e tests on Cirrus Labs runners #5485

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alwx wants to merge 3 commits into
base: main
Choose a base branch
from
Open

DO NOT MERGE: e2e tests on Cirrus Labs runners #5485

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e2397d0
e2e tests on Cirrus Labs runners
alwx Dec 18, 2025
a04f351
Checking another group
alwx Dec 18, 2025
83a3d6a
rolling back last change
alwx Dec 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 7 additions & 16 deletions .github/workflows/e2e-v2.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,11 @@ jobs:
platform: ["ios", "android"]
include:
- platform: ios
runs-on: macos-26
runs-on: ["ghcr.io/cirruslabs/macos-tahoe-xcode:26.2", "runner_group_id:12"]
name: iOS
appPlain: performance-tests/test-app-plain.ipa
- platform: android
# Not using the latest version due to a known issue: https://github.com/getsentry/sentry-react-native/issues/4418
runs-on: ubuntu-22.04
runs-on: ["ghcr.io/cirruslabs/ubuntu-runner-amd64:22.04", "runner_group_id:12"]
name: Android
appPlain: performance-tests/TestAppPlain/android/app/build/outputs/apk/release/app-release.apk
steps:
Comment on lines 47 to 57

This comment was marked as outdated.

Sign in to view

Comment on lines 47 to 57
Copy link

@sentry sentry bot Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The runs-on array syntax ["image", "runner_group_id:12"] is invalid. GitHub Actions will wait indefinitely for a runner with both labels, which will never exist, causing jobs to hang.
Severity: CRITICAL | Confidence: High

🔍 Detailed Analysis

The workflow jobs metrics, react-native-build, and react-native-test use an invalid array syntax for the runs-on key, such as ["ghcr.io/cirruslabs/macos-tahoe-xcode:26.2", "runner_group_id:12"]. GitHub Actions interprets this as requiring a runner that has both labels. However, runner_group_id:12 is not a valid label for Cirrus Runners. As a result, no runner will ever match the criteria, causing the jobs to hang indefinitely and eventually time out, blocking the CI/CD pipeline.

💡 Suggested Fix

Remove the invalid runner_group_id:12 element from the runs-on array. The correct configuration should only specify the container image, like runs-on: "ghcr.io/cirruslabs/macos-tahoe-xcode:26.2". This should be applied to all affected jobs.

🤖 Prompt for AI Agent 
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: .github/workflows/e2e-v2.yml#L47-L57

Potential issue: The workflow jobs `metrics`, `react-native-build`, and
`react-native-test` use an invalid array syntax for the `runs-on` key, such as
`["ghcr.io/cirruslabs/macos-tahoe-xcode:26.2", "runner_group_id:12"]`. GitHub Actions
interprets this as requiring a runner that has both labels. However,
`runner_group_id:12` is not a valid label for Cirrus Runners. As a result, no runner
will ever match the criteria, causing the jobs to hang indefinitely and eventually time
out, blocking the CI/CD pipeline.

Did we get this right? 👍 / 👎 to inform future reviews.
Reference ID: 7703441

Expand All @@ -61,9 +60,6 @@ jobs:
- uses: ./.github/actions/disk-cleanup
if: ${{ matrix.platform == 'android' }}

- run: sudo xcode-select -s /Applications/Xcode_26.1.1.app/Contents/Developer
if: ${{ matrix.platform == 'ios' }}

- run: npm i -g corepack
- uses: actions/setup-node@v6
with:
Expand Down Expand Up @@ -189,15 +185,13 @@ jobs:
# Use Xcode 16 for older RN versions
- platform: ios
rn-version: '0.71.19'
xcode-version: '16.4'
runs-on: macos-15
runs-on: ["ghcr.io/cirruslabs/macos-sequoia-xcode:16.4", "runner_group_id:12"]
# Use Xcode 26 for newer RN versions (0.83.0)
- platform: ios
rn-version: '0.83.0'
xcode-version: '26.1.1'
runs-on: macos-26
runs-on: ["ghcr.io/cirruslabs/macos-tahoe-xcode:26.2", "runner_group_id:12"]
- platform: android
runs-on: ubuntu-latest
runs-on: ["ghcr.io/cirruslabs/ubuntu-runner-amd64:22.04", "runner_group_id:12"]
exclude:
# exclude JSC for new RN versions (keeping the matrix manageable)
- rn-version: '0.83.0'
Expand Down Expand Up @@ -247,9 +241,6 @@ jobs:
echo "SENTRY_RELEASE=$SENTRY_RELEASE"
echo "SENTRY_DIST=$SENTRY_DIST"

- run: sudo xcode-select -s /Applications/Xcode_${{ matrix.xcode-version }}.app/Contents/Developer
if: ${{ matrix.platform == 'ios' }}

- run: npm i -g corepack
- uses: actions/setup-node@v6
with:
Expand Down Expand Up @@ -323,9 +314,9 @@ jobs:
include:
- platform: ios
rn-version: '0.83.0'
runs-on: macos-26
runs-on: ["ghcr.io/cirruslabs/macos-tahoe-xcode:26.2", "runner_group_id:12"]
- platform: android
runs-on: ubuntu-latest
runs-on: ["ghcr.io/cirruslabs/ubuntu-runner-amd64:22.04", "runner_group_id:12"]

steps:
- uses: actions/checkout@v6
Expand Down
Loading