Skip to content

perf(ci): shard cloud parity across 2 BrowserStack sessions via CDP#367

Merged
sebavan merged 2 commits into
masterfrom
raananw-parity-ci-two-sessions
Jul 6, 2026
Merged

perf(ci): shard cloud parity across 2 BrowserStack sessions via CDP#367
sebavan merged 2 commits into
masterfrom
raananw-parity-ci-two-sessions

Conversation

@RaananW

@RaananW RaananW commented Jul 6, 2026

Copy link
Copy Markdown
Member

Why

The Parity (Cloud) job runs all parity scenes serially on a single BrowserStack session behind the browserstack-node-sdk-managed tunnel. This is a small, focused subset of #302 that targets the two biggest, lowest-risk wins without the large surface area of that PR (which also replaced the tunnel with a public static site + 233 scene-URL rewrites).

What changed

  • Direct CDP, no SDK. Parity connects to remote Chrome via Playwright's connectOptions.wsEndpoint (wss://cdp.browserstack.com/playwright) — no browserstack-node-sdk, no browserstack.yml. Each Playwright worker is its own cloud session, so specs shard across workers.
  • Up to 2 parallel sessions. scripts/browserstack-wait.sh grabs up to BSTACK_SESSIONS_REQUIRED (default 2) sessions from the plan, falls back to fewer when the plan is busy, exports CIWORKERS, and retries on queue-size-exceeded.
  • Tunnel kept (unlike perf(ci): cut cloud parity time — public static site + sharded BrowserStack (CDP) #302). A BrowserStack Local tunnel is started by the config's globalSetup (config/browserstack-local-tunnel.ts) so the remote browser still reaches the local Vite dev server. Specs keep using goto("/sceneN.html") unchanged.
  • Perf job untouched. Perf still uses the SDK + config/browserstack.yml + tunnel.

Files

  • config/playwright.parity-cloud.config.ts — SDK launch → CDP connectOptions; workers from CIWORKERS; fullyParallel; trace: "off" (the wsEndpoint embeds the access key).
  • config/browserstack-cdp.ts (new) — shared CDP endpoint builder + localIdentifier.
  • config/browserstack-local-tunnel.ts (new) — globalSetup that starts/stops the Local tunnel; no-op without credentials.
  • scripts/browserstack-wait.sh (new) — session grabber (default 2).
  • package.json — add browserstack-local dep; test:parity-cloud no longer wraps the SDK.
  • azure-pipelines.yml — run parity via browserstack-wait.sh with BSTACK_SESSIONS_REQUIRED=2 and a per-build BSTACK_LOCAL_IDENTIFIER.
  • TESTING.md — document the CDP/sharded setup.

Expected impact

At 2 sessions the parity job should be roughly 2× faster, with no changes to engine src, scenes, or goldens.

Verification

  • Both config paths load via playwright test --list (local-fallback and CDP with creds → 213 tests).
  • bash -n scripts/browserstack-wait.sh clean; azure-pipelines.yml valid YAML.
  • prettier --check clean on changed files; confirmed Playwright 1.58 runs the function returned by globalSetup as teardown, so the tunnel is stopped.
  • Full parity suite needs BrowserStack credentials, so it runs in CI only.

Notes / follow-ups

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Connect the Parity (Cloud) job to remote Chrome directly over CDP
(wss://cdp.browserstack.com/playwright) instead of browserstack-node-sdk,
and shard specs across up to 2 parallel sessions (falling back to fewer
when the plan is busy). This roughly halves parity wall-clock.

- config/playwright.parity-cloud.config.ts: SDK launch -> CDP connectOptions;
  workers from CIWORKERS; fullyParallel; trace off (wsEndpoint embeds the key).
- config/browserstack-cdp.ts: shared CDP endpoint builder + localIdentifier.
- config/browserstack-local-tunnel.ts: globalSetup starts/stops a BrowserStack
  Local tunnel so the remote browser reaches the local dev server; no-op
  without credentials.
- scripts/browserstack-wait.sh: grab up to BSTACK_SESSIONS_REQUIRED (default 2)
  sessions, export CIWORKERS, retry on queue-size-exceeded.
- package.json: add browserstack-local dep; test:parity-cloud no longer wraps
  the SDK. Perf job keeps the SDK (browserstack.yml).
- azure-pipelines.yml: run parity via browserstack-wait.sh with 2 sessions and
  a per-build BSTACK_LOCAL_IDENTIFIER.
- TESTING.md: document the CDP/sharded parity setup.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 6, 2026 13:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR speeds up the Parity (Cloud) Azure Pipelines job by switching it from an SDK-managed single BrowserStack session to direct CDP connections with Playwright, enabling the parity specs to shard across up to 2 parallel BrowserStack sessions while keeping the Local tunnel approach (remote browser still reaches the local Vite dev server).

Changes:

  • Replace browserstack-node-sdk usage in cloud parity with Playwright connectOptions.wsEndpoint (CDP) and drive worker count via CIWORKERS.
  • Add a browserstack-wait.sh helper to poll plan capacity, pick an available session count, export CIWORKERS, and retry on queue contention.
  • Add shared BrowserStack CDP helpers + a Playwright globalSetup to start/stop a browserstack-local tunnel; update CI + docs accordingly.

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
TESTING.md Documents the new CDP + sharded parity-cloud setup and session-wait invocation.
scripts/browserstack-wait.sh New capacity-polling wrapper that selects an available session count and exports CIWORKERS.
pnpm-lock.yaml Locks the added browserstack-local dependency.
package.json Updates parity-cloud script to run Playwright directly; adds browserstack-local.
config/playwright.parity-cloud.config.ts Switches parity-cloud to CDP wsEndpoint, enables parallel workers, and wires in globalSetup.
config/browserstack-local-tunnel.ts New Playwright globalSetup to manage the BrowserStack Local tunnel lifecycle.
config/browserstack-cdp.ts New shared helpers to build the CDP endpoint and keep localIdentifier consistent.
azure-pipelines.yml Runs parity-cloud via the wait script, sets sessions required + per-build local identifier.
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread scripts/browserstack-wait.sh
Comment thread azure-pipelines.yml
…parity artifacts

Address Copilot review on #367:
- browserstack-wait.sh: fail fast when BSTACK_SESSIONS_MIN exceeds the
  (capped) preferred count, which would otherwise silently grant fewer
  than the requested minimum.
- Add scripts/redact-secrets.ts and a ParityCloud pipeline step that
  scrubs BROWSERSTACK_ACCESS_KEY/USERNAME (raw + URL-encoded) from the
  test-results report/JUnit before they are published/uploaded, since
  the CDP wsEndpoint embeds the access key.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@bjsplat

bjsplat commented Jul 6, 2026

Copy link
Copy Markdown

Bundle Size Changes

Decreases

Package Current Master Change
Scene 179 - Clustered Sponza Lights
scene179
47 KB 73 KB -26 KB

Sizes rounded to nearest KB. Run pnpm build:bundle-scenes locally to verify.

@bjsplat

bjsplat commented Jul 6, 2026

Copy link
Copy Markdown

Lab - Static Site

Open deployed site

Build 20260706.9 - merge @ ca7c72f

@sebavan sebavan merged commit 27834f1 into master Jul 6, 2026
11 checks passed
@sebavan sebavan deleted the raananw-parity-ci-two-sessions branch July 6, 2026 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants