Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 50 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
name: Tests

# PER-8195: explicitly use `pull_request` only. `pull_request_target` is
# forbidden — it checks out attacker-controlled code with full secret access.
on: [push, pull_request]

jobs:
build:
basic:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
Expand All @@ -20,3 +24,48 @@
env:
PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}

advanced:
# PER-8195 advanced example. Runs in --testing mode so PR builds (including
# forks and Dependabot) don't require a real PERCY_TOKEN. The `--testing`
# flag is only valid on `percy exec` (not `exec:start`).
runs-on: ubuntu-latest
timeout-minutes: 15
defaults:
run:
working-directory: advanced
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 18
- name: Install jq + yq
run: |
sudo apt-get update -qq
sudo apt-get install -y -qq jq
sudo wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
sudo chmod +x /usr/local/bin/yq
- name: Install advanced/ dependencies
run: npm install
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium
- name: Fetch shared advanced-snapshot assertion helper
# TODO(PER-8195 D8): pin to a tagged commit once percy-public-repos-parent
# publishes the scripts/ dir to a stable URL or to an npm package.
run: |
curl -fsSL -o /tmp/assert-advanced-snapshots.sh \
https://raw.githubusercontent.com/percy/percy-public-repos-parent/main/scripts/assert-advanced-snapshots.sh
chmod +x /tmp/assert-advanced-snapshots.sh
- name: Run playwright advanced tests (--testing) + capture /test/requests
env:
PERCY_TOKEN: fake_token
run: |
npx --no-install percy exec --testing -- bash -c '
npx --no-install playwright test --reporter=dot
ec=$?
curl -fsS http://localhost:5338/test/requests > advanced-requests.json || true
exit $ec
'
- name: Assert matrix-row coverage
env:
PERCY_REQUESTS_FILE: ${{ github.workspace }}/advanced/advanced-requests.json
run: /tmp/assert-advanced-snapshots.sh ./matrix.yml

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium test

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
Comment on lines +31 to +71
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,19 @@

Example app showing integration of [Percy](https://percy.io/) visual testing into Playwright tests.

> **New:** This repo ships an [`advanced/`](./advanced) example covering the full applicable Percy SDK feature surface for `@percy/playwright`. See the [Percy SDK Feature Matrix](https://docs.percy.io/docs/sdk-feature-matrix) for cross-SDK coverage.

Based on the [TodoMVC](https://github.com/tastejs/todomvc) [VanillaJS](https://github.com/tastejs/todomvc/tree/master/examples/vanillajs)
app, forked at commit
[4e301c7014093505dcf6678c8f97a5e8dee2d250](https://github.com/tastejs/todomvc/tree/4e301c7014093505dcf6678c8f97a5e8dee2d250).

## Examples

| Example | What it shows | Run command |
|---|---|---|
| `./` (basic, at repo root) | Minimum viable integration: a `percySnapshot(page, name)` call per test. Start here. | `npm run test-web` |
| [`./advanced/`](./advanced) | Full applicable Percy SDK feature surface: widths, percyCSS, regions, readiness, discovery, domTransformation, etc. See [`advanced/README.md`](./advanced/README.md) for the matrix-row coverage table. | `cd advanced && npm install && npx playwright install --with-deps chromium && npm run test:advanced` |

## Playwright Tutorial

The tutorial assumes you're already familiar with JavaScript and
Expand Down
5 changes: 5 additions & 0 deletions advanced/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
node_modules/
advanced-requests.json
playwright-report/
test-results/
*.log
14 changes: 14 additions & 0 deletions advanced/.percy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# PER-8195 — advanced example global config for @percy/playwright.

version: 2

snapshot:
widths: [375, 1280]
min-height: 1024
percy-css: |
.new-todo::placeholder { color: #999 !important; }

discovery:
allowed-hostnames:
- localhost
network-idle-timeout: 500
57 changes: 57 additions & 0 deletions advanced/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Advanced Percy + Playwright example

This directory exercises the full applicable Percy SDK feature surface for `@percy/playwright`. See the basic example at the repo root for the minimum integration.

## What this example covers

A single Playwright spec (`tests/todomvc_advanced.spec.js`) where each `test(...)` block exercises one row of the [Percy SDK Advanced Feature Matrix](../../../docs/advanced-example-feature-matrix.md). Global SDK config — readiness preset, default widths, discovery — lives in `.percy.yml`.

## Run locally

```bash
cd advanced
npm install
npx playwright install --with-deps chromium
export PERCY_TOKEN="<your project token>" # do NOT commit this
npm run test:advanced
```

To run without a real token (CI assertion mode):

```bash
npm run test:advanced:ci # uses --testing + PERCY_TOKEN=fake_token
```

The CI variant asserts every matrix row appears in the captured POST bodies at the local `/test/requests` endpoint. No real Percy build is created.

## Coverage matrix

States: `Covered` / `N/A — <reason>` / `Planned` / `Deprecated`. Source of truth is [`matrix.yml`](./matrix.yml).

| Feature | State | Test |
|---|---|---|
| widths | Covered | `exercises widths` |
| percyCSS | Covered | `exercises percyCSS` |
| minHeight | Covered | `exercises minHeight` |
| enableJavaScript | Covered | `exercises enableJavaScript` |
| scope | Covered | `exercises scope` |
| discovery options | Covered | `exercises discovery options` |
| domTransformation | Covered | `exercises domTransformation` |
| responsiveSnapshotCapture | Covered | `exercises responsiveSnapshotCapture` |
| labels | Covered | `exercises labels` |
| testCase | Covered | `exercises testCase` |
| devicePixelRatio | Covered | `exercises devicePixelRatio` |
| regions | Covered | `exercises regions` |
| readiness preset | Covered | `exercises readiness preset` |
| browsers override | Covered | `exercises browsers override` |
| cross-origin iframe handling | Covered | automatic via `@percy/playwright >= 1.1.0` |
| cookie capture via `page.context().cookies()` | Covered | automatic via `@percy/playwright` |
| `.percy.yml` global config | Covered | `.percy.yml` consumed at build start |
| environment info reporting | Covered | automatic via `@percy/playwright` client info |
| PERCY_SERVER_ADDRESS via env | Covered | CI advanced job picks up `PERCY_SERVER_ADDRESS` |
| `disableShadowDOM` | Planned | — |
| `enableLayout` | Planned | — |
| `reshuffleInvalidTags` | Planned | — |
| `scopeOptions.scroll` | Planned | — |
| sync mode | Planned | — |
| regions via `createRegion` helper | Planned | — |
85 changes: 85 additions & 0 deletions advanced/matrix.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# PER-8195 Phase 1 — Playwright matrix-row mapping.
# Source of truth for the README table + aggregated docs-site grid.
# Test code: tests/todomvc_advanced.spec.js.

sdk: playwright
package: '@percy/playwright'
language: javascript
sdk_min_version: '1.1.0'
cli_min_version: '1.28.8' # sync option + waitForReady require >= 1.28.8

rows:
- id: widths
state: covered
test: 'TodoMVC Advanced > exercises widths'
- id: percy_css
state: covered
test: 'TodoMVC Advanced > exercises percyCSS'
- id: min_height
state: covered
test: 'TodoMVC Advanced > exercises minHeight'
- id: enable_javascript
state: covered
test: 'TodoMVC Advanced > exercises enableJavaScript'
- id: scope
state: covered
test: 'TodoMVC Advanced > exercises scope'
- id: discovery
state: covered
test: 'TodoMVC Advanced > exercises discovery options'
- id: dom_transformation
state: covered
test: 'TodoMVC Advanced > exercises domTransformation'
- id: responsive_snapshot_capture
state: covered
test: 'TodoMVC Advanced > exercises responsiveSnapshotCapture'
- id: labels
state: covered
test: 'TodoMVC Advanced > exercises labels'
- id: test_case
state: covered
test: 'TodoMVC Advanced > exercises testCase'
- id: device_pixel_ratio
state: covered
test: 'TodoMVC Advanced > exercises devicePixelRatio'
- id: regions
state: covered
test: 'TodoMVC Advanced > exercises regions'
- id: readiness_preset
state: covered
test: 'TodoMVC Advanced > exercises readiness preset'
- id: browsers
state: covered
test: 'TodoMVC Advanced > exercises browsers override'
- id: disable_shadow_dom
state: planned
- id: enable_layout
state: planned
- id: reshuffle_invalid_tags
state: planned
- id: scope_options_scroll
state: planned
- id: sync
state: planned
- id: create_region_helper
state: planned
test: 'TodoMVC Advanced > exercises regions (via createRegion helper)'

# Playwright-specific.
- id: cross_origin_iframe_handling
state: covered
test: 'automatic via @percy/playwright >= 1.1.0'
notes: 'SDK auto-detects cross-origin iframes, parallel frame processing.'
- id: cookie_capture_via_context
state: covered
test: 'automatic via @percy/playwright (page.context().cookies())'

- id: env_percy_server_address
state: covered
test: 'CI: advanced job sets PERCY_SERVER_ADDRESS via env'
- id: percy_yml_global_config
state: covered
test: 'global config consumed via .percy.yml'
- id: environment_info_reporting
state: covered
test: 'automatic via @percy/playwright client info'
Loading
Loading