Skip to content

playwright.md

Latest commit

 

History

History
145 lines (96 loc) · 4.52 KB

playwright.md

File metadata and controls

145 lines (96 loc) · 4.52 KB

Playwright

Playwright provide us the ability to do end-to-end testing, headless browser testing, amongst others; local visual regression testing as an alternative to Chromatic's cloud-based approach.

Use Playwright when you need to:

  • Faster feedback loops during active development
  • Full control over test execution and baselines, e.g. snapshots or screenshots
  • OS specific visual testing

Tip

Use Playwright for rapid local development, (e.g. if you need fast feedback loop) and Chromatic for team reviews and CI/CD.

Requirements

Visual regression testing locally

Visual regression tests are operating system dependent.

Screenshots generated on macOS will differ from Linux or Windows due to font rendering, anti-aliasing, and browser engine differences.

To mitigate differences across operating systems, our visual regression tests are Dockerized for consistent visual regression testing.

Docker (Ubuntu Linux)

All visual tests run in Docker (Ubuntu Linux) by default for consistency with CI:

yarn test:visual          # Run tests
yarn test:visual:update   # Update snapshots
yarn test:visual:ui       # UI mode (opens at http://localhost:8282)
yarn test:visual:report   # View HTML report

Note

The port number is declared in the Docker compose and playwright-docker bash script, which should be in sync.

Snapshots are saved with -linux suffix to match CI environments.

For this reason at time of writing, the following snaptshots are ignored:

# The snapshots which are generally persisted for comparison
**.ts-snapshots

# These are generated during test runner, generally ignored
tests/**/*-actual.png
tests/**/*-diff.png
test-results

Generating snapshots for comparison

You MUST generate the snapshots you want to compare against. At time of writing, the team's using Chromatic for visual regression testing.

Warning

If you are reading this document, you should be aware that this provides you with custom control for advanced needs only! It does not provide you with the setup for cross-operating system, e.g. docker linux. This workflow does NOT expect you to store your favourite OS image/snapshots in the repository, it's for your own usage, or fast feedback loop only!

To generate snapshots, you'll have to manually checkout/switch to the target commit in history and run the test:visual to generate it and return back to your ongoing feature branch. Alternatively, you can store snapshots separately and place them at your need.

Hypothetically, you could use the following workflow to facilitate:

# Switch to a target/stable version of your liking
git checkout <COMMIT-STABLE-VERSION>
# Generate the snapshots against your target version
yarn test:visual
# Check if snapshots generated?
find ./tests -type f -name '**.ts-snapshots*'
# Once happy return to feature branch
git checkout <FEATURE-BRANCH>
# Run the tests
yarn test:visual

When intentional visual changes are made, update baselines:

Warning

Only update baselines when visual changes are intentional and reviewed.

yarn test:visual:update

Running tests locally

Execute all visual regression tests by running:

yarn test:visual

Alternatively, launch UI mode for a browser like experience with timeline, console logs, etc.

yarn test:visual:ui

Run specific test files

yarn test:visual tests/buttons/overview.spec.ts

Show report

To review test reports as HTML run:

yarn test:visual:report

Configuration

The Playwright configuration file contains default configuration values, which you can extend, modify or personalize.

For example, you can modify the default number of workers:

  ...

  workers: process.env.CI ? 4 : undefined,

Migration to Chromatic

Note

At time of writing, there aren't any test source files for Chromatic. Should assume that these are based on Storybook stories: available properties/parameters and tests are not curated.

Playwright tests can be easily migrated to Chromatic as currently (per the example provided), both leverage Storybook stories. Meaning that you can start with Playwright for local development and transition to Chromatic as collaboration or any requirements demand.

Here's a quick example of migration switch:

import { test, expect } from "@playwright/test"; 
import { test, expect } from "@chromatic-com/playwright";

Learn more