Skip to content

getting-started.md

Latest commit

 

History

History
187 lines (140 loc) · 7.21 KB

getting-started.md

File metadata and controls

187 lines (140 loc) · 7.21 KB

Getting Started

This guide is for first-time ui-test users.

Prerequisites

  • Node.js 18+
  • npm

Pick Your Entry Path

Repository checkout

npm run setup

One-off run (no install)

npx -y github:ddv1982/ui-test setup --browsers chromium

Until npm publication, use the repository checkout flow or the one-off GitHub invocation instead of a raw git global install.

All command examples below use either the local checkout flow or the one-off GitHub invocation.

Setup

ui-test setup launches an interactive browser picker. For non-interactive use, pass --browsers:

ui-test setup
ui-test setup --browsers chromium
ui-test setup --browsers chromium --run-play

Test File Format

Test files are YAML documents in the e2e/ directory. Here is e2e/example.yaml:

name: Example Test
description: A visible sample flow to demonstrate headed execution
steps:
  - action: navigate
    url: /
    description: Open the example app
  - action: assertVisible
    description: App root is visible
    target:
      value: "#app"
      kind: css
      source: manual
  - action: fill
    description: Type a name into the input
    target:
      value: "[data-testid='name-input']"
      kind: css
      source: manual
    text: "Codex"
  - action: click
    description: Click the greet button
    target:
      value: "[data-testid='greet-button']"
      kind: css
      source: manual
  - action: assertText
    description: Greeting message is updated
    target:
      value: "[data-testid='message']"
      kind: css
      source: manual
    text: "Hello, Codex!"

Each step has an action type:

  • navigate — go to a URL (relative URLs use the configured baseUrl)
  • click — click the element matched by target
  • fill — type text into the element matched by target
  • press — press a keyboard key on the target element
  • check / uncheck — toggle a checkbox
  • hover — hover over the target element
  • select — pick an option by value from a dropdown
  • assertVisible — verify the target element is visible
  • assertText — verify the target element contains text
  • assertValue — verify the target input has a specific value
  • assertChecked — verify a checkbox is checked (or unchecked)

The target object identifies the element:

  • value — the selector string or locator expression
  • kind — selector type (css, xpath, locatorExpression, playwrightSelector, etc.)
  • source — how the selector was created (manual, codegen, devtools-import; legacy files may still contain codegen-jsonl or codegen-fallback)

optional: true is no longer supported for steps. Remove this field from existing YAML tests. Steps can include timeout: <ms> to override the global step timeout.

Run Tests

ui-test play

If your app is already running and you do not want auto-start:

ui-test play --no-start

Auto-start launches the built-in example app for e2e/example.yaml only.

If replay needs an existing signed-in session, preload a Playwright storage state JSON file:

ui-test play e2e/account.yaml --load-storage .auth/state.json

The same --load-storage <path> flag is available on ui-test improve.

Record and Replay

ui-test record
ui-test play

This opens a browser. Interact with your app, then close the browser to save the recording as a YAML file in the e2e/ directory.

After recording, ui-test runs a review-first auto-improve pass by default. The recorded YAML stays unchanged, and the improve report uses deterministic assertion candidates with the reliable policy. Use --improve-mode apply if you want an improved copy written automatically, or --no-improve to skip post-processing entirely.

On dynamic content pages (for example news homepages), recorder/improve use stability-first defaults:

  • Reliable selector normalization avoids brittle exact: true headline locators when headline text appears dynamic.
  • Improve can adopt safer repair selectors for dynamic targets even on score ties when runtime matching is unique.
  • Improve can convert dynamic internal/playwrightSelector targets into locatorExpression targets via Playwright runtime APIs when uniquely matched.
  • Navigation-like dynamic link clicks avoid deterministic post-click assertVisible insertion; URL/title/snapshot-native assertions are preferred.
  • You can disable runtime selector regeneration/conversion for debugging with UI_TEST_DISABLE_PLAYWRIGHT_RUNTIME_REGEN=1.

Improve Tests

improve upgrades selectors, generates assertion candidates, and classifies runtime-failing interactions (aggressively removes transient dismissal/control click/press failures, retains non-transient and safeguarded content/business interactions as required steps).

ui-test improve e2e/login.yaml                  # prompts to write e2e/login.improved.yaml
ui-test improve e2e/login.yaml --apply          # writes e2e/login.improved.yaml (no prompt)
ui-test improve e2e/login.yaml --apply --in-place
ui-test improve e2e/login.yaml --apply --load-storage .auth/state.json
ui-test improve e2e/login.yaml --no-apply

By default, improve prompts you to confirm before writing an improved copy (<input>.improved.yaml). Use --apply to skip the prompt (CI-friendly), --in-place to overwrite the input file, --output <path> for a custom destination, or --no-apply for a report-only run without prompting.

Apply-mode runs can mark candidates as skipped_policy when policy caps/filters are enforced. Report-only runs (--no-apply) keep candidate status as not_requested.

Selector auto-apply stays intentionally conservative:

  • candidates must be a meaningful improvement over the current selector
  • candidates must resolve uniquely at runtime
  • runtime-derived repairs must pass determinism guards
  • low-confidence recommendations stay report-only so you can review them with ui-test improve
  • for ugly enterprise UIs, improve now also harvests runtime hints such as data-testid, name, id, title, and row context to synthesize better fallback candidates without requiring UI changes

When replaying in headed mode, play registers targeted Playwright locator handlers for consent/non-cookie overlays, keeps targeted pre-step dismissal as fallback, and retries once after a verified overlay dismissal when click interception is detected. When replay still hits an ambiguous/broken locator, the run stops clearly and points you back to ui-test improve <file> --apply instead of silently guessing.

Control assertion generation with --assertions:

ui-test improve e2e/login.yaml --assertions candidates   # default
ui-test improve e2e/login.yaml --assertions none          # skip assertions

Control apply strictness with --assertion-policy:

ui-test improve e2e/login.yaml --assertion-policy balanced    # default
ui-test improve e2e/login.yaml --assertion-policy reliable
ui-test improve e2e/login.yaml --assertion-policy aggressive

Next Steps