Skip to content

Playwright: residual 6 test failures unmasked by #121 (a11y, perf threshold, console errors) #122

@hyperpolymath

Description

@hyperpolymath

Context

#121 fixed the original #118 preload bundle 404 by setting webServer.cwd: '../..' in tools/compat-testing/playwright.config.js. Before #121, all Playwright tests failed at preload bundle init. After #121, 12 tests pass and 6 fail with categorically different errors — pre-existing latent failures that were masked by the preload error.

The 6 residual failures (chromium-1080p)

Group A — Accessibility (4 tests)

  • tests/accessibility.spec.js:33 › ARIA landmarks are present
  • tests/accessibility.spec.js:75 › canvas has accessible name or label
  • tests/accessibility.spec.js:155 › screen reader announcements via aria-live regions
  • tests/accessibility.spec.js:222 › high contrast mode does not break rendering

Root cause: PixiJS canvas has no ARIA attributes by default. The HTML host (<canvas>) needs role="application" + aria-label minimum, and the game probably needs an off-canvas live region for screen reader announcements (alert level changes, menu transitions, etc.).

Group B — Performance threshold (1 test)

  • tests/performance.spec.js:90 › frame rate during gameplay

Result: fps=2.7 (chromium), fps=2.05 (retry). Threshold expects ≥24. Software-rendered WebGL in headless GHA runners cannot hit 24fps. The threshold is too tight for CI; needs an ENV-gated relaxation (e.g. CI_HEADLESS=1 lowers to ≥5fps).

Group C — Console errors (1 test)

  • tests/game-loads.spec.js:47 › page loads without JavaScript errors

5 console.errors emitted during page load. Need to capture the actual messages from a CI artifact trace to triage — could be missing CSP headers in vite dev mode, deprecation warnings, asset 404s for optional assets, etc.

Webkit-1080p

Failed the same set (also includes "WebGL is not yet supported in WebKit headless"). Webkit has known WebGL gaps in CI.

Firefox-1080p

[In progress at time of filing; will update.]

Proposed sub-issues

Each group is a tractable single-PR slice:

  • a11y sub-issue: add ARIA attributes to <canvas id=\"pixi-container\"> + canvas wrapper; create an aria-live=\"polite\" off-canvas region for the dual-alert HUD level changes
  • perf sub-issue: ENV-gate the fps threshold (CI=1 ⇒ ≥5fps, local ⇒ ≥24fps); document why headless GL is slow
  • console-errors sub-issue: download a trace artifact, identify the 5 errors, fix in source or suppress in test

Closes-as-tracking-only

This issue exists to track the unmasked failures. It does not block #121, which is the right fix for the original #118 problem (preload bundle 404). #118 itself can be considered closed once #121 ships, leaving the residual failures here as scope-creep follow-ups.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions