Test: visual-diff GHAs after merge + new post coverage#6
Open
RisingOrange wants to merge 32 commits intomainfrom
Open
Test: visual-diff GHAs after merge + new post coverage#6RisingOrange wants to merge 32 commits intomainfrom
RisingOrange wants to merge 32 commits intomainfrom
Conversation
Captures HTML+asset archives of 15 key routes × 2 viewports on each PR and uploads to Chromatic for server-side rendering and diffing against baseline. Posts a non-blocking "UI Review" check on the PR. - Routes auto-discovered from src/routes/, plus 5 markdown-post representatives; form/admin/content-churn routes explicitly excluded with WHY comments - Deterministic snapshots: /api/news mocked with a fixture, Tally iframes on /statement and /join aborted to avoid async-height churn - Chromium-only, non-blocking by design (exitZeroOnChanges) - Fork PRs supported via plaintext project token in the workflow (Chromatic's recommended pattern) - Playwright report (per-route screenshots) uploaded as GHA artifact
Without production secrets, Notion / Airtable / Substack calls during `pnpm build` (prerender, remote prerender fns) and `pnpm preview` (on-demand SSR, client-side fetches to /api/*) either fail or fall back to a couple of placeholder records — leaving snapshots too sparse to catch layout regressions in populated list/card components. Boot MSW in the Node process via `NPM_CONFIG_NODE_OPTIONS=--import` (the only way to propagate `--import` through pnpm, which otherwise overwrites NODE_OPTIONS with `--experimental-global-webcrypto`; see pnpm/pnpm#6210). The setup file is a no-op unless `VISUAL_TEST=1`. Handlers cover the two Notion databases (press, funding), the three Airtable tables (people, signatories, national-groups), the Substack RSS feed hit by /api/news, plus catch-all empty-result responses for any un-fixtured endpoint. Fake API keys are set in the workflow so the SDKs actually make HTTP requests (they short-circuit to a "no key" error otherwise, bypassing MSW entirely). All fixture data lives under a single `tests/visual/fixtures/` directory — the earlier `tests/visual/fixtures/news.json`, which was intercepted at the browser boundary via `page.route().fulfill()`, is replaced by `substack-feed.xml` served by MSW upstream of the /api/news server code. `page.route()` in smoke.spec.ts is now reserved for aborting third-party widgets (Tally, Mapbox, Luma) whose requests never reach our Node process. Clean split: MSW for fixture data, page.route for cross-origin aborts. Fixtures use realistic-shaped but unambiguously-fake data: first names paired with Example / Sample / Fixture surnames, Lorem-ipsum bios, example.org URLs, Example Foundation / Example News. Chromatic snapshots persist in its UI, so the content leaves no room to mistake a fixture attribution for a real endorsement if an image escapes context. Adds `msw` as a dev dependency and entry-points tests/visual/msw- setup.mjs in knip.config.ts so the indirect import is recognized.
Previously excluded because its chronological listing churns on any new post, same mechanism as the home page's LatestNews section — but the home page is included (its news churn is accepted as one re-approval per news-flagged post). Include /posts on the same trade-off: the cost is a re-approval in Chromatic per post add, which is trivial. Removes an inconsistency in which pages opt out vs which accept content-driven churn.
Four mobile snapshots blew past Chromatic's 25M-pixel snapshot cap with the default Pixel 7 DPR of 2.625: /dear-sir-demis-2025 (52M, 2.1× over), /posts (47M, 1.9× over), /learn (25.8M), /write (25.2M). Override deviceScaleFactor to 1 on the mobile project. The viewport is still 412 CSS px (the layout-relevant dimension); we just capture at 1× instead of 2.6× physical pixels. Mobile layout diffs don't need the hi-DPI oversampling — a layout regression is visible at either DPR. After the change, the largest mobile snapshot is 412 × 18468 ≈ 7.6M, well under the cap.
Addresses the reviewer concern that a green Chromatic check could silently imply coverage we don't actually have. On every PR, post a sticky comment summarizing what the run snapshotted: covered/excluded ratios for pages and posts, what "covered" doesn't mean (fixtures, widget aborts, en-only), and a conditional warning when un-fixtured requests fall through to MSW catch-alls / bypass. Plumbing: - tests/visual/scope-comment.ts renders the comment body from repo state (routes, posts, annotations) + the MSW warning log. - msw-setup.mjs listens for request:unhandled and msw-handlers.ts's catch-alls log to MSW_WARN_LOG when set, so the warning section has data to render. - visual-diff.yml generates the artifact (body.md + pr-number.txt) on every PR run. - visual-diff-comment.yml (new) triggers on workflow_run, runs in the base-repo context with pull-requests: write, downloads the artifact, posts or updates a sticky comment via marocchino/sticky-pull-request- comment. Works for fork PRs without granting forks write permissions. - README updated with a disclaimer about browser-side third-party embeds being out of scope for the MSW-based warning.
The previous page.route setup aborted three specific hosts (tally.so, lu.ma, api.mapbox.com). A new third-party widget added in a future PR would have loaded normally in CI, potentially leaking live state into snapshots with no signal. Replace with a generic rule: abort any cross-origin request whose resourceType is `document` (iframe), `xhr`, or `fetch`. Pass through cross-origin resources — images (Cloudinary), fonts, scripts (inlang CDN), stylesheets — since those are embedded assets whose content is part of the rendered design we want to capture. Server-side external calls remain handled by MSW-node. All three previous specific aborts fall under the generic rule: - tally.so iframes → `document` → aborted - lu.ma iframes → `document` → aborted - api.mapbox.com tiles (via mapbox-gl.js) → `xhr` → aborted Also: - Derive the "same origin" from Playwright's `baseURL` fixture rather than hardcoding `http://localhost:4173`, so a port change in the config doesn't silently invert the rule. - Update the "Not exercised" caveat to reflect the new behavior: containers render empty and reviewers should check the Netlify preview deploy for real rendering.
Codex review (gpt-5.5 high) flagged that the PR-number resolver filters on head_branch and head_repository.id but not base_ref, so a fork branch opened as two PRs against different base branches simultaneously could route the comment to the wrong PR. Fix options were all ugly (iterate + post to each, fail on >1 match); the case is rare enough that documenting the limit is the pragmatic call.
`dotenv.config({ override: true })` was clobbering the fake
AIRTABLE_API_KEY / NOTION_API_KEY that the visual-diff workflow sets,
with the empty values from template.env. The SDKs then short-circuited
on empty key before ever making a request, so MSW had nothing to
intercept and /about, /statement, /communities snapshots rendered their
hardcoded "[FALLBACK DATA]" fallback paths instead of fixture data.
Previously the three `PauseAI Example North/South/East` records didn't match any country name in the static `communities` list, so `nationalGroups.find()` returned undefined for every card and the fixture had no visible effect — snapshots were identical whether the fixture returned records or an empty list. Now one record matches 'France' and carries an inline SVG tricolor as its image, so France's card shows a fixture-provided flag while the rest keep the default icon. The populated-card render path is now covered.
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* Update australia-progress.md * Add files via upload * Update australia-progress.md * Tweaked settings of introductory video in australia-progress.md Added an introductory video for PauseAI Australia. * Revise Australia progress report with new updates Updated the Australia progress report with recent activities and milestones, including volunteer roles, newsletters, and outreach efforts. * Misc updates mainly Dec 2025 * Update Australia progress with recent AI developments * Add files via upload * Update Australia progress with incorporation news Added updates about incorporation and petition efforts. * Assorted activity updates * style: format code with prettier --------- Co-authored-by: Wituareard <87208350+Wituareard@users.noreply.github.com>
…OU (PauseAI#812) Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* Restructure /press to be a Markdown post
Same pattern as /funding: move to src/posts/press.md, fetch coverage via
a prerender remote function, keep the existing PressCoveragePanel as a
legacy leaf component wrapped by a tiny loader that does the await.
Also inherits the l10n pipeline for the intro/materials copy.
* Add PressCoveragePanelLoader to knip entry points
It's imported only from a Markdown file, which knip doesn't scan.
* Restore press page polish styles
Preserve .contact-clarification (1.1rem, bottom margin) and tighter
h2 margin under 'Media Coverage', both lost when converting from the
bespoke route.
* Use {#await} block for press coverage loader
Same fix as was applied to the funding donor list: top-level await in
PressCoveragePanelLoader.svelte created an unscoped async boundary that
would blank the whole /press page subtree during hydration.
Switched to an {#await} template block and dropped the now-unused
compilerOptions.experimental.async flag.
* Drop dead press coverage scaffolding
Remove temporary 'publication' property support, stale .logo-materials CSS rule (class no longer exists after markdown migration), redundant try/catch in the remote function, and the unused PressCoverageData interface.
* Switch press coverage to client-side fetching with loading skeletons
---------
Co-authored-by: Wituareard <87208350+Wituareard@users.noreply.github.com>
* Adjust iframe width and update donation milestone Updated iframe width and donation milestone statement. * Add files via upload * Update australia-progress.md * Add files via upload * Add files via upload * Add files via upload * Update australia-progress.md * Add files via upload * Replace YouTube iframe with no-cookie version Updated the video embed to use a no-cookie version for privacy. * Remove video iframe from australia-progress.md Removed embedded video iframe from Australia progress post. * Enhance Australia page with action buttons Added buttons for more details and progress updates. * Create australia-story.md for AI story sharing Added a new markdown file for sharing stories about AI in Australia. * style: format code with prettier * Replace iframe with Tally component --------- Co-authored-by: Wituareard <87208350+Wituareard@users.noreply.github.com>
…utes in markdown files
Adds @visualDiffEnabled: true to three posts whose embedded components or layout aren't represented by the existing covered set: Donate (donate.md), Doomers (pdoom.md), and the auto-generated SimpleToc (faq.md).
# Conflicts: # knip.config.ts # pnpm-lock.yaml
When a commit lives on the repo's default branch, GitHub's
listPullRequestsAssociatedWithCommit endpoint only returns merged PRs;
open PRs with that commit at their head are filtered out. This is
unlikely in production (the feature branch is never main), but it
silently breaks the scope comment on any fork where the feature branch
happens to be default.
Switch to pulls?state=open&head={owner}:{branch}, which has no such
quirk. Trusted-inputs safety property is unchanged: head owner/branch
come from workflow_run, and we still verify head.repo.id and head.sha
against the trusted payload before posting.
|
Visual diff coverage
Route breakdown
Not exercised
|
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Throwaway PR to exercise the visual-diff GitHub Actions on the merged state of
visual-diff(incl. main merged in and the newdonate/pdoom/faqopt-ins).Not for merging — corresponds to upstream PR PauseAI#785.