Skip to content

ci: add Playwright Appium test health report workflow#31815

Merged
cmd-ob merged 18 commits into
mainfrom
ci/playwright-test-health-report
Jun 18, 2026
Merged

ci: add Playwright Appium test health report workflow#31815
cmd-ob merged 18 commits into
mainfrom
ci/playwright-test-health-report

Conversation

@cmd-ob

@cmd-ob cmd-ob commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Description

Adds a scheduled GitHub Actions workflow that aggregates Playwright JSON artifacts from Appium smoke runs on main (ci.yml) and posts infra/flaky/watch test summaries to Slack via the playwright-test-health-report action (github-tools, merged in #262).

Pinned to MetaMask/github-tools@4a2ddd75. Runs weekdays at 10:00 and 15:00 UTC, with workflow_dispatch and a configurable lookback window. Uses SLACK_WEBHOOK_PLAYWRIGHT_HEALTH_REPORT.

Changelog

CHANGELOG entry: null

Related issues

Refs: Appium CI health reporting follow-up to #31730 / #31756

Manual testing steps

Feature: Playwright Appium test health report

  Scenario: PR trigger posts report to Slack
    Given SLACK_WEBHOOK_PLAYWRIGHT_HEALTH_REPORT_TEST was configured in repo secrets
    And this PR was open against main with a temporary pull_request trigger
    When the pull_request workflow ran on playwright-test-health-report.yml changes
    Then a Playwright Test Health Report (Mobile Appium) message appeared in Slack (see screenshot)

  Scenario: Scheduled report after merge to main
    Given the workflow exists on main with SLACK_WEBHOOK_PLAYWRIGHT_HEALTH_REPORT configured
    When the weekday schedule runs or an engineer triggers workflow_dispatch with lookback_days=1
    Then the report is generated from main ci.yml Playwright JSON artifacts

Screenshots/Recordings

Before

N/A — new workflow only.

After

Slack report from PR test run (1-day lookback on main Appium smoke):

Playwright Test Health Report Slack message

Pre-merge author checklist

Performance checks (if applicable)

  • I've tested on Android
    • Ideally on a mid-range device; emulator is acceptable
  • I've tested with a power user scenario
    • Use these power-user SRPs to import wallets with many accounts and tokens
  • I've instrumented key operations with Sentry traces for production performance metrics

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Made with Cursor

cmd-ob and others added 3 commits June 16, 2026 14:19
Wire up github-tools playwright-test-health-report to post flaky and
broken Appium results to Slack twice on weekdays, pinned to PR #262.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cmd-ob cmd-ob requested a review from a team as a code owner June 16, 2026 14:18
@github-actions github-actions Bot added the pr-not-ready-for-e2e Skip E2E and block merging. Remove this label once the PR is ready to run the E2E tests. label Jun 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@github-actions github-actions Bot added size-S risk:low AI analysis: low risk labels Jun 16, 2026
@cmd-ob cmd-ob removed the pr-not-ready-for-e2e Skip E2E and block merging. Remove this label once the PR is ready to run the E2E tests. label Jun 16, 2026
tommasini
tommasini previously approved these changes Jun 17, 2026
weitingsun
weitingsun previously approved these changes Jun 17, 2026
@cmd-ob cmd-ob dismissed stale reviews from weitingsun and tommasini via 04f3095 June 17, 2026 22:08
@cmd-ob cmd-ob requested a review from a team as a code owner June 17, 2026 22:08
@cmd-ob cmd-ob enabled auto-merge June 17, 2026 22:12
@cmd-ob cmd-ob added this pull request to the merge queue Jun 18, 2026
Merged via the queue into main with commit 4a36741 Jun 18, 2026
84 of 107 checks passed
@cmd-ob cmd-ob deleted the ci/playwright-test-health-report branch June 18, 2026 00:31
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 18, 2026
@metamaskbotv2 metamaskbotv2 Bot added the release-7.83.0 Issue or pull request that will be included in release 7.83.0 label Jun 18, 2026
@tommasini tommasini added release-8.0.0 Issue or pull request that will be included in release 8.0.0 and removed release-7.83.0 Issue or pull request that will be included in release 7.83.0 labels Jun 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

release-8.0.0 Issue or pull request that will be included in release 8.0.0 risk:low AI analysis: low risk size-S team-qa QA team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants