Skip to content

Wire fix/e2e-tests changes into the all-flows CI runner #148

@ElioNeto

Description

@ElioNeto

Issue imported from tinyhumansai/openhuman#1865
Created at: unknown


Summary

app/scripts/e2e-run-all-flows.sh runs each spec in a separate Appium session (Appium restarts per spec). With the single-Appium-session reset model introduced in tinyhumansai#1859, the all-flows runner is now duplicating session setup ~17 times per CI run. It should switch to one session for the whole suite, matching the design called out in app/test/wdio.conf.ts.

Problem

Current e2e-run-all-flows.sh:

run "test/e2e/specs/login-flow.spec.ts" "login"
run "test/e2e/specs/auth-access-control.spec.ts" "auth"
run "test/e2e/specs/telegram-flow.spec.ts" "telegram"
# … 17 specs, each via e2e-run-spec.sh → spins up Appium, mock server, CEF binary

Each run invocation calls e2e-run-spec.sh which calls e2e-run-session.sh which:

  1. Cleans a temp workspace.
  2. Launches CEF.
  3. Waits for CDP.
  4. Starts Appium.
  5. Runs WDIO on the one spec.
  6. Tears it all down.

That's ~30-60s of harness setup per spec × 17 specs = ~10 minutes of pure overhead per CI run.

Now that resetApp(<userId>) produces a fresh-install baseline in-place, all specs can share one session — just run wdio once over the whole specs/**/*.spec.ts glob (which wdio.conf.ts already targets).

Solution

  1. Replace e2e-run-all-flows.sh with a single-session invocation: build once, launch CEF once, start Appium once, run wdio once over the full spec glob.
  2. Ensure spec independence — each spec must call resetApp(...) in before() with a unique userId. Audit + add as part of the spec-conversion sweep (Convert RPC-heavy E2E specs to drive the UI past login tinyhumansai/openhuman#1860).
  3. Per-spec failure isolation — WDIO's mocha.bail = 0 is already set; one failing spec must not stop the suite. Verify via a deliberately-failing canary spec.
  4. Artifact collectione2e/helpers/artifacts.ts already captures per-test failures; confirm they don't collide when many specs run back to back.
  5. CI workflow — update whichever GitHub Actions job invokes the runner so it uses the new single-session script.

Acceptance criteria

  • Single-session runnere2e-run-all-flows.sh (or a new e2e-run-suite.sh) runs all specs in one Appium session.
  • Suite runtime under 10min on CI — measured before/after; expect a 5-10× speedup for the harness-setup portion.
  • Spec independence enforcedresetApp(...) in every spec's before(); covered by a lint or grep CI check.
  • Per-spec failures are isolated — failing canary spec exits non-zero without killing later specs in the same session.
  • Artifact directories don't collide — each failure screenshot/source XML lives under its own per-run, per-test path.
  • Diff coverage ≥ 80% — shell script changes, exercised by running the runner in CI.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions