feat(record): per-test retries + scene-failure hint + slow-scene warning#13
Merged
Conversation
…ing (v1) First half of the long-demo flake mitigation roadmap. Pure additive — no behavior change unless callers opt in. * `video.retries: N` config + `--retries N` CLI flag — embeds Playwright's built-in `retries` in the generated playwright.record.config.mjs so a transient failure (browser hiccup, race in showOverlay) gets a free retry without manual intervention. * When the recording subprocess fails, append the last scene name from `.scene-progress.jsonl` to the error so users can map the failure to a scene without reading the Playwright stack trace. * After a successful pipeline, surface scenes whose duration is more than 1.75× the median as a console warning. Useful intermittent-flake signal — a scene that drifts from 6s → 18s on retries is the most likely culprit. Doesn't fix the "10-minute demo flakes on scene 9, re-records all 10" case — that's v2 (scene-isolated recording + checkpoints), tracked separately. v1 ships first because it's universal and zero-configuration for existing demos. 3 new tests covering the retries embedding (default 0, explicit value, clamp/floor of fractional values).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First half of the long-demo flake mitigation roadmap. Pure additive — no behavior change unless callers opt in.
video.retries: Nconfig +--retries NCLI flag — embeds Playwright's built-inretriesin the generatedplaywright.record.config.mjsso a transient failure (browser hiccup, race inshowOverlay, intermittent network) gets a free retry without manual intervention..scene-progress.jsonlso users can map the failure to a scene without reading the Playwright stack trace.Scope
This is v1 of the long-demo flake roadmap. v2 (scene-isolated recording with state checkpoints +
--resume-from) is deliberately deferred — it's ~5× the scope and the design will be better informed by what failure modes v1's diagnostics surface in production. v1 alone catches a lot of transient failures via the Playwright retry plumbing.Test plan
npm test— 639 tests pass (3 new tests covering retries embedding: default 0, explicit value, clamp/floor of fractional values)npx argo pipeline ci-smoke --retries 1runs cleanly; slow-scene warning fires correctly on the trailing-wait sceneretries(default remains 0, current behavior)