You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add a --retry <n> flag (and BASHUNIT_RETRY env var) that re-runs each failed test up to n extra times before marking it failed. Mitigates flaky tests in CI without hiding real failures.
Motivation
Flaky tests (timing, network, filesystem races) fail intermittently and poison CI. A bounded retry lets a genuinely-flaky test recover while a consistently-broken test still fails after n attempts. --stop-on-failure already exists but does the opposite; there is no retry today (verified: grep -- --retry src/*.sh → none).
Proposed API
bashunit --retry 2 tests/ # each failed test retried up to 2 more times (3 total)
BASHUNIT_RETRY=2 bashunit tests/ # env equivalent
Default: 0 (current behaviour, no retry).
A test is reported passed if any attempt passes; failed only if all attempts fail.
Output should note when a test passed only on retry (e.g. PASSED (retry 2/2)), so flakiness stays visible.
Retries apply per-test, and must work with --parallel.
Files
Flag parsing + env: src/globals.sh, src/console_header.sh/src/helpers.sh (follow how --test-timeout / BASHUNIT_TEST_TIMEOUT is wired — added in Test timeouts #721).
Docs: docs/command-line.md (new flag section) + docs/configuration.md (env var).
Tests: tests/acceptance/bashunit_test.sh (CLI end-to-end), tests/unit/runner_test.sh if unit-testable.
TDD steps
RED: acceptance test — a fixture test that fails on attempt 1 and passes on attempt 2 (use a temp-file counter) is reported PASSED with --retry 1; the same test with --retry 0 is reported FAILED.
RED: a test that always fails is still FAILED after --retry 2 and shows 3 attempts.
GREEN: minimal retry loop around the per-test run.
Summary
Add a
--retry <n>flag (andBASHUNIT_RETRYenv var) that re-runs each failed test up tonextra times before marking it failed. Mitigates flaky tests in CI without hiding real failures.Motivation
Flaky tests (timing, network, filesystem races) fail intermittently and poison CI. A bounded retry lets a genuinely-flaky test recover while a consistently-broken test still fails after
nattempts.--stop-on-failurealready exists but does the opposite; there is no retry today (verified:grep -- --retry src/*.sh→ none).Proposed API
0(current behaviour, no retry).PASSED (retry 2/2)), so flakiness stays visible.--parallel.Files
src/globals.sh,src/console_header.sh/src/helpers.sh(follow how--test-timeout/BASHUNIT_TEST_TIMEOUTis wired — added in Test timeouts #721).src/runner.sh(per-test run/report path).docs/command-line.md(new flag section) +docs/configuration.md(env var).tests/acceptance/bashunit_test.sh(CLI end-to-end),tests/unit/runner_test.shif unit-testable.TDD steps
--retry 1; the same test with--retry 0is reported FAILED.--retry 2and shows 3 attempts../bashunit --parallel --retry 1 tests/.Acceptance criteria
--retry <n>andBASHUNIT_RETRYparsed; default0.ntimes; passes if any attempt passes.--parallel.Constraints
declare -A, no[[ ]], no${var,,}). See.claude/rules/bash-style.md.make sa,make lint,shfmt -w ., and both./bashunit tests/and./bashunit --parallel tests/.