Skip to content

feat(cli): --random-order + --seed to randomize test order #738

Description

@Chemaclass

Summary

Add --random-order to shuffle test execution order, plus --seed <n> (and BASHUNIT_SEED) to reproduce a given shuffle. Surfaces hidden inter-test coupling (tests that only pass because of ordering / shared state).

Motivation

Tests that pass only in declaration order hide real bugs (leaked globals, temp files, ordering deps). Randomizing order catches them; a printed seed makes a failure reproducible. Verified missing: grep -- --random-order src/*.sh → none.

Proposed API

bashunit --random-order tests/              # shuffle; prints the seed used
bashunit --random-order --seed 12345 tests/ # reproduce an exact shuffle
BASHUNIT_SEED=12345 bashunit --random-order tests/
  • Default: current deterministic order.
  • When --random-order is on and no seed given, generate one and print it in the run header so a failing run can be replayed.
  • Shuffle applies to the ordering of test functions (and/or files — pick file+function; document which).

Files

  • Flag/env parsing: src/globals.sh.
  • Test collection/order: src/runner.sh (where the test function list is built before execution).
  • Header print of seed: src/console_header.sh.
  • Docs: docs/command-line.md, docs/configuration.md.
  • Tests: tests/acceptance/bashunit_test.sh.

Implementation notes

  • Deterministic shuffle from a seed without Bash 4 features. A portable Fisher–Yates driven by a simple LCG seeded from --seed works on Bash 3.2; do not rely on $RANDOM alone for reproducibility unless you seed it and confirm portability. Keep the RNG in a small helper (e.g. src/math.sh) with its own unit test.

TDD steps

  1. RED (unit): shuffle helper with a fixed seed returns a deterministic permutation; two different seeds differ; same seed twice is identical.
  2. RED (acceptance): --random-order --seed X twice produces identical run order; header prints the seed.
  3. GREEN: wire shuffle into runner behind the flag.
  4. REFACTOR: confirm no change to default (non-random) ordering.

Acceptance criteria

  • --random-order, --seed <n>, BASHUNIT_SEED parsed.
  • Seed printed in header when shuffling.
  • Same seed → identical order (reproducible); default order unchanged when flag absent.
  • Shuffle helper unit-tested with fixed seeds.
  • Docs + CHANGELOG (Unreleased → Added) updated.

Constraints

  • Bash 3.0+ only; portable RNG (no Bash 4 features). See .claude/rules/bash-style.md.
  • Must pass make sa, make lint, shfmt -w ., ./bashunit tests/, ./bashunit --parallel tests/.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Fields

No fields configured for issues without a type.

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions