Skip to content

test: DRY up the suite with shared builders + parametrized tables#153

Merged
FBumann merged 18 commits into
mainfrom
refactor/test-shared-builders
Jul 1, 2026
Merged

test: DRY up the suite with shared builders + parametrized tables#153
FBumann merged 18 commits into
mainfrom
refactor/test-shared-builders

Conversation

@FBumann

@FBumann FBumann commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

What

Two passes over the test suite to make it DRYer and more scannable.

Pass 1 — shared fixture builders

Every module reimplemented the same synthetic pytest-benchmark shapes under different names. tests/_builders.py now owns three primitives:

  • write_run(path, benchmarks) — write a {"benchmarks": [...]} run file.
  • bm(name, *, t, peak, allocations, total_bytes, rss, params) — one benchmark entry.
  • blob(peak, *, allocations, total_bytes, rss) — a benchmem per-repeat series.

Every module migrated to them (one commit each). Scenario-specific builders stay local.

Pass 2 — parametrized tables + assertion helpers

Collapsed the clone-families (each was N near-identical functions) into parametrized tables, and added output helpers:

  • test_plotting.py: axis-scale (6→1), band-whisker (4→1), where-filter (2→1) tables.
  • test_cli.py: --fail-on gate (4→1), plot bad-flag (5→1), flamegraph usage (4→1) tables; plus capture_view/view_calls fixtures replacing ~9 hand-rolled monkeypatch stubs.
  • test_compare.py: render(runs, **kwargs) -> str helper replacing ~20 StringIO()/compare_runs(out=)/getvalue() triads; unknown-option rejection table (4→1).

Result

  • -183 lines across the test files (462 insertions, 645 deletions) plus the ~70-line builders module.
  • 284 → 260 test functions, but the collapsed ones became parametrized cases, so coverage is unchanged: 309 passed, identical to main.
  • Each commit is scoped to one file / one idea, and pre-commit (ruff + mypy) is green on every commit.

Not included

Grouping test_cli.py into classes — the # --- section --- dividers already give grouping and classes would add churn without functional gain. Easy follow-up if wanted.

🤖 Generated with Claude Code

FBumann and others added 13 commits July 1, 2026 10:32
Every test module reimplements the same three fixture shapes with slightly
different names: writing a {"benchmarks": [...]} run file, building a benchmark
entry, and building a benchmem memory blob. Add tests/_builders.py with
write_run/bm/blob so the shapes are defined once; the per-module migrations
follow in subsequent commits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fold four near-identical 'bad kwarg -> ValueError' tests (columns/group_by/
sort/stat) into one parametrized case over (kwargs, match).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The plot CLI tests each re-declared the same monkeypatch stub — either
capturing the kwargs threaded into one plot_* view, or recording which view
the CLI picks. Two fixtures replace ~9 hand-rolled copies.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@read-the-docs-community

read-the-docs-community Bot commented Jul 1, 2026

Copy link
Copy Markdown

Documentation build overview

📚 pytest-benchmem | 🛠️ Build #33391293 | 📁 Comparing c918e5b against latest (3da9bb1)

  🔍 Preview build  

2 files changed
± changelog/index.html
± compare-plot/index.html

FBumann and others added 3 commits July 1, 2026 11:40
Fold the six axis-scale tests, the four band-whisker tests, and the two
where-filter rejection tests into three parametrized cases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ilies

Fold three families of near-identical CLI tests into parametrized tables:
the --fail-on gate outcomes, the plot bad-flag usage errors, and the
flamegraph usage errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the ~20 repetitions of StringIO()/compare_runs(out=)/getvalue() with
a render(runs, **kwargs) -> str helper, so each table-content test is one line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@FBumann FBumann changed the title test: DRY up the suite with shared fixture builders test: DRY up the suite with shared builders + parametrized tables Jul 1, 2026
FBumann and others added 2 commits July 1, 2026 12:43
The axis-scale table had a y_tozero column that was None for half its rows,
forcing an 'if y_tozero is not None' guard. Keep the table for the uniform
x/y log-type matrix; move the zero-anchor behaviour to its own focused test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapsing these into one table needed a str|tuple|None 'needle' column and
isinstance branching in the body — the cases aren't uniform, so the named
one-liners read better. Revert that one collapse (the homogeneous compare-gate
and flamegraph tables stay).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@FBumann FBumann enabled auto-merge (squash) July 1, 2026 10:44
@FBumann FBumann merged commit 5e359c9 into main Jul 1, 2026
12 checks passed
@FBumann FBumann deleted the refactor/test-shared-builders branch July 1, 2026 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant