test: DRY up the suite with shared builders + parametrized tables#153
Merged
Conversation
Every test module reimplements the same three fixture shapes with slightly
different names: writing a {"benchmarks": [...]} run file, building a benchmark
entry, and building a benchmem memory blob. Add tests/_builders.py with
write_run/bm/blob so the shapes are defined once; the per-module migrations
follow in subsequent commits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fold four near-identical 'bad kwarg -> ValueError' tests (columns/group_by/ sort/stat) into one parametrized case over (kwargs, match). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The plot CLI tests each re-declared the same monkeypatch stub — either capturing the kwargs threaded into one plot_* view, or recording which view the CLI picks. Two fixtures replace ~9 hand-rolled copies. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documentation build overview
|
Fold the six axis-scale tests, the four band-whisker tests, and the two where-filter rejection tests into three parametrized cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ilies Fold three families of near-identical CLI tests into parametrized tables: the --fail-on gate outcomes, the plot bad-flag usage errors, and the flamegraph usage errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the ~20 repetitions of StringIO()/compare_runs(out=)/getvalue() with a render(runs, **kwargs) -> str helper, so each table-content test is one line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The axis-scale table had a y_tozero column that was None for half its rows, forcing an 'if y_tozero is not None' guard. Keep the table for the uniform x/y log-type matrix; move the zero-anchor behaviour to its own focused test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapsing these into one table needed a str|tuple|None 'needle' column and isinstance branching in the body — the cases aren't uniform, so the named one-liners read better. Revert that one collapse (the homogeneous compare-gate and flamegraph tables stay). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two passes over the test suite to make it DRYer and more scannable.
Pass 1 — shared fixture builders
Every module reimplemented the same synthetic pytest-benchmark shapes under different names.
tests/_builders.pynow owns three primitives:write_run(path, benchmarks)— write a{"benchmarks": [...]}run file.bm(name, *, t, peak, allocations, total_bytes, rss, params)— one benchmark entry.blob(peak, *, allocations, total_bytes, rss)— a benchmem per-repeat series.Every module migrated to them (one commit each). Scenario-specific builders stay local.
Pass 2 — parametrized tables + assertion helpers
Collapsed the clone-families (each was N near-identical functions) into parametrized tables, and added output helpers:
test_plotting.py: axis-scale (6→1), band-whisker (4→1), where-filter (2→1) tables.test_cli.py:--fail-ongate (4→1), plot bad-flag (5→1), flamegraph usage (4→1) tables; pluscapture_view/view_callsfixtures replacing ~9 hand-rolled monkeypatch stubs.test_compare.py:render(runs, **kwargs) -> strhelper replacing ~20StringIO()/compare_runs(out=)/getvalue()triads; unknown-option rejection table (4→1).Result
main.Not included
Grouping
test_cli.pyinto classes — the# --- section ---dividers already give grouping and classes would add churn without functional gain. Easy follow-up if wanted.🤖 Generated with Claude Code