Skip to content

feat: benchmem compare --format md (GitHub-flavored markdown table)#147

Open
FBumann wants to merge 1 commit into
mainfrom
feat/compare-md-format
Open

feat: benchmem compare --format md (GitHub-flavored markdown table)#147
FBumann wants to merge 1 commit into
mainfrom
feat/compare-md-format

Conversation

@FBumann

@FBumann FBumann commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

What

Adds --format table|md to benchmem compare. md emits a native GitHub-flavored markdown table to stdout — the readable artifact to pipe into a PR comment or $GITHUB_STEP_SUMMARY, the natural companion to --fail-on / CodSpeed-style CI. Default stays the rich terminal table; --csv (raw values) is unchanged.

benchmem compare base.json head.json --columns time,peak --stat min --format md

pkg::test_x

name time (s) min peak (MiB) min
(base) 1 (1.0) 10.00 (1.0)
(head) 1.5 (1.50) 12.00 (1.20)

Design — deliberately un-opinionated

It's plain: values + the existing (N.NN) multiplier. We discussed colour/emoji and dropped both:

  • GFM strips inline CSS (<span style>), so terminal-style colour can't carry over on GitHub anyway.
  • Emoji (🔴/🟢) would render, but it's content in the cell — can't be stripped, varies by font, has accessibility cost, and bakes one aesthetic into everyone's comments.
  • The (N.NN) multiplier already encodes what colour did: (1.0) is the best, larger is worse, and it's magnitude not just rank. So plain markdown loses nothing essential.

(> out.txt already gives a clean monospace table for code fences; --format md adds the natively-rendered GitHub table. Both documented.)

DRY refactor

Split the per-group table model from the renderers so the two formats can't drift:

  • _build_views computes, once, each group's ordered rows + byte-scaled columns (_scale_columns) + best/worst.
  • _render_rich and _render_markdown differ only in emit; the value math (_cell_body, _mult, _row_label, byte_unit) is shared.
  • The rich path is byte-identical — all existing compare tests pass unchanged. md skips the timed│untimed divider (not a real column) and escapes | in cells.

Tests & docs

New tests: GFM table shape (heading / header / alignment / multiplier / no-ANSI-no-box-chars), pipe-escaping in a multi-id row cell, unknown-format error, and a CLI --format md test. Full suite green (291); ruff format+check clean; mypy clean. Docs gain a "Sharing the table" note covering --format md / > out.txt / --csv.

🤖 Generated with Claude Code

`compare` could only render the rich terminal table (or `--csv` raw values);
there was no readable artifact to drop into a PR comment / $GITHUB_STEP_SUMMARY,
the natural companion to --fail-on and CodSpeed-style CI.

Add `--format table|md`: `md` emits a native GFM table to stdout. It's plain by
design — values + the existing `(N.NN)` multiplier, which already carries best
(1.0) and magnitude — no colour (GFM strips inline styles) and no emoji (that
would bake an opinion into everyone's comments); the multiplier is the signal.

Refactored so both formats render one shared model: `_build_views` computes the
per-group rows + byte-scaled columns + best/worst once, and `_render_rich` /
`_render_markdown` only differ in emit. The fiddly value math (`_scale_columns`,
`_cell_body`, `_mult`, `_row_label`) is shared, so the formats can't drift; the
rich output is byte-identical (existing tests unchanged).

Docs note the three readable paths (--format md, > out.txt, --csv).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 pytest-benchmem | 🛠️ Build #33357458 | 📁 Comparing 5b778a9 against latest (d45506c)

  🔍 Preview build  

3 files changed
± changelog/index.html
± compare-plot/index.html
± reference/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant