Skip to content

Add CodSpeed performance benchmarks and CI integration#2426

Open
codspeed-hq[bot] wants to merge 1 commit into
masterfrom
codspeed/wizard-1782078618443
Open

Add CodSpeed performance benchmarks and CI integration#2426
codspeed-hq[bot] wants to merge 1 commit into
masterfrom
codspeed/wizard-1782078618443

Conversation

@codspeed-hq

@codspeed-hq codspeed-hq Bot commented Jun 21, 2026

Copy link
Copy Markdown

Summary

This PR sets up continuous performance measurement with CodSpeed for returns.

It adds a benchmark suite for the core containers, wires up a GitHub Actions workflow to run the benchmarks on every push and pull request, and adds the CodSpeed badge to the README.

Changes

  • benchmarks/test_benchmarks.py — 9 pytest-codspeed benchmarks exercising the hot paths of the most commonly used parts of the library:
    • Result map/bind chains, failure recovery via .lash/.value_or, and the @safe decorator
    • Maybe map chains and Nothing short-circuiting
    • IO map chains
    • flow pipelines composed with point-free map_/bind
    • Fold.collect over an iterable of Result values
  • .github/workflows/codspeed.yml — workflow running benchmarks in three instruments:
    • CPU Simulation (instrumentation) on ubuntu-latest
    • Memory (memory) on ubuntu-latest
    • Wall time (walltime) on codspeed-macro runners for stable real-time measurements
    • Authentication uses OpenID Connect (id-token: write).
  • pyproject.toml / poetry.lock — added pytest-codspeed to the dev dependency group and a Ruff per-file-ignore for benchmarks/*.py (mirroring the existing tests/*.py rules so benchmark asserts pass linting).
  • README.md — added the CodSpeed badge.

Verification

The benchmarks were run locally before opening this PR:

  • pytest benchmarks/ --codspeed (walltime) — all 9 benchmarks pass
  • codspeed run --mode instrumentation — all 9 benchmarks pass and results upload to the dry-python/returns project

Next steps

  • Merge this PR so CodSpeed can collect a baseline from the master branch.
  • Subsequent pull requests will automatically receive performance reports comparing against that baseline.
  • Add more benchmarks over time to cover additional hot paths (e.g. Future/FutureResult, context containers) as needed.

@codspeed-hq

codspeed-hq Bot commented Jun 21, 2026

Copy link
Copy Markdown
Author

Congrats! CodSpeed is installed 🎉

🆕 18 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks


Open in CodSpeed

@codspeed-hq codspeed-hq Bot marked this pull request as ready for review June 21, 2026 21:55
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.12'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to have the report for different python versions, idk

wdyt @sobolevn?

@codspeed-hq

codspeed-hq Bot commented Jun 21, 2026

Copy link
Copy Markdown
Author

Good question. My recommendation is to keep a single, pinned Python version (the current 3.12) for the CodSpeed job rather than fanning out across multiple versions.

The reasoning:

  • CodSpeed detects regressions in your code over time, by comparing a benchmark against its own history on a consistent environment. It's not designed to compare interpreter versions against each other.
  • Adding a Python-version axis multiplies every benchmark (here, across all three modes) and each version becomes a separate baseline. That's more CI time and more noise without adding regression-detection signal — a change that regresses on 3.11 will almost always regress on 3.12 too.
  • Consistency is what keeps measurements reproducible. Pinning one version avoids cross-version variance masking real changes.

A more common pattern is to keep functional correctness tests (your existing test workflow) running on the full version matrix, and let CodSpeed track performance on one representative version.

If you do specifically want to track per-version performance (e.g. you're tuning for a particular interpreter), it's doable by nesting python-version into the matrix — happy to wire that up. But I'd lean toward the single-version setup as the default. Let me know how you'd like to proceed.

Summary
  • Warming up the performance wizard...
  • Assess multi-version benchmark request — Reviewed the CodSpeed workflow and the question about benchmarking across multiple Python versions.
  • Handling comment — Comment handling finished successfully

Was this response helpful? React with 👍 or 👎 to help us improve CodSpeedBot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants