Conversation
Contributor
There was a problem hiding this comment.
Benchmark Update Review
This is a routine automated benchmark update from 2026-03-24, bumping TUnit from v1.21.0 to v1.21.6. The changes are purely documentation/data — no source code is affected.
Performance Highlights (v1.21.0 → v1.21.6)
Wins:
- MassiveParallelTests: TUnit improved from 658.0ms → 625.3ms (~5% faster); AOT: 228.9ms → 227.0ms
- BuildTime: TUnit improved from 2.621s → 2.580s (~1.6% faster)
- ScaleTests: TUnit improved from 610.8ms → 600.2ms (~1.7% faster)
Regressions to note:
- SetupTeardownTests: TUnit regressed from 489.33ms → 510.39ms (~4.3% slower). Competitors also slowed, but TUnit dropped from faster than MSTest to slower than it — worth keeping an eye on across future runs.
- AsyncTests: Marginal regression (537.6ms → 538.8ms, within noise), but competitors improved significantly, so TUnit's relative advantage narrowed.
- DataDrivenTests: Effectively flat (472.94ms → 474.05ms), within normal run-to-run variance.
Data Integrity
- JSON data files, Markdown docs, and chart axis scales all appear consistently updated.
- The historical.json and summary.json updates look correct.
- No structural changes to the benchmark format; this is a clean data-only update.
No Issues Found
The PR is well-structured as an automated update and the data appears consistent across all files. The SetupTeardownTests regression is worth a follow-up investigation but is not a blocker for merging documentation updates.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated Benchmark Update
This PR updates the benchmark documentation with the latest results from the Speed Comparison workflow.
Benchmarks Produced
Individual benchmark artifacts are available for download:
benchmark-DataDrivenTestsbenchmark-AsyncTestsbenchmark-ScaleTestsbenchmark-MatrixTestsbenchmark-MassiveParallelTestsbenchmark-SetupTeardownTestsbenchmark-BuildTimebenchmark-summary(aggregated overview)Runtime Benchmarks
Build Benchmarks
Changes
docs/static/benchmarks/latest.jsondocs/static/benchmarks/historical.jsondocs/docs/benchmarks/index.mddocs/static/benchmarks/summary.jsonWorkflow Run
🤖 This PR was automatically created and will be merged automatically once CI checks pass.