bench: feature-on baseline for raw_json (keeps field stripping)#269
Draft
stormslowly wants to merge 1 commit into
Draft
bench: feature-on baseline for raw_json (keeps field stripping)#269stormslowly wants to merge 1 commit into
stormslowly wants to merge 1 commit into
Conversation
The benchmark built without the feature, so the raw_json()/init_serde_json code path was never compiled and CodSpeed could not measure changes to it. Enable the feature in the build step so the package.json parsing benchmark exercises the serde_json materialization path.
Merging this PR will degrade performance by 16.16%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Memory | resolver[single-thread] |
8.3 MB | 12.8 MB | -35.29% |
| ❌ | Memory | resolver[[single-threaded]resolve with many extensions] |
13.4 MB | 17.9 MB | -25.25% |
| ❌ | Simulation | resolver[single-thread] |
51.3 ms | 62 ms | -17.34% |
| ❌ | Simulation | resolver[[multi-threaded]resolve] |
60.4 ms | 71.3 ms | -15.3% |
| ❌ | Memory | resolver[tsconfig resolve] |
27.9 KB | 31 KB | -10.04% |
| ❌ | Simulation | resolver[[single-threaded]resolve with many extensions] |
126.6 ms | 137.6 ms | -8.02% |
| ⚡ | Memory | resolver[pnp resolve] |
9.1 KB | 8.8 KB | +3.92% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing bench/raw-json-feature-on-baseline (b9e212a) with main (cb42e9a)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Baseline for measuring the cost of
PackageJson::raw_json(). The benchmark previously built withoutpackage_json_raw_json_api, so theinit_serde_jsonmaterialization path was never compiled and CodSpeed could not see it.This PR only enables the feature in the benchmark build. Source behavior is unchanged —
JSONCell::try_newstill stripsdescription/keywords/scripts/dependencies/… (currentmainbehavior).So this PR's CodSpeed numbers are the feature-on + strip baseline.
What
benchmark.yml:cargo codspeed build→cargo codspeed build --features package_json_raw_json_apiCompanion PR
The change PR #268 is stacked on this branch and removes the stripping. Its CodSpeed report compares directly against this baseline, so reviewers see the keep-vs-strip delta in one table.
This pair is for measurement/review only — not intended to merge as-is.