Skip to content

bench: feature-on baseline for raw_json (keeps field stripping)#269

Draft
stormslowly wants to merge 1 commit into
mainfrom
bench/raw-json-feature-on-baseline
Draft

bench: feature-on baseline for raw_json (keeps field stripping)#269
stormslowly wants to merge 1 commit into
mainfrom
bench/raw-json-feature-on-baseline

Conversation

@stormslowly
Copy link
Copy Markdown
Collaborator

Why

Baseline for measuring the cost of PackageJson::raw_json(). The benchmark previously built without package_json_raw_json_api, so the init_serde_json materialization path was never compiled and CodSpeed could not see it.

This PR only enables the feature in the benchmark build. Source behavior is unchangedJSONCell::try_new still strips description/keywords/scripts/dependencies/… (current main behavior).

So this PR's CodSpeed numbers are the feature-on + strip baseline.

What

  • benchmark.yml: cargo codspeed buildcargo codspeed build --features package_json_raw_json_api
  • No source change.

Companion PR

The change PR #268 is stacked on this branch and removes the stripping. Its CodSpeed report compares directly against this baseline, so reviewers see the keep-vs-strip delta in one table.

This pair is for measurement/review only — not intended to merge as-is.

The benchmark built without the feature, so the raw_json()/init_serde_json
code path was never compiled and CodSpeed could not measure changes to it.
Enable the feature in the build step so the package.json parsing benchmark
exercises the serde_json materialization path.
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Jun 4, 2026

Merging this PR will degrade performance by 16.16%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
❌ 6 regressed benchmarks
✅ 5 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Memory resolver[single-thread] 8.3 MB 12.8 MB -35.29%
Memory resolver[[single-threaded]resolve with many extensions] 13.4 MB 17.9 MB -25.25%
Simulation resolver[single-thread] 51.3 ms 62 ms -17.34%
Simulation resolver[[multi-threaded]resolve] 60.4 ms 71.3 ms -15.3%
Memory resolver[tsconfig resolve] 27.9 KB 31 KB -10.04%
Simulation resolver[[single-threaded]resolve with many extensions] 126.6 ms 137.6 ms -8.02%
Memory resolver[pnp resolve] 9.1 KB 8.8 KB +3.92%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing bench/raw-json-feature-on-baseline (b9e212a) with main (cb42e9a)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant