experiment(pm): swap DiskManifestStore for NoopStore (measurement-only)

[elrrrrrrr]

Side experiment branched off perf/p1-resolve-concurrency at 01d1513.

Question under test: how much of utoo's p1/p3 wall comes from the per-fetch disk-cache existence-check IO that service::registry::UnifiedRegistry issues alongside each manifest fetch.

Swaps Context::manifest_store from DiskManifestStore to NoopStore. NOT for landing — measurement-only branch. Compare against #2916 to read the disk-cache IO cost.

[gemini-code-assist]

Code Review

This pull request introduces performance optimizations and benchmarking tools for manifest preloading, including new benchmark crates and a lean mb_fetch path for lockfile-only resolutions. It optimizes JSON parsing by moving it to tokio::task::spawn_blocking and adds comprehensive timing instrumentation. Reviewers identified several bugs in the new preloading logic, such as potential leaks of sibling specs during fetch failures or cache misses, and misleading success reporting for failed HTTP requests. Additionally, feedback pointed out redundant cache writes and missing client configurations, such as timeouts and TLS settings, in the standalone fetcher.

[gemini-code-assist]

There is a potential leak of sibling specs. If out.fetched is true but the body_cache lookup fails (e.g. due to a race or an earlier error), deferred_by_name.remove() still executes, consuming the siblings. Since they are neither processed nor put back, they are lost. The removal should be decoupled from the processing logic to ensure they are always cleared from the map even on failure.

Suggested change

	if out.fetched
	&& let Some(siblings) = deferred_by_name.remove(&out.name)
	&& let Some(raw) = body_cache.lock().get(&out.name).cloned()
	{
	for sibling_spec in siblings {
	futs.push(spawn_settle(
	out.name.clone(),
	sibling_spec,
	Arc::clone(&raw),
	cache.clone(),
	peer_deps,
	));
	}
	}
	// Drain sibling specs deferred while the fetch was in flight.
	if let Some(siblings) = deferred_by_name.remove(&out.name) {
	if out.fetched {
	if let Some(raw) = body_cache.lock().get(&out.name).cloned() {
	for sibling_spec in siblings {
	futs.push(spawn_settle(
	out.name.clone(),
	sibling_spec,
	Arc::clone(&raw),
	cache.clone(),
	peer_deps,
	));
	}
	}
	}
	}

[gemini-code-assist]

Sibling specs are leaked if the primary fetch fails. The deferred_by_name.remove(&name) call is inside the FetchOutcome::Ok branch. If the outcome is Err, the siblings remain in the map forever, and since no more futures will be spawned for that name, they are never processed.

Suggested change

	if let Some(siblings) = deferred_by_name.remove(&name) {
	for sibling_spec in siblings {
	futs.push(Box::pin(settle_future(
	name.clone(),
	sibling_spec,
	Arc::clone(&full_arc),
	cache.clone(),
	peer_deps,
	)));
	}
	}
	// Sibling specs that were stashed while the
	// fetch was in flight: dispatch each as a
	// separate settle future.
	let siblings = deferred_by_name.remove(&name);
	if let Some(siblings) = siblings {
	for sibling_spec in siblings {
	futs.push(Box::pin(settle_future(
	name.clone(),
	sibling_spec,
	Arc::clone(&full_arc),
	cache.clone(),
	peer_deps,
	)));
	}
	}

[gemini-code-assist]

The reqwest::Client in build_mb_client is missing important configurations that are present in the main ruborist HTTP client and the other benchmark tools in this PR. Specifically, it lacks timeouts and TLS root certificate configuration. Without these, the benchmark might hang indefinitely on network issues or fail to connect to registries using custom CAs.

Suggested change

	fn build_mb_client() -> Result<reqwest::Client> {
	reqwest::Client::builder()
	.no_proxy()
	.pool_max_idle_per_host(256)
	.http1_only()
	.build()
	.context("build reqwest client for mb_resolve")
	}
	fn build_mb_client() -> Result<reqwest::Client> {
	reqwest::Client::builder()
	.no_proxy()
	.pool_max_idle_per_host(256)
	.http1_only()
	.connect_timeout(std::time::Duration::from_secs(10))
	.timeout(std::time::Duration::from_secs(30))
	.build()
	.context("build reqwest client for mb_resolve")
	}

[gemini-code-assist]

Returning fetched: true on an HTTP error status is misleading for instrumentation. This causes the main loop to count failures as successes.

            _ => {
                return FetchOutcome {
                    name,
                    transitives: Vec::new(),
                    fetched: false,
                };
            }

[gemini-code-assist]

Returning fetched: true when failing to read the response body is misleading. This should be marked as a failed fetch.

            Err(_) => {
                return FetchOutcome {
                    name,
                    transitives: Vec::new(),
                    fetched: false,
                };
            }

[gemini-code-assist]

The code contradicts the comment. It overwrites the full_manifest in the cache even though the comment states it shouldn't. This is redundant since the original fetcher already populated this slot.

            Some((_full_arc, resolved, core_arc, transitives)) => {
                // Don't overwrite full_manifest — the original fetcher
                // already set it. Only populate the version-manifest
                // slots so BFS hits the (name, spec) early-return.
                cache.set_version_manifest(name.clone(), spec, Arc::clone(&core_arc));
                cache.set_version_manifest(name.clone(), resolved, core_arc);
                transitives
            }

[gemini-code-assist]

The logic here is redundant as both branches of the if/else if perform the same action. Additionally, it fails to utilize stats.fail because errors are currently reported as fetched: true in spawn_fetch.

        if out.fetched {
            stats.success += 1;
        } else {
            stats.fail += 1;
        }

[github-actions]

📊 pm-bench-phases · c83d1b4 · linux (ubuntu-latest)

Workflow run — ant-design

PMs: utoo (this branch) · utoo-npm (latest published) · bun (latest)

npmjs.org

p0_full_cold

PM	wall	±σ	user	sys	RSS	pgMinor
bun	8.78s	0.16s	9.98s	9.64s	761M	329.6K
utoo-next	8.04s	0.08s	10.14s	11.58s	949M	119.1K
utoo-npm	9.85s	2.55s	10.57s	11.99s	1.36G	176.6K
utoo	7.82s	0.47s	10.17s	11.18s	1.35G	187.2K

PM	vCtx	iCtx	netRX	netTX	cache	node_mod	lock
bun	15.1K	17.5K	1.17G	6M	1.84G	1.72G	1M
utoo-next	122.9K	80.1K	1.14G	5M	1.68G	1.68G	2M
utoo-npm	131.5K	88.6K	1.14G	5M	1.68G	1.68G	2M
utoo	102.4K	91.3K	1.14G	5M	1.68G	1.68G	2M

p1_resolve

PM	wall	±σ	user	sys	RSS	pgMinor
bun	1.90s	0.09s	3.87s	1.05s	490M	170.6K
utoo-next	5.07s	1.85s	5.05s	2.02s	609M	82.0K
utoo-npm	2.90s	0.05s	5.07s	1.95s	600M	79.9K
utoo	2.77s	0.12s	4.90s	1.16s	1.05G	153.3K

PM	vCtx	iCtx	netRX	netTX	cache	node_mod	lock
bun	8.0K	4.6K	202M	3M	105M	-	1M
utoo-next	70.5K	109.3K	199M	2M	7M	3M	2M
utoo-npm	69.4K	112.2K	199M	2M	7M	3M	2M
utoo	43.9K	7.4K	199M	3M	-	3M	2M

p3_cold_install

PM	wall	±σ	user	sys	RSS	pgMinor
bun	6.80s	0.14s	6.14s	9.35s	608M	201.5K
utoo-next	5.82s	0.08s	4.88s	10.10s	449M	56.5K
utoo-npm	6.03s	0.23s	5.28s	10.55s	887M	112.2K
utoo	7.22s	2.94s	4.99s	10.20s	639M	76.0K

PM	vCtx	iCtx	netRX	netTX	cache	node_mod	lock
bun	4.7K	6.4K	995M	3M	1.74G	1.74G	1M
utoo-next	94.0K	44.2K	964M	2M	1.67G	1.67G	2M
utoo-npm	102.5K	61.3K	964M	2M	1.67G	1.67G	2M
utoo	101.2K	70.9K	964M	3M	1.67G	1.67G	2M

p4_warm_link

PM	wall	±σ	user	sys	RSS	pgMinor
bun	3.49s	0.06s	0.17s	2.46s	137M	31.6K
utoo-next	2.26s	0.19s	0.47s	3.73s	79M	18.1K
utoo-npm	2.22s	0.06s	0.52s	3.77s	85M	19.7K
utoo	2.08s	0.04s	0.50s	3.72s	80M	18.2K

PM	vCtx	iCtx	netRX	netTX	cache	node_mod	lock
bun	236	21	3K	6K	1.84G	1.74G	1M
utoo-next	42.4K	18.8K	307K	10K	1.68G	1.68G	2M
utoo-npm	46.8K	20.3K	324K	16K	1.68G	1.68G	2M
utoo	42.7K	20.5K	309K	13K	1.68G	1.68G	2M

npmmirror.com: no output captured.