Skip to content

fix(bench-arena): measure memory churn by allocation sampler, not an uncollected snapshot#407

Merged
ozzyfromspace merged 2 commits into
mainfrom
fix/bench-churn-allocation-sampler
Jun 14, 2026
Merged

fix(bench-arena): measure memory churn by allocation sampler, not an uncollected snapshot#407
ozzyfromspace merged 2 commits into
mainfrom
fix/bench-churn-allocation-sampler

Conversation

@ozzyfromspace

Copy link
Copy Markdown
Contributor

What

The benchmark's memory churn metric read an uncollected heap snapshot, so it measured V8's
GC scheduling rather than allocation. This measures it with the CDP allocation sampler instead.
retained and leak are unchanged.

The bug

In the memory dimension, each cycle reads heap snapshots to derive three figures. retained and
leak run a double-GC before their snapshots; churn did not, reading usedSize immediately after
the keystroke burst (s2 - s1). So churn captured whatever transient garbage V8 had not swept yet at
that instant.

That is GC-scheduling noise, not a memory cost. On a single grid cell, churn could read tens of MB
uncollected
, yet collecting before the snapshot showed the burst truly retains under 100KB (and
retained / leak, which already collect first, confirmed the live cost is parity). Same burst, same
real memory, orders of magnitude apart on a snapshot-timing accident. It quietly skewed the churn
column for the whole cohort.

The fix

Wrap the burst in HeapProfiler.startSampling / stopSampling and sum the sampled allocation. The
sampler records allocations as they happen, independent of when they are collected, so churn now
reflects the bytes a typing burst allocates, GC-timing-independent. retained and leak are untouched
(they already collect before every snapshot).

This was the only metric with the issue: it was the lone uncollected snapshot, and the timed
dimensions measure elapsed time, not heap.

Validation

A local grid N100M8 memory run across the full cohort now reports stable, sane churn proportional to
each library's typing allocation (roughly 50 to 330KB), with no snapshot-timing spikes. Attaform's
churn reads about 327KB, down from one to two MB of uncollected noise before. Bench typecheck
(vue-tsc), eslint, and prettier all pass.

Sequencing

Changing how churn is measured re-bases every library's churn in results.json. The committed numbers
update on the next CI bench refresh (the monthly sharded workflow); the docs render the current numbers
until then.

🤖 Generated with Claude Code

…uncollected snapshot

The memory dimension read churn as the post-burst usedSize delta with no GC before the
snapshot, unlike retained and leak which both collect first. So churn measured whatever
transient garbage V8 had not yet swept at that instant: GC scheduling, not allocation. A
cell could read tens of MB uncollected while the burst's truly-retained memory was under
100KB (collect first and it is), so the figure swung on snapshot timing, not on cost.

Wrap the burst in the CDP allocation sampler (HeapProfiler.startSampling / stopSampling)
and sum the sampled bytes instead. The sampler records allocations as they happen,
independent of when they are collected, so churn reflects what a typing burst allocates,
GC-timing-independent. retained and leak are untouched (they already collect before every
snapshot). Re-bases every library's churn in results.json on the next CI bench refresh.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
attaform Ready Ready Preview, Comment Jun 14, 2026 1:11am

@ozzyfromspace ozzyfromspace merged commit 50523d2 into main Jun 14, 2026
15 checks passed
@ozzyfromspace ozzyfromspace deleted the fix/bench-churn-allocation-sampler branch June 14, 2026 01:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant