diff --git a/README.md b/README.md index b692f077..9e78f852 100644 --- a/README.md +++ b/README.md @@ -592,14 +592,15 @@ Self-measured on every release via CI ([build benchmarks](generated/benchmarks/B | Metric | Latest | |---|---| -| Build speed (native) | **13.3 ms/file** | -| Build speed (WASM) | **13.6 ms/file** | -| Query time | **29ms** | +| Build speed (native) | **12.8 ms/file** | +| Build speed (WASM) | **13.1 ms/file** | +| Query time (native) | **30ms** | +| Query time (WASM) | **31ms** | | No-op rebuild (native) | **8ms** | -| 1-file rebuild (native) | **42ms** | -| Query: fn-deps | **2.2ms** | +| 1-file rebuild (native) | **562ms** | +| Query: fn-deps | **2.1ms** | | Query: path | **2.2ms** | -| ~50,000 files (est.) | **~665.0s build** | +| ~50,000 files (est.) | **~640.0s build** | | Resolution precision | **100.0%** | | Resolution recall | **64.5%** | diff --git a/generated/benchmarks/BUILD-BENCHMARKS.md b/generated/benchmarks/BUILD-BENCHMARKS.md index 8bcea5a4..787c0c6b 100644 --- a/generated/benchmarks/BUILD-BENCHMARKS.md +++ b/generated/benchmarks/BUILD-BENCHMARKS.md @@ -5,6 +5,8 @@ Metrics are normalized per file for cross-version comparability. | Version | Engine | Date | Files | Build (ms/file) | Query (ms) | Nodes/file | Edges/file | DB (bytes/file) | |---------|--------|------|------:|----------------:|-----------:|-----------:|-----------:|----------------:| +| 3.9.0 | native | 2026-04-04 | 567 | 12.8 ↓4% | 30.4 ↑5% | 27.3 ~ | 54 ↑14% | 44666 ↑2% | +| 3.9.0 | wasm | 2026-04-04 | 567 | 13.1 ↓4% | 30.6 ↓4% | 27.3 ~ | 54 ~ | 44623 ↑5% | | 3.8.1 | native | 2026-04-03 | 565 | 13.3 ↑533% | 28.9 ↑113% | 27 ↑4% | 47.5 ↑5% | 43693 ↑42% | | 3.8.1 | wasm | 2026-04-03 | 565 | 13.6 ↑518% | 31.8 ↑108% | 27.2 ↑5% | 53.8 ↑19% | 42417 ↑39% | | 3.8.0 | native | 2026-04-02 | 564 | 2.1 | 13.6 | 26 | 45.3 | 30851 | @@ -55,38 +57,38 @@ Metrics are normalized per file for cross-version comparability. | Metric | Value | |--------|-------| -| Build time | 7.5s | -| Query time | 29ms | -| Nodes | 15,273 | -| Edges | 26,863 | -| DB size | 23.5 MB | -| Files | 565 | +| Build time | 7.3s | +| Query time | 30ms | +| Nodes | 15,483 | +| Edges | 30,609 | +| DB size | 24.2 MB | +| Files | 567 | #### WASM | Metric | Value | |--------|-------| -| Build time | 7.7s | -| Query time | 32ms | -| Nodes | 15,342 | -| Edges | 30,385 | -| DB size | 22.9 MB | -| Files | 565 | +| Build time | 7.5s | +| Query time | 31ms | +| Nodes | 15,483 | +| Edges | 30,610 | +| DB size | 24.1 MB | +| Files | 567 | ### Build Phase Breakdown (latest) | Phase | Native (build) | WASM (build) | Native (1-file) | WASM (1-file) | |-------|---------------:|-------------:|----------------:|--------------:| -| Parse | 559.7 ms | 3341 ms | 0.3 ms | 270.6 ms | -| Insert nodes | 332.8 ms | 338.4 ms | 0.2 ms | 20.4 ms | -| Resolve imports | 4.6 ms | 14 ms | 0.3 ms | 2.3 ms | -| Build edges | 147.8 ms | 204.6 ms | 8.2 ms | 24.1 ms | -| Structure | 0.3 ms | 57.6 ms | 0.3 ms | 30 ms | -| Roles | 65.5 ms | 75.5 ms | 13.6 ms | 56 ms | -| AST nodes | 246.1 ms | 238.9 ms | 2.6 ms | 7.9 ms | -| Complexity | 401.3 ms | 435.6 ms | 0.8 ms | 0.9 ms | -| CFG | 510.3 ms | 487.6 ms | 0.5 ms | 0.4 ms | -| Dataflow | 320.2 ms | 278.9 ms | 0.4 ms | 0.4 ms | +| Parse | 572.6 ms | 3079.6 ms | 50.2 ms | 258.2 ms | +| Insert nodes | 327.4 ms | 336.9 ms | 25.8 ms | 19.3 ms | +| Resolve imports | 3.6 ms | 19.6 ms | 0.9 ms | 1.8 ms | +| Build edges | 168.6 ms | 211 ms | 26.2 ms | 26.9 ms | +| Structure | 54.1 ms | 54.9 ms | 151.7 ms | 28.3 ms | +| Roles | 67.2 ms | 73.2 ms | 65.6 ms | 53.7 ms | +| AST nodes | 246.1 ms | 240 ms | 20.4 ms | 0.6 ms | +| Complexity | 399.7 ms | 420.1 ms | 4.8 ms | 0.7 ms | +| CFG | 541.8 ms | 496.1 ms | 27.7 ms | 0.4 ms | +| Dataflow | 296 ms | 195.9 ms | 15.3 ms | 0.5 ms | ### Estimated performance at 50,000 files @@ -94,15 +96,17 @@ Extrapolated linearly from per-file metrics above. | Metric | Native (Rust) | WASM | |--------|---:|---:| -| Build time | 665.0s | 680.0s | -| DB size | 2083.4 MB | 2022.6 MB | -| Nodes | 1,350,000 | 1,360,000 | -| Edges | 2,375,000 | 2,690,000 | +| Build time | 640.0s | 655.0s | +| DB size | 2129.8 MB | 2127.8 MB | +| Nodes | 1,365,000 | 1,365,000 | +| Edges | 2,700,000 | 2,700,000 | ### Incremental Rebuilds | Version | Engine | No-op (ms) | 1-file (ms) | |---------|--------|----------:|-----------:| +| 3.9.0 | native | 8 ~ | 562 ↑1238% | +| 3.9.0 | wasm | 15 ↓6% | 559 ↓7% | | 3.8.1 | native | 8 ~ | 42 ↑27% | | 3.8.1 | wasm | 16 ↑100% | 600 ↑1718% | | 3.8.0 | native | 8 | 33 | @@ -145,6 +149,8 @@ Extrapolated linearly from per-file metrics above. | Version | Engine | fn-deps (ms) | fn-impact (ms) | path (ms) | roles (ms) | |---------|--------|------------:|--------------:|----------:|----------:| +| 3.9.0 | native | 2.1 ↓5% | 2.2 ~ | 2.2 ~ | 25.9 ↓13% | +| 3.9.0 | wasm | 2.1 ↓5% | 2.2 ~ | 2.1 ↓5% | 26.6 ↓6% | | 3.8.1 | native | 2.2 ↓12% | 2.2 ~ | 2.2 ↑5% | 29.7 ~ | | 3.8.1 | wasm | 2.2 ~ | 2.2 ~ | 2.2 ↑5% | 28.4 ↓9% | | 3.8.0 | native | 2.5 | 2.2 | 2.1 | 29.2 | @@ -186,6 +192,22 @@ Extrapolated linearly from per-file metrics above. ### Notes +**Native 1-file rebuild regression (v3.8.1 42 ms → v3.9.0 562 ms, ↑1238%):** The native incremental +path is re-running graph-wide work on single-file rebuilds. The phase breakdown shows `structureMs` +at 151.7 ms for a 1-file rebuild vs 54.1 ms for the full 567-file build, and `setupMs` at 38.2 ms +vs 5.5 ms. AST/complexity/CFG/dataflow phases also jump from near-zero to 20-28 ms each, +suggesting these phases are not scoped to only the changed file. WASM 1-file rebuild (559 ms) is +superficially similar in total time but has a different root cause: WASM is parse-dominated +(`parseMs` 258.2 ms accounts for nearly half the total) while `structureMs` (28.3 ms), +`astMs`/`cfgMs`/`dataflowMs` (all under 1 ms) are correctly scoped. The native regression is +specifically in graph-wide phases being re-run during incremental builds. The no-op rebuild +(8 ms native, 15 ms WASM) is unaffected, confirming the regression is triggered by actual +file-change detection. + +**Engine edge divergence (v3.9.0, 1 edge):** Native reports 30,609 edges and WASM reports 30,610 +for the same 567-file codebase (node counts match at 15,483). This is a parity bug — tracked in +#855. + **WASM regression (v2.0.0 → v2.1.0, ↑32% — persists in v2.3.0):** The "v2.1.0" entry was measured after the v2.1.0 tag on main, when `package.json` still read "2.1.0" but the codebase already included post-release features: @@ -224,6 +246,153 @@ pre-parse that previously added ~388ms on native builds.