Skip to content

Commit 15ae8d2

Browse files
committed
docs: update README with expanded change-detection metrics
Add helix-db to impact data and update metrics for build/test skip rates, targeted runs, and dependencies unified based on recent measurements.
1 parent c990ab1 commit 15ae8d2

2 files changed

Lines changed: 34 additions & 17 deletions

File tree

README.md

Lines changed: 18 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -6,19 +6,20 @@
66

77
## Why
88

9-
**Documented impact on real repos (tokio, helix, meilisearch):**
9+
**Documented change-detection impact on real repos (tokio, helix, meilisearch, helix-db):**
1010

1111
| Metric | Impact |
1212
|---|---|
13-
| **CI Surface Execution** | 55% fewer surfaces run per merge |
14-
| **Weighted `test`/`build` Units** | 64% reduction in compute units |
15-
| **Dependencies Unified** | 96 across 53 crates (avg 1.8 per crate) |
13+
| **Commits Measured** | 80 (20 per repo) |
14+
| **Could Skip Build** | 21% |
15+
| **Could Skip Tests** | 19% |
16+
| **Targeted (Not Full Run)** | 68% |
17+
| **Dependencies Unified** | 132 across 53 crates |
1618
| **Undeclared `features` Fixed** | 258 silent bugs prevented |
17-
| **Tooling Consolidation** | 6-8 cargo plugins → 1 command |
1819
| **MSRV Computation** | Automatic from dependency graph |
1920

2021
**Compounding effects = massive time/cost savings:**
21-
1. **Change Detection** (`plan`/`run`) reduces what runs — 55% fewer CI surfaces, 64% fewer compute units w/ shown determinism
22+
1. **Change Detection** (`plan`/`run`) reduces what runs — 68% of measured commits avoided full-run behavior, with explicit plan traces.
2223
2. **Dependency Unification** (`unify`) reduces build graph complexity — cleaner deps, smaller build units, fewer rebuilds
2324
3. **Smaller Build Graphs** cargo-rail uses 14 core-deps for automatically removing unused dependencies, pruning truly dead features, unifying undeclared features, computing MSRV, splitting and synching crate/s to new, clean repos, and the entire release workflow w/ changelog generation. This results in fewer tools and less cargo-metadata fetches.
2425

@@ -69,7 +70,7 @@ cargo rail run --merge-base --profile ci # runs ONLY what plan selected
6970
3. Run `run` to execute only selected surfaces — locally or in CI - or wire to `justfile`, `makefile`, `xtask`, or shell scripts.
7071
4. Use `--explain` to understand any decision: "why did this run?" / "why was this skipped?"
7172

72-
Result: **55% fewer CI surface executions, 64% reduction in weighted compute units** (validated on tokio/helix/meilisearch) See: [Examples](examples/change_detection).
73+
Result: **21% build skip, 19% test skip, 68% targeted/non-full runs** across 80 measured commits (tokio/helix/meilisearch/helix-db). See: [Examples](examples/change_detection).
7374

7475
### Dependency Unification (`unify`)
7576

@@ -96,11 +97,11 @@ cargo rail unify --explain # understand each decision
9697

9798
| Repository | Crates | Deps Unified | Undeclared Features | MSRV Computed |
9899
|---|---:|---:|---:|---|
99-
| [tokio-rs/tokio](https://github.com/loadingalias/cargo-rail-testing/tree/unify/tokio) | 10 | 9 | 7 | 1.85.0 |
100-
| [helix-editor/helix](https://github.com/loadingalias/cargo-rail-testing/tree/unify/helix) | 14 | 15 | 19 | 1.87.0 |
101-
| [meilisearch/meilisearch](https://github.com/loadingalias/cargo-rail-testing/tree/unify/meilisearch) | 23 | 54 | 215 | 1.88.0 |
102-
| [helixdb/helix-db](https://github.com/loadingalias/cargo-rail-testing/tree/unify/helix-db) | 6 | 18 | 17 | 1.88.0 |
103-
| **Aggregate** | **53** | **96** | **258** ||
100+
| [tokio-rs/tokio](https://github.com/loadingalias/cargo-rail-testing/tree/unify/tokio) | 10 | 13 | 7 | 1.85.0 |
101+
| [helix-editor/helix](https://github.com/loadingalias/cargo-rail-testing/tree/unify/helix) | 14 | 28 | 19 | 1.87.0 |
102+
| [meilisearch/meilisearch](https://github.com/loadingalias/cargo-rail-testing/tree/unify/meilisearch) | 23 | 70 | 215 | 1.88.0 |
103+
| [helixdb/helix-db](https://github.com/loadingalias/cargo-rail-testing/tree/unify/helix-db) | 6 | 21 | 17 | 1.88.0 |
104+
| **Aggregate** | **53** | **132** | **258** ||
104105

105106
Config files and validation artifacts: [Examples](examples/unify/) | [Validation Forks](https://github.com/loadingalias/cargo-rail-testing)
106107

@@ -171,10 +172,10 @@ All core workflows (`plan`/`run`, `unify`) validated on production repos with fu
171172

172173
| Repository | Crates | Validation | Fork |
173174
|---|---:|---|---|
174-
| tokio-rs/tokio | 10 | Unify (9 deps, 7 features), Plan/run | [Fork](https://github.com/loadingalias/cargo-rail-testing/tree/unify/tokio) |
175-
| helix-editor/helix | 14 | Unify (15 deps, 19 features), Plan/run | [Fork](https://github.com/loadingalias/cargo-rail-testing/tree/unify/helix) |
176-
| meilisearch/meilisearch | 23 | Unify (54 deps, 215 features), Plan/run | [Fork](https://github.com/loadingalias/cargo-rail-testing/tree/unify/meilisearch) |
177-
| helixdb/helix-db | 6 | Unify (18 deps, 17 features), Plan/run | [Fork](https://github.com/loadingalias/cargo-rail-testing/tree/unify/helix-db) |
175+
| tokio-rs/tokio | 10 | Unify (13 deps, 7 features), Plan/run | [Fork](https://github.com/loadingalias/cargo-rail-testing/tree/unify/tokio) |
176+
| helix-editor/helix | 14 | Unify (28 deps, 19 features), Plan/run | [Fork](https://github.com/loadingalias/cargo-rail-testing/tree/unify/helix) |
177+
| meilisearch/meilisearch | 23 | Unify (70 deps, 215 features), Plan/run | [Fork](https://github.com/loadingalias/cargo-rail-testing/tree/unify/meilisearch) |
178+
| helixdb/helix-db | 6 | Unify (21 deps, 17 features), Plan/run | [Fork](https://github.com/loadingalias/cargo-rail-testing/tree/unify/helix-db) |
178179

179180
**Validation forks**: [cargo-rail-testing](https://github.com/loadingalias/cargo-rail-testing) — full configs, integration guides, and reproducible artifacts.
180181

@@ -186,7 +187,7 @@ All core workflows (`plan`/`run`, `unify`) validated on production repos with fu
186187
4. **Real-world scenarios**: Tests run on actual merge commits and real dependency graphs, not synthetic fixtures
187188

188189
**Unify results (4 repos, 53 crates):**
189-
- 96 dependencies unified to `[workspace.dependencies]`
190+
- 132 dependencies unified to `[workspace.dependencies]`
190191
- 258 undeclared features fixed (silent bugs prevented)
191192
- 2 dead features pruned
192193
- MSRV computed for all repos (1.85.0 - 1.88.0)

examples/change_detection/README.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,22 @@ Two approaches based on your project's tooling:
99
| [**with-task-runner/**](with-task-runner/) | just, make, xtask, scripts | `cargo rail plan` + your task runner |
1010
| [**standalone/**](standalone/) | No task runner | `cargo rail run` handles everything |
1111

12+
## Measured Impact (Last 20 Commits per Repo)
13+
14+
Generated with `/Users/mr.wolf/loadingalias/cargo-rail-testing/scripts/measure-impact.sh`.
15+
16+
| Repository | Could Skip Build | Could Skip Tests | Targeted (Not Full Run) |
17+
|---|---:|---:|---:|
18+
| tokio | 10% | 0% | 95% |
19+
| meilisearch | 35% | 35% | 60% |
20+
| helix | 30% | 30% | 40% |
21+
| helix-db | 10% | 10% | 75% |
22+
| **Aggregate (80 commits)** | **21%** | **19%** | **68%** |
23+
24+
Interpretation:
25+
- Task-runner repos (`meilisearch`, `helix`) show the largest immediate test/build skip savings.
26+
- Standalone repos still gain deterministic targeting and explainable plan traces.
27+
1228
## With Task Runner (Recommended for Large Projects)
1329

1430
If you already have **just**, **make**, **xtask**, or shell scripts:

0 commit comments

Comments
 (0)