Skip to content

Commit 89ba415

Browse files
committed
Tighten rust benchmark guardrail threshold to 0.045
1 parent 70cff91 commit 89ba415

5 files changed

Lines changed: 9 additions & 9 deletions

File tree

.github/workflows/ci.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -341,7 +341,7 @@ jobs:
341341

342342
env:
343343
BENCH_RUNS: "1"
344-
BENCH_RUST_TO_PURE_MAX_RATIO: "0.05"
344+
BENCH_RUST_TO_PURE_MAX_RATIO: "0.045"
345345

346346
steps:
347347
- uses: actions/checkout@v4

bin/check-rust-benchmark

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ def usage
1616
BENCH_TOPICS Number of topics (default: 8)
1717
BENCH_MAX_ITER Variational max iterations (default: 20)
1818
BENCH_EM_MAX_ITER EM max iterations (default: 40)
19-
BENCH_RUST_TO_PURE_MAX_RATIO Maximum allowed rust_mean/pure_mean ratio (default: 0.05)
19+
BENCH_RUST_TO_PURE_MAX_RATIO Maximum allowed rust_mean/pure_mean ratio (default: 0.045)
2020
USAGE
2121
end
2222

@@ -40,7 +40,7 @@ benchmark_env = {
4040
"BENCH_MAX_ITER" => ENV.fetch("BENCH_MAX_ITER", "20"),
4141
"BENCH_EM_MAX_ITER" => ENV.fetch("BENCH_EM_MAX_ITER", "40")
4242
}
43-
threshold = Float(ENV.fetch("BENCH_RUST_TO_PURE_MAX_RATIO", "0.05"))
43+
threshold = Float(ENV.fetch("BENCH_RUST_TO_PURE_MAX_RATIO", "0.045"))
4444

4545
stdout, stderr, status = Open3.capture3(benchmark_env, *command, chdir: repo_root)
4646
unless status.success?

docs/modernization-handoff.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -181,7 +181,7 @@ Performance tracking:
181181
- `./bin/benchmark-backends`
182182
- `./bin/check-rust-benchmark`
183183
- `docs/rust-orchestration-guardrails.md`
184-
- CI currently enforces `BENCH_RUST_TO_PURE_MAX_RATIO=0.05` in `benchmark-guardrail`
184+
- CI currently enforces `BENCH_RUST_TO_PURE_MAX_RATIO=0.045` in `benchmark-guardrail`
185185

186186
## CI Jobs Expected
187187

docs/porting-strategy.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ Completed in `codex/experiment-ruby3-modernization`:
6969
- Gem packaging now excludes local Rust cargo build artifacts (`target/**`) for clean release builds.
7070
- Backend benchmark driver added (`bin/benchmark-backends`) to track pure/native/rust runtime deltas.
7171
- Rust orchestration guardrail policy documented (`docs/rust-orchestration-guardrails.md`) with benchmark threshold checker (`bin/check-rust-benchmark`).
72-
- CI benchmark guardrail job added (`benchmark-guardrail`) to enforce Rust/pure runtime ratio on Ubuntu (currently `BENCH_RUST_TO_PURE_MAX_RATIO=0.05`).
72+
- CI benchmark guardrail job added (`benchmark-guardrail`) to enforce Rust/pure runtime ratio on Ubuntu (currently `BENCH_RUST_TO_PURE_MAX_RATIO=0.045`).
7373
- Source install path now has explicit Rust build policy via `LDA_RUBY_RUST_BUILD=auto|always|never`.
7474
- Docker install-policy matrix script added (`bin/docker-test-install-policies`) to verify source install behavior across environments.
7575
- CI now runs install-policy matrix checks on Ubuntu.

docs/rust-orchestration-guardrails.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,10 +26,10 @@ Run:
2626

2727
Default benchmark policy:
2828

29-
- `BENCH_RUST_TO_PURE_MAX_RATIO=0.05`
30-
- i.e., Rust mean runtime must be no worse than 5% of pure mean runtime on the benchmark fixture/config.
29+
- `BENCH_RUST_TO_PURE_MAX_RATIO=0.045`
30+
- i.e., Rust mean runtime must be no worse than 4.5% of pure mean runtime on the benchmark fixture/config.
3131
- CI benchmark guardrail job enforces the same ratio with `BENCH_RUNS=1` for runtime stability.
32-
- latest tightening evidence (2026-03-02): 17 consecutive successful CI benchmark-guardrail runs on `codex/rust-orchestration-phase8` (`22555725309` .. `22557953998`) observed Rust/Pure ratios in `[0.0252, 0.0288]`, supporting a tighter `0.05` threshold with headroom.
32+
- latest tightening evidence (2026-03-05): local Docker guardrail check with `BENCH_RUNS=3` observed Rust/Pure ratio `0.0368` (`rust=0.0758s`, `pure=2.0569s`), and prior CI streak data on `codex/rust-orchestration-phase8` (`22555725309` .. `22557953998`) observed `[0.0252, 0.0288]`, supporting a tighter `0.045` threshold with headroom.
3333

3434
Configurable environment knobs:
3535

@@ -38,7 +38,7 @@ Configurable environment knobs:
3838
- `BENCH_TOPICS` (default `8`)
3939
- `BENCH_MAX_ITER` (default `20`)
4040
- `BENCH_EM_MAX_ITER` (default `40`)
41-
- `BENCH_RUST_TO_PURE_MAX_RATIO` (default `0.05`)
41+
- `BENCH_RUST_TO_PURE_MAX_RATIO` (default `0.045`)
4242

4343
## When to tighten thresholds
4444

0 commit comments

Comments
 (0)