bench(engine): overhead compensation + SCOPE.md (rigor pass items 2 + 7)#36
Open
bench(engine): overhead compensation + SCOPE.md (rigor pass items 2 + 7)#36
Conversation
Item 2 — overhead compensation ============================== Every algo / handoff cycle count emitted to the CSV stream now has a constant `bench_overhead_cycles` subtracted. The constant is the median of 1000 empty `k_cycle_get_32()`-pair measurements taken at boot under irq_lock, before any per-event timing begins. Saturating subtraction at 0 — never report a negative cycle count. The compensation is **visible**, not silent: - Emitted as `overhead_cycles,<value>` in the CSV header. - Tagged through `tag_events.py` and the synth-workflow inline heredocs as `M,R<run>,<variant>,overhead_cycles,<value>`. - Parsed by `analyze.py` into `Meta.overhead_cycles[run]` and surfaced in the report header as "Overhead subtracted (cycles): baseline ...; gale ...". Reviewers can audit the subtraction step and re-add it to recover the raw numbers if needed. Same idiom as Zephyr 4.4 ztest_bench's `ctrl` benchmark pattern (see subsys/testsuite/ztest/benchmark/). **Pre-compensation and post-compensation numbers are different measurements.** The README and SCOPE.md both call this out explicitly. Do not combine them in the same comparison table. Item 7 — SCOPE.md ================= New file `benches/engine_control/SCOPE.md` is the **source of truth** for what the bench measures, what it does NOT measure, and what kind of evidence its numbers constitute. Subsequent published copy imports language from there. Explicit non-claims include peripheral contention, DMA-driven I/O, SMP/multi-core, **WCET** (which requires static analysis tooling like AbsInt aiT, Rapita RapiTime, or OTAWA — explicitly out of scope), power consumption, memory pressure, fault tolerance, and long-duration drift. The WCET distinction is unambiguous and not negotiable in published copy: an observation is not a proof. Worst-case-observed numbers, when added later under work item 6, must be labeled as `worst_observed`, never `wcet`. Local build verification ======================== The local Zephyr workspace at /Users/r/git/pulseengine/z is in a state that doesn't currently build (a Kconfig env var for `hal_espressif` is unset, unrelated to these changes). CI will produce the new compensation-aware Renode reference baseline when this branch lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two parallel items from the bench-rigor work order, both engine_control-scoped. No Rust source touched outside the bench. No changes to flight_control (separate pass).
Item 2 — overhead compensation
`measure_overhead()` runs at boot under `irq_lock`, takes the median of 1000 empty `k_cycle_get_32()`-pair measurements, and stores it as `bench_overhead_cycles`. Every `algo` and `handoff` count emitted to the CSV is the raw measurement minus that constant (saturating at 0).
The compensation is visible, not silent:
Reviewers can audit and re-add the subtraction; the raw number is recoverable. Same idiom as Zephyr 4.4 ztest_bench's `ctrl` benchmark in `subsys/testsuite/ztest/benchmark/`.
Pre-compensation and post-compensation numbers are different measurements. Both the README and SCOPE.md call this out explicitly — do not combine them in the same comparison table.
Item 7 — `benches/engine_control/SCOPE.md` (new)
Source of truth for what the bench measures, what it does NOT measure, and what kind of evidence its numbers constitute. Downstream copy imports from there.
Explicit non-claims:
The file ends with the recommended one-paragraph framing for blog posts and a list of when to update SCOPE.md.
Test plan
Out of scope (deferred per the work order)
🤖 Generated with Claude Code