Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DOCS/FLAGS.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@

- `FORKRUN_RETRY_LIMIT` : Controls how many times a batch will be retried before it is declared poisoned. `0` means declared poisoned after the 1st failure. A negative value means it will never be declared poisoned (and could retry indefinitely). Default is 3.
- `FORKRUN_EXTRA_FUNCS` : Use this to specify required sub-functions to pass into frun's environment.
- EXAMPLE: `hh() { echo "$@"; }; gg() { hh "$@"; }; ff() { gg "$@"; };`. If you call `frun ff <inputs` the definition for `ff` will automatically be available to `frun` but the definitions for `gg` and `hh` will not be. Instead, call `FORKRUN_REQ_FUNCS='gg hh' frun ff <inputs`.
- EXAMPLE: `hh() { echo "$@"; }; gg() { hh "$@"; }; ff() { gg "$@"; };`. If you call `frun ff <inputs` the definition for `ff` will automatically be available to `frun` but the definitions for `gg` and `hh` will not be. Instead, call `FORKRUN_EXTRA_FUNCS='gg hh' frun ff <inputs`.
- `FORKRUN_EXTRA_VARS` : Use this to specify (environment) variables to pass into frun's environment
- EXAMPLE: If your code depends on variable X and X is only defined in your current shell session (and not in the code you are running) then you need to call `frun` via `FORKRUN_EXTRA_VARS='X' frun ...`
- `FORKRUN_EXTRA_SETUP` : Use this to specify raw commands that need to be run in frun's environment during setup
Expand Down
40 changes: 19 additions & 21 deletions DOCS/INVARIANTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,56 +48,54 @@ Workers claim ranges, never individual slots. Overshoot handled *after* claim, n

---

## 4. Signed Batch Size Protocol
## 4. Signed Batch Size Protocol (Hysteresis & Fast-Path Routing)

**Meaning of Sign**
* `batch_size < 0` → **Provisional policy** (scanner may still change its mind)
* `batch_size > 0` → **Finalized contract** (matches stream reality)
* `batch_size < 0` → **Catch-Up / Speculative Mode**. The scanner has geometrically ramped the target batch size (≥ 2x). Workers must speculatively claim multiple older, smaller slots to quickly reach the new magnitude.
* `batch_size > 0` → **Steady-State / Fast-Path Mode**. The scanner is shrinking the batch size or making PID micro-adjustments (< 2x growth). Workers stay on the lock-free fast path and claim exactly 1 slot.

**Scanner Responsibilities**
* May update batch size at any time.
* Must **always** publish negative values (`-abs(N)`).
* Must never publish a positive batch size.
* Must publish a **negative** value (`-N`) ONLY if the new target is at least double the absolute value of the current batch size (`N >= 2 * current`).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Match 2x hysteresis boundary to runtime behavior

Update the invariant to use a strict > 2x threshold instead of >= 2x. The runtime publisher in ring_loadables/forkrun_ring.c (PUBLISH_BATCH_SIZE) emits a negative size only when new_L > abs(current)*2, so an exact 2x increase stays on the positive fast path. Documenting >= here will cause incorrect operator expectations and invalid invariant checks around boundary-sized ramps.

Useful? React with 👍 / 👎.

* Must publish a **positive** value (`+N`) for all other adjustments (shrinking, or growing by less than 2x). This introduces hysteresis, shielding workers from PID jitter.

**Worker Responsibilities**
* Observe the published (negative) batch size.
* May finalize **only** when stream count equals `abs(published_batch_size)`.
* Must use **CAS** to flip `-N → +N`.
* If CAS fails (scanner changed policy), re-evaluate under new policy.
* Observe the published batch size.
* If positive (`> 0`): Bypass all speculative arithmetic and claim exactly 1 ring slot.
* If negative (`< 0`): Enter the speculative slow path to calculate how many older slots satisfy the new magnitude.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove unsupported speculative-claim statement

This line states that negative batch sizes make workers compute multi-slot speculative claims, but the current worker claim path in ring_loadables/forkrun_ring.c initializes claim_count = 1 and does not derive multi-claim count from sbatch < 0. Keeping this statement in the invariants will mislead debugging/tuning and future changes that rely on documented worker behavior.

Useful? React with 👍 / 👎.

* Must use **CAS** to flip `-N → +N` (finalization) once their `read_idx` successfully crosses the `batch_change_idx` boundary where the new size was published.

**Forbidden Transitions**
* Scanner publishing positive batch size
* Worker changing magnitude
* Any positive → negative transition
* Non-CAS sign flip
* Worker changing the magnitude.
* Any positive → negative transition initiated by a worker.
* Non-CAS sign flip by workers (except during explicit Tail Override).

**Correctness Guarantee**
Batch size reflects scanner intent until finalized. Finalization occurs exactly once. Scanner policy changes cannot resurrect stale batch sizes.
Because a worker cannot claim "half a slot," forcing workers into speculative arithmetic when a batch shrinks or slightly grows is a waste of CPU cycles. The signed protocol mathematically guarantees that workers only pay the slow-path cost when a massive geometric ramp actually necessitates combining multiple slots.

---

## 5. Tail-Aware Batch Size Rules

**Definition**
The tail ramp-down begins at `tail_idx`. Remaining data may not satisfy the current batch size.
The tail ramp-down begins at `tail_idx` (EOF). Remaining data may not cleanly satisfy the current batch size.

**Scanner Responsibilities**
* Must not publish batch sizes that correspond to tail batches.
* Once `tail_idx` is established, scanner must not modify batch size.
* When the tail is reached, the scanner must force the batch size to **positive** (`+N`) so that workers naturally downshift to claiming 1 slot at a time without overshooting.

**Worker Responsibilities at Tail**
When `read_idx tail_idx` and `batch_size < 0`:
When `read_idx >= tail_idx` and `batch_size < 0` (e.g., catching a race condition before the scanner's EOF force-flip):
1. Finalize immediately (`-N → +N` via CAS).
2. Bypass all slow paths (no waiting, no escrow).
3. Process exactly one claim using stride metadata.

**Why This Rule Exists**
Guarantees exactly one claim per tail batch, no policy leakage, no duplicate claims.
Guarantees exactly one claim per tail batch, prevents policy leakage, eliminates duplicate claims, and perfectly drains the ring without unnecessary escrow bouncing.

**Forbidden**
* Scanner publishing after entering tail
* Workers applying ramp-down logic based on batch size
* Slow path in tail
* Scanner publishing batch changes after entering the tail.
* Workers applying speculative multi-claim logic in the tail.

**Key Insight**
Batch size is a *policy*, not a property of the tail. Once the tail begins, policy ends and structure takes over.
Expand Down
4 changes: 2 additions & 2 deletions frun.bash
Original file line number Diff line number Diff line change
Expand Up @@ -1193,7 +1193,7 @@ _forkrun_get_arch() {

case "$ARCH0" in
x86_64[-_]v[2-4])
ARCH="${ARCH0//_v/-v}"
ARCH="${ARCH0//-v/_v}"
;;
x86_64)
if grep -qE '( avx512[cdbwdqvlf].*){5}' </proc/cpuinfo; then
Expand All @@ -1204,7 +1204,7 @@ _forkrun_get_arch() {
ARCH='x86_64_v2'
fi
;;
aarch64|armv7|riscv64|s390x|ppcle64)
Comment thread
sourcery-ai[bot] marked this conversation as resolved.
aarch64|riscv64|s390x|ppc64le)
ARCH="$ARCH0"
;;
*)
Expand Down
4 changes: 2 additions & 2 deletions ring_loadables/frun.nob64.bash
Original file line number Diff line number Diff line change
Expand Up @@ -1193,7 +1193,7 @@ _forkrun_get_arch() {

case "$ARCH0" in
x86_64[-_]v[2-4])
ARCH="${ARCH0//_v/-v}"
ARCH="${ARCH0//-v/_v}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The changes in this file are identical to those in frun.bash and ring_loadables/frun.nob64.bash.txt. Maintaining multiple duplicate copies of the same script increases maintenance overhead and the risk of files drifting out of sync. It would be highly beneficial to use symlinks, source a shared script, or use a build/generation step to keep these files synchronized automatically.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ring_loadables/frun.nob64.bash.txt is a copy of frun.bash with the base64 embeddings removed. it is generated using ring_loadables/remove_frun_base64.bash. its purpose is to allow for easier AI review of frun.bash without wasting 100's of thousands of tokens on meaningless embedded base64 strings

;;
x86_64)
if grep -qE '( avx512[cdbwdqvlf].*){5}' </proc/cpuinfo; then
Expand All @@ -1204,7 +1204,7 @@ _forkrun_get_arch() {
ARCH='x86_64_v2'
fi
;;
aarch64|armv7|riscv64|s390x|ppcle64)
aarch64|riscv64|s390x|ppc64le)
ARCH="$ARCH0"
;;
*)
Expand Down
4 changes: 2 additions & 2 deletions ring_loadables/frun.nob64.bash.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1193,7 +1193,7 @@ _forkrun_get_arch() {

case "$ARCH0" in
x86_64[-_]v[2-4])
ARCH="${ARCH0//_v/-v}"
ARCH="${ARCH0//-v/_v}"
;;
x86_64)
if grep -qE '( avx512[cdbwdqvlf].*){5}' </proc/cpuinfo; then
Expand All @@ -1204,7 +1204,7 @@ _forkrun_get_arch() {
ARCH='x86_64_v2'
fi
;;
aarch64|armv7|riscv64|s390x|ppcle64)
aarch64|riscv64|s390x|ppc64le)
ARCH="$ARCH0"
;;
*)
Expand Down
Loading