feat(lib): lib/supervise — observer-native supervision (#409)#527
Merged
Conversation
A supervisor over the #408 cooperative task layer that, beyond BEAM-style crash-restart, also catches the silently WEDGED worker — alive, not crashed, never timing out, but no longer making progress — a signal BEAM structurally cannot see (its processes carry no free per-value trajectory measurement). This is the EigenOS window-manager red-title pattern generalized. Model (no share-nothing messages needed — cooperative tasks share the module-global heap on one thread): - workers publish raw progress with `heartbeat of [slot, value]`; - the SUPERVISOR samples each alive worker every tick and feeds a BOUNDED signature (1.5 + v % 8 — never a raw counter, which hits the #294 flat-entropy blind spot) into that slot's observer trajectory. Sampling on the supervisor side is what makes a wedge visible: a frozen worker stops heart-beating, so only the poller keeps adding the now-constant samples that read `stable`; - a worker that returns normally is `done`; one that raises is caught by the wrapper as `crashed` (so a supervised crash never fails the process, #493); - wedged (alive + stable after a warm-up window) and crashed workers are restarted from their spec, each up to a per-worker restart-intensity cap. API: supervisor_new / supervise of [sup, fn, slot] / supervise_step / supervise_run of [sup, max_ticks]. Deterministic by construction. Eight slots (the WM/task-table precedent; a list element has no observer trajectory, #262, so the signals need named scalars). Flagship example examples/supervisor_tree.eigs (byte-identical across runs). Regression tests/test_supervise.eigs: wedged-detected-and-restarted-once, healthy-never-restarted, crash-restart (exit 0, not fatal), restart cap. Forcing-function gap surfaced (not worked around): a task cannot learn its own id (no task_self), so the message-link supervision variant isn't expressible — filed as #526. Full suite green release + ASan (leaks on), 2703/2703, no leak-tally increase (task kill/respawn clean). Closes #409 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
8c6bd3c to
70ecc1e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A supervisor over the #408 cooperative task layer that, beyond BEAM-style crash-restart, also catches the silently wedged worker — alive, not crashed, never timing out, but no longer making progress — a signal BEAM structurally cannot see (its processes carry no free per-value trajectory measurement). This generalizes the EigenOS window-manager red-title pattern.
Model
Share-nothing messages are unnecessary here — cooperative tasks share the module-global heap on one thread:
heartbeat of [slot, value];1.5 + v % 8— never a raw counter, which hits the Observer predicates classify entropy(value), not value — blind to value-oscillation in flat-entropy regions #294 flat-entropy blind spot) into that slot's observer trajectory. Sampling on the supervisor side is the crux: a frozen worker stops heart-beating, so only the poller keeps adding the now-constant samples that readstable;done; a raise is caught by the wrapper ascrashed, so a supervised crash never fails the process (task: unjoined task death prints error but process exits 0 #493);stableafter a warm-up window) and crashed workers restart from their spec, each up to a per-worker restart-intensity cap.API:
supervisor_new/supervise of [sup, fn, slot]/supervise_step/supervise_run of [sup, max_ticks]. Deterministic by construction. Eight slots (a list element has no observer trajectory, #262, so the signals need named scalars).Verification
examples/supervisor_tree.eigs— a 3-worker pool where one wedges mid-run and is detected + restarted; byte-identical across runs.tests/test_supervise.eigs(6 asserts): wedged-detected-and-restarted-exactly-once, healthy-never-restarted, wedged-recovers; crash-restart (exit 0, not fatal); restart-intensity cap honored.detect_leaks=1): 2703/2703, 0 failed, no leak-tally increase (task kill/respawn is leak-clean).Forcing-function gap surfaced (not worked around)
A task cannot learn its own id (no
task_self), so the BEAM-style message-link supervision variant the issue sketches isn't expressible — the shared-heap model is used instead (idiomatic for this runtime). Filed as #526.Docs
docs/STDLIB.md:lib/supervise.eigssection (satisfies the discoverability gate).CHANGELOG.md: entry under[Unreleased] → Added.Closes #409
🤖 Generated with Claude Code