Deterministic cooperative task layer that rides the trace tape (task_spawn / task_send / task_recv / task_yield)

[InauguralPhysicist]

Why — the biggest contradiction in the language today. recv/try_recv/recv_timeout fail loudly under EIGS_REPLAY (#148, replay_blocks in src/builtins.c; docs/TRACE.md) — deterministic replay, the signature niche, currently EXCLUDES concurrent programs. Meanwhile two consumers independently reinvented the missing layer downstream: EigenOS task_eval (strings-by-copy isolation, window=task, hung-window detection) and liferaft's hand-rolled deterministic scheduler. Upstreaming it converts the contradiction into the headline capability no incumbent has: byte-identical replay of concurrent executions (Go/BEAM/Rust users need rr or Antithesis-class external heroics for this).

What.

• task_spawn / task_send / task_recv / task_yield builtins with per-task isolated state — values-by-copy on every boundary, reusing the existing val_clone_for_send machinery (Cross-thread channel transfer corrupts dict string keys (use-after-free) #293/Cross-thread UAF: a spawned worker's arena-allocated return value is freed by arena_destroy at detach before the joiner reads it #302) that channels already use.
• A seeded deterministic round-robin scheduler; every scheduling decision and message delivery emitted as a tape nondet record, so EIGS_REPLAY re-drives the whole concurrent program byte-identically.
• Single OS thread: g_vm_multithreaded never flips, so the JIT stays ON for concurrent code (fixing the Data races: parallel workers executing the same shared chunk (exec_count, JIT counters, inline caches, shared-env refcounts) #297 perf cliff for the common case), and per-task isolation protects Env::obs binding identity.
• The eigenscript-extend-vm checklist applies (new builtins are nondet + tape-participating).
• OS threads (spawn) keep their honest fail-loudly wall under replay — this is a new tier, not a replacement.
• SPEC.md "Tasks" section with byte-exact examples; examples/task_pipeline.eigs.
• Differential proof: migrate liferaft onto the upstream layer and confirm its byte-for-byte replay-under-chaos results hold.

Acceptance. A multi-task program with message passing records under EIGS_TRACE and replays byte-identically under EIGS_REPLAY; JIT stays enabled (verify via the bench harness MT-vs-task workload); liferaft migration green.

Effort: weeks (VM change; the extend-vm all-N-sites checklist is the discipline). Blocks: lib/supervise.eigs. Source: 2026-07-04 survey (concurrency dimension — the study's single largest strategic item).

[InauguralPhysicist]

Design note: deterministic cooperative task layer (#408)

Requirements were taken from the two downstream reinventions before any design: EigenOS's window=task machinery (EigsState-per-window, eigs_thread_switch, whole-step() granularity, MAX_TASKS=8, the p0–p7 observer slot pool) and liferaft's discrete-event scheduler (the (t, tie, seq) total-order heap, one explicit seeded PRNG, draws-in-fixed-order-even-when-unused, same-seed ⇒ byte-identical two-process output with an empty tape). Plus the existing OS-thread surface (channels deep-copy via val_clone_for_send; recv* fail loudly under replay per #148; first spawn flips g_vm_multithreaded and parks the JIT per #296/#297).

The central design claim: scheduling needs NO tape records

A cooperative scheduler on a single OS thread with a fixed, SPEC-pinned policy and explicit yield points is deterministic by construction. There is no scheduling nondeterminism to record: the interleaving is a pure function of program execution, and the only nondeterminism in a concurrent program remains what the tasks themselves consume — the existing N records, whose tape order is pinned by the deterministic schedule. liferaft is the existence proof: its whole chaos harness replays byte-identically with an empty tape because every choice is a pure seeded draw.

Consequences:

• Zero new tape record kinds for increments 1–2. No Design: trace-tape format versioning — the tape is becoming a persisted artifact #411 format bump. The tape-shipping train is not disturbed.
• The proc_* builtins are invisible to trace/replay — EIGS_REPLAY re-runs live subprocess side effects #148 wall is lifted for tasks, not worked around: task_recv is deterministic, so it is simply legal under EIGS_REPLAY — nothing to record, nothing to refuse. The OS-thread recv family keeps its honest fail-loud wall; this is a new tier, not a replacement.
• Chaos-mode interleaving stays replayable for free: any randomized policy (adversarial reordering, latency, drops) must draw from sources that already ride the tape (random) or from a pure in-language PRNG (liferaft's). The layer itself never consults un-taped entropy. liferaft's discipline — nondet draws in a fixed order even when unused, so consumption is control-flow-independent — is documented as the recommended pattern.
• Divergence detection: EIGS_REPLAY_STRICT=1 already catches N-record name/order drift; a schedule divergence between record and replay surfaces as exactly that.

Execution model

• Single OS thread, always. g_vm_multithreaded never flips. The JIT, OSR, and inline caches stay live for concurrent code — the strongest argument for this tier over spawn (which is interpreter-only, Data races: parallel workers executing the same shared chunk (exec_count, JIT counters, inline caches, shared-env refcounts) #297).
• A task is a reified VM context: own value stack, frame array, sp/ip — the bytecode VM's frames are already heap state, not C recursion, so suspension = save context pointers, resumption = restore them. No C-stack capture. task_yield (and blocking task_recv/task_join) signal vm_run to save the live context and return control to the scheduler.
• Yield is illegal inside a native re-entry (eval, comparator callbacks — anywhere native_call_depth is above the task's entry level): deterministic runtime error. Re-entrant vm_run invocations sit on the C stack and cannot be suspended. This is the same class of restriction Lua places on yields across C boundaries.
• Scope/isolation follows the language's existing rules rather than a new copy regime. The spawned function's writes are already function-local for fresh names (the post-load_file scope asymmetry: a lib function can write a caller global declared BEFORE the load, but not one declared after #373 semantics: a fresh-name is in a function binds locally); only names bound at module level are shared-and-mutable, and single-threaded determinism makes even those deterministic. Message boundaries (task_send/task_join results, spawn args) deep-copy via val_clone_for_send — share-nothing where it matters.
  • This deliberately diverges from the issue's "per-task isolated state on every boundary." Full isolation requires deep-cloning closure env graphs (cycle-prone, heavy) or an EigsState-per-task (that's the embed seam EigenOS already has — heavyweight, numbers/strings-only seams, the pain the survey documented). Natural fresh-name locality + copied messages gets the isolation that matters for a fraction of the machinery. EigenOS keeps EigsState-per-window for its REPL-isolation case (arbitrary user code per window is a different threat model); what it adopts from this layer is what it actually lacks — mid-computation yield and unbounded task count — inside each window's state.
  • Observer note: task-local bindings get per-task trajectories naturally (each task's frame envs are its own — Env::obs identity is safe without any reset machinery). Shared module bindings mutated from several tasks interleave one trajectory — deterministic but mixed; documented, and lib/supervise.eigs — observer-native supervision: predictive, not crash-reactive (blocked on the task layer) #409's supervision seam reads per-task counters instead (below).

API (increment order)

Increment 1 — contexts + scheduling (no messaging):

• task_spawn of [f, arg...] → tid (1-based spawn order; main program is task 0). Args deep-copied.
• task_yield of null — reschedule to the tail.
• task_alive of tid → 0/1.
• task_join of tid → blocks (cooperatively) until tid ends; returns its result deep-copied, or rethrows the task's uncaught error as the same {kind, message, line} it died with (Structured runtime errors: catch binds {kind, message, line} with a closed kind set #406 composes: the error's kind survives the task boundary; throw of e re-raise semantics already work).
• Scheduler policy, SPEC-pinned: one ready deque, round-robin. spawn appends; yield moves to tail; blocking removes from the deque; unblocking appends to tail; run-until-yield/block/end. All ties broken by deterministic queue order — there are no ties left to break (liferaft's seq lesson, made structural).
• Program ends when task 0 ends; outstanding tasks are torn down deterministically (spawn order), and any task that died of an uncaught error without being joined reports to stderr at teardown — no silent task death (the EigenOS "dead window, reason only on serial" pain).

Increment 2 — messaging:

• task_send of [tid, v] — deep-copy append to tid's FIFO mailbox; wakes tid if blocked. v1 mailboxes are unbounded (bounded/backpressure deferred — bounded send introduces send-blocking and a second deadlock shape; add capacity only when a consumer forces it).
• task_recv of null — next message or cooperative block; task_try_recv of null — non-blocking.
• task_kill of tid — deterministic teardown: mailbox dropped, pending join on it rethrows kind interrupt (liferaft's sched_drop_to analog).
• All tasks blocked = deadlock: deterministic, detectable in O(1), raised as a runtime error. This extends the Structured runtime errors: catch binds {kind, message, line} with a closed kind set #406 closed kind set with deadlock — the first designed use of the documented SPEC+DIAGNOSTICS extension process. (Not limit, not internal: consumers will legitimately discriminate it.)

Increment 3 — time + supervision seam:

• task_sleep of n — virtual ticks, discrete-event style: the clock advances only when every runnable task is asleep/blocked (deterministic; subsumes liferaft's t axis and gives EigenOS frame pacing without wall-clock).
• task_stat of tid → {alive, steps, yields, mailbox_len} — deterministic counters. This is lib/supervise.eigs — observer-native supervision: predictive, not crash-reactive (blocked on the task layer) #409's seam: lib/supervise.eigs predicates watch these (N tasks, no p0–p7 slot pool, no 8-task cap).

Increment 4 — tooling attribution (with #418 eigsdap):

• T <tid> context-switch records under EIGS_TRACE, tape format v2 (Design: trace-tape format versioning — the tape is becoming a persisted artifact #411 bump, refuse-on-mismatch as designed). Without them, per-task attribution of A/L records has the docs: TRACE.md says A-records track top-level bindings; function locals also land on the tape (name-only) #467 aggregation problem at task scale; eigsdap's task stepping is the forcing consumer. Designed now, shipped then.

Increment 5 — differential proof:

• liferaft's smallest chaos scenario (3 nodes, adversary on, fixed seed) reproduced on the task layer with byte-equal applied-stream output against its hand-rolled scheduler, in the liferaft repo. Full migration is liferaft's call afterwards — its event heap is ~200 lines of working, forcing-function EigenScript, and "replace it" is not a goal that pays for itself; "the upstream layer can express it byte-exactly" is.

JIT interaction

Suspension can only happen at builtin call sites (task_yield/task_recv/task_join/task_sleep). v1: chunks that contain a suspending builtin are compile-flagged non-JIT — cheap, safe, and the #297 fix claim survives where it matters (compute-heavy task bodies that don't yield mid-loop stay JIT'd; the yield-containing driver fn is typically cold). Full JIT-state suspend/resume (materializing a JIT frame into a resumable VM context, OSR-in-reverse) is deliberately deferred until the bench harness shows the flag costs real workloads. Measured, not assumed (perf-attribution discipline).

Top implementation risks (flagged for increment 1, in order)

1. Arena lifetime across suspension. The value arena is per-thread and window-based; a suspended task's live stack values must not sit in an arena window another task's execution reclaims. Options: promote-on-suspend (heap-promote the suspended stack's arena values — cost proportional to live stack), per-task arenas (memory growth), or arena-pinning while any task is suspended (defeats the arena). Needs the memory-design pass before coding; this is the Initialize EigenScript repository with official README and project structure #1 way increment 1 goes sideways.
2. Per-task VM context memory: VM_STACK_MAX-sized stacks per task don't scale to N tasks on the 4 GB box; contexts need small grow-on-demand stacks (task stack overflow = existing limit kind).
3. Handle-table and teardown interaction: task teardown must compose with handle_table_drain and the leak-tally gate (channel/thread handles created inside tasks).
4. prev of / temporal history is name-keyed process-wide — same-named task locals interleave one history stream (the docs: TRACE.md says A-records track top-level bindings; function locals also land on the tape (name-only) #467 aggregation, at task scale). Documented in inc 1; truly fixed by T-records + per-task keying if eigsdap demands it.

Non-goals (standing)

Preemption (that's EigenOS M14, kernel-side); replacing OS-thread spawn/channels (parallelism tier stays, with its honest replay wall); wall-clock timeouts in the task API (virtual ticks only — wall-clock re-enters via taped monotonic_* if a program insists); cross-process tasks; work-stealing or any scheduler cleverness that trades determinism for throughput.

Acceptance (from the issue, sharpened)

• A multi-task message-passing program records under EIGS_TRACE and replays byte-identically under EIGS_REPLAY — and also replays byte-identically with no tape at all when it consumes no nondet builtins (the liferaft property, now upstream).
• task_recv legal under replay; OS-thread recv still refuses (proc_* builtins are invisible to trace/replay — EIGS_REPLAY re-runs live subprocess side effects #148 wall intact).
• JIT stays enabled: bench-harness MT-vs-task workload shows task-tier compute at JIT speed while the same workload under spawn runs interpreter-only.
• Extend-vm checklist satisfied (new builtins, VM context field, scheduler state — all N sites).
• SPEC.md "Tasks" section with byte-exact examples; examples/task_pipeline.eigs; deadlock + join-rethrow EM-style corpus checks.
• Increment 5's byte-equal liferaft scenario.

Open decisions (answers change the design)

1. Isolation: this note recommends shared-env + natural fresh-name locality + copied messages, against the issue's "per-task isolated state on every boundary." If hard isolation is wanted anyway, the honest implementation is EigsState-per-task (embed seam), and the layer becomes a thin scheduler over it — different beast, different costs.
2. Mailbox bounds: unbounded v1 vs bounded-with-backpressure (second deadlock shape, more liferaft-faithful).
3. Task-0 exit policy: kill-outstanding (recommended, Go-style, deterministic) vs drain-until-all-done.
4. deadlock kind: confirm extending the Structured runtime errors: catch binds {kind, message, line} with a closed kind set #406 closed set is preferred over overloading limit.

[InauguralPhysicist]

#408 design rulings — prior-art research verdict

Two adversarially-verified research rounds (deep-research harness; round 1 settled the architecture on 23 confirmed claims, round 2 settled mailboxes + JIT on 25 more). Each ruling names the system it's borrowed from. This resolves the four open decisions the design note left for Jon and hardens two technical risks. Net: the design note survives — with one sharpening (scheduler is three-layer, not plain round-robin), one revision (dead-task send is silent-drop, not error), and one confirmed new closed-world hole (#471).

The core claim survives adversarial verification (high confidence)

Every deterministic-simulation / controlled-concurrency system examined — TigerBeetle VOPR, madsim, AWS Shuttle, Microsoft Coyote, PCT (ASPLOS'10) — reproduces a run from at most a seed or a compact choice sequence; none records per-decision scheduling state. VOPR: "we can perfectly reproduce any bugs discovered in testing" from seed + git commit. Coyote — the one system with an exploratory scheduler — records exactly the scheduling/choice sequence and nothing else, which confirms scheduling decisions are the only thing that would ever need recording; a fixed input-independent policy has an empty decision sequence. ⇒ zero new tape record kinds for increments 1–2, no #411 bump. Conditional on the closed-world invariant below.

Decision 1 — Isolation: RULED → shared heap, against the issue text (high confidence)

Borrowed from: TigerBeetle VOPR / FoundationDB environment-injection. DST systems draw the determinism boundary at nondeterminism sources (stubbed clock, network, disk) — VOPR: "all non-deterministic parts of the system are stubbed out. This includes the clock, network, and disk operations" — not at per-task heaps; production code runs unchanged on a shared heap on one thread. Antithesis names "clocks, thread interleaving, and system-provided sources of randomness" as the nondeterminism classes; heap sharing is not on the list.

This confirms the design note's recommendation against the issue's "per-task isolated state on every boundary." Decisive secondary benefit: a Value keeps object identity across a yield, so Env::obs binding identity survives with no reset machinery — the observer window (keyed to the Value object, #262) answers correctly across suspension for free. Per-task-isolated heaps (BEAM-style) would break that and are what no examined DST system chose. Deep-copy at message boundaries stays a policy choice the evidence neither mandates nor forbids — keep it for task_send/task_join results (share-nothing where it's cheap and safe), don't extend it to a per-task heap.

Decision 2 — Scheduler: RULED, SHARPENED beyond the note → three layers (high confidence)

Plain round-robin alone is a poor explorer — every DST framework layers seeded randomization above the deterministic substrate. So, three layers, not one:

• (a) Substrate: serialized one-task-at-a-time. Coyote (TACAS'23): sequentializing workers to run one-at-a-time is "fairly standard in CCT tools"; PCT: "only one thread is scheduled to execute in each step." EigenScript's single-OS-thread cooperative model is this substrate. Borrowed from: Coyote / PCT.
• (b) Default policy: fixed round-robin for the EigenOS window=task consumer (predictable, debuggable).
• (c) Pluggable seeded-strategy hook for the liferaft-DST consumer — seed is an ordinary input, still zero tape records. Borrowed from: Shuttle's PctScheduler/RandomScheduler + madsim's first-in-random-out queue. Coyote's production data: task-aware PCT found 17/21 Azure bugs vs 9 for random walk — "a combination of schedulers is likely required." This hook is what lets liferaft's (t,tie,seq) heap express on top without semantic drift.
• (d) Virtual time: discrete-event, clock jumps when all tasks are blocked. madsim assigns each event "a deterministic occurrence time"; VOPR "can speed up time arbitrarily"; Polar Signals' Go DST advances time only when all goroutines are blocked. task_sleep in ticks over a timestamp heap — subsumes liferaft's t axis exactly. Borrowed from: madsim Timer + TigerBeetle tick model.

Decision 3 — Mailboxes: RULED (high confidence)

• Bounds: unbounded FIFO for v1, borrowed from Erlang — but for a different and better-founded reason than Erlang's. Erlang keeps mailboxes unbounded because "tracking queue length is too expensive under SMP/NUMA" (OTP maintainer, otp#1032) and pushes flow control to userland (pobox/jobs/safetyvalve drop-on-overflow). That cost reason does not apply to us — a single-threaded cooperative scheduler tracks mailbox length in O(1) for free. So unbounded is the right v1 default (matches the note, avoids the second deadlock shape bounded-send introduces), and bounded-with-backpressure is a clean, cheap later add if a consumer forces it — we are not permanently boxed the way BEAM is. Erlang's documented failure mode (unbounded mailbox → node memory-death) is real but doesn't bite the DST/OS consumers at their scale, and determinism means it's reproducible if it ever does.
• Send-to-dead-task: REVISION to the note → silent drop + observability counter, NOT an error. Strong, unanimous prior art: Erlang cast (async) "returns ok immediately, ignoring if the destination ... does not exist"; Akka is at-most-once with undeliverable messages routed to a synthetic /deadLetters actor, "not an error at the send site." Borrowed from: Akka dead-letters + Erlang cast. For a deterministic system silent-drop is also the right call — an error path here would be a nondeterminism magnet. Ruling: task_send to a dead/unknown tid is a deterministic no-op, counted in task_stat (a dead-letter count) so it's observable without being fatal. (Note the asymmetry Erlang draws: synchronous call to a dead process is an error, noproc. We have no synchronous send in the API, so silent-drop is uniformly correct.)
• Deadlock: CONFIRMED → structured error, the deadlock kind. Borrowed from: Go. Go's runtime raises fatal error: all goroutines are asleep - deadlock! on total block; asyncio's alternative — hang silently, no detector (verified: asyncio's debug diagnostics list has nothing for mutual block) — is precisely the anti-pattern for a language whose thesis is loud, reproducible failure. All-tasks-blocked is O(1) detectable on our single thread. This confirms extending the Structured runtime errors: catch binds {kind, message, line} with a closed kind set #406 closed kind set with deadlock — the first designed use of the documented SPEC+DIAGNOSTICS extension process. (Distinct from Node.js's clean exit-0 when the loop legitimately empties with no blocked work — that's normal termination, not deadlock; we mirror it: task 0 ending with no blocked tasks is a clean exit.)

Decision 4 — Task-0 exit: RULED → kill-outstanding (medium confidence)

Design default: when task 0 ends, outstanding tasks are torn down deterministically in spawn order (Go-style — the main goroutine returning ends the program). Node.js offers the contrast (drain until the loop empties); rejected for v1 because "drain" makes program lifetime depend on task quiescence, which is harder to reason about and a hang risk. A task that wanted to keep the program alive should be joined by task 0. Low-stakes, reversible, no strong external precedent forcing either way — flagging as the one ruling made on design taste rather than verified prior art.

Risk A — Arena across suspension: memory-design pass required, tractable (medium confidence)

Literature is thin (no verified external prior art on region allocators coexisting with coroutine suspension — the honest finding). But our own machinery makes it tractable: arena reset is program-driven (arena_reset builtin at explicit arena_mark points), not an automatic per-statement window, so a suspended task's live arena values are only at risk across an explicit reset, not pervasively. The design input is EigenScript's own arena/heap ref-asymmetry rule (the make_list_heap lesson): a suspended task's live stack must not hold bare arena values that another task's arena_reset reclaims. Recommended v1: promote-on-suspend (heap-promote the suspending task's live arena stack values — cost proportional to live stack depth, which is small at a yield point). This is the #1 way increment 1 goes sideways and gets the eigenscript-memory-design pass before any coding. Confirmed as a real gate, not a blocker.

Risk B — JIT + yield: RULED → non-JIT flag on suspending chunks (high confidence)

Borrowed from: V8 (pre-5.6) + PyPy (pre-2.0). "Don't optimize functions with suspension points, keep them in the baseline tier" is not a hack — it was V8's shipping strategy for years: before v5.6 (Dec 2016) generators "could not be optimized at all and ran only in the baseline tier"; PyPy's tracing JIT did not handle greenlet/stacklet context switches until 2.0 beta 2. V8's Crankshaft bailed to a non-optimizing path on every construct it couldn't handle (try-catch, with, generators) — exact prior art for "bail out of the optimizing JIT on unsupported constructs."

The measured cost is the one caution: routing generators through Ignition+TurboFan (i.e. finally optimizing them) tripled generator runtime vs the un-optimized handling. So the non-JIT penalty is real (~3×) on the suspending function body itself — not the whole program. This confirms the note's v1 plan and its deferral logic: V8 lived with un-optimized generators for years before investing in the hard suspend/resume path — measure before building it. One red flag the measurement surfaces: the note assumed yield-containing fns are "typically cold." If a hot loop yields every iteration (a tight producer that task_yields per item), the ~3× lands on the hot path. So the non-JIT flag ships behind a perf-attribution gate (n=5, real task workload) that checks the assumption before we call it free — not an unmeasured "typically cold."

Standing precondition (the load-bearing red flag) — one already actioned

Zero-tape-records holds only under a closed-world invariant: every remaining nondeterminism source taped or forbidden, and the scheduler consulting no hidden input (no pointer addresses, no hash-iteration order in the ready queue / mailbox). Coyote states it: tests must be "deterministic modulo scheduling" or "replay will fail"; madsim: "any missed sources of randomness can undermine the determinism of the entire system"; S2.dev documents schedulers silently broken by HashMap iteration order and ASLR. The no-untraced-FFI veto is now formally load-bearing for #408. Concretely: ready-queue and mailbox ordering must be pure functions of task-id / insertion order, never addresses.

Already actioned: filed #471 — args returns argv untaped (a real closed-world hole, and the likely delivery path for chaos seeds). Recommend an audit pass for any other untaped nondet source before increment 1 lands.

Net effect on the design note

• Confirmed: single-thread + JIT-stays-on; shared heap (against the issue); zero tape records inc 1–2; deadlock extends Structured runtime errors: catch binds {kind, message, line} with a closed kind set #406; non-JIT flag on suspending chunks; virtual-time task_sleep.
• Sharpened: scheduler is three-layer (substrate + round-robin default + seeded-strategy hook), not plain round-robin — the seeded hook is what makes liferaft expressible without drift.
• Revised: send-to-dead-task is silent-drop-plus-counter (Akka/Erlang-cast), not an error.
• Gated: arena-across-suspension → memory-design pass first (promote-on-suspend); non-JIT flag → perf gate on the hot-yield case.
• The four open decisions are now answered. Increment 1 is unblocked pending the memory-design arena pass. Want me to route that pass next, or hold for your read on the rulings?

[InauguralPhysicist]

#408 increment-1 memory plan — arena-across-suspension gate + per-task context sizing

Design pass, code-grounded (arena.c, eigenscript.c, vm.h read before deciding). The rulings addendum flagged arena-lifetime-across-suspension as the top increment-1 risk. Walking the actual layout turned up a second, larger memory risk in the same gate — per-task VM context size — so this memo settles both. Both resolutions preserve object identity across a yield, which the isolation ruling banks on.

What the code actually says (the two facts that drive everything)

1. The arena is OFF by default. g_arena.active is 0 except between an explicit arena_mark and arena_reset builtin call (arena.c:188/219). The default allocation path is heap-refcounted (xcalloc), not arena. make_str/make_list/etc. only draw from the arena when g_arena.active. Ecosystem grep: the only arena users are iLambdaAi's token_train.eigs, EigenGauntlet's memory_lab.eigs, and the arena test files — all single-task ML training / stress loops. Neither #408 consumer (liferaft, EigenOS) touches the arena at all.

2. promote_if_arena is shallow — it promotes NUM and STR/JSON_RAW only (eigenscript.c:771-793); lists, dicts, functions are explicitly left arena-allocated ("complex deep copy... callers should avoid storing arena-allocated complex types"). The full arena→heap escape is val_clone_for_send (the #302 deep clone used at the spawn/thread_join boundary), which walks deep container graphs and mints new objects (changes identity).

3. sizeof(VM) = 1.28 MB (measured): a 512 KB embedded by-value stack[65536] + 800 KB frames[4096] (CallFrame is 200 B). Each thread's VM is one xcalloc(1, sizeof(VM)) (vm.c:354).

Question A — arena across suspension: RULED → FORBID suspension while the arena is active (not promote-on-suspend)

The rulings addendum defaulted to promote-on-suspend. Reading the code flips that recommendation. Promote-on-suspend is the wrong v1 default here, for four grounded reasons:

1. It's incomplete as-built. promote_if_arena skips containers. A suspending task with an arena-allocated list on its stack would need val_clone_for_send — the deep walk the design note explicitly feared, on the 4 GB box.
2. It breaks the property the isolation ruling banks on. Any promotion mints a new heap object → changes Value identity → breaks Env::obs binding-identity-across-yield. The whole reason shared-heap won decision 1 was "observer window survives suspension for free." Deep-cloning on suspend throws that away.
3. Nobody needs the combination. The arena is off by default and used only by single-task ML training loops that have no reason to suspend mid-training-step; the two Deterministic cooperative task layer that rides the trace tape (task_spawn / task_send / task_recv / task_yield) #408 consumers never activate it. Paying deep-clone cost (and identity loss) to support a combination zero consumers exercise is backwards on a constrained box.
4. The arena mark is a single global position, not a stack (mark_block/mark_offset, arena.c:183) — so cross-task arena_mark/arena_reset interleaving is already ill-defined by construction. Forbidding suspension inside the scope sidesteps that entirely instead of layering promotion on top of a fragile primitive.

Ruling: task_yield/task_recv/task_join/task_sleep raise a deterministic structured runtime error (kind value, #406) if g_arena.active at the call. Message: "cannot suspend inside an arena scope (arena_mark…arena_reset); the task layer and the training arena are separate tools". Precedent: Lua's attempt to yield across a C-call boundary — the same "you can't suspend while a certain state is on the stack" restriction — and the design note already forbids yield inside native re-entry (eval, comparators) for the identical structural reason. This adds one more member to a restriction class that already exists; it costs one branch at the suspend point, zero memory, zero promotion, and preserves identity.

Promote-on-suspend is documented as the escape path if a future consumer genuinely needs arena+suspend — but explicitly deferred until one does (YAGNI, and it can't ship without first extending promote_if_arena to containers or accepting the identity loss, a decision no current consumer forces).

Question B — per-task context memory: RULED → copying-stack, not per-task VM structs

The design note's risk #2 ("VM_STACK_MAX-sized stacks per task don't scale") is now quantified: a naive per-task VM is 1.28 MB. 1000 tasks = 1.28 GB of context on a 4 GB box — dead on arrival. Per-task arenas are worse (16 MB ARENA_BLOCK_SIZE × N).

Ruling: tasks share the main VM's embedded execution surface; a suspended task's live slice is copied out to a right-sized heap buffer. (Go's copying-stack model, which the round-1 research named.) Mechanics:

• On suspend, memcpy the task's live stack region [task_bp0, sp) and its CallFrame region [frame0, frame_count) into a heap save-buffer sized to actual depth. On resume, memcpy back. Memory = Σ(live depths), not N × 512 KB — a yield point is typically shallow (the design note's "cost proportional to live stack depth, small in practice" — now literally true: it's a memcpy of the live slice).
• The JIT is untouched — the main embedded stack[]/frames[] stay the single execution surface the JIT addresses by offset. Only save/restore is added, at suspend points (never inside a thunk).
• This composes with ruling A: because suspension-inside-arena is forbidden, every copied slot is either an immediate (NaN-boxed num/null/bool — trivially copyable) or a heap-refcounted Value*. No arena pointers ever enter a save buffer, so the copy is a pure shallow slot move — no deep walk, no promotion, no identity change. Ownership moves with the bytes: the save buffer owns the refs while suspended; the main stack re-owns them on resume. No incref/decref on the move itself.
• Task stack overflow (a task growing past a per-task cap) raises the existing limit kind.

Composition with the cycle collector, refcounts, and teardown (the load-bearing part)

The collector runs single-threaded (tasks keep g_vm_multithreaded at 0 — good). But a suspended task's save-buffer holds the only owned refs to its Env*/Value* while it's off the main stack. Therefore:

• The scheduler's task table (all suspended save-buffers) MUST be a GC root. GC_FOR_EACH_CHILD and root-marking must walk each suspended task's saved slots and frames. This is exactly CLAUDE.md's rule: a new owning edge out of the runtime goes into both the walker and gc_clear_node, and only counted edges may be traversed. Miss it → a closure cycle reachable only from a suspended stack gets swept (frees live memory) or trips the accounting abort. This is the real integration work and the highest-risk step.
• Teardown (task_kill, program exit): decref every saved slot and frame, then free the buffer — sequenced with handle_table_drain. Any channel/task handles a task created compose through the existing drain.
• Refcount ownership of a reified context's stack is unchanged from the main VM's: each stack slot holds one owned ref; moving the slice moves the refs.

Validation gate (with the planted fault that proves it isn't vacuous)

Two instruments, two planted faults:

1. Arena guard (ruling A). Test: a task calls arena_mark, allocates an arena value, then calls task_yield → must raise the value-kind error, deterministically. Planted fault: temporarily delete the g_arena.active guard, run the same program under EIGS_VALGRIND (arena-interior NOACCESS poisoning, arena.c:209) + ASan — a second task's arena_reset then a resume-read of the freed arena slot must trip Valgrind NOACCESS / ASan. Confirm the guard suppresses it and its removal reintroduces it → the guard is load-bearing, not decoration.
2. Save-buffer GC root (ruling B). Test: a task suspends holding the only ref to a heap list that participates in an env↔closure cycle; another task runs and triggers gc_collect; the first task resumes and reads the list. Must be clean under ASan + the leak-tally gate (stays 0; section [87] strict). Planted fault: remove the save-buffer-as-GC-root registration → ASan use-after-free on resume (collector swept the live cycle). Confirm registration prevents it and removal reintroduces the UAF → the root walk is load-bearing.

Both ride the existing gates: make asan + detect_leaks=1, test_arena_ownership.eigs, test_leak_guard.sh, and the EIGS_VALGRIND job. No new instrument needed — the arena already poisons its interior for exactly this class.

Net for increment 1

• Arena hazard: closed by construction — forbid the combination (one branch), don't pay to support what no consumer needs. Preserves identity.
• Context memory: copying-stack — Σ(live depths) not N×1.28 MB, JIT untouched, and it depends on ruling A (no arena ptrs in save buffers → shallow copy).
• One genuinely hard step: registering suspended save-buffers as GC roots and extending GC_FOR_EACH_CHILD — that's where increment 1 can silently free live memory, and it gets the planted-fault gate above.
• No new error kind (reuses value); deadlock from the rulings addendum remains the only Structured runtime errors: catch binds {kind, message, line} with a closed kind set #406 extension.

Increment-1 coding is unblocked with this plan. Recommend the copying-stack + GC-root work land as the first PR (smallest that exercises suspend/resume with a real cycle), arena guard as a trivial rider, before any task_send/mailbox work.

[InauguralPhysicist]

#408 increment-1 implementation blueprint (code-grounded)

Read the full vm_run mechanism, the handle table, and the GC roots before writing this. Every line reference is real. This is the build plan the suspend-core surgery executes against — the architecture is settled, so the surgery starts from a concrete design, not a blank page.

The enabling fact (why suspension is even possible here)

vm_run(chunk, env) (vm.c:2058) sets base_frame = g_vm.frame_count on entry and unwinds back to it to return. EigenScript→EigenScript calls push onto the shared g_vm.frames[] array within one vm_run invocation (CASE(CALL) VAL_FN path) — they do not re-enter vm_run on the C stack. Only builtins that call back (dispatch, eval, sort comparators) re-enter. So a task's entire call depth is a contiguous slice g_vm.frames[base .. frame_count) + g_vm.stack[bp0 .. sp) — save the slice, you've saved the task. This is why the copying-stack model works with zero C-stack capture.

Structures (new, in vm.h + a new src/task.c/.h, or fold into vm.c)

typedef enum { TASK_READY, TASK_RUNNING, TASK_SUSPENDED, TASK_DONE, TASK_DEAD } TaskState;

typedef struct {
    int        id;               /* 1-based; task 0 = main program (implicit, not in table) */
    TaskState  state;
    /* not-yet-started: the entry closure + deep-copied args (val_clone_for_send) */
    Value     *entry_fn;         /* owned VAL_FN */
    Value    **args;             /* owned, deep-copied at spawn */
    int        argc;
    int        started;          /* 0 until first scheduled */
    /* copying-stack save-buffer (valid while TASK_SUSPENDED) */
    EigsSlot  *saved_stack;      /* heap, right-sized: sp - bp0 slots */
    int        saved_stack_len;
    CallFrame *saved_frames;     /* heap, right-sized: frame_count - base slots */
    int        saved_frame_count;
    int        saved_current_line;
    /* completion */
    Value     *result;           /* owned, deep-copied on task end */
    int        has_error;        /* died of uncaught error */
    Value     *error_value;      /* the {kind,message,line} dict, owned */
} Task;

Scheduler state on EigsState (mirrors handle_table — one per state, single-thread):

Task    *tasks[MAX_TASKS];   /* or grow-on-demand; keyed by id */
int      task_count;
int      ready_head, ready_tail; int ready_queue[...];  /* round-robin deque of ids */
int      current_task_id;    /* -1 = main (task 0) */
int      scheduler_active;    /* 0 until first task_spawn */

Register each Task in the handle table as HANDLE_TASK (add to the HandleType enum, eigenscript.h:429) so handle_table_drain reclaims them — the memo's teardown composition.

The two vm_run edit points (the whole suspend mechanism)

Exit — suspend (new label vm_suspend_halt, sibling of vm_error_halt at vm.c:5015). A g_task_suspend flag is set by the suspending builtin (task_yield/recv/join/sleep). CHECK_ERROR() (vm.c:2166) already runs at every DISPATCH(); add a sibling one-line check right after it in the DISPATCH macro (vm.c:2206): if (__builtin_expect(g_task_suspend,0)) goto vm_suspend_halt;. At vm_suspend_halt, do NOT run the vm_error_halt per-frame drain (that frees the frames — we're preserving them). Instead: memcpy g_vm.frames[base_frame..frame_count) → task->saved_frames, g_vm.stack[bp0..sp) → task->saved_stack (bp0 = g_vm.frames[base_frame].bp), record counts + current_line, then restore g_vm.frame_count = base_frame, g_vm.sp = bp0, clear g_task_suspend, return NULL (a distinct sentinel — the scheduler distinguishes suspend from a real null result via g_task_suspend-was-set / a scheduler flag, not the return value).

Ownership on the memcpy: the saved slots/frames own their refs (moved with the bytes) — no incref/decref, sp/frame_count just retreat so the main stack no longer references them. Symmetric on resume.

Entry — resume (new vm_run_resume(Task*) beside vm_run). Instead of pushing a fresh base frame (vm.c:2069), memcpy the saved frames/stack back onto g_vm.frames[base..]/g_vm.stack[bp0..], set frame_count/sp/current_line, take frame = &g_vm.frames[frame_count-1], chunk = frame->chunk, ip = frame->ip, and goto the dispatch loop head (continue from the saved ip — the byte right after the task_yield CALL, so task_yield's already-pushed null return sits on the restored stack and execution flows on). Factor vm_run's dispatch body so both fresh-entry and resume-entry reach the same for(;;)/computed-goto.

The scheduler (sits just above the top-level vm_run — flat C stack)

vm_execute (vm.c:5036) currently: vm_init(); return vm_run(chunk, env);. New:

vm_init();
Value *r = vm_run(chunk, env);          // task 0
if (!scheduler_active) return r;        // no task ever spawned → ZERO change, zero cost
// task 0 suspended (or ended with tasks live): enter the scheduler loop
return scheduler_drive(r);              // round-robin resume/run until all tasks done

scheduler_drive: pop next ready id; if !started → vm_run_fresh(task->entry_fn's chunk, bound env), else vm_run_resume(task); on suspend re-enqueue at tail; on normal return record task->result (deep-copied); on error record task->error_value. Loop until the ready deque drains. All-blocked with none ready = deadlock → raise deadlock kind (the #406 extension). Task 0 ending kills outstanding tasks (kill-outstanding ruling). C-stack depth stays flat: vm_execute → scheduler_drive → vm_run(one task) — one vm_run live at a time regardless of ping-pong count.

GC roots (the load-bearing step — CLAUDE.md's both-sites rule)

The collector (gc_collect, eigenscript.c) marks from roots via GC_FOR_EACH_CHILD + gc_clear_node. A suspended task's saved_stack/saved_frames and a not-started task's entry_fn/args/result hold the only owned refs to those Envs/Values. So:

• Add the scheduler's task table to the root set the collector marks from.
• Extend root-marking to walk, per task: saved_stack[i] (slot_is_ptr → the Value), each saved_frames[i].env / .fn_env / .closure_val / .chunk, plus entry_fn / args[i] / result / error_value.
• This is the "new owning edge goes into both the walker and gc_clear_node" rule. Miss it → a cycle reachable only from a suspended stack is swept (frees live memory) or trips the accounting abort.

Validation (the two planted faults from the memory memo)

1. Arena guard: task marks arena, allocs, calls task_yield → must raise value kind. Planted fault: delete the g_arena.active guard, run under make asan + EIGS_VALGRIND → a second task's arena_reset + resume-read trips NOACCESS/UAF. Guard suppresses; removal reintroduces → load-bearing.
2. Save-buffer GC root: task suspends holding the only ref to a list in an env↔closure cycle; another task triggers gc_collect; first task resumes and reads it. Clean under ASan + leak-tally 0 (section [87] strict). Planted fault: remove the root registration → ASan UAF on resume. Registration prevents; removal reintroduces → load-bearing.

Extend-vm checklist coverage (all N sites)

• 4 builtins registered in register_builtins (builtins.c ~5300s), beside spawn/channels.
• HANDLE_TASK added to HandleType (eigenscript.h:429) + a case in handle_table_drain (builtins.c:4446).
• No new opcode / AST node / CallFrame field → the compiler-walker and frame-init checklists don't apply. (task_yield suspends via a flag checked in the existing DISPATCH, not a new opcode — keeps the opcode ABI and JIT scanner untouched; the JIT simply never runs a chunk mid-suspend because suspending chunks can be non-JIT-flagged per the JIT ruling, or the flag-check lives on the interpreter back-edge only.)
• Replay/trace: task_spawn/yield/alive/join take NO tape records (the deterministic-by-construction ruling) — they are not wrapped in TRACE_NONDET_RET, and that's correct, because scheduling order is a pure function of program order, not host nondeterminism. Documented so a future reviewer doesn't "fix" the missing wrap.

Staging (validated increments within increment 1)

1. Stage 1a — structs + HANDLE_TASK + task_spawn (deep-copy args, hold closure) + task_alive + GC roots for held refs + teardown. task_join/task_yield stubbed. Gate: compiles, spawn+alive test, leak 0. No suspend machinery yet — lowest risk, validates the table/args/GC/teardown independently.
2. Stage 1b — the vm_run suspend/resume surgery + scheduler_drive + real task_yield + task_join (blocking). Gate: two-task ping-pong interleaving test (deterministic order), the GC-root planted fault, leak 0.
3. Stage 1c — arena guard + deadlock detection + SPEC "Tasks" section (executable) + examples/task_pipeline.eigs + both planted faults + full gate. PR.

Each stage is independently green on release + ASan detect_leaks=1. Stage 1b is the hard one and gets its own focused build.

[InauguralPhysicist]

Increment 1a landed (PR #472, d5300e9): the foundational slice — Task struct, HANDLE_TASK, task_spawn/task_alive, leak-clean teardown. No suspend machinery yet (that's 1b). Release 2610/2610, ASan tally-0 (test_tasks.eigs proves an unrun task holding a closure cycle + list args is reclaimed clean at teardown), all 17 CI checks.

One finding from reading the collector that de-risks 1b: the cycle collector is trial-deletion (Bacon–Rajan), not mark-from-roots. A counted ref auto-keeps its object live (its true refcount exceeds the internal-edge count within the collector's candidate set U, exactly as ThreadHandle->fn does) — so held refs need zero root registration. This corrects the blueprint's 'register save-buffers as GC roots' framing: the real 1b requirement is just that refs stay genuinely counted through the copying-stack memcpy-move and that teardown decrefs them — much less delicate than a mark-phase root walk.

Next: increment 1b — the vm_run suspend/resume surgery (vm_suspend_halt label + vm_run_resume re-entry), scheduler_drive above vm_execute, real task_yield/task_join, and the arena-active suspension guard. Focused core-VM session.

[InauguralPhysicist]

Increment 1b landed (PR #473, 485992e): the core suspend/resume surgery. task_yield/task_join now actually work — mid-computation suspension via the copying-stack model, a serialized round-robin scheduler above the outermost vm_execute (flat C stack across any ping-pong count), blocking join that delivers the deep-copied result or re-raises the worker's #406 error across the boundary, kill-outstanding at main's end, the arena-scope suspension guard, and a new deadlock kind (loud, not a hang).

The enabling fact that made it tractable: vm_run is already a flat-frame machine (calls push onto shared frames[] within one vm_run, unwind to base_frame), so a task's whole depth is one contiguous slice — vm_suspend_halt memcpy's it out (memory = live depth, not 1.28MB/task), vm_run_ex's resume path memcpy's it back and continues from the saved ip. The 1a trial-deletion-GC finding held: the save-buffer's counted refs auto-protect, no root registration needed.

Verified: release 2611/2611, ASan detect_leaks=1 2611/2611 tally 0, all 17 CI. test_tasks.eigs (22 checks) pins deterministic interleaving order, join result delivery, deep-copy-arg isolation, cross-boundary error rethrow, and the arena guard; ping-pong / mutual-join deadlock / byte-identical determinism hand-verified ASan-clean.

Increment 1 is functionally complete. Remaining polish for 1c: the two adversarial planted-fault harnesses (arena-UAF under a removed guard; save-buffer lifetime under a removed teardown-decref) and the SPEC executable Tasks section. Then increment 2: mailboxes / task_send / task_recv.