Skip to content

feat(tasks): #408 increment 1b — suspend/resume, scheduler, join, deadlock#473

Merged
InauguralPhysicist merged 1 commit into
mainfrom
feat-408-tasks-inc1b
Jul 7, 2026
Merged

feat(tasks): #408 increment 1b — suspend/resume, scheduler, join, deadlock#473
InauguralPhysicist merged 1 commit into
mainfrom
feat-408-tasks-inc1b

Conversation

@InauguralPhysicist

Copy link
Copy Markdown
Collaborator

Refs #408. The core of the cooperative task layer — real mid-computation suspension, a serialized round-robin scheduler, blocking task_join, and loud deadlock detection. Builds on the 1a foundation (#472).

The enabling fact

vm_run is a flat-frame machine: EigenScript→EigenScript calls push onto the shared frames[] array within one vm_run and unwind to base_frame — they don't re-enter vm_run on the C stack. So a task's entire call depth is one contiguous slice frames[0..fc) + stack[0..sp). Save it, you've saved the task. No C-stack capture.

Core surgery (vm.c)

  • vm_run_ex(chunk, env, resume) — a resume path restores a suspended task's copying-stack slice and continues from the saved ip (vm_resume_dispatch label) instead of pushing a fresh base frame. vm_run() is the fresh-entry wrapper (every non-task call is unchanged).
  • vm_suspend_halt — sibling of vm_error_halt but the opposite: it PRESERVES the frames, memcpy'ing the live slice into the task's right-sized save-buffer (memory = live depth, not 1.28 MB/task) and retreating the VM to empty. Refs move with the bytes — no incref/decref, no deep walk.
  • Suspend actioned at the CASE(CALL) builtin site, gated to base_frame==0 — legal only at a task's top level; inside a nested eval/comparator/import it raises value (the C stack can't be captured there — Lua's "yield across a C boundary").
  • scheduler_trampoline sits just above the outermost vm_execute — C-stack depth stays flat (vm_execute → scheduler → one vm_run) across any number of ping-pongs. Task 0 (main) is a schedulable peer; it ending kills outstanding tasks (kill-outstanding, Go-style).
  • JIT gated off while the scheduler is active (the non-JIT-suspending-code ruling, coarsened for v1 — V8 shipped un-optimized generators for years) so no thunk runs mid-suspend.

Semantics

  • task_yield of null — cooperative reschedule; task_join of id — block for the deep-copied result, or re-raise the worker's uncaught error as its Structured runtime errors: catch binds {kind, message, line} with a closed kind set #406 {kind, message, line} (verified: a worker's undefined_name surfaces at the join). Joining a finished task returns immediately.
  • New deadlock kind (extends the Structured runtime errors: catch binds {kind, message, line} with a closed kind set #406 closed set per the rulings): all tasks blocked with none runnable is loud, exit 1 — not a silent hang (the asyncio anti-pattern the research flagged).
  • Arena guard: suspending inside arena_markarena_reset is a value error — the memory-design pass ruled forbid-over-promote (promoting would break observer identity + need a deep walk).
  • GC: the trial-deletion collector auto-protects the save-buffer's counted refs (the 1a finding held) — no root registration; teardown decrefs a killed task's saved slice.

Verification

  • Release 2611/2611; ASan+UBSan detect_leaks=1 2611/2611, leak tally 0
  • test_tasks.eigs (22 checks): deterministic interleaving order, join result delivery, deep-copy-arg isolation (task saw the copy, not the caller's later mutation), error rethrow across the boundary, re-join of a finished task, arena guard
  • Hand-verified + ASan-clean: two-task ping-pong, blocking join collecting results, join error rethrow, byte-identical determinism across runs, mutual-join deadlock detection
  • doc-drift, freestanding (EigenOS profile), jit-smoke green; examples/task_pipeline.eigs; BUILTINS/DIAGNOSTICS/CHANGELOG updated

Not in this PR

Mailboxes / task_send (increment 2), virtual time (task_sleep), the pluggable seeded-strategy hook for the DST consumer, the two adversarial planted-fault harnesses (arena-UAF, save-buffer lifetime under a planted removed-guard) — those ride increment 1c / determinism-verification. The SPEC "Tasks" executable section also lands in 1c.

🤖 Generated with Claude Code

…dlock

The core of the cooperative task layer: real mid-computation suspension via
the copying-stack model, a serialized round-robin scheduler, blocking
task_join, and loud deadlock detection. Deterministic by construction —
byte-identical across runs, no trace records.

The enabling fact: vm_run is a flat-frame machine (EigenScript calls push
onto the shared frames[] within one vm_run and unwind to base_frame), so a
task's entire call depth is one contiguous slice — save it, you've saved the
task. No C-stack capture.

Core surgery (vm.c):
- vm_run -> vm_run_ex(chunk, env, resume): a resume path restores a suspended
  task's copying-stack slice and continues from the saved ip (vm_resume_
  dispatch label) instead of pushing a fresh base frame.
- vm_suspend_halt: sibling of vm_error_halt, but the OPPOSITE — it PRESERVES
  the frames, memcpy'ing the live [0,fc)/[0,sp) slice into the task's right-
  sized save-buffer (memory = live depth, not 1.28 MB/task) and retreating
  the VM to empty. Refs move with the bytes — no incref/decref, no deep walk.
- The suspend request is actioned at the CASE(CALL) builtin site, gated to
  base_frame==0: suspension is only legal at a task's top level; inside a
  nested eval/comparator/import (base_frame>0) it raises (value) — the C
  stack there can't be captured (Lua's 'yield across a C boundary').
- scheduler_trampoline sits just above the outermost vm_execute, so C-stack
  depth stays flat (vm_execute -> scheduler -> one vm_run) across any number
  of ping-pongs. Task 0 (main) is a schedulable peer; it ending kills
  outstanding tasks (kill-outstanding, Go-style).
- JIT gated off while the scheduler is active (the non-JIT-suspending-code
  ruling, coarsened for v1) so no thunk runs mid-suspend.

Semantics:
- task_yield: cooperative reschedule; task_join(id): block for the result
  (deep-copied) or re-raise the worker's uncaught error as its #406
  {kind,message,line}; joining a finished task returns immediately.
- New EK_DEADLOCK kind (extends the #406 closed set per the rulings): all
  tasks blocked with none runnable is loud, not a hang.
- Arena guard: task_yield/task_join inside arena_mark..arena_reset is a value
  error — a suspended task's arena values would be reclaimed by a sibling's
  arena_reset; the memory-design pass ruled forbid-over-promote.
- GC: the trial-deletion collector auto-protects the save-buffer's counted
  refs (they exceed the internal-edge count in U) — no root registration; the
  1a finding held. Teardown decrefs a killed task's saved slice (leak 0).

Verification: release 2611/2611, ASan+UBSan detect_leaks=1 2611/2611 tally 0.
test_tasks.eigs (22 checks) covers deterministic interleaving order, join
result delivery, deep-copy-arg isolation, error rethrow across the boundary,
re-join of a finished task, and the arena guard; ping-pong / join / deadlock
/ determinism verified by hand and ASan-clean. examples/task_pipeline.eigs.
doc-drift, freestanding, jit-smoke green. BUILTINS/DIAGNOSTICS/CHANGELOG updated.

Refs #408

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@InauguralPhysicist InauguralPhysicist merged commit 485992e into main Jul 7, 2026
17 checks passed
@InauguralPhysicist InauguralPhysicist deleted the feat-408-tasks-inc1b branch July 7, 2026 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant