Rewrite local_task executor and queue as process-based cooperative scheduler.

[benvanik]

Replaces the coordinator-based task executor and DAG-based queue with a process-based cooperative scheduling model. The old system routed every operation through 4 thread hops (wait -> issue -> dispatch shards -> retire), each involving coordinator mutex acquisition, futex wake syscalls, and context switches. The new system eliminates the coordinator entirely and reduces dispatch latency from ~300us to ~5us.

Architecture

Process model: The universal work unit is iree_task_process_t — a cooperative drainable entity with a drain function, suspend count, worker budget, and dependent list. Processes replace the previous task DAG (NOP, CALL, BARRIER, FENCE, DISPATCH, DISPATCH_SHARD types) with a single abstraction.

Executor: Workers scan compute slots and an immediate list instead of going through a coordinator. The coordinator mutex, incoming ready slist, and work-stealing infrastructure are removed. Workers use a Dekker-style sleeping protocol with adaptive spin timeouts based on aggregate worker budget.

Block ISA: A compact bytecode representation for command buffer operations that workers cooperatively execute through a block processor. Supports direct and indirect command buffers, VM dispatch fallback (VMVX), reusable recordings, and deferred fixups for buffers not mappable at recording time.

Queue: Each device queue is a persistent budget-1 process that pops operations from an MPSC ready list. Command buffer recordings are delegated to a persistent budget-N compute process for multi-worker tile distribution. Two-phase completion (eager semaphore signaling + deferred resource release) ensures low-latency signal propagation without use-after-free.

Performance

Dispatch latency on a 96-core EPYC (NUMA-pinned):

• Single worker wake: 5.1us (target was <10us)
• Warm worker reuse: 4.9us
• 4-worker cold wake: 27us (<30us target for non-dominated path)

local-task w=1 overhead vs local-sync: eliminated (was +3.4% / 287M mostly spinning/contention extra instructions on Qwen3.5-4B decode, now noise-equivalent at -1% median). The fast-path empty check on slist pop/flush and the pump loop reorder (drain compute slots before immediate list) are the key contributors.

For a more direct comparison:

  ┌─────────┬──────────────┬──────────────┬─────────────────┐
  │ Threads │ IREE level3  │ ik-llama.cpp │ stock llama.cpp │
  ├─────────┼──────────────┼──────────────┼─────────────────┤
  │ 1       │ 185.0 ms/tok │ 132.7 ms/tok │ 243.0 ms/tok    │
  ├─────────┼──────────────┼──────────────┼─────────────────┤
  │ 4       │ 83.7 ms/tok  │ 73.0 ms/tok  │ 160.4 ms/tok    │
  ├─────────┼──────────────┼──────────────┼─────────────────┤
  │ 16      │ 76.8 ms/tok  │ 101.8 ms/tok │ 142.4 ms/tok    │
  └─────────┴──────────────┴──────────────┴─────────────────┘

IREE now wins at t=16 — ik-llama.cpp degrades past 4 threads (73→102ms) while we keep scaling (84→77ms). Stock llama.cpp doesn't appear to use VNNI at all despite the flags. This being despite the slower single-core time (less optimal quantized matvec).

Correctness

• Full HAL CTS passes under both ASAN and TSAN (7 test suites each)
• Two TSAN races found and fixed:
  • Transient buffer commit was overwriting metadata fields on the worker thread while HAL submission validation read them synchronously. Fixed by not overwriting — the creation-time params are always a conservative-correct subset.
  • Compute process two-phase completion had an ordering gap where the eager completer's atomic exchange on the context raced with the deferred releaser's free. Fixed by consuming the result before the CLOSED fetch_or while the worker is still registered.
• Native queue operations (alloca, dealloca, fill, copy, update, read, write, dispatch) with proper semaphore wait satisfaction, frontier tracking, and error propagation
• Async proactor file I/O with synchronous pread/pwrite fallback when async import fails
• 2+GB file read/write support via per-operation size capping and retry loops across all three proactor backends (io_uring, IOCP, POSIX)

What was removed

The entire old task system: coordinator mutex, task DAG types (NOP/CALL/BARRIER/FENCE/DISPATCH/SHARD), shard pool, post batch routing, worker mailboxes, work stealing, submission batching, and the old task command buffer. The old queue emulation path for fill/copy/update/dispatch is no longer used by local_task (still used by GPU drivers).