perf(server): remove the WAL coordination ceiling + 1M-stream cliff; memory mode 4× faster; crash-sim + 3 recovery fixes by balegas · Pull Request #4675 · electric-sql/electric

balegas · 2026-07-01T23:19:57Z

What & why

The write path stopped scaling well before the hardware did — for three independent
reasons, each found, fixed and validated by a ds-bench campaign on this branch. This PR
removes all three ceilings, plus a seeded crash/fault simulation and the three
correctness fixes it found.

Coordination ceiling (T1a + T1c + T2a) — the WAL path plateaued at ~80% CPU under
saturation, capped by group-commit + durability-wakeup coordination, not compute or
fsync. Baseline: ds-bench/results/run-durable-pool2/FINDINGS.md.
High-cardinality cliff (200k–1M streams) — per-op costs that were supposed to be
amortized "once per stream per checkpoint interval" became per-append costs when
ops/stream/interval dropped below 1. Findings: CARDINALITY_1M.md.
Memory-mode splice path + SSE wake collapse — --durability memory was slower
than wal at every cardinality, and live SSE delivery collapsed past ~16k writes/s.
Merged from perf(server): drop splice append path; per-stream SSE wake coalescing — memory mode 4x faster #4684; findings: MIXED_WORKLOAD_VALIDATION.md.

Changes

1. Remove the coordination ceiling (all in `src/wal/`)

T2a — dedicated committer thread(s). Move the group-commit committer off the shared
Tokio runtime onto a dedicated OS thread and drop the per-commit spawn_blocking hop.
Biggest single win; the runtime round-trip — not fsync — was capping commit cadence.
T1a — lock-free register_dirty. Epoch-gated atomic dirty registration; a stream
already marked this interval does zero map work (no per-append Mutex<HashMap> insert).
T1c — coalesced durability wakeups. Wake only satisfied waiters instead of
broadcasting the shard's entire watch waiter set on every commit.

Supporting: --wal-stats <secs> (dependency-free, runtime-gated contention
telemetry: WAL_CONT / WAL_CKPT per-interval lines; one relaxed atomic load when off)
and --worker-threads <n> (pin the runtime pool size; defaults to
available_parallelism(), which reads the cgroup cpu limit).

2. Fix the high-cardinality write cliff

O(1) checkpoint drain critical section (capture moved outside the shard dirty mutex —
the same mutex every append's epoch transition takes).
Checkpoint fully off the async runtime (spawn_blocking, shards concurrent via
JoinSet), with a memory-resident tails map instead of a re-read/re-sort/rewrite of
the cumulative tails file per tick.
Meta sidecar flush moved from per-append to the checkpoint — the per-append
JSON + create + rename had all workers spinning on the data-dir inode rwsem
(38–46% of ALL server CPU). WAL-staged appends now only mark meta_dirty.
Single registry lookup per append (was two: 2× SipHash + cold DashMap walk at 1M keys).

3. Memory-mode append path + SSE wake coalescing (merged from #4684)

Remove the zero-copy splice append path entirely. Memory mode had routed every
binary append through a fresh open() per append, pipe2 + 2×splice(2), and
socket-readiness parking under the per-stream appender lock, with the tail cache
force-disabled. At sync-sized payloads that loses badly to the buffered path's single
write(2) — the campaign measured memory losing to wal at every cardinality. Memory
mode is now simply the buffered wal path with the WAL stage/wait skipped (~250 lines of
subtle lock-held-await handling deleted); the Linux-only restriction and the forced
tail-cache-off go with it.
Per-stream SSE wake coalescing (StreamSubs::wake_pending latch). Wal's group
commit was acting as an implicit fan-out batcher; without it, one reactor wake (mutex
push + eventfd syscall) per append drowned the reactor. wake_stream now skips
queue+eventfd while a wake is in flight; the reactor clears the latch before
reading the tail (a racing publish is either covered by the flush or re-queues), and
registration resets it so a gen-stale dropped wake can never strand a fresh subscriber.
Idle streams keep per-append wakes (no latency floor); hot streams converge to one wake
per reactor cycle. Helps wal mode at saturation too.

4. Crash/fault simulation + the three correctness fixes it found

Seeded randomized crash/recovery simulation for the WAL path (src/wal/sim_tests.rs,
design in docs/superpowers/specs/2026-07-02-wal-crash-simulation-design.md, writeup in
CRASH_SIM_FINDINGS.md). Each seed drives the real handler path with a random workload,
crashes by dropping the runtime + committers, injects power-loss disk faults constrained
to the documented fault model, reboots through the real recovery sequence, and checks a
no-loss/no-torn/consistency oracle over 4 crash generations. cargo test runs a 4-seed
deterministic smoke (~4s); long hunts scale via DS_SIM_SEEDS/DS_SIM_GENS/
DS_SIM_STEPS. No new dependencies; zero shipped-binary footprint.

It found (each fixed here with a deterministic regression test):

CRITICAL — multi-segment recovery data loss (first seed tried): Shard::open
re-preallocated 1.wal unconditionally, so a sealed first segment grew a zero
tail that replay mis-read as end-of-log (dropping every later segment's acked records,
then truncating the per-stream files to the stale frontier), and a checkpoint-
recycled 1.wal was recreated empty. One 128 MiB segment roll between boots was
the only trigger — plain process crash, no power loss needed. Fix: non-destructive
boot open (FileSegment::open_existing).
Torn un-acked tails exposed on WAL-quiet streams after power loss (seed 20230): a
stream created after the last checkpoint had no truncation proof, so recovery exposed
a torn fragment of a never-acked record. Fix: the .meta sidecar persists a
durable_tail proof (rides on existing fsynced meta writes — no new hot-path fsyncs).
Acked DELETE was not durable (seed 20387, clean crash): the 204 returned while the
unlinks ran on a detached task; a crash after the ack resurrected the stream with all
its data. Fix: unlinks + parent-dir fsync (or the durable soft-delete meta flag) are
awaited before the 204.

Plus a wrong recovery debug_assert (panicked debug builds on a healthy multi-boot
recovery) and stale ARCHITECTURE.md/README.md claims fixed (visibility gating, and the
removed splice path / Linux-only memory-mode restriction).

Measured impact

Coordination ceiling — pool saturation ladder vs the 2026-06-30 baseline:

scale	metric	baseline	combined
32 vCPU, 200k streams	throughput	1.48M @ 52 pods	2.37M @ 40 pods (+60%)
32 vCPU, 200k streams	p99	41 ms	12 ms
32 vCPU, 500k streams	throughput	1.15M @ 52 pods	≥1.78M (+55%)

Direct counters at 4 vCPU (--wal-stats) localize what is removed — the win is
coordination, not lock waits: waiters_woken/commit ~340–378 → ~16–18 (~21× fewer);
fsync/s 83–192 → 1021–1178 (~10× more).

High cardinality — 1M-stream write scaling, 16 vCPU (GKE, 256 B, batch 1):

streams	pods	before	after
1M	32	862k, p99 32 ms, max 405 ms	899k, p99 30.5 ms, max 150 ms
1M	64	—	1.11M ops/s (unsaturated, still climbing)

Memory mode — local kind, 2-CPU server, mixed suites, same binary A/B:

	wal	memory (before)	memory (after)
Write ceiling, 50 streams	23.3k ops/s	29.7k	96.3k (4.1× wal)
Writes w/ 100 SSE subs	13.9k	29.2k, delivery lagging	21.4k
Delivery completeness	2.00 at every level	collapse past ~16k w/s	2.00 through 15k w/s
Delivery p99 @ ~14-15k w/s	18.2 ms	200+ ms	13.2 ms

Read/write interference validation of the wal-mode changes: MIXED_WORKLOAD_VALIDATION.md
(PR #4679).

Changeset

One consolidated patch changeset for @electric-ax/durable-streams-server-rust
(.changeset/wal-write-path-perf-and-recovery-fixes.md) covering the perf work and the
three user-facing fixes.

Reviewer notes

src/wal/shard.rs is the core and deserves an adversarial concurrency review
(lock-free dirty set + wakeup coalescing + committer handoff + checkpoint drain).
The crate's unit/e2e/sim tests cover it; CI is the source of truth.
src/sse_reactor.rs wake coalescing — the clear-then-read ordering is the
correctness argument (see the comment at the latch); worth a careful look.
DS_BENCH_FAST_FSYNC (store.rs, segment.rs) is a bench-only, non-durable fsync
shortcut (plain fsync vs F_FULLFSYNC on macOS) — env-gated, never default on. Flag
if we'd rather split it out.
Docs/provenance on the branch: CONTENTION_INVESTIGATION.md, CARDINALITY_1M.md,
CRASH_SIM_FINDINGS.md, MIXED_WORKLOAD_VALIDATION.md, scripts/contention-repro*.sh.
Can stay or move to a docs-only PR.
Not included: T1b (atomic reserve) — measured inner_wait_load ≈ 0 at 4–16 vCPU,
so it adds lock-restructuring risk for ~no gain. T3 (shared-nothing) — design-only,
confirmed unnecessary at ≤32 cores.
Validation: crate suite 100 passed / 0 failed; 1000-seed × 4-generation crash-sim
sweep clean with all fault types enabled; clippy clean. Read+write mix validated in
docs(bench): mixed read/write interference validation of the perf branch #4679; at the fully saturated memory-mode max level delivery ratio dips to 1.55
(suspected client-bound on the single-VM kind node) — a remote mixed-delivery run
will confirm.

🤖 Generated with Claude Code

Phase 0 of the write-saturation contention investigation. Adds always-on, dependency-free per-shard contention telemetry (independent of the OTLP `telemetry` feature) and a local reproduction harness, so candidate architecture changes can be judged on whether they lift the commit-path ceiling AND drop the contention they target. - ShardStats: inner/dirty lock-wait nanos + acquire counts, records staged, durability waiters woken (src/wal/telemetry.rs). - Instrument the append hot path: register_dirty, reserve_and_stage (both inner acquisitions), publish_durable wakeup fan-out (src/wal/shard.rs). - `--wal-stats <secs>`: runtime gate (one relaxed load when off, no clock reads in a default run) + stderr WAL_CONT emitter with per-interval rates. - scripts/contention-repro.sh: drive the server with the ds-bench pool client, report throughput + CPU + steady-state contention. - DS_BENCH_FAST_FSYNC (bench-only): plain fsync over F_FULLFSYNC on macOS so a RAM-disk data dir gives cheap fsync (the Linux+NVMe lock-bound regime). - CONTENTION_INVESTIGATION.md: hypothesis, method, candidate architectures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

contention-repro-linux.sh runs the server in a cpuset-pinned container with a tmpfs data dir (cheap fdatasync = the Linux+NVMe regime) driven by a separate client container, so the throughput ceiling is not confounded by client/server CPU co-location. Incremental in-container builds via a per-worktree target volume keep iteration fast. Baseline reproduces the findings' signature: a hard ~45-49k ops/s ceiling at ~80% CPU, barely helped by more shards — the gate is the commit + durability- wakeup coordination machinery, not compute or the lock (at 6 cores). Recorded in CONTENTION_INVESTIGATION.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…+ t2a (dedicated committer threads) Combine three orthogonal WAL performance changes that all overlap in src/wal/shard.rs into one build. Hand-integrated onto t2a's shard.rs (structural committer rewrite), then layered t1c and t1a back in; the single-owner files were taken verbatim from each source branch. - t1a (lock-free register_dirty): StreamState.dirty_epoch (store.rs), Shard.dirty_epoch + dirty: Mutex<Vec<Arc<StreamState>>>, epoch-CAS register_dirty, epoch-bump+drain in checkpoint's dirty-lock section. - t1c (coalesced durability wakeups): durable_lsn: AtomicU64 + waiters: Mutex<WaiterReg> (min-heap of oneshots) replacing the watch channel; fast-path/register/re-check wait_durable; prefix-drain publish_durable recording waiters_woken = fired; telemetry.rs doc updates. - t2a (dedicated committer OS threads): CommitSignal (Mutex+Condvar) replacing Notify, synchronous run_committer + commit_once (direct fdatasync, no spawn_blocking), spawn_committer/CommitterHandle, WalSet spawn/stop_committers, main.rs shutdown wiring, recovery/e2e adapters. Integration points: the Shard struct holds all of dirty_epoch+dirty Vec, durable_lsn+waiters, and commit_signal; the constructor inits all of them (watch channel + Notify fully removed). publish_durable does t1c's coalesced drain AND retains sealed_pending, and is called from t2a's synchronous committer thread (atomic store + oneshot fires are thread-safe, no tokio handle). checkpoint reads durable_lsn via the atomic and does t1a's epoch-bump+drain. t1a's post-drain test was adapted to the spawn_committer/stop API. cargo test: 95 passed; 0 failed. cargo build --release + cargo clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The ds-bench pool write-saturation suites pass `--worker-threads 32` to size the tokio runtime independently of the cgroup cpu limit (available_parallelism reads cpu.max, so under a limit it would under-size the pool). Add the flag (defaults to available_parallelism); also used as the default WAL shard count. Required for the combined branch to boot under the canonical remote config. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… + per-append meta flush) At high stream cardinality (200k-1M) nearly every append is its stream's first touch of the checkpoint interval, which turned three amortized costs into per-op costs. Local repro (6 cores, 256 conns, 400k streams): 16.0k -> 44.2k ops/s (+2.8x), p99 144ms -> 27ms. - checkpoint drain: O(1) dirty-lock critical section (take + epoch bump only); the O(touched) tail/file capture ran under the lock and stalled every appender on the shard for 25-140ms per tick. - checkpoint body: run capture + per-stream fdatasyncs + tails/ckpt persist + recycle in ONE spawn_blocking task (was: capture/tails/recycle on the async runtime); shards checkpoint concurrently (JoinSet) instead of serially. - tails map: resident in memory (seeded once from disk); stop re-reading, re-parsing and re-sorting the whole cumulative file every 3s. - meta sidecar: append path no longer debounce-schedules write_meta_sync (JSON + File::create + rename per append -> all workers spinning on the data-dir inode rwsem, ~40% of server CPU in perf at every cardinality). WAL-staged appends just mark meta_dirty; the checkpoint flushes sidecars for drained streams after recycle. Memory-mode appends keep the debounced flush. Producer/access lag bound moves from the 100ms debounce to the checkpoint cadence (already a documented non-durable, lagging flush). - handle_append: drop the redundant second registry lookup for the is_json metric label (2x cold DashMap walk per append at 1M keys). - telemetry: WAL_CKPT per-shard checkpoint phase line (touched/tails/meta counts, capture/fsync/tails/rest/meta timings) behind --wal-stats; repro script summarizes it, plus --tmpfs / --wal-stats flags. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…sults Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzV7avRUMhPqmm1SgU4z1J

…ery gap assert; add randomized crash/recovery simulation Two recovery-path bugs found by the new seeded crash/fault simulation (src/wal/sim_tests.rs), plus the harness itself: 1. Shard::open re-preallocated 1.wal unconditionally. A SEALED first segment (exactly-packed at roll) grew a zero tail that replay read as end-of-log, silently dropping every later segment's acked records and truncating the per-stream files to the stale frontier; a RECYCLED 1.wal was recreated as all-zeros, making replay recover NOTHING. Boot now opens the newest existing segment non-destructively (FileSegment::open_existing) and only preallocates on a fresh dir. Regression tests: e2e_multi_segment_acked_records_after_first_seal_ survive_crash, e2e_recycled_first_segment_acked_records_survive_crash. 2. recovery.rs debug_assert claimed a stream rebuilt purely from replay must start at file_base — false after a previous boot's recover+reset cycle (post-boot WAL records legitimately start at the recovered tail with no persisted-tails entry until the next checkpoint). Debug builds panicked on a healthy recovery. The assert now checks the real invariant: no hole between the pre-replay file end and the first replayed record. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzV7avRUMhPqmm1SgU4z1J

… in ARCHITECTURE.md ARCHITECTURE.md still described pre-fsync tail publication; the code gates reader visibility on durability (publish_durable_tail after wait_durable_lsn, PROTOCOL.md §4.1). Also clippy nits in the new sim/recovery code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzV7avRUMhPqmm1SgU4z1J

…car durable_tail proof Confirmed by the crash sim (seed 20230): a stream created after the last checkpoint whose only in-flight append was torn by power loss had NO truncation proof — zero surviving WAL records, no checkpoint tails entry — so the sidecar pass trusted tail = file size and exposed the torn fragment to readers (the C1 shape the WAL exists to prevent). The .meta sidecar now persists durable_tail, captured from Shared.durable_tail (only ever advances post-fsync, so captured values are honest) and riding along on writes that already happen: fsynced at create/close, refreshed by the checkpoint meta flush and by recovery's reconcile — zero new hot-path fsyncs. Recovery seeds every stream's frontier with max(meta proof, checkpoint tails, replayed ends); a fast path skips streams already exactly at their frontier so 1M-stream boots add no I/O; the proof is durably re-persisted before reset_after_recovery wipes the WAL + tails file. Old sidecars without the field keep the previous trust-the-file-size behavior (serde default). Regression test: e2e_wal_quiet_stream_torn_unacked_tail_truncated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzV7avRUMhPqmm1SgU4z1J

Found by the crash sim (seed 20387, clean crash, no fault injection): handle_delete acked 204 while the file + sidecar unlinks ran on a detached spawn_blocking task with no parent-dir fsync — a crash after the ack left both files on disk and the stream resurrected with all its data on reboot. DELETE now awaits delete_or_soft_delete_durable (synchronous unlinks + one parent-dir fsync, or the durable soft-delete meta write) and acks 500 on failure. The expiry sweep keeps the detached non-durable variant (re-expiry on next access is harmless). Regression test: e2e_acked_delete_is_durable_no_resurrection_after_crash. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzV7avRUMhPqmm1SgU4z1J

codecov · 2026-07-02T09:56:48Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 29.99%. Comparing base (ed04ee4) to head (f54bc38).
⚠️ Report is 7 commits behind head on main.
✅ All tests successful. No failed tests found.

❗ There is a different number of reports uploaded between BASE (ed04ee4) and HEAD (f54bc38). Click for more details.

HEAD has 9 uploads less than BASE

Flag BASE (ed04ee4) HEAD (f54bc38)

elixir 1 0

electric-telemetry 1 0

unit-tests 13 10

typescript 12 10

packages/agents-runtime 1 0

packages/agents-server 1 0

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #4675       +/-   ##
===========================================
- Coverage   60.15%   29.99%   -30.16%     
===========================================
  Files         410      258      -152     
  Lines       44348    18172    -26176     
  Branches    12582     6308     -6274     
===========================================
- Hits        26677     5451    -21226     
+ Misses      17593    12656     -4937     
+ Partials       78       65       -13

Flag	Coverage Δ
electric-telemetry	`?`
elixir	`?`
packages/agents	`72.64% <ø> (ø)`
packages/agents-mcp	`77.70% <ø> (ø)`
packages/agents-mobile	`80.67% <ø> (ø)`
packages/agents-runtime	`?`
packages/agents-server	`?`
packages/agents-server-ui	`8.32% <ø> (ø)`
packages/electric-ax	`51.06% <ø> (ø)`
packages/experimental	`87.73% <ø> (ø)`
packages/react-hooks	`86.48% <ø> (ø)`
packages/start	`82.83% <ø> (ø)`
packages/typescript-client	`91.75% <ø> (-0.08%)`	⬇️
packages/y-electric	`56.05% <ø> (ø)`
typescript	`29.99% <ø> (-30.01%)`	⬇️
unit-tests	`29.99% <ø> (-30.16%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@10k

…es per stream The 2026-07-02 ds-bench campaign showed --durability memory LOSING to wal on writes at every cardinality (575k vs 816k @10k streams; 1.33M vs 2.05M @500k on 4 CPUs) and its SSE delivery collapsing past ~16k writes/s (16-20k del/s, p50 200ms+, against 40-62k writes — same client sustained 65k del/s under wal). Two causes, both fixed here: 1. --durability memory silently routed every binary append through the splice intercept: a fresh open() per append, pipe2 + 2x splice(2) when the body split past the head buffer, socket-readiness parking under the per-stream appender lock, and the tail cache force-disabled. At sync-sized payloads (256B) that fixed overhead loses to the buffered path's single write(2) plus an amortized group-commit fdatasync share. Most workloads have small payloads — remove the splice path entirely rather than size-gate it: memory mode is now the buffered wal path with the WAL stage/wait skipped, the Linux-only restriction is gone, and the tail cache is orthogonal again. 2. Memory mode sent one reactor wake (mutex push + eventfd write) per append — wal's group commit had been acting as an implicit fan-out batcher, and without it the reactor drowned. StreamSubs gains a wake_pending latch: wake_stream skips queue+eventfd while a wake is in flight; the reactor clears the latch BEFORE reading the tail (clear-then-read, so a racing publish is either covered by the flush or re-queues); registration resets it so a gen-stale drop can never strand a fresh subscriber. Idle streams keep per-append wakes (no added latency); hot streams converge to one wake per reactor cycle. Benefits wal mode at saturation too. cargo test: 100 passed. Bench validation to follow (ds-bench mixed suites). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…nch (#4679) ## What Adds `packages/durable-streams-rust/MIXED_WORKLOAD_VALIDATION.md` — the evidence record for running ds-bench's new **mixed read/write interference workload** (concurrent writers + paced catch-up readers + SSE subscribers on shared streams) against this branch's server build. Local kind validation (server pinned to 2 CPU); the remote GKE run comes when the workload is promoted into the full suite. ## Why Before adding the mixed workload to the standard benchmark matrix we validated its two premises against the perf branch, and the results are worth recording with the branch they measured: - **Bounded catch-up read load does not cost write throughput.** A 60%-of-ceiling pinned write load (17.75k ops/s) holds to within noise under up to 128 paced readers pulling 123 MiB/s of replay bandwidth; only p99 tails couple (6.5 → 53 ms at the top level). Unpaced hot-loop readers (adversarial mode) fair-share writes down instead — and in both modes the server never sheds load (zero 429/503), which is worth a deliberate decision at some point. - **Live SSE delivery latency tracks commit latency + 1–3 ms all the way to write saturation**, in both `wal` and `memory` durability. The "~14 ms delivery floor" seen in the first run was WAL fsync cost, **not** the SSE reactor — no server-side change came out of this validation. - Baseline: this branch's build is ~1.5× faster than the previous build at 50-stream write saturation on the same box (29.7k vs 19.6k appends/s). Harness, suites and full result grids: `electric-sql/ds-bench` @ `3bcc0f2` (`suites/mixed-{cal,writes,delivery}-local.json`, `results/mixed-*/FINDINGS.md`). ## No code changes Docs only — the validation found nothing on this branch needing a fix. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…to perf/combined-t1a-t1c-t2a

…angesets into one The splice append path removal (#4684) left ARCHITECTURE.md and README.md describing a zero-copy socket→file path and a Linux-only restriction that no longer exist. Also folds the four per-fix changesets into a single patch changeset covering the whole branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ettier-format docs The splice append path deletion removed the last Linux user of the engine_raw AtomicBool import; the cfg-gated import is invisible to native macOS checks but fails clippy -D warnings on Linux CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

balegas and others added 7 commits July 1, 2026 00:14

docs(wal): cardinality-fix findings + 16 vCPU 1M-stream validation re…

0411530

…sults Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chore: changeset for the wal cardinality perf fixes

f6bc68b

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

balegas mentioned this pull request Jul 2, 2026

perf(wal): fix the high-stream-cardinality write cliff (1M streams → 1.11M ops/s @ 16 vCPU) #4676

Merged

balegas and others added 5 commits July 2, 2026 07:59

docs: design for WAL crash/recovery randomized simulation

15ac85e

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzV7avRUMhPqmm1SgU4z1J

balegas and others added 4 commits July 2, 2026 22:41

Merge remote-tracking branch 'origin/perf/memory-mode-drop-splice' in…

d4622da

…to perf/combined-t1a-t1c-t2a

balegas changed the title ~~perf(wal): remove the write-saturation coordination ceiling (T1a + T1c + T2a)~~ perf(server): remove the WAL coordination ceiling + 1M-stream cliff; memory mode 4× faster; crash-sim + 3 recovery fixes Jul 3, 2026

balegas force-pushed the perf/combined-t1a-t1c-t2a branch from f54bc38 to d80bb1b Compare July 3, 2026 11:32

samwillis approved these changes Jul 6, 2026

View reviewed changes

chore: remove crash-sim spec doc from PR

41bb37a

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf(server): remove the WAL coordination ceiling + 1M-stream cliff; memory mode 4× faster; crash-sim + 3 recovery fixes#4675

perf(server): remove the WAL coordination ceiling + 1M-stream cliff; memory mode 4× faster; crash-sim + 3 recovery fixes#4675
balegas wants to merge 18 commits into
mainfrom
perf/combined-t1a-t1c-t2a

balegas commented Jul 1, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jul 2, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

balegas commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What & why

Changes

1. Remove the coordination ceiling (all in src/wal/)

2. Fix the high-cardinality write cliff

3. Memory-mode append path + SSE wake coalescing (merged from #4684)

4. Crash/fault simulation + the three correctness fixes it found

Measured impact

Changeset

Reviewer notes

Uh oh!

codecov Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

balegas commented Jul 1, 2026 •

edited

Loading

1. Remove the coordination ceiling (all in `src/wal/`)

codecov Bot commented Jul 2, 2026 •

edited

Loading