Parallelize the snapshot rebuild across cores (opt-in, validated)

[ptrlrd]

Speeds up the ~80-minute snapshot rebuild by parallelizing the blob walk across cores. The walk reads every run blob (~750k files) and is CPU-bound in pure-Python accumulation, so it's a natural map-reduce.

How

Split _build_cache_data into _accumulate(rows) → raw accumulators and _finalize(raw) → snapshot. The walk can now run either serially (as before) or as a map-reduce: each core accumulates a chunk of runs in its own process, the raw accumulators are merged, and the result is finalized once. Every accumulator is a plain sum (or min/max for the fastest-win / longest-run / biggest-deck records), so the merge is lossless — added merge() to community/charts/encounter stats and the entity-cache merge in run_entity_stats.

Safety — this is the whole point

Given how central this code is, I built it to be provably correct and default-off:

• OFF by default. ENTITY_STATS_REBUILD_WORKERS=1 (serial, unchanged behavior). Set it to N>1 to use N cores. Deploying this PR alone changes nothing — the serial path is byte-identical to today, so no snapshot version bump and no forced rebuild.
• Serial fallback. Any error in the parallel path logs and falls back to serial.
• spawn, not fork. This runs in the leader's refresher thread; forking a multithreaded process can deadlock the child and hang the refresher (the exact failure mode from earlier). spawn re-imports cleanly.
• Prove-it tool. backend/scripts/validate_parallel_rebuild.py runs the full walk both ways in one process and deep-compares (ints exact, floats within 1e-9; serialized-dict lists compared order-insensitively since the merge orders them differently). Run it against the real run data before enabling.

Validated locally

• Refactored serial == original serial: byte-identical (row order + hash seed pinned).
• Parallel (spawn, 4 workers) == serial: identical.
• The validation caught two real merge bugs before they could touch data (charts enc_hist/death_room are int counters not lists; relic act_picks/act_wins arrays) — now fixed.

Rollout

1. Merge + deploy (nothing changes — still serial).
2. In the container: python -m scripts.validate_parallel_rebuild 4 — runs the full walk twice on your real 750k runs and prints IDENTICAL (or the diffs). This takes a while (two full walks) but proves correctness on your data.
3. If IDENTICAL, set ENTITY_STATS_REBUILD_WORKERS=4 and the next rebuild uses 4 cores.

Expected ~3-4x at scale (per-run work dominates over the spawn/pickle overhead once you're past a few thousand runs). Doesn't touch the walk being file-by-file — moving blobs into Mongo is a separate, bigger win if you want it later.