From 3335bc6b5c6b26b71894ac0329af3befc64e2cf0 Mon Sep 17 00:00:00 2001 From: booth-algo Date: Tue, 2 Jun 2026 00:31:20 +0100 Subject: [PATCH 1/2] docs: track sub-64 multi-batch decoder runs in results matrix; mark old lineage stale Adds four 2026-06-01 native SmolVLM2-256M-text decoder rows (batch=2 @64/64/16, 16/16/4, 32/32/4 and the batch=1 baseline) with ISA/wall/allclose/MAE, surfacing that 32/32/4 sub-64 emulation is ~720x slower than 16/16/4 (3.37h vs 17s) despite fewer ISA lines -- host wall time is not ISA-proportional at sub-64 mlen=32. Adds a lineage note flagging that all rows <= 2026-05-26 are on the superseded exp/roll-attention-head-batch lineage (45c02b8) and their Repro@HEAD is unverified against current main (marked stale rather than re-run). --- doc/EMULATOR_MODEL_CONFIG_MATRIX.md | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/doc/EMULATOR_MODEL_CONFIG_MATRIX.md b/doc/EMULATOR_MODEL_CONFIG_MATRIX.md index fb6737b6..f8ddeccf 100644 --- a/doc/EMULATOR_MODEL_CONFIG_MATRIX.md +++ b/doc/EMULATOR_MODEL_CONFIG_MATRIX.md @@ -1,6 +1,6 @@ # PLENA Emulator Model/Config Results Matrix -Generated: `2026-05-22` | Last updated: `2026-05-25` +Generated: `2026-05-22` | Last updated: `2026-06-02` Simulator checkout: ```text @@ -14,6 +14,15 @@ compiler submodule HEAD: 287f6fb > **Note:** History was squashed on 2026-05-27. All prior sim commits in the > results table now map to `45c02b8`. +> **⚠️ Lineage / staleness note (2026-06-02):** Every row dated **≤ 2026-05-26** was produced on +> the now-superseded `exp/roll-attention-head-batch` lineage (sim `45c02b8`). `main` has since +> advanced well past it — sub-64 single-batch (PLENA_Compiler #51/#54), native non-packed +> **batch_size>1 incl. true sub-64** (#55 / Simulator #79 / Tools #6), and the accelerator +> refactors (#68/#71/#73/#75). Those rows' **`Repro @HEAD?` is therefore unverified against current +> `main`** and should be treated as stale unless re-run. New rows below are pinned to their actual +> producing commits. (Marked stale rather than re-run: the older matrix is 40+ rows, several +> multi-hour on the shared box.) + Key fixes in this update (2026-05-25): - **Golden RoPE bug fix**: golden was using X_embed (un-normed) instead of X_norm for RoPE cos term, causing 82% FAIL on SmolLM2/SmolVLM2 due to peaked-softmax amplification. Now 100% at 1L. - **Q/O projection for mlen=256**: `linear_projection` ISA ops + W_q/W_o weight loading. All 5 decoder models pass 100% at mlen=256 through 10 layers. @@ -45,6 +54,10 @@ All rows sorted by date descending. Allclose bar is 90% unless noted. | Date | Sim commit | Comp commit | Mode | Model | Arch | L | MLEN | BLEN | seq | hidden | inter | batch | ISA | Wall | Allclose | Pass | MAE | Repro @HEAD? | Notes | |---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| 2026-06-01 | `56ed1e7`‡ | `0f55571`‡ | native | SmolVLM2-256M-text | LlamaForCausalLM | 1 | 16 | 4 | 4 | 576 | 1536 | 2 | 1.24M | 17s | **99.07%** | PASS | 0.037 | YES | **true sub-64** head_dim 64>mlen 16 (4 col-blocks). Was `NotImplementedError` before #55. Multi-batch via kernel batch-loop + rpb-strided comparison (Tools #6) | +| 2026-06-01 | `56ed1e7`‡ | `0f55571`‡ | native | SmolVLM2-256M-text | LlamaForCausalLM | 1 | 32 | 4 | 4 | 576 | 1536 | 2 | 1.16M | **3.37h** | **99.67%** | PASS | 0.034 | YES | true sub-64 head_dim 64>mlen 32 (2 col-blocks). ⚠️ Wall NOT ISA-proportional: **720× slower than 16/16/4 (17s)** despite *fewer* ISA lines — cycle-accurate sub-64 mlen=32 is pathologically slow | +| 2026-06-01 | `56ed1e7`‡ | `0f55571`‡ | native | SmolVLM2-256M-text | LlamaForCausalLM | 1 | 64 | 16 | 4 | 576 | 1536 | 2 | 148K | 34s | **97.37%** | PASS | 0.052 | YES | head_dim==mlen batched path. Byte-identical ISA before/after the kernel edit (no-op regression proof). Was a 29% false-negative before the Tools #6 comparison fix | +| 2026-06-01 | `56ed1e7`‡ | `0f55571`‡ | native | SmolVLM2-256M-text | LlamaForCausalLM | 1 | 64 | 16 | 4 | 576 | 1536 | 1 | 73K | — | **99.96%** | PASS | 0.031 | YES | batch=1 baseline (no regression); seq Date: Tue, 2 Jun 2026 00:53:30 +0100 Subject: [PATCH 2/2] docs: report simulated latency (not host wall) for sub-64 multi-batch rows The earlier version recorded only host wall time, which is box-load-dependent and not an accelerator metric. The 32/32/4 "3.37h / 720x slower / pathologically slow" note was wrong: that was box contention. An identical quiet-box re-run took 106s and the simulated latency is a normal 16.42ms (in line with 16/16/4's 10.03ms). Recover and record simulated latency (executor.now()) for all four 2026-06-01 rows (64/64/16 b2 = 2.39ms, 16/16/4 b2 = 10.03ms, 32/32/4 b2 = 16.42ms, 64/64/16 b1 = 1.28ms); rename the column to Sim lat and document that earlier rows hold host wall time and are stale. The harness now captures sim_latency_ns automatically. --- doc/EMULATOR_MODEL_CONFIG_MATRIX.md | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/doc/EMULATOR_MODEL_CONFIG_MATRIX.md b/doc/EMULATOR_MODEL_CONFIG_MATRIX.md index f8ddeccf..73715c0c 100644 --- a/doc/EMULATOR_MODEL_CONFIG_MATRIX.md +++ b/doc/EMULATOR_MODEL_CONFIG_MATRIX.md @@ -52,12 +52,20 @@ Key fixes in this update (2026-05-25): All rows sorted by date descending. Allclose bar is 90% unless noted. -| Date | Sim commit | Comp commit | Mode | Model | Arch | L | MLEN | BLEN | seq | hidden | inter | batch | ISA | Wall | Allclose | Pass | MAE | Repro @HEAD? | Notes | +> **`Sim lat` column = simulated latency** (`executor.now()`, the modeled hardware time — the real +> accelerator metric). For 2026-06-01+ rows this is the emulator's `Simulation completed. Latency …` +> value; host wall time (box-load-dependent, **not** a hardware metric) is noted inline only as +> context. **Earlier rows show host *wall* time in this column** (it was labelled `Wall`) and are +> stale per the lineage note — sim latency was never captured for them because the harness ran the +> emulator at `--log-level warn`, which suppressed the INFO latency line. That harness gap is now +> fixed (`RUST_LOG=warn,transactional_emulator=info`), so `sim_latency_ns` is recorded automatically. + +| Date | Sim commit | Comp commit | Mode | Model | Arch | L | MLEN | BLEN | seq | hidden | inter | batch | ISA | Sim lat | Allclose | Pass | MAE | Repro @HEAD? | Notes | |---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| -| 2026-06-01 | `56ed1e7`‡ | `0f55571`‡ | native | SmolVLM2-256M-text | LlamaForCausalLM | 1 | 16 | 4 | 4 | 576 | 1536 | 2 | 1.24M | 17s | **99.07%** | PASS | 0.037 | YES | **true sub-64** head_dim 64>mlen 16 (4 col-blocks). Was `NotImplementedError` before #55. Multi-batch via kernel batch-loop + rpb-strided comparison (Tools #6) | -| 2026-06-01 | `56ed1e7`‡ | `0f55571`‡ | native | SmolVLM2-256M-text | LlamaForCausalLM | 1 | 32 | 4 | 4 | 576 | 1536 | 2 | 1.16M | **3.37h** | **99.67%** | PASS | 0.034 | YES | true sub-64 head_dim 64>mlen 32 (2 col-blocks). ⚠️ Wall NOT ISA-proportional: **720× slower than 16/16/4 (17s)** despite *fewer* ISA lines — cycle-accurate sub-64 mlen=32 is pathologically slow | -| 2026-06-01 | `56ed1e7`‡ | `0f55571`‡ | native | SmolVLM2-256M-text | LlamaForCausalLM | 1 | 64 | 16 | 4 | 576 | 1536 | 2 | 148K | 34s | **97.37%** | PASS | 0.052 | YES | head_dim==mlen batched path. Byte-identical ISA before/after the kernel edit (no-op regression proof). Was a 29% false-negative before the Tools #6 comparison fix | -| 2026-06-01 | `56ed1e7`‡ | `0f55571`‡ | native | SmolVLM2-256M-text | LlamaForCausalLM | 1 | 64 | 16 | 4 | 576 | 1536 | 1 | 73K | — | **99.96%** | PASS | 0.031 | YES | batch=1 baseline (no regression); seqmlen 16 (4 col-blocks). Was `NotImplementedError` before #55. Multi-batch via kernel batch-loop + rpb comparison (Tools #6). Host ~17s | +| 2026-06-01 | `56ed1e7`‡ | `0f55571`‡ | native | SmolVLM2-256M-text | LlamaForCausalLM | 1 | 32 | 4 | 4 | 576 | 1536 | 2 | 1.16M | 16.42ms | **99.67%** | PASS | 0.034 | YES | true sub-64 head_dim 64>mlen 32 (2 col-blocks). Sim latency normal (cf 16/4 = 10.03ms) — **no pathology**. The original host run took 3.37h **under heavy box contention**; an identical quiet-box re-run was 106s (114× faster). Host wall is unreliable; sim latency is the metric | +| 2026-06-01 | `56ed1e7`‡ | `0f55571`‡ | native | SmolVLM2-256M-text | LlamaForCausalLM | 1 | 64 | 16 | 4 | 576 | 1536 | 2 | 148K | 2.39ms | **97.37%** | PASS | 0.052 | YES | head_dim==mlen batched path. Byte-identical ISA before/after the kernel edit (no-op proof). Was a 29% false-negative before Tools #6. Host ~33s | +| 2026-06-01 | `56ed1e7`‡ | `0f55571`‡ | native | SmolVLM2-256M-text | LlamaForCausalLM | 1 | 64 | 16 | 4 | 576 | 1536 | 1 | 73K | 1.28ms | **99.96%** | PASS | 0.031 | YES | batch=1 baseline (no regression); seq