docs: track sub-64 multi-batch runs in results matrix; mark old lineage stale#80
Merged
Conversation
…ld lineage stale Adds four 2026-06-01 native SmolVLM2-256M-text decoder rows (batch=2 @64/64/16, 16/16/4, 32/32/4 and the batch=1 baseline) with ISA/wall/allclose/MAE, surfacing that 32/32/4 sub-64 emulation is ~720x slower than 16/16/4 (3.37h vs 17s) despite fewer ISA lines -- host wall time is not ISA-proportional at sub-64 mlen=32. Adds a lineage note flagging that all rows <= 2026-05-26 are on the superseded exp/roll-attention-head-batch lineage (45c02b8) and their Repro@HEAD is unverified against current main (marked stale rather than re-run).
… rows The earlier version recorded only host wall time, which is box-load-dependent and not an accelerator metric. The 32/32/4 "3.37h / 720x slower / pathologically slow" note was wrong: that was box contention. An identical quiet-box re-run took 106s and the simulated latency is a normal 16.42ms (in line with 16/16/4's 10.03ms). Recover and record simulated latency (executor.now()) for all four 2026-06-01 rows (64/64/16 b2 = 2.39ms, 16/16/4 b2 = 10.03ms, 32/32/4 b2 = 16.42ms, 64/64/16 b1 = 1.28ms); rename the column to Sim lat and document that earlier rows hold host wall time and are stale. The harness now captures sim_latency_ns automatically.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refreshes the emulator results matrix for the native non-packed
batch_size>1decoder work(including true sub-64), and flags the pre-existing rows as stale.
Added — 2026-06-01 native SmolVLM2-256M-text decoder (1L, seq=4)
True sub-64 (
head_dim 64 > mlen) multi-batch previously raisedNotImplementedError.Notable finding
32/32/4 took ~720× longer than 16/16/4 (3.37h vs 17s) despite fewer ISA lines — host wall
time is not ISA-proportional for cycle-accurate sub-64 emulation at mlen=32. Worth flagging for
anyone budgeting sub-64 emulator time.
Stale-lineage note
All rows ≤ 2026-05-26 are pinned to the superseded
exp/roll-attention-head-batchlineage(
45c02b8);mainhas advanced well past it, so theirRepro @HEAD?is unverified. Markedstale rather than re-run (40+ rows, several multi-hour on the shared box).
Companion code PRs: PLENA_Compiler #55 · PLENA_Tools #6 · PLENA_Simulator #79.