Support sub-64 MLEN in the emulator (relax 64-byte HBM burst model)#61
Merged
booth-algo merged 1 commit intoMay 28, 2026
Merged
Conversation
The compiler already compiles at MLEN<64; the emulator's 64-byte HBM burst model blocked end-to-end runs (MLEN=8/16/32). Make the emulator + HBM writer handle tightly-packed sub-64 rows, with all changes no-ops at MLEN>=64. Emulator (transactional_emulator/src/main.rs): - transfer_mx_from_hbm: relax assert is_multiple_of(8*64) -> is_multiple_of(8) (whole-bytes); rewrite the element read to cover [element_addr, +len) by reading the 64-byte-aligned word(s) and extracting the real bytes for any byte offset / boundary span. Replaces the addr.is_multiple_of(64) assertion (reads are aligned by construction). At MLEN>=64 element_addr is 64-aligned and len is a multiple of 64, so this reduces to one aligned read per 64 B. - transfer_mx_to_hbm: symmetric — relax the store assert and read-modify-write the aligned word(s) so tightly-packed sub-64 neighbours aren't clobbered. At MLEN>=64 each store fills whole 64-byte words (RMW == plain write). HBM writer (testbench/sim_env_utils.py): - Pack element rows tightly (hbm_row_bytes = element_row_bytes; drop the 64-byte burst floor) to match the compiler's tight HBM stride and scale offset. - Drop the per-tensor pad-to-64; it shifted the NEXT tensor past its compiler-assigned base whenever a tensor's size was not a 64-multiple (only at sub-64 MLEN, e.g. X(16,16)=288 B -> 320), which made weight prefetches read a zero region. The emulator's MemoryBacked capacity (hbm-size, 64-multiple, ~2x preload) zero-covers the final aligned-block read. No-op at MLEN>=64 where sizes are already 64-multiples. Guards (testbench/aten/configurable.py): - Remove the MLEN%64 guards in setup_hw and HardwareConfig.from_args; sub-64 is now supported (8-bit MXFP rows are always byte-aligned). Verified end-to-end: ffn/linear/softmax/rms_norm/embedding_add/rope pass at MLEN=8/16/32; ffn(128), softmax(256), linear(256) unchanged (no regression). No compiler changes were required. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The compiler already compiles at MLEN<64; the emulator's 64-byte HBM burst model blocked end-to-end runs at MLEN=8/16/32. This makes the emulator + Python HBM writer handle tightly-packed sub-64 rows. No compiler changes required, and every change is a no-op at MLEN≥64.
Designed via a 3-phase analyze→synthesize→adversarially-verify workflow; the verifiers caught that a naive assertion-relaxation was unsound (stride/row-layout mismatch), which led to this layout-consistent approach.
Emulator (
transactional_emulator/src/main.rs)transfer_mx_from_hbm: relaxis_multiple_of(8*64)→is_multiple_of(8)(whole-bytes); rewrite the element read to cover[element_addr, +len)by reading the 64-byte-aligned word(s) and extracting the real bytes for any byte offset / boundary span. Theaddr.is_multiple_of(64)assert is removed (reads are aligned by construction). At MLEN≥64 it reduces to one aligned read per 64 B.transfer_mx_to_hbm: symmetric — relax the store assert and read-modify-write the aligned word(s) so tightly-packed sub-64 neighbours aren't clobbered. At MLEN≥64 each store fills whole 64-byte words (RMW == plain write).HBM writer (
testbench/sim_env_utils.py)hbm_row_bytes = element_row_bytes; drop the 64-byte burst floor) to match the compiler's tight HBM stride + scale offset.X(16,16)=288 B → 320), making weight prefetches read a zero region. The emulator'sMemoryBackedcapacity (64-multiple, ~2× preload) zero-covers the final aligned-block read.Guards (
testbench/aten/configurable.py)MLEN%64guards insetup_hw/from_args.Test plan
Note
Stacked on #60 (configurable MLEN + bugfixes). Rebase onto main after #60 merges.
🤖 Generated with Claude Code