Skip to content

Support sub-64 MLEN in the emulator (relax 64-byte HBM burst model)#61

Merged
booth-algo merged 1 commit into
fix/aten-test-configurable-mlenfrom
feat/emulator-mlen-sub64
May 28, 2026
Merged

Support sub-64 MLEN in the emulator (relax 64-byte HBM burst model)#61
booth-algo merged 1 commit into
fix/aten-test-configurable-mlenfrom
feat/emulator-mlen-sub64

Conversation

@booth-algo

Copy link
Copy Markdown
Collaborator

Summary

The compiler already compiles at MLEN<64; the emulator's 64-byte HBM burst model blocked end-to-end runs at MLEN=8/16/32. This makes the emulator + Python HBM writer handle tightly-packed sub-64 rows. No compiler changes required, and every change is a no-op at MLEN≥64.

Designed via a 3-phase analyze→synthesize→adversarially-verify workflow; the verifiers caught that a naive assertion-relaxation was unsound (stride/row-layout mismatch), which led to this layout-consistent approach.

Emulator (transactional_emulator/src/main.rs)

  • transfer_mx_from_hbm: relax is_multiple_of(8*64)is_multiple_of(8) (whole-bytes); rewrite the element read to cover [element_addr, +len) by reading the 64-byte-aligned word(s) and extracting the real bytes for any byte offset / boundary span. The addr.is_multiple_of(64) assert is removed (reads are aligned by construction). At MLEN≥64 it reduces to one aligned read per 64 B.
  • transfer_mx_to_hbm: symmetric — relax the store assert and read-modify-write the aligned word(s) so tightly-packed sub-64 neighbours aren't clobbered. At MLEN≥64 each store fills whole 64-byte words (RMW == plain write).

HBM writer (testbench/sim_env_utils.py)

  • Pack element rows tightly (hbm_row_bytes = element_row_bytes; drop the 64-byte burst floor) to match the compiler's tight HBM stride + scale offset.
  • Drop the per-tensor pad-to-64: it shifted the next tensor past its compiler-assigned base whenever a tensor's size wasn't a 64-multiple (only at sub-64 MLEN, e.g. X(16,16)=288 B → 320), making weight prefetches read a zero region. The emulator's MemoryBacked capacity (64-multiple, ~2× preload) zero-covers the final aligned-block read.

Guards (testbench/aten/configurable.py)

  • Remove the MLEN%64 guards in setup_hw / from_args.

Test plan

  • End-to-end PASS at MLEN=8 (ffn), MLEN=16 (ffn, linear, softmax, rms_norm, embedding_add, rope), MLEN=32 (ffn, linear)
  • No regression: ffn(128), softmax(256), linear(256) still pass
  • ruff + cargo fmt clean

Note

Stacked on #60 (configurable MLEN + bugfixes). Rebase onto main after #60 merges.

🤖 Generated with Claude Code

The compiler already compiles at MLEN<64; the emulator's 64-byte HBM burst
model blocked end-to-end runs (MLEN=8/16/32). Make the emulator + HBM writer
handle tightly-packed sub-64 rows, with all changes no-ops at MLEN>=64.

Emulator (transactional_emulator/src/main.rs):
- transfer_mx_from_hbm: relax assert is_multiple_of(8*64) -> is_multiple_of(8)
  (whole-bytes); rewrite the element read to cover [element_addr, +len) by
  reading the 64-byte-aligned word(s) and extracting the real bytes for any
  byte offset / boundary span. Replaces the addr.is_multiple_of(64) assertion
  (reads are aligned by construction). At MLEN>=64 element_addr is 64-aligned
  and len is a multiple of 64, so this reduces to one aligned read per 64 B.
- transfer_mx_to_hbm: symmetric — relax the store assert and read-modify-write
  the aligned word(s) so tightly-packed sub-64 neighbours aren't clobbered. At
  MLEN>=64 each store fills whole 64-byte words (RMW == plain write).

HBM writer (testbench/sim_env_utils.py):
- Pack element rows tightly (hbm_row_bytes = element_row_bytes; drop the 64-byte
  burst floor) to match the compiler's tight HBM stride and scale offset.
- Drop the per-tensor pad-to-64; it shifted the NEXT tensor past its
  compiler-assigned base whenever a tensor's size was not a 64-multiple (only at
  sub-64 MLEN, e.g. X(16,16)=288 B -> 320), which made weight prefetches read a
  zero region. The emulator's MemoryBacked capacity (hbm-size, 64-multiple, ~2x
  preload) zero-covers the final aligned-block read. No-op at MLEN>=64 where
  sizes are already 64-multiples.

Guards (testbench/aten/configurable.py):
- Remove the MLEN%64 guards in setup_hw and HardwareConfig.from_args; sub-64 is
  now supported (8-bit MXFP rows are always byte-aligned).

Verified end-to-end: ffn/linear/softmax/rms_norm/embedding_add/rope pass at
MLEN=8/16/32; ffn(128), softmax(256), linear(256) unchanged (no regression).
No compiler changes were required.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@booth-algo booth-algo merged commit d034fa1 into fix/aten-test-configurable-mlen May 28, 2026
@booth-algo booth-algo deleted the feat/emulator-mlen-sub64 branch May 28, 2026 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant