Skip to content

Enable batch_size>1 native decode (incl. true sub-64); bump compiler + tools#79

Merged
booth-algo merged 1 commit into
mainfrom
feat/multibatch-decoder
Jun 2, 2026
Merged

Enable batch_size>1 native decode (incl. true sub-64); bump compiler + tools#79
booth-algo merged 1 commit into
mainfrom
feat/multibatch-decoder

Conversation

@booth-algo

Copy link
Copy Markdown
Collaborator

Summary

Enables batch_size>1 native decode end-to-end, including true sub-64
(mlen 16/32 < head_dim 64). Pairs the PLENA_Compiler decoder/kernel fix with the
PLENA_Tools comparison reader.

  • emulator_runner.py: forward rows_per_batch/active_seq_per_batch from
    comparison_params to compare_vram_with_golden, so a batch>1 output compares
    only the active (rpb-strided) rows rather than all physical rows (which include
    tile-align padding and false-negatived the match rate, e.g. 29% on a ~99%-correct run).
  • Bump PLENA_Compilerfeat/multibatch-decoder (batch>1 MHA decoder + head_dim>mlen kernel).
  • Bump PLENA_Toolsfeat/multibatch-comparison (rpb-strided active-row reader).

Companion PRs

Submodule pointers reference those PR branches; they'll be re-bumped to the merged
main commits once the companions land.

Test plan (numerical, allclose PASS)

  • batch=2 @64/64/16 seq=4 = 97.37% (was a 29% false-negative before the harness fix)
  • batch=2 @16/16/4 seq=4 = 99.07%, @32/32/4 = 99.67% (sub-64, no NaN)
  • batch=1 @64/64/16 seq=4 = 99.96% (no regression)

Forward rows_per_batch/active_seq_per_batch from comparison_params to
compare_vram_with_golden so a batch>1 decoder output compares only the active
(rpb-strided) rows. Bump PLENA_Compiler and PLENA_Tools to their merged main
commits: the batch>1 MHA decoder + head_dim>mlen kernel, and the rpb-strided
comparison reader.
@booth-algo booth-algo force-pushed the feat/multibatch-decoder branch from 230042a to 86e7a3e Compare June 2, 2026 01:44
@booth-algo booth-algo merged commit e7c10bd into main Jun 2, 2026
4 checks passed
@booth-algo booth-algo deleted the feat/multibatch-decoder branch June 2, 2026 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant