fix(im2col): enable the V_SHIFT_V im2col path for non-64-aligned columns (native-dim vision)#57
Merged
Merged
Conversation
…mns (native-dim vision) The shift im2col already loads each patch at its exact pixel_col (patch at [0:K]) and places it with mask + right-shift; only a stale compiler-side assertion required pixel_col to be 64-aligned. That assertion predated the emulator's byte-block HBM gather (dma::transfer_mx_from_hbm, #62), which now walks the load one 64-byte block at a time and handles non-64-aligned starts. Removing the assertion lets the shift path (CONV_USE_SHIFT=1) run for stride-16 patch embedding. Verified: SmolVLM2 vision 64/64/16 1L emulates with allclose 100% PASS. No emulator change. Still opt-in (default stays im2col_asm_no_shift).
booth-algo
force-pushed
the
fix/im2col-shift-native
branch
from
d38a66b to
f037c35
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
V_SHIFT_Vim2col path (asm_templates/im2col_asm.py, opt-in viaCONV_USE_SHIFT=1) already loads each patch at its exactpixel_col, so the K patch elements land at[0:K]of the loaded vector and are placed with a mask + right-shift. The only thing blocking it for SmolVLM2/SigLIP patch embedding (stride=16 → non-64-aligned pixel columns) was a stale compiler-side assertion requiring everyow*strideto be 64-aligned. That assertion predated the emulator's byte-block HBM gather (dma::transfer_mx_from_hbm, #62), which now walks each load one 64-byte block at a time and clamps each read to a block boundary — so a non-64-alignedH_PREFETCH_Vstart is loaded correctly with no realignment needed. Removing the assertion is the entire fix; there is no emulator change.Verified on SmolVLM2 vision (1 layer, both im2col modes) across all native dims —
allclose100% PASS in every cell, and the shift path is both smaller and lower modeled latency:The reduction is largest where the patch-embed im2col is a big share of the program (64/256: −74/−75% lines, −12/−17% sim_lat) and smaller at mlen=16 (−12% lines) where attention/projection dominate the 2.4M-line program — but it is a win on both axes everywhere. Still opt-in: the default stays
im2col_asm_no_shift;CONV_USE_SHIFT=1selects the shift path. This branch is rebased onmain(post #56), so the mlen=16 row compiles via the O(n²) ISA-emit fix.