Skip to content

read checkpoint in as float data#13

Open
garrisonhh wants to merge 1 commit intocgbur:mainfrom
garrisonhh:read-checkpoint-as-floats
Open

read checkpoint in as float data#13
garrisonhh wants to merge 1 commit intocgbur:mainfrom
garrisonhh:read-checkpoint-as-floats

Conversation

@garrisonhh
Copy link
Copy Markdown

Resolves #12 by avoiding page alignment in the first place.

@cgbur
Copy link
Copy Markdown
Owner

cgbur commented Jun 10, 2025

Sorry I had missed that this was opened, thank you for the fix. Did you run any benchmarks with this change? I’ll have to do some checks but I think the reason for page alignment was to have perfect divisions of the memory so that all the simd and usage of weights wouldn’t cross/straddle the page boundaries. I’d bet in practice it makes minimal to no difference but I’d want to check before merging. That may have been only a thing when mmaping but it’s been awhile since I’ve looked into the details.

@garrisonhh
Copy link
Copy Markdown
Author

Of course, thanks for your excellent work. I didn't do anything more extensive than running both versions several times with the same input. The token generation speed was basically identical

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mem.page_size has been removed

2 participants