Skip to content

Define ST_F8_E8M0#3448

Merged
zcbenz merged 1 commit intoml-explore:mainfrom
pcuenca:f8_e8m0
May 5, 2026
Merged

Define ST_F8_E8M0#3448
zcbenz merged 1 commit intoml-explore:mainfrom
pcuenca:f8_e8m0

Conversation

@pcuenca
Copy link
Copy Markdown
Contributor

@pcuenca pcuenca commented Apr 24, 2026

Opening for discussion. I saw #3374, so perhaps this should go there instead. The goal is just to be able to load the native DeepSeek V4 safetensors files, since the attention scales use F8_E8M0.

If this direction is ok, I'm happy to add tests or update docs as needed.

Proposed changes

Define ST_F8_E8M0 so loading safetensor files that include this type succeed. Handling would be deferred to sanitize functions in mlx_lm and other user code.

Checklist

Put an x in the boxes that apply.

  • I have read the CONTRIBUTING document
  • I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the necessary documentation (if needed)

No conversion is performed, this just allows safetensors files to be loaded. I think this should help load the native DeepSeek checkpoints without conversion, as the attention scales use F8_E8M0.
Copy link
Copy Markdown
Collaborator

@zcbenz zcbenz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

Adding test is tricky because the from_fp8/to_fp8 ops assume e4m3, we could extend the APIs to e8m0 but I think maybe we should just add fp8 support instead. So I'm good with no test for this one.

@pcuenca
Copy link
Copy Markdown
Contributor Author

pcuenca commented Apr 25, 2026

So I'm good with no test for this one.

👍 Sounds good!

I searched for type mentions in the docs but I found nothing to update either. Let me know if anything else is missing.

@zcbenz zcbenz merged commit b439659 into ml-explore:main May 5, 2026
16 checks passed
@pcuenca pcuenca deleted the f8_e8m0 branch May 5, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants