Assume less about whisper vocab #2000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

jordimas merged 1 commit into OpenNMT:master from sssshhhhhh:whisper

Feb 1, 2026

Contributor

sssshhhhhh commented Jan 31, 2026 •

edited

Loading

Check token ids instead of magic numbers on vocab size. Values are same for all 3 openai tokenizer variations


          Assume less about whisper vocab

f17baa7

Collaborator

jordimas commented Feb 1, 2026

Thanks, more robust approach

jordimas merged commit 57c053a into OpenNMT:master

17 checks passed

sssshhhhhh deleted the whisper branch

February 1, 2026 12:10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet