-
Notifications
You must be signed in to change notification settings - Fork 0
dsuyu1/token-optimization
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
The goal is to build a vocabulary that minimizes the number of tokens needed to represent text while keeping the total vocabulary size limited to control memory and embedding size.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published