feature: add routed MoE emulator substrate#91
Open
Happymic wants to merge 4 commits into
Open
Conversation
Happymic
force-pushed
the
feat/routed-moe-emulator-substrate
branch
2 times, most recently
from
f285d19 to
79b3b2b
Compare
Happymic
force-pushed
the
feat/routed-moe-emulator-substrate
branch
from
79b3b2b to
8a40efc
Compare
Comment on lines
+30
to
+38
| unsafe { | ||
| tch::Tensor::from_blob( | ||
| data.as_ptr() as *const u8, | ||
| &[data.len() as i64], | ||
| &[], | ||
| tch::Kind::Float, | ||
| tch::Device::Cpu, | ||
| ) | ||
| .internal_to_copy((tch::Kind::Float, tch::Device::Cpu), false) |
Collaborator
There was a problem hiding this comment.
Why unsafe here?
Comment on lines
+282
to
+333
| fn classify_comment(comment: &str, current: StageKind) -> StageKind { | ||
| let text = comment.to_ascii_lowercase(); | ||
| if text.contains("gpt-oss router") | ||
| || text.contains("router token") | ||
| || text.contains("router dot token") | ||
| { | ||
| StageKind::RouterTopk | ||
| } else if text.contains("gpt-oss vram scatter-add") || text.contains("_scatter") { | ||
| StageKind::ScatterCombine | ||
| } else if text.contains("allocate vram matrix step6_pair") && text.contains("_route") { | ||
| StageKind::ExpertRouteWeight | ||
| } else if text.contains("materialize route weight") | ||
| || text.contains("vram matrix mul") | ||
| || (text.contains("true-zero vram rows") && matches!(current, StageKind::ExpertRouteWeight)) | ||
| { | ||
| StageKind::ExpertRouteWeight | ||
| } else if text.contains("step6_device_routing_acc") || text.contains("true-zero vram rows") { | ||
| StageKind::AccumulatorInit | ||
| } else if text.contains("gpt-oss gather token rows") | ||
| || text.contains("gather pair") | ||
| || text.contains("clear gather padding") | ||
| || (text.contains("allocate vram matrix step6_pair") && text.contains("_gather")) | ||
| { | ||
| StageKind::Gather | ||
| } else if text.contains("dynamic expert bias add") { | ||
| StageKind::ExpertBias | ||
| } else if text.contains("allocate vram matrix step6_pair") && text.contains("_sigmoid") { | ||
| StageKind::ExpertActivation | ||
| } else if text.contains("tile row min fp") | ||
| || text.contains("tile row max fp") | ||
| || matches!(current, StageKind::ExpertActivation) | ||
| && (text.contains("vram fill zero") | ||
| || text.contains("vram matrix add") | ||
| || text.contains("vram matrix mul")) | ||
| { | ||
| StageKind::ExpertActivation | ||
| } else if text.contains("dynamic hbm weight prefetch") | ||
| || text.contains("expert_id_to_weight_base") | ||
| { | ||
| StageKind::ExpertWeightAddress | ||
| } else if text.contains("subblock [") { | ||
| StageKind::ExpertWeightPrefetch | ||
| } else if text.contains("sub projection") | ||
| || text.contains("vram block add") | ||
| || text.contains("vram block") | ||
| || (text.contains("allocate vram matrix step6_pair") && !text.contains("_gather")) | ||
| { | ||
| StageKind::ExpertProjection | ||
| } else { | ||
| current | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the transactional-emulator substrate for routed MoE execution.
Changes
Scope notes
PLENA_Compilerworktree state is not part of this PR.Validation
Ran locally from
transactional_emulator:cargo fmt --check cargo test