Skip to content

Commit 3774aa3

Browse files
authored
[PyTorch] Add ops for MoE grouped MLP (#2664)
* Add ops for MoE grouped MLP Signed-off-by: Tim Moon <tmoon@nvidia.com> * Move testing utility functions to util submodule Signed-off-by: Tim Moon <tmoon@nvidia.com> * Tweak docs Signed-off-by: Tim Moon <tmoon@nvidia.com> * Change order of tensor compatibility checks in noop_cat Review suggestion from @ptrendx. Signed-off-by: Tim Moon <tmoon@nvidia.com> * Add support for GLU interleaving in clamped SwiGLU Signed-off-by: Tim Moon <tmoon@nvidia.com> --------- Signed-off-by: Tim Moon <tmoon@nvidia.com>
1 parent 93d51c8 commit 3774aa3

7 files changed

Lines changed: 1744 additions & 101 deletions

File tree

0 commit comments

Comments
 (0)