MCORE merged this https://github.com/NVIDIA/Megatron-LM/pull/2685 and we should support it.
MCORE merged this NVIDIA/Megatron-LM#2685 and we should support it.