Would like to use --fast_moe but the v2.5.0 image is missing the acceleration plugin moe

[VassilisVassiliadis]

Is your feature request related to a problem? Please describe.

We would like to use --fast_moe but that commandline parameter requires the "moe" acceleration plugin (

fms-hf-tuning/tuning/config/acceleration_configs/acceleration_framework_config.py

Lines 94 to 102 in 59a72cd

	fast_moe: Annotated[
	FastMoe,
	ConfigAnnotation(
	path="training.moe",
	key="scattermoe",
	standalone=True,
	experimental=True,
	required_packages=["moe"],
	),

).

However, the moe plugin is not in the 2.5.0 image:

fms-hf-tuning/build/Dockerfile

Lines 146 to 151 in 59a72cd

	RUN if [[ "${ENABLE_FMS_ACCELERATION}" == "true" ]]; then \
	python -m pip install --user "$(head bdist_name)[fms-accel]"; \
	python -m fms_acceleration.cli install fms_acceleration_peft; \
	python -m fms_acceleration.cli install fms_acceleration_foak; \
	python -m fms_acceleration.cli install fms_acceleration_aadp; \
	fi

Describe the solution you'd like

Install the moe plugin in the Dockerfile

Describe alternatives you've considered

1. Build a custom image with support for --fast_moe. This is problematic because the community will not have access too it.

Additional context

[willmj]

We also require a manual install of this dependency currently to use accelerated MoE, so will need to include this.