CUDA: fuse muls

[am17an]

Overview

Add fusion for mul operator, same as adds. This is useful for gemma4 models which have a down expert scale which can be fused with mul, this saves a full roundtrip of used_experts x expert_dims in f32 from global memory, so it seems to help PP more than TG surprisingly. Additionally, we can fuse mul-mat + (epilogue), which would benefit all MoE models, however that is not a simple change since we have account for all the different mul-mat-id paths we take.

on a 4090

Model	Test	t/s cuda_fast_hash	t/s cuda_mul_fused	Speedup
gemma4 ?B Q4_0	pp2048	9525.77	10065.06	1.06
gemma4 ?B Q4_0	pp2048@d16384	7084.46	7335.44	1.04
gemma4 ?B Q4_0	pp2048@d32768	5819.71	6010.60	1.03
gemma4 ?B Q4_0	tg128	193.45	194.73	1.01
gemma4 ?B Q4_0	tg128@d16384	173.40	174.48	1.01
gemma4 ?B Q4_0	tg128@d32768	156.48	157.38	1.01

Additional information

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES, continuing on my letting AI run in a loop, it found this for gemma 4, it found some other ones as well but those don't work. I wrote the code

[am17an]

BTW @ggerganov, this model seems to have down_exps_scale even without nvfp4, how would it work if someone made an nvfp4 version of gemma4?

[CISC]

BTW @ggerganov, this model seems to have down_exps_scale even without nvfp4, how would it work if someone made an nvfp4 version of gemma4?

I think we will just have to fix that on conversion.