Question about loss behavior

[MuZhao2333]

Hi,

Thanks for the interesting work! I have a quick question regarding the loss formulation in the drifting model.

When considering the single-temperature case, it seems that gen_scaled and old_gen_scaled are numerically identical (with the only difference being that the former carries gradients). In that case, the loss reduces to something like ||V_norm||^2, where V_norm is the normalized force.

Since the force is normalized, its magnitude is approximately constant, which suggests that the loss value itself may not decrease during training. However, the gradient (which still depends on the unnormalized force) could remain meaningful and drive optimization.

Am I understanding this correctly, or am I missing something in how the loss evolves during training? Also, have you observed in practice that the loss does not decrease (or stays roughly constant) during training?

Thanks in advance!

[phamtrongthang123]

The constant loss behavior aligns with my experience as well. phamtrongthang123/drifting_policy#1