GCE implementation

[sunniesuhyoung]

Why is GCE implemented as

F.cross_entropy(logits, targets, reduction='none') * (Yg.squeeze().detach()**self.q)*self.q

instead of (1 - Yg.squeeze().detach()**self.q) / self.q ?

[mvandenhi]

Hey Sunnie,
in your formulation you mustn't use detach(), as you still need to take the derivative there. While in their formula, the authors notice that the derivative of the GCE consists of the CE-derivative as well as the "loss_weight"(see p.5 of their paper). Hence, by fixing and detaching the "loss_weight" from the graph, for the backward pass they now only need to compute the gradient the cross_entropy function which is probably much faster.
I don't understand why they multiply by the constant q though, do you?
Best,
Moritz