Fix _get_batch_logps NaN on fully-masked sequences (DPO)

[finbarrtimbers]

Clamp the divisor in _get_batch_logps when average_log_prob=True so a sequence with every label masked (-100) returns 0.0 instead of NaN (supersedes #1625).

[gemini-code-assist]

Code Review

This pull request fixes a division-by-zero issue in _get_batch_logps within open_instruct/dpo_utils.py by clamping the denominator to 1 when calculating average log probabilities for fully masked sequences. The changes also include regression tests and a changelog update. Feedback suggests using masked_fill to more robustly handle potential -inf values in masked positions, which could otherwise lead to NaN results during multiplication.

[gemini-code-assist]

While clamping the denominator prevents division by zero, the expression (per_token_logps * loss_mask) can still produce NaN if any masked token has a log probability of -inf (since -inf * 0 = NaN in PyTorch). Although rare with standard log_softmax outputs, using masked_fill is a more robust way to ensure masked positions are zeroed out regardless of their value.

Suggested change

	return (per_token_logps * loss_mask).sum(-1) / loss_mask.sum(-1).clamp(min=1)
	return per_token_logps.masked_fill(~loss_mask, 0.0).sum(-1) / loss_mask.sum(-1).clamp(min=1)