diff --git a/CHANGELOG.md b/CHANGELOG.md index 44283b53..12875a78 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,13 @@ changelog does not include internal changes that do not affect the user. ### Added +- Added `SDMGradWeighting` from + [Direction-oriented Multi-objective Learning: Simple and Provable Stochastic + Algorithms](https://arxiv.org/pdf/2305.18409) + (NeurIPS 2023). It is a stateful `Weighting` that solves for task weights via a simplex-projected + inner loop on a cross-batch matrix `A = J_1 @ J_2.T` (computed from two independent mini-batches + using `autojac.jac`), with a direction-oriented regularizer pulling the descent direction toward + a preference direction. - Added `DWA` (Dynamic Weight Average) from [End-to-End Multi-Task Learning with Attention](https://openaccess.thecvf.com/content_CVPR_2019/papers/Liu_End-To-End_Multi-Task_Learning_With_Attention_CVPR_2019_paper.pdf) (CVPR 2019), a stateful `Scalarizer` that weights each value by the relative rate at which its @@ -25,7 +32,6 @@ changelog does not include internal changes that do not affect the user. ### Added -- Added `SDMGradWeighting` from [Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms](https://arxiv.org/pdf/2305.18409) (NeurIPS 2023). It is a stateful `Weighting` that solves for task weights via a simplex-projected inner loop on a cross-batch matrix `A = J_1 @ J_2.T` (computed from two independent mini-batches using `autojac.jac`), with a direction-oriented regularizer pulling the descent direction toward a preference direction. - Added `IMTL-L` (the loss-balancing variant of Impartial Multi-Task Learning) from [Towards Impartial Multi-Task Learning](https://openreview.net/pdf?id=IMPnRXEWpvr) (ICLR 2021), a stateful `Scalarizer` that learns a per-task scale `s_i` and combines the values as