Description of the problem/new feature
Three new general Baseline[2D] methods should be added to allow calculating polynomials, penalized spline smoothing, and Whittaker smoothing with a user-input weighting function. The weighting function would be expected to have the following signature:
weight_func(residual: np.ndarray, iteration: int, mask: np.ndarray | None) -> tuple[np.ndarray, bool]
with the output being the updated weights and a Boolean to designate if early termination is required.
This encompasses a lot of ideas I've wanted to add over the years but couldn't do in a non-clunky way:
-
Making weight functions is comparatively easy, so users can implement things I otherwise wouldn't add as a Baseline method, and gives users more control to do generalized fits.
- take for example the rrlPLS method, which uses a weighting similar to
arpls, just with a different asymmetry and shift of the logistic curve; it's not really different enough where I'd want to add it as a separate method, is fairly limited in the data that it's focused on, and allowing inputting asymmetry and shift values to arpls would make it no longer arpls (and why not add asymmetry and shifts to all other relevant methods...).
- also covers weighting schemes like Cadusch's Poisson weighted APLS and APoly and Zhang's normalized Poisson weighted PD-AsLS methods where the weighting is heavily dependent on the scaling of the data such that I don't want to implement them.
-
Allows doing things like fixing weights at the endpoints to prevent edge effects; I've considered in the past adding an argument like fixed_weights, but I never felt like any option was very clean. With a custom weight function, users can just write their own wrapper on an existing weight function that allows fixing some number of points at the edges; it's obviously not quite the same as implementing fixed weights for ones that do additional things within their methods like drpls, but it's a good enough compromise
-
Enables all the weighting schemes from literature for Whittaker smoothing/penalized least squares to be used for polynomials; I've wanted to do this before, but there really wasn't a nice way to do it like I did with P-splines since polynomials are not penalized such that their functional forms wouldn't match, so naming something like poly_arpls would be misleading.
- Will need to make
_weighting public and make appropriate wrappers that match the expected weight_func signature for most of the current weighting schemes
-
The methods would be setup such that they integrate with existing optimizer methods, so all the benefits of using collab_pls, optmize_extended_range, optimize_pls, etc. are directly available
The one downside is that there's less discoverability for these simpler papers which just tweak weighting schemes for Whittaker smoothing since a Baseline method might no longer added be for them. To make up for this, I could add a page in the docs to cover other weighting functions from literature, which seems like a nice middle ground since, as I said in the first bullet point, I don't want to actually add/support all the different weighting schemes going forward.
Description of the problem/new feature
Three new general Baseline[2D] methods should be added to allow calculating polynomials, penalized spline smoothing, and Whittaker smoothing with a user-input weighting function. The weighting function would be expected to have the following signature:
with the output being the updated weights and a Boolean to designate if early termination is required.
This encompasses a lot of ideas I've wanted to add over the years but couldn't do in a non-clunky way:
Making weight functions is comparatively easy, so users can implement things I otherwise wouldn't add as a Baseline method, and gives users more control to do generalized fits.
arpls, just with a different asymmetry and shift of the logistic curve; it's not really different enough where I'd want to add it as a separate method, is fairly limited in the data that it's focused on, and allowing inputting asymmetry and shift values to arpls would make it no longer arpls (and why not add asymmetry and shifts to all other relevant methods...).Allows doing things like fixing weights at the endpoints to prevent edge effects; I've considered in the past adding an argument like
fixed_weights, but I never felt like any option was very clean. With a custom weight function, users can just write their own wrapper on an existing weight function that allows fixing some number of points at the edges; it's obviously not quite the same as implementing fixed weights for ones that do additional things within their methods likedrpls, but it's a good enough compromiseEnables all the weighting schemes from literature for Whittaker smoothing/penalized least squares to be used for polynomials; I've wanted to do this before, but there really wasn't a nice way to do it like I did with P-splines since polynomials are not penalized such that their functional forms wouldn't match, so naming something like
poly_arplswould be misleading._weightingpublic and make appropriate wrappers that match the expected weight_func signature for most of the current weighting schemesThe methods would be setup such that they integrate with existing optimizer methods, so all the benefits of using
collab_pls,optmize_extended_range,optimize_pls, etc. are directly availableThe one downside is that there's less discoverability for these simpler papers which just tweak weighting schemes for Whittaker smoothing since a Baseline method might no longer added be for them. To make up for this, I could add a page in the docs to cover other weighting functions from literature, which seems like a nice middle ground since, as I said in the first bullet point, I don't want to actually add/support all the different weighting schemes going forward.