Add an `fN::mul_add` variant that might round twice

[scottmcm]

Proposal

I could have sworn there was conversation about this before, but couldn't find it anywhere, so opening this. If I totally missed it, sorry, and please close this as dup.

Problem statement

fN::mul_add is specifically defined to be fusedMultiplyAdd and *+ rounds twice, which is good for determinism.

However, sometimes people want to give up that determinism for speed, and we should give them that option, letting the codegen backend pick (based on target or whatever) the fastest approach.

Motivating examples or use cases

Easing functions are often defined as polynomials, but when used only for (say) UI smoothing (as opposed to replicated physics) there's no need to calculate them perfectly accurately. It would be nice if they could just be written using Horner's method and let the compiler do it the fastest way, rather than needing to think about it.

Solution sketch

impl f16/f32/f64/f128 {
    /// Either `Self::mul_add` or `*+`, nondeterministically.
    fn mul_add_relaxed(self, multiplicand: Self, addend: Self) -> Self;
}

The intrinsic for this was added in rust-lang/rust#124874 but has never been exposed in even an unstable non-intrinsic method.

Alternatives

• Use a different name
• Offer different versions of */+ instead so people write, say, x.contracting_mul(y).contracting_add(z) instead.
• Say we don't care about the only-contract version and tell people that if they want nondet they can use algebraic_* instead.

Links and related work

https://doc.rust-lang.org/nightly/core/arch/wasm/fn.f32x4_relaxed_min.html uses the "relaxed" name.

This issue prompted by #t-compiler/llvm > float::min/max vs SNaN @ 💬.

algebraic_*: rust-lang/rust#136469

What happens now?

This issue contains an API change proposal (or ACP) and is part of the libs-api team feature lifecycle. Once this issue is filed, the libs-api team will review open proposals as capability becomes available. Current response times do not have a clear estimate, but may be up to several months.

Possible responses

The libs team may respond in various different ways. First, the team will consider the problem (this doesn't require any concrete solution or alternatives to have been proposed):

• We think this problem seems worth solving, and the standard library might be the right place to solve it.
• We think that this probably doesn't belong in the standard library.

Second, if there's a concrete solution:

• We think this specific solution looks roughly right, approved, you or someone else should implement this. (Further review will still happen on the subsequent implementation PR.)
• We're not sure this is the right solution, and the alternatives or other materials don't give us enough information to be sure about that. Here are some questions we have that aren't answered, or rough ideas about alternatives we'd want to see discussed.

[programmerjake]

I've been wanting this for a while (not because I have a specific application for it at the moment, but more because LLVM has llvm.fmuladd.* and I think users should have the option to use it). I'm kinda annoyed that Rust decided mul_add is fma since it has the same name as llvm.fmuladd.* but different semantics.

[scottmcm]

LLVM has both llvm.fma and llvm.fmuladd which I think is a horrible naming scheme, so wouldn't make any decisions based on it :P

Note that, for nightly experimentation, one can always use rust-lang/rust#29602 to call it (even if we didn't have the core::intrinsics for it already).

---

Another thing I came up with for a discussion topic here: would you want rust to be able to simplify x.mul_add_relaxed(y, 0) to x * y? Or would only x.mul_add_relaxed(y, -0) to x * y be legal?

[hanna-kruppe]

Another thing I came up with for a discussion topic here: would you want rust to be able to simplify x.mul_add_relaxed(y, 0) to x * y? Or would only x.mul_add_relaxed(y, -0) to x * y be legal?

I don't think adding more non-determinism beyond "choose between one or two roundings" makes sense for this API. IMO the main reason why this new variant is potentially useful is because it takes the drudgery out of "use FMA if it's fast, otherwise do mul -> add" without opening additional cans of worms. The non-determinism is sufficiently constrained that you actually have a chance of reasoning precisely about the possible results. We already have the "algebraic" operations, if you want more rewrites. In fact, at a glance I don't even know what would be a useful/natural spec that allows playing fast and loose with signed zeros but doesn't allow everything allowed by x.algebraic_mul(y).algebraic_add(z).

[RalfJung]

There was a bit of discussion in rust-lang/rust#124874 but the bikeshed was avoided by adding only the intrinsics as a first step... and then nobody ever made the second step.

[RalfJung]

In fact, at a glance I don't even know what would be a useful/natural spec that allows playing fast and loose with signed zeros but doesn't allow everything allowed by x.algebraic_mul(y).algebraic_add(z).

To me the main question is, what is a usecase where you're fine with the non-determinism allowed by this operation, but you're not fine with x.algebraic_mul(y).algebraic_add(z)? The example given in the ACP of UI computations surely would be fine with either.

[programmerjake]

To me the main question is, what is a usecase where you're fine with the non-determinism allowed by this operation, but you're not fine with x.algebraic_mul(y).algebraic_add(z)?

how about when you're trying to make code run quickly on all platforms but still fit within specific error bounds? you can relatively easily compute those bounds for non-deterministically choosing fma or * and + since those both give specified answers, but algebraic_mul/add is basically asking LLVM to give you anything that vaguely resembles the input code and can have arbitrarily large errors.

[hanna-kruppe]

Although in that case you’d probably prefer if every "FMA or mul+add" operation in your algorithm coordinated to make the same choice, since that simplifies the analysis (or lets you reuse existing analysis for either case). Which is essentially what you get today if you laboriously write both variants and choose between the based on target arch and target features.

[programmerjake]

Although in that case you’d probably prefer if every "FMA or mul+add" operation in your algorithm coordinated to make the same choice, since that simplifies the analysis (or lets you reuse existing analysis for either case). Which is essentially what you get today if you laboriously write both variants and choose between the based on target arch and target features.

yes, but using #[cfg] doesn't handle when your code is inlined into some other function that has the required target features. it also doesn't handle when rustc adds a new target that you didn't write the #[cfg] to handle.

[hanna-kruppe]

Sure, that's why the LLVM intrinsic is defined the way it is. But I do wonder how many use cases need that (and can deal with the per-op non-determinism it implies) yet can't use algebraic_* operations. On paper those use cases should exist, but the niche may be relatively small.