Between Vector<T> and Vector128/256/512<> what should I use?

[DrkWzrd]

I read the guidelines (https://github.com/dotnet/runtime/blob/main/docs/coding-guidelines/vectorization-guidelines.md) but I didn't found any clear recommendation ("If you have already vectorized your code with Vector you can use the new APIs to check if they can produce better code-gen")

I know I should test and all, but shouldn't be clear if Vector<> is capable of generate most-performant kind of vectorization (512 in 512, 256 in 256, etc...)? Maybe I'm missing something.

[tannergooding]

The TL;DR is that if you're manually writing perf code, there is no simple answer. Perf takes explicit effort and measurement.

---

Vector<T> is not explicitly generating the most performant vectorization, it doesn't automatically use 512-bit vectors if they're available as the trivial example (you must opt-in). The size being "variable" also makes it very difficult to use for some types of algorithms, you couldn't easily use it to build a Matrix4x4 for example.

In general, Vector128<T> is the common base type that is going to be accelerated on any platform that has SIMD support, it is the minimum thing to target.

Vector256/512 are then optional additional paths to provide if you know you're targeting hardware that may support it (such as x64 or x86) and want additional performance.

Vector<T> can be used to provide a single algorithm, rather than 3, but it may also leave perf on the table. It may even map to say SVE on Arm64 in which case it may be 128-bits in width but have different perf characteristics from Vector128<T> (which maps to AdvSimd instead).

---

In general if you just want acceleration on the most platforms, then support V128

Then consider V256, V512, and Vector if you want to have additional acceleration opportunities on other platforms.

I would generally recommend ordering it as this, which handles all platforms currently supported and favors known vector lengths over unknown ones, but prefers an unknown length if it is bigger than V128 (i.e. SVE256 > AdvSimd > SVE128):

if (Vector512.IsHardwareAccelerated) { }
if (Vector256.IsHardwareAccelerated) { }
if (Vector.IsHardwareAccelerated && (Vector<T>.Count != Vector128<T>.Count)) { }
if (Vector128.IsHardwareAccelerated) { }

There may be an API in the future that tells you if Vector<T> is the "preferred" vector type, in which case you'd then do the Vector<T> check first, but that needs to be designed/considered.

Vector64<T> is mostly a type that exists for hardware that supports it, it isn't something most code should use (outside of platform specific intrinsics).

[DrkWzrd]

so this type of chaining is "good"?

[MethodImpl(MethodImplOptions.AggressiveOptimization)]
public static Box2 GetBounds(ReadOnlySpan<Vector2> points)
{
    if (points.IsEmpty)
        return Box2.NaN;

    if (points.Length == 1)
    {
        Vector2 p = points[0];
        return new Box2(p.X, p.Y, 0, 0);
    }
    else if (points.Length == 2)
    {
        return GetBounds(points[0], points[1]);
    }

    ReadOnlySpan<float> values = MemoryMarshal.Cast<Vector2, float>(points);
    if (values.Contains(float.NaN))
        return Box2.NaN;

    ref float rValues = ref MemoryMarshal.GetReference(values);

    float minX = float.MaxValue, minY = float.MaxValue;
    float maxX = float.MinValue, maxY = float.MinValue;
    int i = 0;

    if (Vector512.IsHardwareAccelerated && values.Length >= 32)
    {
        Vector512<float> vMin1 = Vector512.Create(float.MaxValue), vMin2 = vMin1;
        Vector512<float> vMax1 = Vector512.Create(float.MinValue), vMax2 = vMax1;

        for (; i <= values.Length - 32; i += 32)
        {
            vMin1 = Vector512.MinNative(vMin1, Vector512.LoadUnsafe(ref rValues, (nuint)i));
            vMin2 = Vector512.MinNative(vMin2, Vector512.LoadUnsafe(ref rValues, (nuint)(i + 16)));
            vMax1 = Vector512.MaxNative(vMax1, Vector512.LoadUnsafe(ref rValues, (nuint)i));
            vMax2 = Vector512.MaxNative(vMax2, Vector512.LoadUnsafe(ref rValues, (nuint)(i + 16)));
        }

        vMin1 = Vector512.MinNative(vMin1, vMin2);
        vMax1 = Vector512.MaxNative(vMax1, vMax2);

        if (i <= values.Length - 16)
        {
            vMin1 = Vector512.MinNative(vMin1, Vector512.LoadUnsafe(ref rValues, (nuint)i));
            vMax1 = Vector512.MaxNative(vMax1, Vector512.LoadUnsafe(ref rValues, (nuint)i));
            i += 16;
        }

        ReduceVector512(vMin1, vMax1, ref minX, ref minY, ref maxX, ref maxY);
    }
    else if (Vector256.IsHardwareAccelerated && values.Length >= 16)
    {
        Vector256<float> vMin1 = Vector256.Create(float.MaxValue), vMin2 = vMin1;
        Vector256<float> vMax1 = Vector256.Create(float.MinValue), vMax2 = vMax1;

        for (; i <= values.Length - 16; i += 16)
        {
            vMin1 = Vector256.MinNative(vMin1, Vector256.LoadUnsafe(ref rValues, (nuint)i));
            vMin2 = Vector256.MinNative(vMin2, Vector256.LoadUnsafe(ref rValues, (nuint)(i + 8)));
            vMax1 = Vector256.MaxNative(vMax1, Vector256.LoadUnsafe(ref rValues, (nuint)i));
            vMax2 = Vector256.MaxNative(vMax2, Vector256.LoadUnsafe(ref rValues, (nuint)(i + 8)));
        }

        vMin1 = Vector256.MinNative(vMin1, vMin2);
        vMax1 = Vector256.MaxNative(vMax1, vMax2);

        if (i <= values.Length - 8)
        {
            vMin1 = Vector256.MinNative(vMin1, Vector256.LoadUnsafe(ref rValues, (nuint)i));
            vMax1 = Vector256.MaxNative(vMax1, Vector256.LoadUnsafe(ref rValues, (nuint)i));
            i += 8;
        }

        ReduceVector256(vMin1, vMax1, ref minX, ref minY, ref maxX, ref maxY);
    }
    else if (Vector128.IsHardwareAccelerated && values.Length >= 8)
    {
        Vector128<float> vMin1 = Vector128.Create(float.MaxValue), vMin2 = vMin1;
        Vector128<float> vMax1 = Vector128.Create(float.MinValue), vMax2 = vMax1;

        for (; i <= values.Length - 8; i += 8)
        {
            vMin1 = Vector128.MinNative(vMin1, Vector128.LoadUnsafe(ref rValues, (nuint)i));
            vMin2 = Vector128.MinNative(vMin2, Vector128.LoadUnsafe(ref rValues, (nuint)(i + 4)));
            vMax1 = Vector128.MaxNative(vMax1, Vector128.LoadUnsafe(ref rValues, (nuint)i));
            vMax2 = Vector128.MaxNative(vMax2, Vector128.LoadUnsafe(ref rValues, (nuint)(i + 4)));
        }

        vMin1 = Vector128.MinNative(vMin1, vMin2);
        vMax1 = Vector128.MaxNative(vMax1, vMax2);

        if (i <= values.Length - 4)
        {
            vMin1 = Vector128.MinNative(vMin1, Vector128.LoadUnsafe(ref rValues, (nuint)i));
            vMax1 = Vector128.MaxNative(vMax1, Vector128.LoadUnsafe(ref rValues, (nuint)i));
            i += 4;
        }

        ReduceVector128(vMin1, vMax1, ref minX, ref minY, ref maxX, ref maxY);
    }

    for (; i < values.Length - 1; i += 2)
    {
        float x = Unsafe.Add(ref rValues, i);
        float y = Unsafe.Add(ref rValues, i + 1);
        minX = float.MinNative(minX, x);
        minY = float.MinNative(minY, y);
        maxX = float.MaxNative(maxX, x);
        maxY = float.MaxNative(maxY, y);
    }

    return new Box2(minX, minY, maxX - minX, maxY - minY);
}

//reduce methods chaining until shuffle 2,3,0,1

[tannergooding]

That is fine. There are other ways to do it as well, such as pulling the V128/V256/V512 code into their own methods. i.e.

if (Vector512.IsHardwareAccelerated)
{
    if (remainder >= Vector512<T>.Count)
    {
        Vectorized512(...)
    }
    else
    {
        VectorizedSmall(...);
    }
}
else if (Vector256.IsHardwareAccelerated)
{
    if (remainder >= Vector256<T>.Count)
    {
        Vectorized256(...)
    }
    else
    {
        VectorizedSmall(...);
    }
}
else if (Vector128.IsHardwareAccelerated)
{
    if (remainder >= Vector128<T>.Count)
    {
        Vectorized128(...)
    }
    else
    {
        VectorizedSmall(...);
    }
}
else
{
    SoftwareFallback(...);
}

This makes it easier for it to be inlined to just the code that matters. You can see an example of this approach in the TensorPrimitives helpers: https://source.dot.net/#System.Numerics.Tensors/System/Numerics/Tensors/netcore/Common/TensorPrimitives.IUnaryOperator.cs,fdab74764af40a1e