-
Notifications
You must be signed in to change notification settings - Fork 317
Fix | SqlVector: Explicitly perform little-endian multibyte writes #3861
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR refactors the SqlVector<T> implementation to use explicit little-endian byte operations when serializing and deserializing vector data. The change replaces platform-specific memory marshaling/copying approaches with consistent loop-based operations using BinaryPrimitives methods to ensure correct endianness.
Key Changes:
- Replaced manual bit manipulation and platform-specific
Buffer.BlockCopy/MemoryMarshaloperations with explicit little-endian read/write methods - Introduced platform-specific handling:
BinaryPrimitives.WriteSingleLittleEndianfor .NET andBitConverterCompatible.SingleToInt32Bits+BinaryPrimitives.WriteInt32LittleEndianfor .NET Framework - Ensured symmetry between serialization (
MakeTdsBytes) and deserialization (MakeArray) operations
| if (typeof(T) == typeof(float)) | ||
| { | ||
| Buffer.BlockCopy(values.ToArray(), 0, result, TdsEnums.VECTOR_HEADER_SIZE, values.Length * _elementSize); | ||
| } | ||
| for (int i = 0, currPosition = TdsEnums.VECTOR_HEADER_SIZE; i < values.Length; i++, currPosition += _elementSize) | ||
| { | ||
| #if NET | ||
| BinaryPrimitives.WriteSingleLittleEndian(result.AsSpan(currPosition), (float)(object)valueSpan[i]); | ||
| #else | ||
| // Fast span-based copy. | ||
| var byteSpan = MemoryMarshal.AsBytes(values.Span); | ||
| byteSpan.CopyTo(result.AsSpan(TdsEnums.VECTOR_HEADER_SIZE)); | ||
| BinaryPrimitives.WriteInt32LittleEndian(result.AsSpan(currPosition), BitConverterCompatible.SingleToInt32Bits((float)(object)valueSpan[i])); | ||
| #endif | ||
| } | ||
| } |
Copilot
AI
Dec 24, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The conditional check for typeof(T) == typeof(float) should include an else clause that throws an exception for unsupported types. Currently, if T is not float, the method would return a byte array with only the header populated and no element data. While GetTypeFieldsOrThrow() prevents non-float types at construction time, adding an explicit else clause here would make the code more defensive and maintainable for future type additions. Consider adding: else { throw SQL.VectorTypeNotSupported(typeof(T).FullName); }
| result[i] = (T)(object)BitConverterCompatible.Int32BitsToSingle(BinaryPrimitives.ReadInt32LittleEndian(_tdsBytes.AsSpan(currPosition))); | ||
| #endif | ||
| } | ||
| } |
Copilot
AI
Dec 24, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The conditional check for typeof(T) == typeof(float) should include an else clause that throws an exception for unsupported types. Currently, if T is not float, the method would return an uninitialized array. While GetTypeFieldsOrThrow() prevents non-float types at construction time, adding an explicit else clause here would make the code more defensive and maintainable for future type additions. Consider adding: else { throw SQL.VectorTypeNotSupported(typeof(T).FullName); }
| } | |
| } | |
| else | |
| { | |
| throw SQL.VectorTypeNotSupported(typeof(T).FullName); | |
| } |
Description
This deals with a minor point which came out of the original implementation PR: when we build (or read) the byte array representing a
vector({size}, float32), we now explicitly do so using BinaryPrimitives' little-endian methods.There are two slightly unusual points:
BinaryPrimitives.WriteSingleLittleEndianmethod. Instead, we fall back to the existingBitConverterCompatible.SingleToInt32Bitson this target.(float)(object)valueSpan[i]. This is another variation of the same pattern used elsewhere of(T)(object)item, and the same pattern holds: the JIT sees thatvalueSpanis actually aReadOnlySpan<float>and eliminates the redundant cast. An example of this on Sharplab is here.Issues
Fixes #3790.
Testing
Automated tests continue to pass.