feat: add Windows ARM64 (MSVC) build support

[Aanerud]

Summary

Enables pip install . to produce a native ARM64 Python wheel on Windows ARM64 with MSVC + Visual Studio Build Tools 2022. Four small, targeted changes — three in zvec, one new patch file applied to the bundled Arrow 21.0 tree.

The Python SDK build docs currently list Windows as x86_64-only with MSVC 2022+. This PR adds the ARM64 arm of that — everything else (Linux ARM64, macOS ARM64) is already working.

Changes

File	Why
src/ailego/CMakeLists.txt	The arm/arm64 branch of AUTO_DETECT_ARCH had if(MSVC) return() endif(), which bailed before cc_library(zvec_ailego ...) was reached — downstream then failed with No target "zvec_ailego". The -march=armv8-a / NEON glob setup is still GCC/Clang-only, so wrap just that in if(NOT MSVC) and let the target definition proceed.
src/ailego/internal/cpu_features.cc	MSVC branch unconditionally called __cpuidex (x86/x64 intrinsic); GCC branch was gated only by !__ARM_ARCH. MSVC ARM64 matched neither and failed with __cpuidex: identifier not found. Scope both branches to x86/x64 explicitly so MSVC ARM64 falls through to the existing empty-stub ctor.
thirdparty/arrow/CMakeLists.txt	Arrow 21.0 ships xsimd 13.0, which does not implement xsimd::make_sized_batch_t for MSVC ARM64. Pass -DARROW_SIMD_LEVEL=NONE -DARROW_RUNTIME_SIMD_LEVEL=NONE only when CMAKE_SYSTEM_PROCESSOR is ARM64 — x64 MSVC keeps its default SSE4.2 path untouched. Also wires in the new patch below.
thirdparty/arrow/arrow.windows-arm64.patch (new)	Arrow's vendored PCG header (arrow/vendored/pcg/pcg_uint128.hpp) uses an x86-only endianness check and calls _umul128, which is not available on MSVC ARM64 — the equivalent intrinsic is __umulh. Add _M_ARM64/_M_ARM/__aarch64__/__arm__ to the little-endian case, branch to __umulh on ARM, and guard #pragma intrinsic(_umul128) so it is not referenced on ARM. Applied via the existing apply_patch_once mechanism, scoped to MSVC+ARM64.

Non-goals

• x64 MSVC behavior is unchanged. All new logic is gated on CMAKE_SYSTEM_PROCESSOR MATCHES "^(ARM64|arm64|aarch64)$".
• No SIMD path for MSVC ARM64 yet. This PR deliberately disables Arrow's xsimd SIMD and skips zvec's NEON math glob for MSVC. A follow-up could add a NEON path guarded on _M_ARM64 using MSVC's <arm_neon.h>, but that's additive and out of scope here.
• CPU feature detection returns all-zero on MSVC ARM64. That's correct — x86 features like SSE/AVX don't exist on ARM. A future change could populate ARM-specific flags via IsProcessorFeaturePresent or compile-time __ARM_FEATURE_* macros.

Test plan

• Built on Windows 11 ARM64 (native) with MSVC 14.44 (VS Build Tools 2022, Microsoft.VisualStudio.Component.VC.Tools.ARM64) — pip install . succeeds and produces a working wheel
• python -c "import zvec; print(zvec)" imports cleanly on the resulting venv (zvec: OK (0.3.2.dev2))
• git apply --check thirdparty/arrow/arrow.windows-arm64.patch clean against the bundled Arrow 21.0 tree
• Needs validation: Windows x64 MSVC still builds unchanged (should be — all changes are gated)
• Needs validation: Linux/macOS builds unchanged (should be — all changes gated on MSVC)

Error trail for reviewers

For context on what each change fixes (in build order):

1. CMake Error at src/binding/c/CMakeLists.txt:109 (target_link_options): No target "zvec_ailego" → fixed by ailego CMakeLists change.
2. error C3861: '__cpuidex': identifier not found → fixed by cpu_features.cc change.
3. error C2039: 'make_sized_batch_t': is not a member of 'xsimd' (~20 errors in arrow_compute_core_static and arrow_util_static) → fixed by ARROW_SIMD_LEVEL=NONE.
4. error C1189: #error: Unable to determine target endianness in arrow/vendored/pcg/pcg_uint128.hpp → fixed by endianness hunk of the new arrow patch.
5. error C3861: '_umul128': identifier not found in the same file → fixed by the __umulh hunk of the new arrow patch.

After those five errors are resolved, the rest of the build (protobuf, rocksdb, arrow, zvec_core, zvec_db, zvec_ailego, zvec_turbo, roaring, Python binding) completes cleanly on MSVC ARM64.

[CLAassistant]

All committers have signed the CLA.

[feihongxu0824]

hi, @Aanerud, it would be beneficial to introduce a CI workflow for Windows ARM64 to guarantee ongoing integration support.

[Aanerud]

Thanks for the review @feihongxu0824 — CLA signed, and just pushed 3f586d1 which adds windows-11-arm to the matrix in .github/workflows/05-windows-build.yml.

Kept it minimal to match the existing style:

• Third row in the existing matrix: platform: windows-11-arm, msvc_arch: arm64, python_version: '3.11' (Python 3.10 has no official Windows-on-ARM installer; 3.11 is the first).
• Parameterized ilammy/msvc-dev-cmd on matrix.msvc_arch and actions/setup-python on matrix.python_version. x64 rows still use Python 3.10 + MSVC x64 — no change to existing behavior.
• Build / C++ unittest / pytest / examples steps are identical; fail-fast: false was already set, so an ARM64 regression won't mask x64 and vice versa.

Happy to split the ARM64 row into its own reusable workflow file (e.g. 07-windows-arm64-build.yml, mirroring how 03-macos-linux-build.yml is parameterized in 01-ci-pipeline.yml) if that's a better fit for your CI organization — just let me know.

[Aanerud]

The windows-11-arm run surfaced two failing tests in hnsw_streamer_test:

• TestKnnSearchCosine — expected i=150, got 149
• TestFetchVectorCosine — expected i=174, got 173

Both asserts are ASSERT_EQ(i, linearResult[0].key()) after a brute-force cosine search where vector i is the query, i.e. they expect the query vector to be its own top-1. On MSVC ARM64 the top-1 comes back as i - 1 for one or two indices.

Root cause is test sensitivity, not an index bug. The dataset vectors are constructed with small inter-vector deltas (fixed_value + i * add_on), so after cosine normalization dot(v[i], v[i]) and dot(v[i], v[i-1]) are within ~1 ULP. The NEON math kernels in src/ailego/math/*_neon.cc are gated on __ARM_NEON, which MSVC does not predefine (MSVC uses _M_ARM64), so the scalar fallback runs on MSVC ARM64 and happens to hit a tie on these two queries.

Signal pattern that makes me confident in this diagnosis:

• Same two tests pass on linux-arm64 and macos-arm64 (GCC/Clang with NEON) and on x64 MSVC (SSE/AVX2) in this same run.
• The other five cosine variants on MSVC ARM64 — TestFetchVectorCosineHalfFloatConverter, Fp16Converter, Int8Converter, Int4Converter, TestKnnSearchL2 etc. — all pass.

Just pushed 3d13553 which adds GTEST_SKIP() to those two tests under #if defined(_MSC_VER) && defined(_M_ARM64) with a TODO pointing at the right follow-up: wire up a MSVC-ARM64 NEON kernel that uses <arm_neon.h> gated on _M_ARM64 instead of __ARM_NEON. Once that's in, the skips come off and these two tests should pass for real.

If you'd rather I do the MSVC-ARM64 NEON work in this PR instead of skipping + follow-up, I'm happy to — let me know.

[feihongxu0824]

Don't we need to consider -march optimizations for MSVC on ARM64 here?

[feihongxu0824]

If we add an else() branch after the if(NOT MSVC) block in src/ailego/CMakeLists.txt to compile the *_neon.cc files for MSVC ARM64, and update the source code guards from __ARM_NEON to __ARM_NEON || _M_ARM64, the NEON kernels will be included. This may resolve the precision issues, allowing us to re-enable these two tests?

[feihongxu0824]

Apologies for the delay in reviewing this PR. Thank you for your contribution and for providing such a clear description. I’ve submitted some suggestions for your reference. Also, I noticed that the CLA hasn’t been signed yet.

[Aanerud]

Rebased onto current main (51c6d9e) and force-pushed with three commits:

1. b37e324 — feat: original Windows ARM64 build support (zvec_ailego target, cpu_features stub, arrow SIMD/patch)
2. 97d6ba6 — ci: add windows-11-arm to the build matrix (rebased over the recent pytest-xdist + zvec.dll-copy CI changes — clean merge)
3. aa76ffa — feat(ailego): enable NEON math kernels on MSVC ARM64 per your inline review on src/ailego/CMakeLists.txt

For commit 3, instead of leaving the kernels gated on the GCC/Clang-only macros, I expanded the source guards as you suggested:

• defined(__ARM_NEON) → (defined(__ARM_NEON) || defined(_M_ARM64)) at 51 sites
• defined(__aarch64__) → (defined(__aarch64__) || defined(_M_ARM64)) at 26 sites (so the AArch64-specific NEON paths like vaddvq_f32 get used, and the ARMv7-only polyfills get correctly skipped under MSVC ARM64)
• <arm_neon.h> now included on the MSVC branch of platform.h when _M_ARM64 is defined
• src/ailego/CMakeLists.txt: added an explicit else() branch after the existing if(NOT MSVC) with a comment explaining MSVC ARM64 doesn't need -march (NEON is the ARMv8 baseline)

One subtlety: __fp16 is a GCC/Clang extension type that MSVC does not provide even on ARM64. The Float16 wrapper class in src/include/zvec/ailego/utility/float_helper.h and the helper functions in src/ailego/utility/float_helper.cc were previously gated only on __aarch64__, so my generalized sed accidentally tried to compile the __fp16 variants under MSVC ARM64. Those two specific guards now read ... && !defined(_MSC_VER), so MSVC ARM64 falls through to the uint16_t-storage Float16 and the F16C/scalar conversion paths. The FP16 NEON math kernels (*_fp16_neon.cc) themselves operate on float16_t from <arm_neon.h> and only enter the ARMv8.2 FP16-arithmetic path when __ARM_FEATURE_FP16_VECTOR_ARITHMETIC is defined (which MSVC does not predefine), so they should fall through to the FP32-conversion variant on MSVC ARM64.

I also dropped the old test(hnsw): skip ... cosine self-match tests on MSVC ARM64 commit, since the NEON enablement should make those tests pass natively rather than skipping them.

The CLA-bot pending status was because my earlier commits were authored as aaanerud@users.noreply.github.com (a noreply that doesn't actually map to a GitHub account). All three commits in this push are now authored as Andreas Martin Aanerud <a.m.aanerud@gmail.com>, which is the email I signed the CLA with on my personal Aanerud account.

Waiting on the windows-11-arm CI run to confirm the NEON path compiles cleanly and the previously-failing HnswStreamerTest.TestKnnSearchCosine, TestFetchVectorCosine, and NormMatrix.Norm1_General now pass. If anything else surfaces I'll iterate.