EDIT after merging #1795: The final conclusions was "Working As Intended, update the documentation"
Summary
On Windows, the pathfinder's load_with_system_search() fails to find all CUDA Toolkit DLLs when CUDA_HOME/CUDA_PATH are not set. The DLLs are on PATH but LoadLibraryExW never finds them.
Root cause
Python 3.8+ calls SetDefaultDllDirectories(LOAD_LIBRARY_SEARCH_DEFAULT_DIRS) at process startup, which restricts LoadLibraryExW(name, NULL, 0) to searching only:
- The application directory (Python's install directory)
- The system directory (
System32)
- Directories explicitly added via
AddDllDirectory()
PATH is excluded entirely. The current code calls LoadLibraryExW(dll_name, None, 0) with a bare DLL name, so it silently fails to find any CUDA DLL that isn't in one of those three locations.
Note: This was initially misdiagnosed as a dependency-resolution issue (error 126 for co-located DLLs like nvrtc-builtins). Experiments uncovered the problem is more fundamental: LoadLibraryExW never finds the DLL in the first place, affecting all CTK libraries equally.
Why this went unnoticed
- Our CI Windows test environments always have
CUDA_HOME or CUDA_PATH set.
- The
CUDA_HOME/CUDA_PATH search step runs before system search and successfully locates the DLLs, so the PATH-based fallback was never exercised.
References
EDIT after merging #1795: The final conclusions was "Working As Intended, update the documentation"
Summary
On Windows, the pathfinder's
load_with_system_search()fails to find all CUDA Toolkit DLLs whenCUDA_HOME/CUDA_PATHare not set. The DLLs are onPATHbutLoadLibraryExWnever finds them.Root cause
Python 3.8+ calls
SetDefaultDllDirectories(LOAD_LIBRARY_SEARCH_DEFAULT_DIRS)at process startup, which restrictsLoadLibraryExW(name, NULL, 0)to searching only:System32)AddDllDirectory()PATHis excluded entirely. The current code callsLoadLibraryExW(dll_name, None, 0)with a bare DLL name, so it silently fails to find any CUDA DLL that isn't in one of those three locations.Why this went unnoticed
CUDA_HOMEorCUDA_PATHset.CUDA_HOME/CUDA_PATHsearch step runs before system search and successfully locates the DLLs, so thePATH-based fallback was never exercised.References
pathfinderfor dynamic libraries numba-cuda#308