Skip to content

Only require numpy for the vectorized UDF path#538

Open
yvaishalirao wants to merge 1 commit into
duckdb:mainfrom
yvaishalirao:fix/create-function-numpy-requirement
Open

Only require numpy for the vectorized UDF path#538
yvaishalirao wants to merge 1 commit into
duckdb:mainfrom
yvaishalirao:fix/create-function-numpy-requirement

Conversation

@yvaishalirao

Copy link
Copy Markdown

Fixes duckdb/duckdb#17052 (duckdb/duckdb#17052)

create_function() was unnecessarily requiring numpy even for native UDFs that never use it. GetFunction() in src/python_udf.cpp imports/version-checks/warms-up numpy before determining if the vectorized mode was chosen. Only the vectorized path actually needs numpy, since it round-trips through pyarrow — the native path never touches either.

Fix is one line, scoping that block inside if (vectorized).

Two new tests cover both directions:

  • test_scalar_native.py::test_native_udf_without_numpy
  • test_scalar_arrow.py::test_arrow_udf_requires_numpy

Both run in a subprocess rather than monkeypatching sys.modules in-process — numpy's import cache (PythonImportCacheItem) lives for the whole interpreter process, so if numpy's already been imported by another test in the same worker, an in-process monkeypatch won't make the cache forget it. A fresh subprocess sidesteps that.

Verified the test actually catches the regression by stashing the fix and running it both ways, numpy uninstalled in both:

Build numpy installed? Result
Pre-fix No fails — 'numpy' is required for this operation, but it wasn't installed
Post-fix No passes

…, but GetFunction() checked for numpy unconditionally before even branching on `vectorized`. Move the check inside that branch so native UDFs work without numpy installed.Fixes duckdb/duckdb#17052 (duckdb/duckdb#17052)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

duckdb.create_function in Python requires numpy to be installed in v1.2.2

1 participant