error: incompatible integer to pointer conversion assigning to 'PyObject *'

[yurivict]

Build fails:

./numpoly/cfunctions/cdispach.c:3036:13: warning: call to undeclared function 'PyDataType_ELSIZE'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3036 |   __pyx_r = PyDataType_ELSIZE(__pyx_v_self);
      |             ^
./numpoly/cfunctions/cdispach.c:3070:13: warning: call to undeclared function 'PyDataType_ALIGNMENT'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3070 |   __pyx_r = PyDataType_ALIGNMENT(__pyx_v_self);
      |             ^
./numpoly/cfunctions/cdispach.c:3108:15: warning: call to undeclared function 'PyDataType_FIELDS'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3108 |   __pyx_t_1 = PyDataType_FIELDS(__pyx_v_self);
      |               ^
./numpoly/cfunctions/cdispach.c:3108:13: error: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
 3108 |   __pyx_t_1 = PyDataType_FIELDS(__pyx_v_self);
      |             ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./numpoly/cfunctions/cdispach.c:3150:15: warning: call to undeclared function 'PyDataType_NAMES'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3150 |   __pyx_t_1 = PyDataType_NAMES(__pyx_v_self);
      |               ^
./numpoly/cfunctions/cdispach.c:3150:13: error: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
 3150 |   __pyx_t_1 = PyDataType_NAMES(__pyx_v_self);
      |             ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./numpoly/cfunctions/cdispach.c:3188:13: warning: call to undeclared function 'PyDataType_SUBARRAY'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3188 |   __pyx_r = PyDataType_SUBARRAY(__pyx_v_self);
      |             ^
./numpoly/cfunctions/cdispach.c:3188:11: error: incompatible integer to pointer conversion assigning to 'PyArray_ArrayDescr *' (aka 'struct _arr_descr *') from 'int' [-Wint-conversion]
 3188 |   __pyx_r = PyDataType_SUBARRAY(__pyx_v_self);
      |           ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./numpoly/cfunctions/cdispach.c:3222:13: warning: call to undeclared function 'PyDataType_FLAGS'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3222 |   __pyx_r = PyDataType_FLAGS(__pyx_v_self);
      |             ^
./numpoly/cfunctions/cdispach.c:3256:13: warning: call to undeclared function 'PyArray_MultiIter_NUMITER'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3256 |   __pyx_r = PyArray_MultiIter_NUMITER(__pyx_v_self);
      |             ^
./numpoly/cfunctions/cdispach.c:3290:13: warning: call to undeclared function 'PyArray_MultiIter_SIZE'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3290 |   __pyx_r = PyArray_MultiIter_SIZE(__pyx_v_self);
      |             ^
./numpoly/cfunctions/cdispach.c:3324:13: warning: call to undeclared function 'PyArray_MultiIter_INDEX'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3324 |   __pyx_r = PyArray_MultiIter_INDEX(__pyx_v_self);
      |             ^
./numpoly/cfunctions/cdispach.c:3358:13: warning: call to undeclared function 'PyArray_MultiIter_NDIM'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3358 |   __pyx_r = PyArray_MultiIter_NDIM(__pyx_v_self);
      |             ^
./numpoly/cfunctions/cdispach.c:3392:13: warning: call to undeclared function 'PyArray_MultiIter_DIMS'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3392 |   __pyx_r = PyArray_MultiIter_DIMS(__pyx_v_self);
      |             ^
./numpoly/cfunctions/cdispach.c:3392:11: error: incompatible integer to pointer conversion assigning to 'npy_intp *' (aka 'long *') from 'int' [-Wint-conversion]
 3392 |   __pyx_r = PyArray_MultiIter_DIMS(__pyx_v_self);
      |           ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./numpoly/cfunctions/cdispach.c:3426:13: warning: call to undeclared function 'PyArray_MultiIter_ITERS'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
 3426 |   __pyx_r = PyArray_MultiIter_ITERS(__pyx_v_self);
      |             ^
./numpoly/cfunctions/cdispach.c:3426:11: error: incompatible integer to pointer conversion assigning to 'void **' from 'int' [-Wint-conversion]
 3426 |   __pyx_r = PyArray_MultiIter_ITERS(__pyx_v_self);
      |           ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
13 warnings and 5 errors generated.
error: command '/usr/local/libexec/ccache/cc' failed with exit code 1

ERROR Backend subprocess exited when trying to invoke build_wheel

Version: 1.3.0
Python-3.11
numpy-1.26.4
FreeBSD 14.1

[jonathf]

Can't reproduce on Linux or MacOS.

From the error and the version this is obvious the introduction to Cython in the project.

@oarcelus, any chance you are familiar with BSD or know what we would get this error for BSD and not the other platforms?

@yurivict, can we get some more information about your cc compiler (assuming that it comes from freebsd and not Cython itself)?

[yurivict]

cc is clang-18

[yurivict]

Where are functions PyDataType_xx, like PyDataType_ELSIZE, declared?

[jonathf]

Is it possible to try again and setting an up-to-date gcc as your compiler?

PyDataType_xx are set in numpoly/cfuntions/cdispach.c. It is auto-generated by Cython from numpoly/cfunctions/cdistpach.pyx.

[oarcelus]

Those errors are related to the NumPy C-API I believe. @yurivict could you try running the np.get_include() function to see where it points to?. In principle this should be correctly configured already in setup.py buu just to be sure. Otherwise I would try setting up 'gcc' as the compiler.

Python-3.12
numpy-1.26.4
Cython 3.0.11

The gcc compiler is the following:
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

Anyways the errors are coming from the cdispach module. Which is not in use in numpoly right now. Maybe the rest of modules work correctly?

[yurivict]

could you try running the np.get_include() function to see where it points to?

>>> numpy.get_include() 
'/usr/local/lib/python3.11/site-packages/numpy/core/include'

[yurivict]

PyDataType_xx are set in numpoly/cfuntions/cdispach.c

PyDataType_ELSIZE is called in numpoly/cfunctions/cdispach.c, but where is it declared and defined?

[oarcelus]

It looks like the C headers are correctly located. I am not familiar with BSD let me look into it. In the meantime could you try by using the gcc compiler instead?

You could temporarily change compiler by running export CC=gcc export CXX=g++.

About the definition and declaration of the cdispatch.c, as pointed out by @jonathf, these files are automatically generated by cython from .pyx files. If the building process is failing then you may not be generating these files, but they should be located at /usr/local/lib/python3.11/site-packages/numpoly/cfunctions/.

[yurivict]

It builds successfully with gcc.

[oarcelus]

Nice!

[karlr-vestas]

I think I have a similar error. I'm trying to install chaospy (and therefore numpoly) in databricks (a cloud environment where I dont fully control the installation process).

Here is the relevant part of the error output.

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [99 lines of output]
      /databricks/python/lib/python3.11/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-7nezif7s/numpoly_a372760575304122a650fd7ab2794a79/numpoly/cfunctions/cdispach.pyx
        tree = Parsing.p_module(s, pxd, full_module_name)
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
      cimport numpy as np
      import numpoly
      from typing import List
      
      
      cdef List[np.ndarray] cprepare_input(
          ^
      ------------------------------------------------------------
      
      numpoly/cfunctions/cdispach.pyx:10:5: 'List' is not a type identifier
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
      import numpoly
      from typing import List
      
      
      cdef List[np.ndarray] cprepare_input(
          List[np.ndarray] inputs,
         ^
      ------------------------------------------------------------
      
      numpoly/cfunctions/cdispach.pyx:11:4: 'List' is not a type identifier
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          return output
      
      
      def cloop_function(
              object numpy_ufunc,
              List[np.ndarray] inputs,
             ^
      ------------------------------------------------------------
      
      numpoly/cfunctions/cdispach.pyx:26:8: 'List' is not a type identifier
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
      cdef List[np.ndarray] cprepare_input(
          List[np.ndarray] inputs,
          str name,
      ):
          cdef np.ndarray poly
          cdef List[np.ndarray] output = []
              ^
      ------------------------------------------------------------
      
      numpoly/cfunctions/cdispach.pyx:15:9: 'List' is not a type identifier
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
      cdef List[np.ndarray] cprepare_input(
          List[np.ndarray] inputs,
          str name,
      ):
          cdef np.ndarray poly
          cdef List[np.ndarray] output = []
                                        ^
      ------------------------------------------------------------
      
      numpoly/cfunctions/cdispach.pyx:15:35: Cannot coerce list to type '<error>'
      Compiling ./numpoly/cfunctions/cdispach.pyx because it depends on /databricks/python/lib/python3.11/site-packages/numpy/__init__.pxd.
      Compiling ./numpoly/cfunctions/cvalues.pyx because it depends on /databricks/python/lib/python3.11/site-packages/numpy/__init__.pxd.
      Compiling ./numpoly/cfunctions/cmultiply.pyx because it depends on /databricks/python/lib/python3.11/site-packages/numpy/__init__.pxd.
      Compiling ./numpoly/cfunctions/cfrom_attributes.pyx because it depends on /databricks/python/lib/python3.11/site-packages/numpy/__init__.pxd.
      [1/4] Cythonizing ./numpoly/cfunctions/cdispach.pyx
      Traceback (most recent call last):
        File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/databricks/python/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/databricks/python/lib/python3.11/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "/databricks/python/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 17, in <module>
        File "/databricks/python/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1127, in cythonize
          cythonize_one(*args)
        File "/databricks/python/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1250, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: ./numpoly/cfunctions/cdispach.pyx
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

Sorry if this is not the same error, I can create another issue.
I just saw Cython and build issues mentioned, so I assume they are related

[oarcelus]

Hi! Which is the Cython version that you have installed? It looks like it does not support the List type annotation from the typing module. In principle current Cython versions should support this. Honestly I am not very familiar with the databricks environment.

@jonathf, I see that both build issues are related to the cdispach.pyx. Since these functions are not really included in the current version of numpoly maybe we can delete them for the moment and see if errors with other modules (cvalues.pyx, cmultiply.pyx, ...) persist.

[karlr-vestas]

From here it looks like I have Cython 0.29.32

[oarcelus]

That might be a problem. That cython version is old. I am using the 3.0.11 version. Indeed the error messague is already hinting the problem
/databricks/python/lib/python3.11/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2).

It is trying to read the .pyx files as if it were Python2. From the cython documentation.

language_level (2/3/3str), default=None
Globally set the Python language level to be used for module compilation. Default is None, indicating compatibility with Python 3 in Cython 3.x and with Python 2 in Cython 0.x.

So since you are using Cython 0.x version the syntax of the current .pyx files is incompatible with Python 2 and hence your error.

I would try to update cython if you can.

@jonathf, how do you think we should proceed.

[karlr-vestas]

I dont have a lot of control of the environment, unfortunately. I can see that there is a newer version of the databricks runtime with cython version 3.x. I will try that when I have the opportunity.

While trying to debug the error, I noticed a recent commit where you change to only building a source distribution, instead of also including a wheel. 6370b92

I have used chaospy in databricks previously without any problems, so could this be the reason for this build failure?