[GLUTEN-11828][VL] Use immutable gpu config and add cuda runtime detection#11830
[GLUTEN-11828][VL] Use immutable gpu config and add cuda runtime detection#11830marin-ma wants to merge 1 commit intoapache:mainfrom
Conversation
|
Run Gluten Clickhouse CI on x86 |
1 similar comment
|
Run Gluten Clickhouse CI on x86 |
|
Run Gluten Clickhouse CI on x86 |
|
Verified using gpu build + spark.gluten.sql.columnar.cudf=true on cpu node. @jinchengchenghh Could you help to review? Thanks! |
| bool hasCudaRuntimeAndDevice() { | ||
| #ifdef GLUTEN_ENABLE_GPU | ||
| int count = 0; | ||
| cudaError_t err = cudaGetDeviceCount(&count); |
There was a problem hiding this comment.
Looks like this code would execute on CPU node, but is the function executes by header, if not, the cuda library does not exist in CPU node, I'm not sure if it can run successfully. If you very it can run well on CPU node without CUDA environment, we may need to add a comment on it.
And the common way is to check if nvidia-smi command exists, if exists, we can check further.
There was a problem hiding this comment.
If we execute the GPU build on a cpu node without CUDA Runtime installed, the process will fail early when loading libvelox.so and reporting the cuda library is missing.
There was a problem hiding this comment.
This is a new problem, we should not require user to install CUDA in CPU node, the build pipeline may also need to be updated
There was a problem hiding this comment.
Please create an issue to track this, thanks!
2 participants
Related issue: #11828