TDA-ris capable of 30000 nbf (3000 atoms)

[John-zzh]

major updates compared to 1.4.1

1. 3c2e engine updated with 3c2e_bdiv
2. on the fly Coulomb iajb instead of storing tensor
3. store Davidson trial vectors in CPU mem
4. many other optimizations

As of date, some version of MKL-numpy is not compatible with cupy 13.4.1 and cutensor 2.2.0.0, contract engine will complaine about STATUS,
while openblas-numpy-scipy has segmentation fault when solving large FC=SCE.

To use TDA-ris on large system, use

mamba install -c conda-forge \
  cupy=13.4.1 \
  cutensor=2.2.0 \
  numpy scipy \
  blas=*=*mkl

(cupy_mkl_py311) root@volcengine:~/jojo/software/gpu4pyscf_ris/gpu4pyscf/tdscf# ls /root/jojo/miniconda3/envs/cupy_mkl_py311/lib | grep blas
libblas.so
libblas.so.3
libcblas.so
libcblas.so.3
libcublasLt.so.12
libcublasLt.so.12.9.1.4
libcublas.so.12
libcublas.so.12.9.1.4
libnvblas.so.12
libnvblas.so.12.9.1.4

 python -c "import numpy as np; np.show_config()"
... 
  "Build Dependencies": {
    "blas": {
      "name": "blas",
      "found": true,
      "version": "3.9.0",
      "detection method": "pkgconfig",
      "include directory": "/root/jojo/miniconda3/envs/cupy_mkl_py311/include",
      "lib directory": "/root/jojo/miniconda3/envs/cupy_mkl_py311/lib",
      "openblas configuration": "unknown",
      "pc file directory": "/root/jojo/miniconda3/envs/cupy_mkl_py311/lib/pkgconfig"
    },
    "lapack": {
      "name": "lapack",
      "found": true,
      "version": "3.9.0",
      "detection method": "pkgconfig",
      "include directory": "/root/jojo/miniconda3/envs/cupy_mkl_py311/include",
      "lib directory": "/root/jojo/miniconda3/envs/cupy_mkl_py311/lib",
      "openblas configuration": "unknown",
      "pc file directory": "/root/jojo/miniconda3/envs/cupy_mkl_py311/lib/pkgconfig"
    }
...

[jeanwsr]

Wraning, this will not install a mkl-numpy-scipy, the backend is an unknown verision of BLAS,

What if you replace ls by ls -l ? If you see libblas.so is a link to libmkl_rt.so, then numpy is really using mkl.

[John-zzh]

Wraning, this will not install a mkl-numpy-scipy, the backend is an unknown verision of BLAS,

What if you replace ls by ls -l ? If you see libblas.so is a link to libmkl_rt.so, then numpy is really using mkl.

you are right, seems it is really using MKL.

(cupy_mkl_py311) root@volcengine:~/jojo/software/gpu4pyscf_ris/gpu4pyscf/tdscf# ls -l /root/jojo/miniconda3/envs/cupy_mkl_py311/lib | grep mkl
lrwxrwxrwx  1 root root        14 Dec 25 16:49 libblas.so -> libmkl_rt.so.2
lrwxrwxrwx  1 root root        14 Dec 25 16:49 libblas.so.3 -> libmkl_rt.so.2
lrwxrwxrwx  1 root root        14 Dec 25 16:49 libcblas.so -> libmkl_rt.so.2
lrwxrwxrwx  1 root root        14 Dec 25 16:49 libcblas.so.3 -> libmkl_rt.so.2
lrwxrwxrwx  1 root root        14 Dec 25 16:49 liblapacke.so -> libmkl_rt.so.2
lrwxrwxrwx  1 root root        14 Dec 25 16:49 liblapacke.so.3 -> libmkl_rt.so.2
lrwxrwxrwx  1 root root        14 Dec 25 16:49 liblapack.so -> libmkl_rt.so.2
lrwxrwxrwx  1 root root        14 Dec 25 16:49 liblapack.so.3 -> libmkl_rt.so.2
-rwxrwxr-x  3 root root  49788392 Oct 11 02:32 libmkl_avx2.so.2
-rwxrwxr-x  3 root root  73555016 Oct 11 02:32 libmkl_avx512.so.2
lrwxrwxrwx  1 root root        32 Dec 25 16:49 libmkl_blacs_intelmpi_ilp64.so -> libmkl_blacs_intelmpi_ilp64.so.2
-rwxrwxr-x  3 root root    581872 Oct 11 02:32 libmkl_blacs_intelmpi_ilp64.so.2
lrwxrwxrwx  1 root root        31 Dec 25 16:49 libmkl_blacs_intelmpi_lp64.so -> libmkl_blacs_intelmpi_lp64.so.2
-rwxrwxr-x  3 root root    305744 Oct 11 02:32 libmkl_blacs_intelmpi_lp64.so.2
lrwxrwxrwx  1 root root        31 Dec 25 16:49 libmkl_blacs_openmpi_ilp64.so -> libmkl_blacs_openmpi_ilp64.so.2
-rwxrwxr-x  3 root root    586312 Oct 11 02:32 libmkl_blacs_openmpi_ilp64.so.2
lrwxrwxrwx  1 root root        30 Dec 25 16:49 libmkl_blacs_openmpi_lp64.so -> libmkl_blacs_openmpi_lp64.so.2
-rwxrwxr-x  3 root root    301992 Oct 11 02:32 libmkl_blacs_openmpi_lp64.so.2
lrwxrwxrwx  1 root root        21 Dec 25 16:49 libmkl_cdft_core.so -> libmkl_cdft_core.so.2
-rwxrwxr-x  3 root root    195056 Oct 11 02:32 libmkl_cdft_core.so.2
lrwxrwxrwx  1 root root        16 Dec 25 16:49 libmkl_core.so -> libmkl_core.so.2
-rwxrwxr-x  3 root root  72157040 Oct 11 02:32 libmkl_core.so.2
-rwxrwxr-x  3 root root  41036168 Oct 11 02:32 libmkl_def.so.2
lrwxrwxrwx  1 root root        20 Dec 25 16:49 libmkl_gf_ilp64.so -> libmkl_gf_ilp64.so.2
-rwxrwxr-x  3 root root  15185440 Oct 11 02:32 libmkl_gf_ilp64.so.2
lrwxrwxrwx  1 root root        19 Dec 25 16:49 libmkl_gf_lp64.so -> libmkl_gf_lp64.so.2
-rwxrwxr-x  3 root root  17181208 Oct 11 02:32 libmkl_gf_lp64.so.2
lrwxrwxrwx  1 root root        22 Dec 25 16:49 libmkl_gnu_thread.so -> libmkl_gnu_thread.so.2
-rwxrwxr-x  3 root root  28829056 Oct 11 02:32 libmkl_gnu_thread.so.2
lrwxrwxrwx  1 root root        23 Dec 25 16:49 libmkl_intel_ilp64.so -> libmkl_intel_ilp64.so.2
-rwxrwxr-x  3 root root  18290680 Oct 11 02:32 libmkl_intel_ilp64.so.2
lrwxrwxrwx  1 root root        22 Dec 25 16:49 libmkl_intel_lp64.so -> libmkl_intel_lp64.so.2
-rwxrwxr-x  3 root root  20294504 Oct 11 02:32 libmkl_intel_lp64.so.2
lrwxrwxrwx  1 root root        24 Dec 25 16:49 libmkl_intel_thread.so -> libmkl_intel_thread.so.2
-rwxrwxr-x  3 root root  42925416 Oct 11 02:32 libmkl_intel_thread.so.2
-rwxrwxr-x  3 root root  49999992 Oct 11 02:32 libmkl_mc3.so.2
lrwxrwxrwx  1 root root        14 Dec 25 16:49 libmkl_rt.so -> libmkl_rt.so.2
-rwxrwxr-x  3 root root  18291704 Oct 11 02:32 libmkl_rt.so.2
lrwxrwxrwx  1 root root        27 Dec 25 16:49 libmkl_scalapack_ilp64.so -> libmkl_scalapack_ilp64.so.2
-rwxrwxr-x  3 root root   7675808 Oct 11 02:32 libmkl_scalapack_ilp64.so.2
lrwxrwxrwx  1 root root        26 Dec 25 16:49 libmkl_scalapack_lp64.so -> libmkl_scalapack_lp64.so.2
-rwxrwxr-x  3 root root   7695368 Oct 11 02:32 libmkl_scalapack_lp64.so.2
lrwxrwxrwx  1 root root        22 Dec 25 16:49 libmkl_sequential.so -> libmkl_sequential.so.2
-rwxrwxr-x  3 root root  25645064 Oct 11 02:32 libmkl_sequential.so.2
lrwxrwxrwx  1 root root        22 Dec 25 16:49 libmkl_tbb_thread.so -> libmkl_tbb_thread.so.2
-rwxrwxr-x  3 root root  36164656 Oct 11 02:32 libmkl_tbb_thread.so.2
-rwxrwxr-x  3 root root  15804448 Oct 11 02:32 libmkl_vml_avx2.so.2
-rwxrwxr-x  3 root root  16413888 Oct 11 02:32 libmkl_vml_avx512.so.2
-rwxrwxr-x  3 root root   8124792 Oct 11 02:32 libmkl_vml_cmpt.so.2
-rwxrwxr-x  3 root root   9461344 Oct 11 02:32 libmkl_vml_def.so.2
-rwxrwxr-x  3 root root  13247632 Oct 11 02:32 libmkl_vml_mc3.so.2

(cupy_mkl_py311) root@volcengine:~/jojo/software/gpu4pyscf_ris/gpu4pyscf/tdscf# python - << 'EOF'
import ctypes

mkl = ctypes.CDLL('libmkl_rt.so')
buf = ctypes.create_string_buffer(200)
mkl.MKL_Get_Version_String(buf, 200)
print(buf.value.decode())
EOF
Intel(R) oneAPI Math Kernel Library Version 2025.3-Product Build 20251007 for Intel(R) 64 architecture applications

In short,
use conda-forge

mamba install -c conda-forge cupy=13.4.1  cutensor=2.2.0 numpy scipy blas=*=*mkl

and do not use default channel

mamba install -c default  cupy cutensor numpy scipy blas=*=*mkl   # contract engine will complain about CUTENSOR STATUS
mamba install -c default  cupy=13.4.1 cutensor=2.2.0 numpy scipy blas=*=*mkl  # no such version combination

Maybe conda default channel have verison mismatch with contract engine

[John-zzh]

wired, test_ris.py and test_krylov.py works fine from my side