Concoct endless warning due to OpenBLAS

[HouyuZhang]

Dear devlopers,
Thank you for developing this package. I met endless warnings OpenBLAS Warning: Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.
I have checked through each page on the internet related to this issue:

1. Have gone through GitHub endless OpenBLAS Warning #232, and I got a sense that the OpenMP and OpenBLAS modules have problems. But, neither of conda install -y blas=2.5=mkl, conda install -y blas=*=*mkl works for me.
2. endless OpenBLAS Warning again #316. I installed the concoct in a new environment using py3, instead of py2, but still did not work.
3. Request: Please create version with make USE_OPENMP=1  conda-forge/openblas-feedstock#79. I tried to install OpenBLAS and openMP against in conda by conda install libopenblas=*=openmp --channel conda-forge, not work.

Lastly, I used -t 1 to give up parallel running, but also did not work.

Do you have further suggestions on this issue? Also, it will help if you tell me an approximation of the running time of concoct.
Thanks in advance!

Houyu

[HouyuZhang]

Below is my conda env:
name: metawrap-env
channels:

intel
bioconda/label/cf201901
conda-forge
ursky
bioconda
defaults
dependencies:
_libgcc_mutex=0.1=conda_forge
_openmp_mutex=4.5=2_kmp_llvm
biopython=1.81=py310h1fa729e_0
blas=2.106=mkl
bowtie2=2.5.1=py310ha0a81b8_2
brotli=1.0.9=h166bdaf_9
brotli-bin=1.0.9=h166bdaf_9
brotli-python=1.0.9=py310hd8f1fbe_9
bwa=0.7.17=he4a0461_11
bzip2=1.0.8=h7f98852_4
c-ares=1.19.1=hd590300_0
ca-certificates=2023.7.22=hbcca054_0
certifi=2023.7.22=pyhd8ed1ab_0
charset-normalizer=3.2.0=pyhd8ed1ab_0
checkm-genome=1.2.2=pyhdfd78af_1
concoct=1.1.0=py310h74abf4b_3
contourpy=1.1.0=py310hd41b1e2_0
cycler=0.11.0=pyhd8ed1ab_0
cython=3.0.0=py310hc6cd4ac_0
dendropy=4.6.1=pyhdfd78af_0
fonttools=4.42.1=py310h2372a71_0
fraggenescan=1.31=h031d066_6
freetype=2.12.1=hca18f0e_1
gettext=0.21.1=h27087fc_0
gsl=2.7=he838d99_0
hmmer=3.3.2=hdbdd923_4
htslib=1.17=h81da01d_2
icu=72.1=hcb278e6_0
idba=1.1.3=1
idna=3.4=pyhd8ed1ab_0
intel-openmp=2023.1.0=hdb19cb5_46305
jellyfish=1.1.12=h2d50403_0
joblib=1.3.0=pyhd8ed1ab_1
keyutils=1.6.1=h166bdaf_0
kiwisolver=1.4.4=py310hbf28c38_1
kraken=1.1=h470a237_2
krb5=1.21.1=h659d440_0
krona=2.7=0
lcms2=2.15=haa2dc70_1
ld_impl_linux-64=2.40=h41732ed_0
lerc=4.0.0=h27087fc_0
libblas=3.9.0=6_mkl
libbrotlicommon=1.0.9=h166bdaf_9
libbrotlidec=1.0.9=h166bdaf_9
libbrotlienc=1.0.9=h166bdaf_9
libcblas=3.9.0=6_mkl
libcurl=8.2.0=hca28451_0
libdeflate=1.18=h0b41bf4_0
libedit=3.1.20191231=he28a2e2_2
libev=4.33=h516909a_1
libffi=3.4.2=h7f98852_5
libgcc=7.2.0=h69d50b8_2
libgcc-ng=13.1.0=he5830b7_0
libgfortran-ng=13.1.0=h69a702a_0
libgfortran5=13.1.0=h15d22d2_0
libgomp=13.1.0=he5830b7_0
libhwloc=2.9.1=nocuda_h7313eea_6
libiconv=1.17=h166bdaf_0
libidn2=2.3.4=h166bdaf_0
libjpeg-turbo=2.1.5.1=h0b41bf4_0
liblapack=3.9.0=6_mkl
liblapacke=3.9.0=6_mkl
libnghttp2=1.52.0=h61bc06f_0
libnsl=2.0.0=h7f98852_0
libopenblas=0.3.23=openmp_h2a4e791_0
libpng=1.6.39=h753d276_0
libsqlite=3.42.0=h2797004_0
libssh2=1.11.0=h0841786_0
libstdcxx-ng=13.1.0=hfd8a6a1_0
libtiff=4.5.1=h8b53f26_0
libunistring=0.9.10=h7f98852_0
libuuid=2.38.1=h0b41bf4_0
libwebp-base=1.3.1=hd590300_0
libxcb=1.15=h0b41bf4_0
libxml2=2.11.4=h0d562d8_0
libzlib=1.2.13=hd590300_5
llvm-openmp=16.0.6=h4dfa4b3_0
matplotlib-base=3.7.1=py310he60537e_0
maxbin2=2.2.1=0
metabat2=2.12.1=0
mkl=2020.4=h726a3e6_304
mpi=1.0=openmpi
munkres=1.0.7=py_1
ncurses=6.4=hcb278e6_0
nose=1.3.7=py_1006
numpy=1.25.1=py310ha4c1d20_0
openblas=0.3.23=openmp_hef934df_0
openjpeg=2.5.0=hfec8fc6_2
openmpi=4.1.5=h414af15_101
openssl=3.1.2=hd590300_0
packaging=23.1=pyhd8ed1ab_0
pandas=2.0.3=py310h7cbd5c2_1
perl=5.32.1=4_hd590300_perl5
perl-threaded=5.22.0=13
pillow=10.0.0=py310h582fbeb_0
pip=23.2.1=pyhd8ed1ab_0
platformdirs=3.9.1=pyhd8ed1ab_0
pooch=1.7.0=pyha770c72_3
pplacer=1.1.alpha19=h9ee0642_2
prodigal=2.6.3=h031d066_6
pthread-stubs=0.4=h36c2ea0_1001
pyparsing=3.1.1=pyhd8ed1ab_0
pysam=0.21.0=py310h41dec4a_1
pysocks=1.7.1=pyha2e5f31_6
python=3.10.12=hd12c33a_0_cpython
python-dateutil=2.8.2=pyhd8ed1ab_0
python-tzdata=2023.3=pyhd8ed1ab_0
python_abi=3.10=3_cp310
pytz=2023.3=pyhd8ed1ab_0
readline=8.2=h8228510_1
rename=1.601=hdfd78af_1
requests=2.31.0=pyhd8ed1ab_0
samtools=1.17=hd87286a_1
scikit-learn=1.2.2=py310hf7d194e_2
scipy=1.11.1=py310ha4c1d20_0
setuptools=68.0.0=pyhd8ed1ab_0
six=1.16.0=pyh6c4a22f_0
spades=3.13.0=0
sqlite=3.42.0=h2c6b66d_0
tbb=2021.9.0=hf52228f_0
threadpoolctl=3.2.0=pyha21a80b_0
tk=8.6.12=h27826a3_0
typing-extensions=4.7.1=hd8ed1ab_0
typing_extensions=4.7.1=pyha770c72_0
tzdata=2023c=h71feb2d_0
unicodedata2=15.0.0=py310h5764c6d_0
urllib3=2.0.4=pyhd8ed1ab_0
wget=1.20.3=ha35d2d1_1
wheel=0.41.0=pyhd8ed1ab_0
xorg-libxau=1.0.11=hd590300_0
xorg-libxdmcp=1.1.3=h7f98852_0
xz=5.2.6=h166bdaf_0
zlib=1.2.13=hd590300_5
zstd=1.5.2=hfc55251_7
prefix: /lustre/home/tangmoran_2021phd/anaconda3/envs/metawrap-env

[jvollme]

since i just had the same problem and there was no answer here yet, i'll share what helped in my case (even if i do not quite get why):
For me concoct simply stopped working (has started generating this error) even though the conda environment had been successfully used in the past for a long time. maybe someone updated something there and it broke? So i created a new environment to ensure an openMP version of openBlas

I see HouyuZhang has alread tried installing the openMP version of libopenBlas and it did not help him. However in my case it did so i wonder if it may have to do also with the exact concoct-version or the exact libopenblas build?

this is the command that gave me a new environment where concoct seems to work again (note that i already have bioconda subscribed in my standard channel list):
conda create -y -n concoct libopenblas=0.3.28=openmp_hd680484_1 concoct

the following packages were installed:

_libgcc_mutex      conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
  _openmp_mutex      conda-forge/linux-64::_openmp_mutex-4.5-2_kmp_llvm
  biopython          conda-forge/linux-64::biopython-1.84-py312h9a8786e_0
  bzip2              conda-forge/linux-64::bzip2-1.0.8-h4bc722e_7
  c-ares             conda-forge/linux-64::c-ares-1.34.3-hb9d3cd8_1
  ca-certificates    conda-forge/linux-64::ca-certificates-2024.8.30-hbcca054_0
  concoct            bioconda/linux-64::concoct-1.1.0-py312h245ed52_6
  cython             conda-forge/linux-64::cython-3.0.11-py312h8fd2918_3
  gsl                conda-forge/linux-64::gsl-2.7-he838d99_0
  htslib             bioconda/linux-64::htslib-1.21-h5efdd21_0
  joblib             conda-forge/noarch::joblib-1.4.2-pyhd8ed1ab_0
  keyutils           conda-forge/linux-64::keyutils-1.6.1-h166bdaf_0
  krb5               conda-forge/linux-64::krb5-1.21.3-h659f571_0
  ld_impl_linux-64   conda-forge/linux-64::ld_impl_linux-64-2.43-h712a8e2_2
  libblas            conda-forge/linux-64::libblas-3.9.0-25_linux64_openblas
  libcblas           conda-forge/linux-64::libcblas-3.9.0-25_linux64_openblas
  libcurl            conda-forge/linux-64::libcurl-8.10.1-hbbe4b11_0
  libdeflate         conda-forge/linux-64::libdeflate-1.21-h4bc722e_0
  libedit            conda-forge/linux-64::libedit-3.1.20191231-he28a2e2_2
  libev              conda-forge/linux-64::libev-4.33-hd590300_2
  libexpat           conda-forge/linux-64::libexpat-2.6.4-h5888daf_0
  libffi             conda-forge/linux-64::libffi-3.4.2-h7f98852_5
  libgcc             conda-forge/linux-64::libgcc-14.2.0-h77fa898_1
  libgcc-ng          conda-forge/linux-64::libgcc-ng-14.2.0-h69a702a_1
  libgfortran        conda-forge/linux-64::libgfortran-14.2.0-h69a702a_1
  libgfortran5       conda-forge/linux-64::libgfortran5-14.2.0-hd5240d6_1
  liblapack          conda-forge/linux-64::liblapack-3.9.0-25_linux64_openblas
  libnghttp2         conda-forge/linux-64::libnghttp2-1.64.0-h161d5f1_0
  libnsl             conda-forge/linux-64::libnsl-2.0.1-hd590300_0
  libopenblas        conda-forge/linux-64::libopenblas-0.3.28-openmp_hd680484_1
  libsqlite          conda-forge/linux-64::libsqlite-3.47.0-hadc24fc_1
  libssh2            conda-forge/linux-64::libssh2-1.11.1-hf672d98_0
  libstdcxx          conda-forge/linux-64::libstdcxx-14.2.0-hc0a3c3a_1
  libstdcxx-ng       conda-forge/linux-64::libstdcxx-ng-14.2.0-h4852527_1
  libuuid            conda-forge/linux-64::libuuid-2.38.1-h0b41bf4_0
  libxcrypt          conda-forge/linux-64::libxcrypt-4.4.36-hd590300_1
  libzlib            conda-forge/linux-64::libzlib-1.3.1-hb9d3cd8_2
  llvm-openmp        conda-forge/linux-64::llvm-openmp-19.1.4-h024ca30_0
  ncurses            conda-forge/linux-64::ncurses-6.5-he02047a_1
  nose               conda-forge/noarch::nose-1.3.7-py_1006
  numpy              conda-forge/linux-64::numpy-2.1.3-py312h58c1407_0
  openssl            conda-forge/linux-64::openssl-3.4.0-hb9d3cd8_0
  pandas             conda-forge/linux-64::pandas-2.2.2-py312h1d6d2e6_1
  pip                conda-forge/noarch::pip-24.3.1-pyh8b19718_0
  python             conda-forge/linux-64::python-3.12.7-hc5c86c4_0_cpython
  python-dateutil    conda-forge/noarch::python-dateutil-2.9.0.post0-pyhff2d567_0
  python-tzdata      conda-forge/noarch::python-tzdata-2024.2-pyhd8ed1ab_0
  python_abi         conda-forge/linux-64::python_abi-3.12-5_cp312
  pytz               conda-forge/noarch::pytz-2024.2-pyhd8ed1ab_0
  readline           conda-forge/linux-64::readline-8.2-h8228510_1
  samtools           bioconda/linux-64::samtools-1.21-h50ea8bc_0
  scikit-learn       conda-forge/linux-64::scikit-learn-1.5.2-py312h7a48858_1
  scipy              conda-forge/linux-64::scipy-1.14.1-py312h62794b6_1
  setuptools         conda-forge/noarch::setuptools-75.6.0-pyhff2d567_0
  six                conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  threadpoolctl      conda-forge/noarch::threadpoolctl-3.5.0-pyhc1e730c_0
  tk                 conda-forge/linux-64::tk-8.6.13-noxft_h4845f30_101
  tzdata             conda-forge/noarch::tzdata-2024b-hc8b5060_0
  wheel              conda-forge/noarch::wheel-0.45.1-pyhd8ed1ab_0
  xz                 conda-forge/linux-64::xz-5.2.6-h166bdaf_0
  zstd               conda-forge/linux-64::zstd-1.5.6-ha6fb4c9_0

hope this helps...

[hugolefeuvre]

Using libopenblas=0.3.28=openmp_hd680484_1 in my conda environment work just fine for me, thank you for your answer @jvollme !