question: how to develop based on qe-compiler

[Zhaoyilunnn]

Hi Team,

Currently I still cannot figure out how to make this project dependent on qe-compiler. qe-compiler is an excellent project, I believe many users would like to develop their own full compilers based on this framework. Thus it would be helpful if you could provide some instructions on how to achieve this. I am also very willing to contribute if it requires more effort to make this project a conan package that other projects can depend on.

Here are two ways I have tried but failed. Note that I can seamlessly build qe-compiler locally, but I just cannot build a conan package of it and then make my own project dependent on qe-compiler.

01 Manually

This is done by copying conandata.yml and conanfile.py to the path ./conan/qec and create a conan_deps.sh, which essentially calls conan export

Direclly use the original conanfile.py does not work because it sets exports_sources = "*", resulting in a non-empty directory ~/.conan/data/qss-compiler/0.1.0/qss-compiler/stable/export_source/. Then conan will copy conandata.yml, conanfile.py, conanmanifest.txt into ~/.conan/data/qss-compiler/0.1.0/qss-compiler/stable/source/. But conan cannot delete conandata.yml, resulting in a non-empty source dir, preventing cloning source file of qe-compiler.

Steps

Manually create a conan package and export it to local cache and then build it

./conan_deps.sh

Then install qe-compiler through conan

mkdir build && cd build
cd build
conan install .. --build=outdated -pr:h default -pr:b default

This method finally results in

CMake Error at /root/miniconda3/envs/quos-dev/lib/python3.10/site-packages/cmake/data/share/cmake-3.23/Modules/CMakeDetermineSystem.cmake:130 (message):
  Could not find toolchain file: conan_toolchain.cmake
Call Stack (most recent call first):
  CMakeLists.txt:46 (enable_language)


CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_ASM_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

02 Conan

Then I tried using conan create to create a local conan package.

In the root directory of qe-compiler

cd build
conan create .. zhaoyilun/qe-compiler --build=outdated -pr:h default -pr:b default -e QSSC_VERSION=0.1.0

Initially, I met following error

fatal: not a git repository (or any of the parent directories): .git
-- Root CMake project Git directory (used for version):
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/root/miniconda3/envs/quos-dev/lib/python3.10/site-packages/setuptools_scm/__init__.py", line 148, in get_version
    _version_missing(config)
  File "/root/miniconda3/envs/quos-dev/lib/python3.10/site-packages/setuptools_scm/__init__.py", line 108, in _version_missing
    raise LookupError(
LookupError: setuptools-scm was unable to detect version for /root/.conan/data/qss-compiler/0.1.0/zhaoyilun/qe-compiler/build/d28c692a69e3ccd7dc8cd629cfb34b8fd367905b.

Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

The root cause is that conan will not copy .git directory into the local cache, resulting in the error of setuptools_scm. After checking the CMakeLists.txt, I found that I can explicitly set VERSION_STRING to avoid using setuptools_scm.

After fixing the above error, it results in a cyclic dependency error

CMake Error: The inter-target dependency graph contains the following strongly connected component (cycle):
  "QSSPythonSources.Dialects.quir" of type INTERFACE_LIBRARY
    depends on "QSSPythonModules.sources.QSSPythonSources.Dialects.quir" (strong)
  "QSSPythonModules.sources.QSSPythonSources.Dialects.quir" of type UTILITY
    depends on "QSSPythonSources.Dialects.quir" (strong)
At least one of these targets is not a STATIC_LIBRARY.  Cyclic dependencies are allowed only among static libraries.
CMake Generate step failed.  Build files cannot be regenerated correctly.

System info

PRETTY_NAME="Ubuntu Jammy Jellyfish (development branch)"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

[Zhaoyilunnn]

I made some changes to existing config files main...Zhaoyilunnn:qe-compiler:zhaoyilun-local-conan-package

Then a local conan package can be built through conan create . zhaoyilun/qe-compiler --build=outdated -pr:h default -pr:b default -o qss-compiler:pythonlib=False

[taalexander]

Hi @Zhaoyilunnn, thank you for writing up this issue. The qe-compiler is still in its infancy of being an opensource project and we certainly have a lot of work to do in migrating documentation that needs to be resumed.

Note that I can seamlessly build qe-compiler locally, but I just cannot build a conan package of it and then make my own project dependent on qe-compiler.

In particular, we have not attempted this path to development yet, but are very interested in this. Internally, we are using the project as a submodule with integration through the Cmake build system.

I made some changes to existing config files main...Zhaoyilunnn:qe-compiler:zhaoyilun-local-conan-package
Then a local conan package can be built through conan create . zhaoyilun/qe-compiler --build=outdated -pr:h default -pr:b default -o qss-compiler:pythonlib=False

We would be very interested if you could make a draft PR demonstrating this work so that we could discuss how it might be incorporated in-tree.

[taalexander]

An obfuscated version of our top-level CMakeLists to integrate the project

cmake_minimum_required(VERSION 3.25)
project(target-compiler)
set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/build_tools/cmake ${CMAKE_MODULE_PATH})

set(QSSC_SYSTEMS_DIR
        "${CMAKE_CURRENT_SOURCE_DIR}/targets/systems"
        CACHE STRING
        "The directory containing the common systems"
        )

option(QSSC_WITH_MOCK_TARGET "Available targets include the built-in mock target" ON)
option(QSSC_WITH_SYSTEM_CONFIG "Use system_config to retrieve configuration for targets" OFF)
option(QSSC_BUILD_DOCS "Build the documentation for the compiler" ON)

# List of directories containing compiler targets. Subtargets should be
# listed first so that they are built before top-level targets
set(QSSC_TARGET_DIRS
        ${CMAKE_CURRENT_SOURCE_DIR}/targets/systems/YourTarget;
        CACHE PATH
        "List of directories containing compiler targets."
        )

set(QSSC_TARGET_TEST_DIRS
        ${CMAKE_CURRENT_SOURCE_DIR}/targets/systems/YourTarget/test;
        CACHE PATH
        "List of directories containing compiler LIT test suites."
        )

set(QSSC_TARGET_PYTHON_DIRS
        ${CMAKE_CURRENT_SOURCE_DIR}/targets/systems/YourTarget/python_lib/
        CACHE PATH
        "List of directories containing compiler target python additions."
        )

# Build type
if (NOT CMAKE_BUILD_TYPE)
    set(CMAKE_BUILD_TYPE Debug)
endif()
set(CMAKE_COMPILE_WARNING_AS_ERROR ON)

# Here is the submodule which we integrate through
# the Cmake build system with
add_subdirectory(qe-compiler)

In this way our internal project contains our targets in a very similar way to https://github.com/openqasm/qe-compiler/tree/main/targets/systems/mock with a top-level cmake file which we use to generate internal binaries. The rough project structure is

• target-compiler/
  • qe-compiler/ (submodule)
  • targets/systems/YourTarget/
  • CMakeLists.txt

We could be very interested in enabling the pathway to remove the qe-compiler submodule and replace it with a Conan dependency.

[Zhaoyilunnn]

An obfuscated version of our top-level CMakeLists to integrate the project

cmake_minimum_required(VERSION 3.25)
project(target-compiler)
set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/build_tools/cmake ${CMAKE_MODULE_PATH})

set(QSSC_SYSTEMS_DIR
        "${CMAKE_CURRENT_SOURCE_DIR}/targets/systems"
        CACHE STRING
        "The directory containing the common systems"
        )

option(QSSC_WITH_MOCK_TARGET "Available targets include the built-in mock target" ON)
option(QSSC_WITH_SYSTEM_CONFIG "Use system_config to retrieve configuration for targets" OFF)
option(QSSC_BUILD_DOCS "Build the documentation for the compiler" ON)

# List of directories containing compiler targets. Subtargets should be
# listed first so that they are built before top-level targets
set(QSSC_TARGET_DIRS
        ${CMAKE_CURRENT_SOURCE_DIR}/targets/systems/YourTarget;
        CACHE PATH
        "List of directories containing compiler targets."
        )

set(QSSC_TARGET_TEST_DIRS
        ${CMAKE_CURRENT_SOURCE_DIR}/targets/systems/YourTarget/test;
        CACHE PATH
        "List of directories containing compiler LIT test suites."
        )

set(QSSC_TARGET_PYTHON_DIRS
        ${CMAKE_CURRENT_SOURCE_DIR}/targets/systems/YourTarget/python_lib/
        CACHE PATH
        "List of directories containing compiler target python additions."
        )

# Build type
if (NOT CMAKE_BUILD_TYPE)
    set(CMAKE_BUILD_TYPE Debug)
endif()
set(CMAKE_COMPILE_WARNING_AS_ERROR ON)

# Here is the submodule which we integrate through
# the Cmake build system with
add_subdirectory(qe-compiler)

In this way our internal project contains our targets in a very similar way to https://github.com/openqasm/qe-compiler/tree/main/targets/systems/mock with a top-level cmake file which we use to generate internal binaries. The rough project structure is

• target-compiler/

  • qe-compiler/ (submodule)
  • targets/systems/YourTarget/
  • CMakeLists.txt

We could be very interested in enabling the pathway to remove the qe-compiler submodule and replace it with a Conan dependency.

@taalexander Thanks a lot for your example CMakeLists.txt, actually I gave up making qe-compiler a conan package and adopted a similar top-level CMakeLists.txt ^_^

I am still in the progress of finding the correct way to make it a Conan package. I will make a PR once I figure it out

[taalexander]

Thank you @Zhaoyilunnn, happy to help however you might like. In general, as you're making progress I'd love to chat to understand better what you are trying to do and how to enable it.

[anurag-quantum]

I was trying to build with Mac M1 pro but seeing the error below:

(qiskit_env) anuragpateriya@anurags-MBP build % conan install .. --build=outdated -pr:h default -pr:b default
Conan 1 is on a deprecation path, please consider migrating to Conan 2
Configuration (profile_host):
[settings]
arch=armv8
arch_build=armv8
build_type=Release
os=Macos
os_build=Macos
[options]
[build_requires]
[env]

Configuration (profile_build):
[settings]
arch=armv8
arch_build=armv8
build_type=Release
os=Macos
os_build=Macos
[options]
[build_requires]
[env]

ERROR: qss-compiler/None: 'settings.compiler' value not defined
(qiskit_env) anuragpateriya@anurags-MBP build % 

can you help how to resolve this?

[taalexander]

This seems like a conan issue. It appears to not have been able to detect your compiler version. Could you please try

conan profile new default --detect --force

To generate a new profile? This is following this issue

[anurag-quantum]

Thanks , that worked. Can we add this in README for future?

[taalexander]

Sure, would you mind creating a PR quickly and I'll approve. Something along the lines of making sure a profile is setup with

conan profile new default --detect

[anurag-quantum]

Raised the PR 339

[anurag-quantum]

Sure, would you mind creating a PR quickly and I'll approve. Something along the lines of making sure a profile is setup with

conan profile new default --detect

Hi @taalexander ,
Did you get time to check the PR?339

[anurag-quantum]

After the error resolved regarding the profile setup , i ran next step , one other error is seen!!

CMake Error at /Users/anuragpateriya/Library/Python/3.9/lib/python/site-packages/cmake/data/share/cmake-3.30/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
  Could NOT find Python3 (missing: Python3_NumPy_INCLUDE_DIRS NumPy) (found
  suitable version "3.9.6", minimum required is "3.0")
Call Stack (most recent call first):

@taalexander

anuragpateriya@anurags-mbp build % conan install .. --build=outdated -pr:h default -pr:b default
/Users/anuragpateriya/Library/Python/3.9/lib/python/site-packages/requests/__init__.py:86: RequestsDependencyWarning: Unable to find acceptable character detection dependency (chardet or charset_normalizer).
  warnings.warn(
Conan 1 is on a deprecation path, please consider migrating to Conan 2
Configuration (profile_host):
[settings]
arch=armv8
arch_build=armv8
build_type=Release
compiler=apple-clang
compiler.libcxx=libc++
compiler.version=15
os=Macos
os_build=Macos
[options]
[build_requires]
[env]

Configuration (profile_build):
[settings]
arch=armv8
arch_build=armv8
build_type=Release
compiler=apple-clang
compiler.libcxx=libc++
compiler.version=15
os=Macos
os_build=Macos
[options]
[build_requires]
[env]

WARN: libzip/1.10.1: requirement zlib/[>=1.2.11 <2] overridden by qss-compiler/None to zlib/1.2.13 
WARN: openssl/3.3.2: requirement zlib/[>=1.2.11 <2] overridden by libzip/1.10.1 to zlib/1.2.13 
WARN: mpc/1.3.1: requirement mpfr/4.2.0 overridden by qasm/0.3.3@qss/stable to mpfr/4.1.0 
bzip2/1.0.8: Package is up to date
clang-tools-extra/17.0.5-0: Package is up to date
gmp/6.3.0: Package is up to date
gtest/1.11.0: Package is up to date
m4/1.4.19: Package is up to date
nlohmann_json/3.9.1: Package is up to date
pybind11/2.11.1: Package is up to date
xz_utils/5.4.5: Package is up to date
zlib/1.2.13: Package is up to date
zstd/1.5.5: Package is up to date
flex/2.6.4: Package is up to date
llvm/17.0.5-0: WARN: Package binary is corrupted, removing: 60e1ca9cf1a1629140c76766aba050bf24034339
Version ranges solved
    Version range '>=1.1 <4' required by 'libzip/1.10.1' resolved to 'openssl/3.3.2' in local cache

conanfile.py (qss-compiler/None): Installing package
Requirements
    bison/3.8.2 from 'conancenter' - Cache
    bzip2/1.0.8 from 'conancenter' - Cache
    clang-tools-extra/17.0.5-0 from local cache - Cache
    flex/2.6.4 from 'conancenter' - Cache
    gmp/6.3.0 from 'conancenter' - Cache
    gtest/1.11.0 from 'conancenter' - Cache
    libzip/1.10.1 from 'conancenter' - Cache
    llvm/17.0.5-0 from local cache - Cache
    m4/1.4.19 from 'conancenter' - Cache
    mpc/1.3.1 from 'conancenter' - Cache
    mpfr/4.1.0 from 'conancenter' - Cache
    nlohmann_json/3.9.1 from 'conancenter' - Cache
    openssl/3.3.2 from 'conancenter' - Cache
    pybind11/2.11.1 from 'conancenter' - Cache
    qasm/0.3.3@qss/stable from local cache - Cache
    xz_utils/5.4.5 from 'conancenter' - Cache
    zlib/1.2.13 from 'conancenter' - Cache
    zstd/1.5.5 from 'conancenter' - Cache
Packages
    bison/3.8.2:f64928b7d16a6272dea208460e11e509fc79caf3 - Build
    bzip2/1.0.8:e678d526ab22540179f155f996fc23a941dfe05a - Cache
    clang-tools-extra/17.0.5-0:b94b3e20dcf84d923f5281ae5fda07413382811b - Cache
    flex/2.6.4:f64928b7d16a6272dea208460e11e509fc79caf3 - Cache
    gmp/6.3.0:25f8261418ab8ee78955db9f8bb0ae52224fe34f - Cache
    gtest/1.11.0:2e0989b314164006a0afad237a653afc8f947c64 - Cache
    libzip/1.10.1:b35e5ecdf3dbe261c8a36112e7ea50c9faf495ee - Build
    llvm/17.0.5-0:60e1ca9cf1a1629140c76766aba050bf24034339 - Build
    m4/1.4.19:3eb32b976fb0626f3c40a8b87ace72156aa44204 - Cache
    mpc/1.3.1:5ea6fa268e74fdea4cfcee1403284796a164f88a - Build
    mpfr/4.1.0:ea30eefad43ad1c77a109ec9f8edcc64fcbf62a5 - Build
    nlohmann_json/3.9.1:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    openssl/3.3.2:8a73dad672d6dfb98774f9915bc06ab2bd085a90 - Build
    pybind11/2.11.1:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    qasm/0.3.3@qss/stable:2d563367076c6819862296bac5f3576d6c5b0c1d - Build
    xz_utils/5.4.5:6f12be38b2e454b75ce1f2efb692e5b77c8cdee3 - Cache
    zlib/1.2.13:6f12be38b2e454b75ce1f2efb692e5b77c8cdee3 - Cache
    zstd/1.5.5:ff9e8482a36218a5d5b20ac2c64b497df2eecc5e - Cache
Build requirements
    bison/3.8.2 from 'conancenter' - Cache
    clang-tools-extra/17.0.5-0 from local cache - Cache
    flex/2.6.4 from 'conancenter' - Cache
    llvm/17.0.5-0 from local cache - Cache
    m4/1.4.19 from 'conancenter' - Cache
    zlib/1.2.13 from 'conancenter' - Cache
Build requirements packages
    bison/3.8.2:f64928b7d16a6272dea208460e11e509fc79caf3 - Build
    clang-tools-extra/17.0.5-0:b94b3e20dcf84d923f5281ae5fda07413382811b - Cache
    flex/2.6.4:f64928b7d16a6272dea208460e11e509fc79caf3 - Cache
    llvm/17.0.5-0:d4f312835a1ce7e89c6bb6988129dfa3484c79cb - Build
    m4/1.4.19:3eb32b976fb0626f3c40a8b87ace72156aa44204 - Cache
    zlib/1.2.13:6f12be38b2e454b75ce1f2efb692e5b77c8cdee3 - Cache

Installing (downloading, building) binaries...
bzip2/1.0.8: Already installed!
clang-tools-extra/17.0.5-0: Already installed!
gmp/6.3.0: Already installed!
gtest/1.11.0: Already installed!
m4/1.4.19: Already installed!
nlohmann_json/3.9.1: Already installed!
pybind11/2.11.1: Already installed!
xz_utils/5.4.5: Already installed!
zlib/1.2.13: Already installed!
zstd/1.5.5: Already installed!
flex/2.6.4: Already installed!
flex/2.6.4: Appending PATH environment variable: /Users/anuragpateriya/.conan/data/flex/2.6.4/_/_/package/f64928b7d16a6272dea208460e11e509fc79caf3/bin
flex/2.6.4: Setting LEX environment variable: /Users/anuragpateriya/.conan/data/flex/2.6.4/_/_/package/f64928b7d16a6272dea208460e11e509fc79caf3/bin/flex
flex/2.6.4: Appending PATH environment variable: /Users/anuragpateriya/.conan/data/flex/2.6.4/_/_/package/f64928b7d16a6272dea208460e11e509fc79caf3/bin
flex/2.6.4: Setting LEX environment variable: /Users/anuragpateriya/.conan/data/flex/2.6.4/_/_/package/f64928b7d16a6272dea208460e11e509fc79caf3/bin/flex
flex/2.6.4: Appending PATH environment variable: /Users/anuragpateriya/.conan/data/flex/2.6.4/_/_/package/f64928b7d16a6272dea208460e11e509fc79caf3/bin
flex/2.6.4: Setting LEX environment variable: /Users/anuragpateriya/.conan/data/flex/2.6.4/_/_/package/f64928b7d16a6272dea208460e11e509fc79caf3/bin/flex
flex/2.6.4: Appending PATH environment variable: /Users/anuragpateriya/.conan/data/flex/2.6.4/_/_/package/f64928b7d16a6272dea208460e11e509fc79caf3/bin
flex/2.6.4: Setting LEX environment variable: /Users/anuragpateriya/.conan/data/flex/2.6.4/_/_/package/f64928b7d16a6272dea208460e11e509fc79caf3/bin/flex
llvm/17.0.5-0: WARN: Build folder is dirty, removing it: /Users/anuragpateriya/.conan/data/llvm/17.0.5-0/_/_/build/60e1ca9cf1a1629140c76766aba050bf24034339
llvm/17.0.5-0: Configuring sources in /Users/anuragpateriya/.conan/data/llvm/17.0.5-0/_/_/source
Cloning into 'llvm-project'...
remote: Enumerating objects: 5462185, done.
remote: Counting objects: 100% (8494/8494), done.
remote: Compressing objects: 100% (1123/1123), done.
Receiving objects: 100% (5462185/5462185), 1.03 GiB | 3.62 MiB/s, done.
remote: Total 5462185 (delta 8166), reused 7388 (delta 7371), pack-reused 5453691 (from 1)
Resolving deltas: 100% (4512802/4512802), done.
Note: switching to '98bfdac5ce82d1679f8af9a57501471812ab68d7'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

Updating files: 100% (132077/132077), done.
llvm/17.0.5-0: Building your package in /Users/anuragpateriya/.conan/data/llvm/17.0.5-0/_/_/build/60e1ca9cf1a1629140c76766aba050bf24034339
llvm/17.0.5-0: WARN: 
     ************************************************
     The 'cmake' generator is deprecated.
     Please update your code and remove it.
     *************************************************

llvm/17.0.5-0: Generator cmake created conanbuildinfo.cmake
llvm/17.0.5-0: Aggregating env generators
llvm/17.0.5-0: Calling build()
llvm/17.0.5-0: WARN: **** The 'from conans import CMake' helper is deprecated. Please update your code and remove it. ****
-- The C compiler identification is AppleClang 15.0.0.15000309
-- The CXX compiler identification is AppleClang 15.0.0.15000309
-- The ASM compiler identification is AppleClang
-- Found assembler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- bolt project is disabled
-- clang project is disabled
-- clang-tools-extra project is disabled
-- compiler-rt project is disabled
-- cross-project-tests project is disabled
-- libc project is disabled
-- libclc project is disabled
-- lld project is enabled
-- lldb project is disabled
-- mlir project is enabled
-- openmp project is disabled
-- polly project is disabled
-- pstl project is disabled
-- flang project is disabled
-- Found libtool - /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool
-- Found Python3: /Applications/Xcode.app/Contents/Developer/usr/bin/python3 (found suitable version "3.9.6", minimum required is "3.0") found components: Interpreter
-- Looking for dlfcn.h
-- Looking for dlfcn.h - found
-- Looking for errno.h
-- Looking for errno.h - found
-- Looking for fcntl.h
-- Looking for fcntl.h - found
-- Looking for link.h
-- Looking for link.h - not found
-- Looking for malloc/malloc.h
-- Looking for malloc/malloc.h - found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for signal.h
-- Looking for signal.h - found
-- Looking for sys/ioctl.h
-- Looking for sys/ioctl.h - found
-- Looking for sys/mman.h
-- Looking for sys/mman.h - found
-- Looking for sys/param.h
-- Looking for sys/param.h - found
-- Looking for sys/resource.h
-- Looking for sys/resource.h - found
-- Looking for sys/stat.h
-- Looking for sys/stat.h - found
-- Looking for sys/time.h
-- Looking for sys/time.h - found
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for sysexits.h
-- Looking for sysexits.h - found
-- Looking for termios.h
-- Looking for termios.h - found
-- Looking for unistd.h
-- Looking for unistd.h - found
-- Looking for valgrind/valgrind.h
-- Looking for valgrind/valgrind.h - not found
-- Looking for fenv.h
-- Looking for fenv.h - found
-- Looking for FE_ALL_EXCEPT
-- Looking for FE_ALL_EXCEPT - found
-- Looking for FE_INEXACT
-- Looking for FE_INEXACT - found
-- Performing Test HAVE_BUILTIN_THREAD_POINTER
-- Performing Test HAVE_BUILTIN_THREAD_POINTER - Success
-- Looking for mach/mach.h
-- Looking for mach/mach.h - found
-- Looking for CrashReporterClient.h
-- Looking for CrashReporterClient.h - not found
-- Performing Test HAVE_CRASHREPORTER_INFO
-- Performing Test HAVE_CRASHREPORTER_INFO - Success
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Looking for pthread_rwlock_init in pthread
-- Looking for pthread_rwlock_init in pthread - found
-- Looking for pthread_mutex_lock in pthread
-- Looking for pthread_mutex_lock in pthread - found
-- Looking for dlopen in dl
-- Looking for dlopen in dl - found
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - not found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found ZLIB: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk/usr/lib/libz.tbd (found version "1.2.12")
-- Looking for compress2
-- Looking for compress2 - found
-- Looking for xar_open in xar
-- Looking for xar_open in xar - found
-- The xar file format has been deprecated: LLVM_HAVE_LIBXAR might be removed in the future.
-- Looking for arc4random
-- Looking for arc4random - found
-- Looking for backtrace
-- Looking for backtrace - found
-- backtrace facility detected in default set of libraries
-- Found Backtrace: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk/usr/include
-- Performing Test C_SUPPORTS_WERROR_UNGUARDED_AVAILABILITY_NEW
-- Performing Test C_SUPPORTS_WERROR_UNGUARDED_AVAILABILITY_NEW - Success
-- Looking for __register_frame
-- Looking for __register_frame - found
-- Looking for __deregister_frame
-- Looking for __deregister_frame - found
-- Looking for __unw_add_dynamic_fde
-- Looking for __unw_add_dynamic_fde - found
-- Looking for _Unwind_Backtrace
-- Looking for _Unwind_Backtrace - found
-- Looking for getpagesize
-- Looking for getpagesize - found
-- Looking for sysconf
-- Looking for sysconf - found
-- Looking for getrusage
-- Looking for getrusage - found
-- Looking for setrlimit
-- Looking for setrlimit - found
-- Looking for isatty
-- Looking for isatty - found
-- Looking for futimens
-- Looking for futimens - found
-- Looking for futimes
-- Looking for futimes - found
-- Looking for mallctl
-- Looking for mallctl - not found
-- Looking for mallinfo
-- Looking for mallinfo - not found
-- Looking for mallinfo2
-- Looking for mallinfo2 - not found
-- Looking for malloc_zone_statistics
-- Looking for malloc_zone_statistics - found
-- Looking for getrlimit
-- Looking for getrlimit - found
-- Looking for posix_spawn
-- Looking for posix_spawn - found
-- Looking for pread
-- Looking for pread - found
-- Looking for sbrk
-- Looking for sbrk - found
-- Looking for strerror_r
-- Looking for strerror_r - found
-- Looking for strerror_s
-- Looking for strerror_s - not found
-- Looking for setenv
-- Looking for setenv - found
-- Performing Test HAVE_STRUCT_STAT_ST_MTIMESPEC_TV_NSEC
-- Performing Test HAVE_STRUCT_STAT_ST_MTIMESPEC_TV_NSEC - Success
-- Performing Test HAVE_STRUCT_STAT_ST_MTIM_TV_NSEC
-- Performing Test HAVE_STRUCT_STAT_ST_MTIM_TV_NSEC - Failed
-- Looking for __GLIBC__
-- Looking for __GLIBC__ - not found
-- Looking for pthread_getname_np
-- Looking for pthread_getname_np - found
-- Looking for pthread_setname_np
-- Looking for pthread_setname_np - found
-- Looking for dlopen
-- Looking for dlopen - found
-- Looking for dladdr
-- Looking for dladdr - found
-- Looking for proc_pid_rusage
-- Looking for proc_pid_rusage - found
-- Performing Test HAVE_CXX_ATOMICS_WITHOUT_LIB
-- Performing Test HAVE_CXX_ATOMICS_WITHOUT_LIB - Success
-- Performing Test HAVE_CXX_ATOMICS64_WITHOUT_LIB
-- Performing Test HAVE_CXX_ATOMICS64_WITHOUT_LIB - Success
-- Performing Test LLVM_HAS_ATOMICS
-- Performing Test LLVM_HAS_ATOMICS - Success
-- Performing Test SUPPORTS_VARIADIC_MACROS_FLAG
-- Performing Test SUPPORTS_VARIADIC_MACROS_FLAG - Success
-- Performing Test SUPPORTS_GNU_ZERO_VARIADIC_MACRO_ARGUMENTS_FLAG
-- Performing Test SUPPORTS_GNU_ZERO_VARIADIC_MACRO_ARGUMENTS_FLAG - Success
-- Native target architecture is X86
-- Threads enabled.
-- Doxygen disabled.
-- Ninja version: 1.11.1.git.kitware.jobserver-1
-- Found ld64 - /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld
-- Could NOT find OCaml (missing: OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH) 
-- OCaml bindings disabled.
-- Found Python module pygments
-- Found Python module pygments.lexers.c_cpp
-- Found Python module yaml
-- LLVM host triple: x86_64-apple-darwin23.5.0
-- LLVM default target triple: x86_64-apple-darwin23.5.0
-- Performing Test C_SUPPORTS_FPIC
-- Performing Test C_SUPPORTS_FPIC - Success
-- Performing Test CXX_SUPPORTS_FPIC
-- Performing Test CXX_SUPPORTS_FPIC - Success
-- Building with -fPIC
-- Performing Test C_SUPPORTS_FNO_SEMANTIC_INTERPOSITION
-- Performing Test C_SUPPORTS_FNO_SEMANTIC_INTERPOSITION - Failed
-- Performing Test CXX_SUPPORTS_FNO_SEMANTIC_INTERPOSITION
-- Performing Test CXX_SUPPORTS_FNO_SEMANTIC_INTERPOSITION - Failed
-- Performing Test SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG
-- Performing Test SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG - Success
-- Performing Test C_SUPPORTS_WERROR_DATE_TIME
-- Performing Test C_SUPPORTS_WERROR_DATE_TIME - Success
-- Performing Test CXX_SUPPORTS_WERROR_DATE_TIME
-- Performing Test CXX_SUPPORTS_WERROR_DATE_TIME - Success
-- Performing Test CXX_SUPPORTS_WERROR_UNGUARDED_AVAILABILITY_NEW
-- Performing Test CXX_SUPPORTS_WERROR_UNGUARDED_AVAILABILITY_NEW - Success
-- Performing Test C_SUPPORTS_CMAKE_CXX_FLAGS
-- Performing Test C_SUPPORTS_CMAKE_CXX_FLAGS - Failed
-- Performing Test CXX_SUPPORTS_CMAKE_CXX_FLAGS
-- Performing Test CXX_SUPPORTS_CMAKE_CXX_FLAGS - Failed
-- Performing Test LINKER_SUPPORTS_COLOR_DIAGNOSTICS
-- Performing Test LINKER_SUPPORTS_COLOR_DIAGNOSTICS - Failed
-- Looking for os_signpost_interval_begin
-- Looking for os_signpost_interval_begin - found
-- Performing Test macos_signposts_usable
-- Performing Test macos_signposts_usable - Success
-- Linker detection: unknown
-- Performing Test HAS_WERROR_GLOBAL_CTORS
-- Performing Test HAS_WERROR_GLOBAL_CTORS - Success
-- Performing Test LLVM_HAS_NOGLOBAL_CTOR_MUTEX
-- Performing Test LLVM_HAS_NOGLOBAL_CTOR_MUTEX - Success
-- Looking for __x86_64__
-- Looking for __x86_64__ - not found
-- Found Git: /usr/bin/git (found version "2.39.3 (Apple Git-146)")
-- Targeting X86
-- Targeting PowerPC
-- Targeting AArch64
-- LLD version: 17.0.5
-- Performing Test C_SUPPORTS_WERROR_IMPLICIT_FUNCTION_DECLARATION
-- Performing Test C_SUPPORTS_WERROR_IMPLICIT_FUNCTION_DECLARATION - Success
-- Performing Test C_SUPPORTS_WERROR_MISMATCHED_TAGS
-- Performing Test C_SUPPORTS_WERROR_MISMATCHED_TAGS - Success
-- Found Python3: /Applications/Xcode.app/Contents/Developer/usr/bin/python3 (found suitable version "3.9.6", minimum required is "3.0") found components: Interpreter Development Development.Module Development.Embed
CMake Error at /Users/anuragpateriya/Library/Python/3.9/lib/python/site-packages/cmake/data/share/cmake-3.30/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
  Could NOT find Python3 (missing: Python3_NumPy_INCLUDE_DIRS NumPy) (found
  suitable version "3.9.6", minimum required is "3.0")
Call Stack (most recent call first):
  /Users/anuragpateriya/Library/Python/3.9/lib/python/site-packages/cmake/data/share/cmake-3.30/Modules/FindPackageHandleStandardArgs.cmake:603 (_FPHSA_FAILURE_MESSAGE)
  /Users/anuragpateriya/Library/Python/3.9/lib/python/site-packages/cmake/data/share/cmake-3.30/Modules/FindPython/Support.cmake:4001 (find_package_handle_standard_args)
  /Users/anuragpateriya/Library/Python/3.9/lib/python/site-packages/cmake/data/share/cmake-3.30/Modules/FindPython3.cmake:602 (include)
  /Users/anuragpateriya/.conan/data/llvm/17.0.5-0/_/_/source/llvm-project/mlir/cmake/modules/MLIRDetectPythonEnv.cmake:21 (find_package)
  /Users/anuragpateriya/.conan/data/llvm/17.0.5-0/_/_/source/llvm-project/mlir/CMakeLists.txt:156 (mlir_configure_python_dev_packages)


-- Configuring incomplete, errors occurred!
llvm/17.0.5-0: 
llvm/17.0.5-0: ERROR: Package '60e1ca9cf1a1629140c76766aba050bf24034339' build failed
llvm/17.0.5-0: WARN: Build folder /Users/anuragpateriya/.conan/data/llvm/17.0.5-0/_/_/build/60e1ca9cf1a1629140c76766aba050bf24034339
ERROR: llvm/17.0.5-0: Error in build() method, line 251
        cmake.configure(source_folder=self._source_subfolder)
        ConanException: Error 1 while executing cd '/Users/anuragpateriya/.conan/data/llvm/17.0.5-0/_/_/build/60e1ca9cf1a1629140c76766aba050bf24034339' && cmake -G "Ninja" -DCMAKE_BUILD_TYPE="Release" -DCMAKE_OSX_ARCHITECTURES="arm64" -DCMAKE_OSX_SYSROOT="/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk" -DCONAN_IN_LOCAL_CACHE="ON" -DCONAN_COMPILER="apple-clang" -DCONAN_COMPILER_VERSION="15" -DCONAN_LIBCXX="libc++" -DBUILD_SHARED_LIBS="False" -DCMAKE_INSTALL_PREFIX="/Users/anuragpateriya/.conan/data/llvm/17.0.5-0/_/_/package/60e1ca9cf1a1629140c76766aba050bf24034339" -DCMAKE_INSTALL_BINDIR="bin" -DCMAKE_INSTALL_SBINDIR="bin" -DCMAKE_INSTALL_LIBEXECDIR="bin" -DCMAKE_INSTALL_LIBDIR="lib" -DCMAKE_INSTALL_INCLUDEDIR="include" -DCMAKE_INSTALL_OLDINCLUDEDIR="include" -DCMAKE_INSTALL_DATAROOTDIR="share" -DCONAN_CMAKE_POSITION_INDEPENDENT_CODE="ON" -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY="ON" -DCONAN_EXPORTED="1" -DLLVM_ENABLE_PROJECTS="mlir;lld" -DCMAKE_SKIP_RPATH="True" -DCMAKE_POSITION_INDEPENDENT_CODE="True" -DDISABLE_LLVM_LINK_LLVM_DYLIB="True" -DLLVM_TARGET_ARCH="host" -DLLVM_TARGETS_TO_BUILD="X86;PowerPC;AArch64" -DLLVM_BUILD_LLVM_DYLIB="False" -DLLVM_DYLIB_COMPONENTS="all" -DLLVM_ENABLE_PIC="True" -DLLVM_ABI_BREAKING_CHECKS="WITH_ASSERTS" -DLLVM_ENABLE_WARNINGS="False" -DLLVM_ENABLE_PEDANTIC="True" -DLLVM_ENABLE_WERROR="False" -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN="True" -DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO="False" -DLLVM_BUILD_INSTRUMENTED_COVERAGE="False" -DLLVM_OPTIMIZED_TABLEGEN="True" -DLLVM_REVERSE_ITERATION="False" -DLLVM_ENABLE_BINDINGS="False" -DLLVM_CCACHE_BUILD="False" -DLLVM_BUILD_TOOLS="True" -DLLVM_INCLUDE_TOOLS="True" -DLLVM_INSTALL_UTILS="True" -DLLVM_INCLUDE_EXAMPLES="False" -DLLVM_INCLUDE_TESTS="False" -DLLVM_INCLUDE_BENCHMARKS="False" -DLLVM_APPEND_VC_REV="False" -DLLVM_BUILD_DOCS="False" -DLLVM_ENABLE_IDE="False" -DLLVM_ENABLE_EH="True" -DLLVM_ENABLE_RTTI="True" -DLLVM_ENABLE_THREADS="True" -DLLVM_ENABLE_LTO="Off" -DLLVM_STATIC_LINK_CXX_STDLIB="False" -DLLVM_ENABLE_UNWIND_TABLES="True" -DLLVM_ENABLE_EXPENSIVE_CHECKS="False" -DLLVM_ENABLE_ASSERTIONS="False" -DLLVM_ENABLE_TERMINFO="False" -DLLVM_USE_NEWPM="False" -DLLVM_USE_OPROFILE="False" -DLLVM_USE_PERF="False" -DLLVM_USE_SANITIZER="" -DLLVM_ENABLE_Z3_SOLVER="False" -DLLVM_ENABLE_LIBPFM="False" -DLLVM_ENABLE_LIBEDIT="False" -DLLVM_ENABLE_FFI="False" -DLLVM_ENABLE_ZLIB="True" -DLLVM_ENABLE_LIBXML2="False" -DLLVM_PARALLEL_LINK_JOBS="4" -DMLIR_ENABLE_BINDINGS_PYTHON="True" -DPython3_EXECUTABLE="/Applications/Xcode.app/Contents/Developer/usr/bin/python3" -Wno-dev '/Users/anuragpateriya/.conan/data/llvm/17.0.5-0/_/_/source/llvm-project/llvm'
anuragpateriya@anurags-mbp build % 

[taalexander]

Likely the issue is that you have multiple versions of python installed on your system.

I would recommend building and configuring the compiler within a virtualenv.

You may also pass to cmake the correct python path with  -DPython_EXECUTABLE=/path/to/python.

What does

which python3
python3
>>> import numpy as np

Give you?

[anurag-quantum]

I found only one python3 installed with brew

anuragpateriya@anurags-MBP build % 
anuragpateriya@anurags-MBP build % which python3
/opt/homebrew/bin/python3
anuragpateriya@anurags-MBP build % python3
Python 3.13.0 (main, Oct  7 2024, 05:02:14) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'
>>>