Skip to content

Commit 1d2d0be

Browse files
robertodrmaxscheurer
robertodr
and
maxscheurer
authored
Skbuild (#44)
* skbuild * dynamic version, remove metadata.{hh,cc} * adapt workflow * working on ci * bit less verbose * ci * force ninja * add dependabot * fix * files for setuptools_scm * ci * refresh publish.yml * add flags for visual studio compiler * try with msvc * auto-ize * test verbosely * no openmp on windows * clang-cl w/o openmp * dobule quotes * quotes * quotes * quotes * openmp is not the problem on windows, apparently * install eigen * trigger * single quotes * cibuildwheel * install a recent eigen on linux * environment * tinker * cancel older * no pypy * env vars * env * Add fallback version * forward slashed * foo * Eigen3_ROOT * remove continue-on-error * reorg a bit * reorg * reorg * reorg * reorg * dammit * NO_EXTRAS * auto-ize * decltype instead of auto * Update .github/workflows/ci.yml * fix warning * eigen3 path on windows * eigen3 path on windows * eigen3 path on windows * setup-miniconda conf * update checkout action * update ci.yml * update ci.yml * testing command * testing command * non-shallow clone in ci.yml * skip test_fields * skip test_fields * skip all test that use cache * fetch eigen3 for sdist * skip C compiler checks when fetching Eigen * blasted eigen * blasted eigen * Try aarch64 ppc64le * One job per wheel * fix action version * skip CPython 3.12 * chore: update action versions * chore: update action versions * chore: update toml and publish.yml * fix: option available only in latest scikit-build-core * fix: step name * fix: skip musl, do 3.12 * chore: exclude more variants * chore: exclude aarch64 on linux * test: wheel upload * test: wheel upload * fix: use tarball for eigen * use correct action --------- Co-authored-by: Maximilian Scheurer <max.scheurer@me.com>
1 parent d1bc43f commit 1d2d0be

37 files changed

+514
-654
lines changed

‎.clang-format.ignore‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎.git_archival.txt‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
node: $Format:%H$
2+
node-date: $Format:%cI$
3+
describe-name: $Format:%(describe:tags=true,match=*[0-9]*)$
4+
ref-names: $Format:%D$
5+

‎.gitattributes‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
.git_archival.txt export-subst
2+

‎.github/dependabot.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
version: 2
2+
updates:
3+
# Maintain dependencies for GitHub Actions
4+
- package-ecosystem: "github-actions"
5+
directory: "/"
6+
schedule:
7+
interval: "weekly"
8+
ignore:
9+
# cibuildwheel updates often and cannot be tracked by
10+
# a major version tag like 'v1'. Mute updates since
11+
# this is not a critical component
12+
- dependency-name: "pypa/cibuildwheel*"

‎.github/environment.yml‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,6 @@ name: cppe-gha
22
channels:
33
- conda-forge
44
dependencies:
5-
- cmake
65
- cxx-compiler
76
- eigen
8-
- h5py
9-
- ninja
10-
- numpy
11-
- pandas
127
- pip
13-
- pybind11
14-
- pytest
15-
- scipy

‎.github/workflows/ci.yml‎

Lines changed: 59 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -10,19 +10,17 @@ on:
1010
branches:
1111
- master
1212

13-
env:
14-
BUILD_TYPE: Release
15-
1613
concurrency:
1714
group: ${{ github.workflow }}-${{ github.ref }}
1815
cancel-in-progress: true
1916

17+
env:
18+
FORCE_COLOR: 3
19+
2020
jobs:
2121
build:
2222
runs-on: ${{ matrix.os }}
2323

24-
continue-on-error: true
25-
2624
strategy:
2725
fail-fast: false
2826
matrix:
@@ -32,84 +30,64 @@ jobs:
3230
- windows-latest
3331
python-version:
3432
- "3.8"
35-
- "3.11"
33+
- "3.12"
3634

3735
defaults:
3836
run:
39-
shell: bash -l {0}
40-
41-
steps:
42-
- uses: actions/checkout@v3
43-
44-
# this will set the system compiler;
45-
# This must be done *before* setting up miniconda, see:
46-
# https://github.com/ilammy/msvc-dev-cmd/issues/34
47-
- name: Set Windows env
48-
if: matrix.os == 'windows-latest'
49-
uses: ilammy/msvc-dev-cmd@v1
37+
shell: bash -el {0}
5038

51-
- name: Cache conda
52-
uses: actions/cache@v3
53-
env:
54-
# Increase this value to reset cache if .github/environment.yml has not changed
55-
CACHE_NUMBER: 1
56-
with:
57-
path: ~/conda_pkgs_dir
58-
key:
59-
${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{ hashFiles('.github/environment.yml') }}
39+
env:
40+
NINJA_STATUS: "[Built edge %f of %t in %e sec] "
6041

61-
- uses: conda-incubator/setup-miniconda@v2
62-
with:
63-
mamba-version: "*"
64-
auto-update-conda: true
65-
auto-activate-base: false
66-
activate-environment: cppe-gha
67-
environment-file: .github/environment.yml
68-
channel-priority: strict
69-
python-version: ${{ matrix.python-version }}
70-
71-
- name: Select CMake CLI options
72-
run: |
73-
echo "We are running on ${{ matrix.os }}"
74-
if [ "${{ matrix.os }}" == "ubuntu-latest" ]; then
75-
echo "CXX=g++" >> $GITHUB_ENV
76-
elif [ "${{ matrix.os }}" == "macos-latest" ]; then
77-
echo "CXX=clang++" >> $GITHUB_ENV
78-
else
79-
echo "CXX=clang-cl" >> $GITHUB_ENV
80-
fi
81-
82-
- name: Configure
83-
run: |
84-
cmake -S. \
85-
-Bbuild \
86-
-GNinja \
87-
-DCMAKE_BUILD_TYPE=${{ env.BUILD_TYPE }} \
88-
-DCMAKE_CXX_COMPILER=${CXX} \
89-
-DCMAKE_INSTALL_PREFIX=~/Software
90-
91-
- name: Build and install
92-
run: |
93-
cmake --build build --config ${{ env.BUILD_TYPE }} --target install -- -v -d stats
94-
95-
- name: Test
96-
run: |
97-
python -m pip install git+https://gitlab.com/robertodr/polarizationsolver.git@master
98-
99-
cd
100-
101-
if [ "${{ matrix.os }}" == "windows-latest" ]; then
102-
export KMP_DUPLICATE_LIB_OK=TRUE
103-
export PYTHONPATH=Software/Lib/site-packages
104-
python -m pytest --capture=no --log-cli-level=INFO --color=yes --pyargs cppe
105-
else
106-
export PYTHONPATH=$PYTHONPATH:Software/lib/python${{ matrix.python-version }}/site-packages
107-
python -m pytest --capture=no --log-cli-level=INFO --color=yes --pyargs cppe
108-
fi
109-
110-
# TODO move to separate workflow
111-
#- name: Build and Test Setuptools
112-
# run: |
113-
# rm -rf build
114-
# python setup.py install
115-
# py.test
42+
steps:
43+
- uses: actions/checkout@v4
44+
with:
45+
fetch-depth: 0 # such that setuptools_scm can do its job correctly
46+
47+
# this will set the system compiler;
48+
# This must be done *before* setting up miniconda, see:
49+
# https://github.com/ilammy/msvc-dev-cmd/issues/34
50+
- name: Set Windows env
51+
if: matrix.os == 'windows-latest'
52+
uses: ilammy/msvc-dev-cmd@v1
53+
env:
54+
KMP_DUPLICATE_LIB_OK: "TRUE"
55+
56+
- name: Cache conda
57+
uses: actions/cache@v4
58+
env:
59+
# Increase this value to reset cache if .github/environment.yml has not changed
60+
CACHE_NUMBER: 2
61+
with:
62+
path: ~/conda_pkgs_dir
63+
key:
64+
${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{ hashFiles('.github/environment.yml') }}
65+
66+
- uses: conda-incubator/setup-miniconda@v3
67+
with:
68+
python-version: ${{ matrix.python-version }}
69+
auto-update-conda: true
70+
auto-activate-base: false
71+
channels: conda-forge
72+
channel-priority: true
73+
activate-environment: cppe-gha
74+
environment-file: .github/environment.yml
75+
76+
- name: Select CMake CLI options
77+
run: |
78+
echo "We are running on ${{ matrix.os }}"
79+
if [ "${{ matrix.os }}" == "ubuntu-latest" ]; then
80+
echo "SKBUILD_CMAKE_ARGS=-DCMAKE_CXX_COMPILER=g++;-GNinja" >> $GITHUB_ENV
81+
elif [ "${{ matrix.os }}" == "macos-latest" ]; then
82+
echo "SKBUILD_CMAKE_ARGS=-DCMAKE_CXX_COMPILER=clang++;-GNinja" >> $GITHUB_ENV
83+
else
84+
echo "SKBUILD_CMAKE_ARGS=-DCMAKE_CXX_COMPILER=clang-cl;-GNinja" >> $GITHUB_ENV
85+
fi
86+
87+
- name: Configure, build, install
88+
run: |
89+
python -m pip install -v .[test]
90+
91+
- name: Test
92+
run: |
93+
python -m pytest -vvv --ignore=tests/test_fields.py --ignore=tests/test_functionality.py --ignore=tests/test_gradients.py --ignore=tests/test_solver.py

‎.github/workflows/publish.yml‎

Lines changed: 130 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,140 @@
11
name: Publish package
22

33
on:
4-
push:
5-
branches: [master]
64
workflow_dispatch:
5+
pull_request:
6+
push:
7+
branches:
8+
- master
9+
release:
10+
types:
11+
- published
12+
13+
concurrency:
14+
group: ${{ github.workflow }}-${{ github.ref }}
15+
cancel-in-progress: true
16+
17+
env:
18+
FORCE_COLOR: 3
719

820
jobs:
9-
publish:
21+
build_sdist:
22+
name: Build source distribution
1023
runs-on: ubuntu-latest
1124
steps:
12-
- uses: actions/checkout@v2
13-
- uses: actions/setup-python@v1
25+
- uses: actions/checkout@v4
1426
with:
15-
python-version: '3.7'
16-
- name: Install setuptools and wheel
17-
run: |
18-
python -m pip install --user setuptools wheel
19-
- name: Build a source tarball
27+
fetch-depth: 0 # such that setuptools_scm can do its job correctly
28+
29+
- name: Build SDist
30+
run: pipx run build --sdist
31+
32+
- uses: actions/upload-artifact@v4
33+
with:
34+
name: cibw-sdist
35+
path: dist/*.tar.gz
36+
37+
generate-wheels-matrix:
38+
name: Generate wheels matrix
39+
runs-on: ubuntu-latest
40+
outputs:
41+
include: ${{ steps.set-matrix.outputs.include }}
42+
steps:
43+
- uses: actions/checkout@v4
44+
- name: Install cibuildwheel
45+
run: pipx install cibuildwheel==2.19.2
46+
- id: set-matrix
2047
run: |
21-
pip install pytest numpy h5py scipy pandas scikit-build pybind11
22-
pip install git+https://gitlab.com/reinholdt/polarizationsolver.git@master
23-
python setup.py install
24-
pytest --pyargs cppe
25-
python setup.py sdist
26-
- name: Publish distribution 📦 to Test PyPI
27-
if: startsWith(github.ref, 'refs/tags')
28-
uses: pypa/gh-action-pypi-publish@master
29-
with:
30-
user: __token__
31-
password: ${{ secrets.pypi_password }}
48+
MATRIX=$(
49+
{
50+
cibuildwheel --print-build-identifiers --platform linux \
51+
| jq -nRc '{"only": inputs, "os": "ubuntu-latest"}' \
52+
&& cibuildwheel --print-build-identifiers --platform macos \
53+
| jq -nRc '{"only": inputs, "os": "macos-latest"}' \
54+
&& cibuildwheel --print-build-identifiers --platform windows \
55+
| jq -nRc '{"only": inputs, "os": "windows-latest"}'
56+
} | jq -sc
57+
)
58+
echo "Full build matrix"
59+
echo "$MATRIX"
60+
echo "include=$MATRIX" >> $GITHUB_OUTPUT
61+
env:
62+
CIBW_ARCHS_LINUX: x86_64 # aarch64 # skipping as h5py stopped providing pre-build wheels for aarch64 on PyPI
63+
CIBW_ARCHS_MACOS: x86_64 arm64
64+
CIBW_ARCHS_WINDOWS: AMD64
65+
# skip musl builds
66+
CIBW_SKIP: "*-musllinux_*"
67+
# disable free-threaded support
68+
CIBW_FREE_THREADED_SUPPORT: False
69+
# exclude latest Python beta
70+
CIBW_PRERELEASE_PYTHONS: False
71+
72+
build_wheels:
73+
name: Build ${{ matrix.only }}
74+
needs: generate-wheels-matrix
75+
strategy:
76+
matrix:
77+
include: ${{ fromJson(needs.generate-wheels-matrix.outputs.include) }}
78+
runs-on: ${{ matrix.os }}
79+
steps:
80+
- uses: actions/checkout@v4
81+
with:
82+
fetch-depth: 0 # such that setuptools_scm can do its job correctly
83+
84+
- name: Set up QEMU
85+
if: runner.os == 'Linux'
86+
uses: docker/setup-qemu-action@v3
87+
with:
88+
platforms: all
89+
90+
# this will set the system compiler
91+
- name: Set Windows env
92+
if: matrix.os == 'windows-latest'
93+
uses: ilammy/msvc-dev-cmd@v1
94+
95+
- uses: pypa/cibuildwheel@v2.19.2
96+
env:
97+
# skip testing PyPy builds
98+
CIBW_TEST_SKIP: "pp*"
99+
100+
CIBW_ENVIRONMENT_LINUX: >
101+
SKBUILD_CMAKE_ARGS="-DENABLE_ARCH_FLAGS=OFF;-DENABLE_OPENMP=OFF"
102+
103+
CIBW_ENVIRONMENT_MACOS: >
104+
SKBUILD_CMAKE_ARGS="-DENABLE_ARCH_FLAGS=OFF;-DENABLE_OPENMP=OFF"
105+
106+
CIBW_ENVIRONMENT_WINDOWS: >
107+
SKBUILD_CMAKE_ARGS="-GNinja;-DCMAKE_CXX_COMPILER=clang-cl;-DENABLE_ARCH_FLAGS=OFF;-DENABLE_OPENMP=OFF"
108+
with:
109+
only: ${{ matrix.only }}
110+
111+
- name: Verify clean directory
112+
run: git diff --exit-code
113+
shell: bash
114+
115+
- uses: actions/upload-artifact@v4
116+
with:
117+
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
118+
path: ./wheelhouse/*.whl
119+
120+
upload_all:
121+
name: Upload if release
122+
needs:
123+
- build_wheels
124+
- build_sdist
125+
environment: pypi
126+
permissions:
127+
id-token: write
128+
runs-on: ubuntu-latest
129+
#if: github.event_name == 'release' && github.event.action == 'published'
130+
131+
steps:
132+
- uses: actions/download-artifact@v4
133+
with:
134+
pattern: cibw-*
135+
path: dist
136+
merge-multiple: true
137+
138+
- uses: pypa/gh-action-pypi-publish@release/v1
139+
with:
140+
repository-url: https://test.pypi.org/legacy/

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,3 +42,6 @@ __pycache__
4242
*.egg-info
4343
*.eggs
4444
.ipynb_checkpoints/
45+
46+
# autogenerated by setuptools-scm
47+
_version.py

0 commit comments

Comments
 (0)