Skip to content

Gmsh: Implement fallback mirror for Gmsh download #152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kopperp wants to merge 2 commits into
base: main
Choose a base branch
from
+97 −40
Open

Gmsh: Implement fallback mirror for Gmsh download #152

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
53c454c
Gmsh: Implement fallback mirror for Gmsh download
kopperp Jun 17, 2026
f7e4ade
CI/CD: Implement fallback mirror for Gmsh download
kopperp Jun 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/setup/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ runs:
source .venv/bin/activate
if [[ "${{ inputs.minimal }}" != "true" ]]; then
uv pip install --no-cache-dir https://gitlab.iag.uni-stuttgart.de/libs/python-gmsh/-/raw/master/gmsh-4.15.2.post1-py3-none-linux_x86_64.whl
- uv pip install --no-cache-dir https://gitlab.iag.uni-stuttgart.de/libs/python-gmsh/-/raw/master/gmsh-${GMSH_VERSION}-py3-none-linux_x86_64.whl || uv pip install https://piclas.boltzplatz.eu/public-libs/python-gmsh/-/raw/master/gmsh-${GMSH_VERSION}-py3-none-linux_x86_64.whl
uv pip install --no-cache-dir -e .
fi
Expand Down
24 changes: 16 additions & 8 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,14 @@ concurrency:
# Environment variables
env:
UV_LINK_MODE: copy
# Define the Gmsh version
GMSH_VERSION: "4.15.2.post1"
# Define the Python version
# PYTHON310_VERSION: 3.10.20-slim-trixie
# PYTHON311_VERSION: 3.11.15-slim-trixie
# PYTHON312_VERSION: 3.12.13-slim-trixie
# PYTHON313_VERSION: 3.13.14-slim-trixie
# PYTHON314_VERSION: 3.14.6-slim-trixie

jobs:
# ----------------------------------------------------------------------------------------------------------------------------------------------------
Expand All @@ -58,7 +66,7 @@ jobs:
name: Lint / ruff
runs-on: ubuntu-latest
container:
image: python:3.14.5-slim-trixie
image: python:3.14.6-slim-trixie
steps:
- uses: actions/checkout@v6
# with:
Expand All @@ -78,7 +86,7 @@ jobs:
name: Lint / ty
runs-on: ubuntu-latest
container:
image: python:3.14.5-slim-trixie
image: python:3.14.6-slim-trixie
steps:
# Git LFS requires git
- name: Install Git LFS
Expand All @@ -102,7 +110,7 @@ jobs:
name: Lint / vulture
runs-on: ubuntu-latest
container:
image: python:3.14.5-slim-trixie
image: python:3.14.6-slim-trixie
steps:
- uses: actions/checkout@v6
# with:
Expand Down Expand Up @@ -157,8 +165,8 @@ jobs:
- { python-version: "3.10", image: "python:3.10.20-slim-trixie" }
- { python-version: "3.11", image: "python:3.11.15-slim-trixie" }
- { python-version: "3.12", image: "python:3.12.13-slim-trixie" }
- { python-version: "3.13", image: "python:3.13.13-slim-trixie" }
- { python-version: "3.14", image: "python:3.14.5-slim-trixie" }
- { python-version: "3.13", image: "python:3.13.14-slim-trixie" }
- { python-version: "3.14", image: "python:3.14.6-slim-trixie" }
container:
image: ${{ matrix.image }}
steps:
Expand Down Expand Up @@ -190,7 +198,7 @@ jobs:
runs-on: ubuntu-latest
needs: [compatibility]
container:
image: python:3.14.5-slim-trixie
image: python:3.14.6-slim-trixie
steps:
# Git LFS requires git
- name: Install Git LFS
Expand Down Expand Up @@ -264,7 +272,7 @@ jobs:
runs-on: ubuntu-latest
needs: [compatibility]
container:
image: python:3.14.5-slim-trixie
image: python:3.14.6-slim-trixie
steps:
# Git LFS requires git
- name: Install Git LFS
Expand Down Expand Up @@ -351,7 +359,7 @@ jobs:
runs-on: ubuntu-latest
needs: [examples, healthcheck]
container:
image: python:3.14.5-slim-trixie
image: python:3.14.6-slim-trixie
steps:
- uses: actions/checkout@v6
# with:
Expand Down
35 changes: 22 additions & 13 deletions .gitlab-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,25 @@
# Continuous Integration/Continuous Deployment
# ==================================================================================================================================

# Official language image. Look for the different tagged releases at:
# > https://hub.docker.com/r/library/python/tags/
# image: ${CI_REGISTRY_NAME}/nrg-python:3.14.5-trixie-x86_64
image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:3.14.5-trixie-x86_64

# Change pip's cache directory to be inside the project directory since we can only cache local items.
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
PIP_CACHE_DIR: "${CI_PROJECT_DIR}/.cache/pip"
# Enable the faster OverlayFS driver
DOCKER_DRIVER: overlay2
# Define the Gmsh version
GMSH_VERSION: "4.15.2.post1"
# Define the Python version
PYTHON310_VERSION: 3.10.20-trixie
PYTHON311_VERSION: 3.11.15-trixie
PYTHON312_VERSION: 3.12.13-trixie
PYTHON313_VERSION: 3.13.13-trixie
PYTHON314_VERSION: 3.14.5-trixie

# Official language image. Look for the different tagged releases at:
# > https://hub.docker.com/r/library/python/tags/
# image: ${CI_REGISTRY_NAME}/nrg-python:${PYTHON314_VERSION}-x86_64
image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:${PYTHON314_VERSION}-x86_64


# https://pip.pypa.io/en/stable/topics/caching/
# .venv_cache: &venv_cache
Expand Down Expand Up @@ -85,8 +94,8 @@ variables:
# Setup Python virtual environment
- uv venv .venv
- source .venv/bin/activate
# Regression checks needs NRG Gmsh, pre-install it
- uv pip install --no-cache-dir https://gitlab.iag.uni-stuttgart.de/libs/python-gmsh/-/raw/master/gmsh-4.15.2.post1-py3-none-linux_x86_64.whl
# Regression checks needs NRG Gmsh, pre-install it (try two different mirrors for downloading the .whl file)
- uv pip install --no-cache-dir https://gitlab.iag.uni-stuttgart.de/libs/python-gmsh/-/raw/master/gmsh-${GMSH_VERSION}-py3-none-linux_x86_64.whl || uv pip install https://piclas.boltzplatz.eu/public-libs/python-gmsh/-/raw/master/gmsh-${GMSH_VERSION}-py3-none-linux_x86_64.whl
# Install all pre-requisites
- uv pip install --no-cache-dir -e .
# Install coverage
Expand Down Expand Up @@ -183,31 +192,31 @@ pre-commit:
python:3.10:
extends: .defaults
stage: compatibility
image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:3.10.20-trixie-x86_64
image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:${PYTHON310_VERSION}-x86_64
<<: *compatibility-script
# Python 3.11 ----------------------------------------------------------------------------------------------------------------------------------------
python:3.11:
extends: .defaults
stage: compatibility
image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:3.11.15-trixie-x86_64
image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:${PYTHON311_VERSION}-x86_64
<<: *compatibility-script
# Python 3.12 ----------------------------------------------------------------------------------------------------------------------------------------
python:3.12:
extends: .defaults
stage: compatibility
image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:3.12.13-trixie-x86_64
image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:${PYTHON312_VERSION}-x86_64
<<: *compatibility-script
# Python 3.13 ----------------------------------------------------------------------------------------------------------------------------------------
python:3.13:
extends: .defaults
stage: compatibility
image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:3.13.13-trixie-x86_64
image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:${PYTHON313_VERSION}-x86_64
<<: *compatibility-script
# Python 3.14 ----------------------------------------------------------------------------------------------------------------------------------------
python:3.14:
extends: .defaults
stage: compatibility
# image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:3.14.3-trixie-x86_64 # default version at the moment
# image: registry.iag.uni-stuttgart.de/flexi/codes/pyhope/nrg-python:${PYTHON314_VERSION}-x86_64 # default version at the moment
<<: *compatibility-script

# ----------------------------------------------------------------------------------------------------------------------------------------------------
Expand Down
8 changes: 6 additions & 2 deletions pyhope/common/common_vars.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,9 +138,13 @@ def commit(self) -> str:
@final
class Gitlab:
# Gitlab "python-gmsh" access
LIB_GITLAB: str = 'gitlab.iag.uni-stuttgart.de'
LIB_GITLAB: tuple[str] = ('gitlab.iag.uni-stuttgart.de',
'piclas.boltzplatz.eu' ,
)
# LIB_PROJECT = 'libs/python-gmsh'
LIB_PROJECT: str = '797'
LIB_PROJECT: tuple[int] = (797,
26 ,
)
LIB_VERSION: dict[str, dict[str, str]] = { # noqa: RUF012
'linux': {
'x86_64' : '4.15.2.post1',
Expand Down
65 changes: 50 additions & 15 deletions pyhope/gmsh/gmsh_install.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,9 @@
import os
import platform
import shutil
import subprocess
from importlib import metadata
from packaging.version import Version
from typing import Optional, cast
from typing import Final, Optional, cast
# ----------------------------------------------------------------------------------------------------------------------------------
# Third-party libraries
# ----------------------------------------------------------------------------------------------------------------------------------
Expand Down Expand Up @@ -135,10 +134,13 @@ def PkgsInstallGmsh(system: str, arch: str, version: str) -> None:
# Standard libraries -----------------------------------
import sys
import hashlib
import subprocess
import tempfile
import requests
# Local imports ----------------------------------------
import pyhope.output.output as hopout
from pyhope.common.common_vars import Gitlab
from pyhope.common.common_progress import ProgressBar
# ------------------------------------------------------
# Get our package manager
# > Check if 'uv' is available
Expand All @@ -163,6 +165,9 @@ def PkgsInstallGmsh(system: str, arch: str, version: str) -> None:
lfs = 'yes'
lib = f'gmsh-{Gitlab.LIB_VERSION[system][arch]}-py3-none-{system}_{arch}.whl'

noPing: Final[int] = 1 # Number of pings
toPing: Final[int] = 5 # Timeout of pings

# Create a temporary directory
with tempfile.TemporaryDirectory() as path:
# On macOS add major version string to filename an rename darwin to macosx in whl filename
Expand All @@ -173,20 +178,50 @@ def PkgsInstallGmsh(system: str, arch: str, version: str) -> None:
else:
pkgs = os.path.join(path, lib)

curl = [f'curl https://{Gitlab.LIB_GITLAB}/api/v4/projects/{Gitlab.LIB_PROJECT}/repository/files/{lib}/raw?lfs={lfs} --output {pkgs}'] # noqa: E501
_ = subprocess.run(curl, check=True, shell=True)

# Compare the hash
# > Initialize a new sha256 hash
sha256 = hashlib.sha256()
with open(pkgs, 'rb') as f:
# Read and update hash string value in blocks of 4K
for chunk in iter(lambda: f.read(4096), b""):
sha256.update(chunk)
# Try the hosts in order, we generally expect the first host to be up and up-to-date
for idGit, (gitURL, gitID) in enumerate(zip(Gitlab.LIB_GITLAB, Gitlab.LIB_PROJECT, strict=True)):
try:
subprocess.check_call(('ping', f'-c{noPing}', f'-w{toPing}', gitURL),
stdout = subprocess.DEVNULL, # noqa: E251
stderr = subprocess.DEVNULL, # noqa: E251
timeout = toPing) # noqa: E251
except subprocess.CalledProcessError: # noqa: PERF203
continue

# Host is responding, attempt to get the file
# urllib.request.urlretrieve(url = f'https://{gitURL}/api/v4/projects/{Gitlab.LIB_PROJECT}/repository/files/{lib}/raw?lfs={lfs}', # noqa: E251
# filename = pkgs # noqa: E251
# )
request = requests.get(f'https://{gitURL}/api/v4/projects/{gitID}/repository/files/{lib}/raw?lfs={lfs}',
stream=True)
size = int(request.headers['content-length'])
bar = ProgressBar(title = f'│ Downloading Gmsh [v{Gitlab.LIB_VERSION[system][arch]}] from {gitURL}', # noqa: E251
value = int(size/1024), # noqa: E251
threshold = 0) # noqa: E251

with open(pkgs, 'wb') as f:
for chunk in request.iter_content(chunk_size=1024*50):
if chunk: # filter out keep-alive new chunks
bar.step(int(len(chunk)/1024))
f.write(chunk)

bar.close()

# Compare the hash
# > Initialize a new sha256 hash
sha256 = hashlib.sha256()
with open(pkgs, 'rb') as f:
# Read and update hash string value in blocks of 4K
for chunk in iter(lambda: f.read(4096), b""):
sha256.update(chunk)

if sha256.hexdigest() == Gitlab.LIB_SUPPORT[system][arch]:
hopout.info('Hash matches, installing Gmsh wheel...')
break

if idGit < len(Gitlab.LIB_GITLAB) - 1:
hopout.info('Hash mismatch, trying next mirror...')

if sha256.hexdigest() == Gitlab.LIB_SUPPORT[system][arch]:
hopout.info('Hash matches, installing Gmsh wheel...')
else:
hopout.error('Hash mismatch, exiting...')

# Remove the old version
Expand Down
3 changes: 2 additions & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,15 @@ maintainers = [
]
dependencies = [
'alive-progress',
'gmsh',
'h5py',
'hilbertcurve',
'meshio',
'numpy>=2.0.0',
'packaging',
'plotext',
'requests',
'scipy',
'gmsh',
'typing-extensions',
# 'threadpoolctl'
]
Expand Down
Loading