Skip to content

macOS: centralise Python build dependencies using uv #2539

@tallpsmith

Description

@tallpsmith

Problem

Python build dependencies for macOS PCP are defined in at least four places with no single source of truth:

Location Purpose pip3 deps
.github/workflows/macOS.yml CI build lxml openpyxl OrderedDict psycopg2-binary prometheus_client pyarrow pyodbc requests
.cirrus.yml Tart VM build none
qa/admin/package-lists/Darwin+* QA test deps requests setuptools openpyxl libvirt-python Pillow-PIL prometheus_client elasticsearch psycopg2
Developer's machine local build whatever they remembered

Additionally, Homebrew Python 3.14+ returns externally-managed-environment on bare pip3 install, making system-level installs impossible. The GitHub Actions CI already works around this with a manual venv (python3 -m venv pybuilddeps), but the Cirrus/Tart build and local development don't.

The CI also hardcodes PYTHONPATH="/usr/local/lib/python3.13/site-packages" (line 256 of macOS.yml) and pins Python 3.13 via actions/setup-python — both fragile assumptions that will break on version bumps.

Proposed Solution

Mandate uv (brew install uv) as the Python package manager for macOS builds. No pip fallback — one tool, one path.

New files

build/mac/requirements-build.txt — Python packages that affect what ./configure probes for:

requests
setuptools

build/mac/requirements-test.txt — additional packages for QA/testing:

-r requirements-build.txt
lxml
openpyxl
prometheus_client
psycopg2-binary

build/mac/setup-python-env.sh — single script used by both CI pipelines and developers:

#!/bin/bash
set -e
VENV_DIR="${1:-.pcp-venv}"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
REQS="${2:-$SCRIPT_DIR/requirements-build.txt}"

uv venv "$VENV_DIR"
uv pip install --python "$VENV_DIR/bin/python3" -r "$REQS"

echo "Python build environment ready. Activate with:"
echo "  source $VENV_DIR/bin/activate"

CI updates

.github/workflows/macOS.yml:

  • Remove actions/setup-python@v6 step (use Homebrew Python instead of pinning a version)
  • Add uv to brew install step
  • Replace inline pip install block with ./build/mac/setup-python-env.sh
  • Remove hardcoded PYTHONPATH="/usr/local/lib/python3.13/site-packages" hack
  • Remove hardcoded Python version references

.cirrus.yml:

  • Add uv to brew install line in populate_script
  • Add Python venv setup step before build
  • Activate venv in pcp_build_script

Also update

  • .gitignore — add .pcp-venv
  • build/mac/CLAUDE.md — add uv to prerequisites
  • build/mac/MACOS_DEVELOPMENT.md — developer setup instructions

Why uv?

  • brew install uv — first-class Homebrew citizen
  • 10-100x faster than pip for installs
  • Handles venv creation natively (uv venv)
  • Standalone Rust binary — no Python package needed

Verification

  • ./configure output shows PMDA_OPENMETRICS ... yes and PMDA_OPENTELEMETRY ... yes
  • python3 -c 'import requests' succeeds inside the venv
  • Both CI pipelines build successfully
  • No hardcoded Python version strings remain in CI files

Metadata

Metadata

Assignees

Labels

macOSFor issues specific or related to macOS

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions