feat: fall back to system Python when pre-built binaries unavailable#1289
feat: fall back to system Python when pre-built binaries unavailable#1289gounthar wants to merge 3 commits intoactions:mainfrom
Conversation
When pre-built Python binaries are not available for the current architecture (e.g., riscv64), try using system Python as a fallback instead of failing with an error. This enables setup-python to work on architectures that don't yet have pre-built binaries in actions/python-versions, such as riscv64 self-hosted runners. The fallback only activates when: 1. No cached version is found 2. No manifest entry exists for the architecture 3. System python3 exists and satisfies the requested version spec A warning is emitted so users know the fallback was used. Fixes actions#1288 Signed-off-by: Bruno Verachten <gounthar@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR adds a fallback path in actions/setup-python so that when no pre-built CPython binary can be found for the requested version/architecture, the action can optionally use a compatible system python3 instead of failing (targeting unsupported architectures like riscv64).
Changes:
- Add system
python3fallback resolution inuseCpythonVersionwhen toolcache + manifest lookup fails. - Emit a warning when falling back to system Python.
- Update the bundled
distoutput to include the same logic.
Reviewed changes
Copilot reviewed 1 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| src/find-python.ts | Adds system python3 fallback when no cached/downloadable CPython is found. |
| dist/setup/index.js | Compiled distribution output reflecting the new fallback behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/find-python.ts
Outdated
| if (!installDir) { | ||
| // Try system Python as fallback (e.g., on architectures without pre-built binaries) | ||
| try { | ||
| const {exitCode, stdout} = await exec.getExecOutput('python3', [ | ||
| '-c', | ||
| 'import sys; print(sys.prefix)' | ||
| ]); | ||
| if (exitCode === 0) { | ||
| const systemPrefix = stdout.trim(); | ||
| const systemVersion = await exec.getExecOutput('python3', [ | ||
| '-c', | ||
| 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}")' | ||
| ]); | ||
| if ( | ||
| systemVersion.exitCode === 0 && | ||
| semver.satisfies(systemVersion.stdout.trim(), semanticVersionSpec) | ||
| ) { | ||
| installDir = systemPrefix; | ||
| core.warning( | ||
| `Pre-built Python not available for architecture '${architecture}'. Using system Python ${systemVersion.stdout.trim()} at ${systemPrefix}.` | ||
| ); | ||
| } | ||
| } | ||
| } catch { | ||
| // System Python not available, fall through to error | ||
| } | ||
| } |
There was a problem hiding this comment.
Addressed — tests will be added in a follow-up. The structural fixes in this push are the priority.
src/find-python.ts
Outdated
| if (!installDir) { | ||
| // Try system Python as fallback (e.g., on architectures without pre-built binaries) | ||
| try { | ||
| const {exitCode, stdout} = await exec.getExecOutput('python3', [ | ||
| '-c', | ||
| 'import sys; print(sys.prefix)' | ||
| ]); | ||
| if (exitCode === 0) { | ||
| const systemPrefix = stdout.trim(); | ||
| const systemVersion = await exec.getExecOutput('python3', [ | ||
| '-c', | ||
| 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}")' | ||
| ]); | ||
| if ( | ||
| systemVersion.exitCode === 0 && | ||
| semver.satisfies(systemVersion.stdout.trim(), semanticVersionSpec) | ||
| ) { | ||
| installDir = systemPrefix; | ||
| core.warning( | ||
| `Pre-built Python not available for architecture '${architecture}'. Using system Python ${systemVersion.stdout.trim()} at ${systemPrefix}.` | ||
| ); | ||
| } |
There was a problem hiding this comment.
Fixed — added !freethreaded guard. The fallback is now skipped entirely for free-threaded builds.
src/find-python.ts
Outdated
| 'import sys; print(sys.prefix)' | ||
| ]); | ||
| if (exitCode === 0) { | ||
| const systemPrefix = stdout.trim(); | ||
| const systemVersion = await exec.getExecOutput('python3', [ | ||
| '-c', | ||
| 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}")' | ||
| ]); | ||
| if ( | ||
| systemVersion.exitCode === 0 && | ||
| semver.satisfies(systemVersion.stdout.trim(), semanticVersionSpec) | ||
| ) { | ||
| installDir = systemPrefix; | ||
| core.warning( | ||
| `Pre-built Python not available for architecture '${architecture}'. Using system Python ${systemVersion.stdout.trim()} at ${systemPrefix}.` |
There was a problem hiding this comment.
Fixed — the fallback now returns early with outputs captured directly from sys.executable, sys.prefix, and os.path.dirname(sys.executable). No longer sets installDir or relies on versionFromPath()/binDir().
src/find-python.ts
Outdated
| if (!installDir) { | ||
| // Try system Python as fallback (e.g., on architectures without pre-built binaries) | ||
| try { | ||
| const {exitCode, stdout} = await exec.getExecOutput('python3', [ | ||
| '-c', | ||
| 'import sys; print(sys.prefix)' | ||
| ]); | ||
| if (exitCode === 0) { | ||
| const systemPrefix = stdout.trim(); | ||
| const systemVersion = await exec.getExecOutput('python3', [ | ||
| '-c', | ||
| 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}")' | ||
| ]); | ||
| if ( | ||
| systemVersion.exitCode === 0 && | ||
| semver.satisfies(systemVersion.stdout.trim(), semanticVersionSpec) | ||
| ) { | ||
| installDir = systemPrefix; | ||
| core.warning( | ||
| `Pre-built Python not available for architecture '${architecture}'. Using system Python ${systemVersion.stdout.trim()} at ${systemPrefix}.` | ||
| ); | ||
| } | ||
| } | ||
| } catch { | ||
| // System Python not available, fall through to error | ||
| } | ||
| } |
There was a problem hiding this comment.
Fixed — the fallback now only triggers when the manifest has zero entries for the current architecture (!archHasManifestEntries). On x86_64/arm64 where the manifest has entries, the original error behavior is preserved.
- Gate fallback on architecture having no manifest entries, preventing false triggers on supported architectures with missing versions - Skip fallback for free-threaded Python builds - Return early with correct outputs (sys.executable, version, bin dir) instead of relying on toolcache path parsing which breaks with system prefix paths like /usr - Set environment variables (pythonLocation, Python_ROOT_DIR, etc.) correctly for system Python paths Signed-off-by: Bruno Verachten <gounthar@gmail.com>
|
Pushed a v2 addressing all review feedback:
|
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
|
Tests added in cb5cf22: 5 new test cases covering the system Python fallback:
All 13 tests pass (8 existing + 5 new). |
| try { | ||
| const sysInfo = await exec.getExecOutput('python3', [ | ||
| '-c', | ||
| 'import sys, os; print(sys.executable + "\\n" + f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}" + "\\n" + sys.prefix + "\\n" + os.path.dirname(sys.executable))' | ||
| ]); | ||
| if (sysInfo.exitCode === 0) { | ||
| const [sysExecutable, sysVersion, sysPrefix, sysBinDir] = sysInfo.stdout.trim().split('\n'); | ||
| if (semver.satisfies(sysVersion, semanticVersionSpec)) { | ||
| core.warning(`Pre-built Python not available for architecture '${baseArchitecture}'. Using system Python ${sysVersion} at ${sysExecutable}.`); |
There was a problem hiding this comment.
Is this somewhat hardcoded to whatever the default Python version on the runner image is at the moment? I.e. Python 3.12 on the ubuntu-24.04-riscv images? I was testing this branch out using python-version: "3.13" at weiji14/cog3pio@c40e8af, but got the following error output:
Run gounthar/setup-python@cb5cf223af5f1712fe2914b9c669d2008e051022
Installed versions
Version 3.13 was not found in the local cache
/usr/bin/python3 -c import sys, os; print(sys.executable + "\n" + f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}" + "\n" + sys.prefix + "\n" + os.path.dirname(sys.executable))
/usr/bin/python3
3.12.13
/opt/python-3.12
/usr/bin
Error: The version '3.13' with architecture 'riscv64' was not found for Ubuntu 24.04.
The list of all available versions can be found here: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json
0s
If I change to using python-version: "3.x" (see this commit), it picks up Python 3.12:
Version 3.x was not found in the local cache
/usr/bin/python3 -c import sys, os; print(sys.executable + "\n" + f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}" + "\n" + sys.prefix + "\n" + os.path.dirname(sys.executable))
/usr/bin/python3
3.12.13
/opt/python-3.12
/usr/bin
Warning: Pre-built Python not available for architecture 'riscv64'. Using system Python 3.12.13 at /usr/bin/python3.
Successfully set up CPython (3.12.13)
but I hit into another error with PyO3/maturin-action (which is a separate issue 🙂). I'm wondering how much effort it would be to allow a different Python version?
|
Thanks for testing this. You are right that it only picks up whatever Python version is on the runner. The RISE runner currently ships 3.12 only, so requesting 3.13 fails. The fallback checks whether the system Python satisfies the requested version spec and bails out if it does not, which is why 3.13 gives the original error while 3.x works. Supporting multiple versions would require the RISE runner image to pre-install them (like the standard ubuntu-24.04 runners do with 3.9-3.13 in the toolcache). I have mentioned this to the RISE team but not sure when it will happen. For now, |
I did see Python 3.13 in this Dockerfile (https://github.com/riseproject-dev/riscv-runner-images/blame/598f920e51bba243f1f3f75efa2e98f6c7eb9a0c/runner/Dockerfile.ubuntu#L158-L182), as well as 3.10-3.14 and 3.13t/3.14t, but maybe I'm not looking at the correct place? |
|
Good find. The Dockerfile does install Python 3.10-3.14 in /opt/python-3.x/. I did not check those carefully enough. When I tried using them on the actual runner, they were missing standard library modules (e.g. math), which broke venv and most tools. That is why the fallback ended up using system Python instead. I reported the broken /opt/python issue to the RISE team (riseproject-dev/riscv-runner-images#21). Once those installations work properly, the fallback should look there for the requested version instead of only checking system Python. |
Summary
When pre-built Python binaries are not available for the current architecture, fall back to system Python instead of failing with an error.
This enables
setup-pythonto work on architectures likeriscv64that don't yet have pre-built binaries inactions/python-versions. RISC-V self-hosted runners are becoming available through programs like RISE RISC-V runners, used by numpy, llama.cpp, and pytorch.Changes
src/find-python.ts: Before throwing "version not found" error, check if systempython3exists and satisfies the requested version spec. If so, use it and emit a warning.Behavior
Test plan
npm run build)Fixes #1288