Skip to content

feat: Python version matrix for Docker images + error-masking fix #76

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
deanq merged 15 commits into from
Mar 10, 2026
Merged

feat: Python version matrix for Docker images + error-masking fix #76

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
23a6f5c
feat: add Python version matrix for Docker images
deanq Mar 6, 2026
a20be9e
fix: stop masking deployment errors by falling back to Live Serverless
deanq Mar 6, 2026
0c56199
fix: update handler tests for RuntimeError + sync uv.lock
deanq Mar 6, 2026
f917b44
feat: add Python version to startup banner
deanq Mar 6, 2026
851c66a
fix(review): address PR #76 feedback
deanq Mar 6, 2026
37a90c0
fix(review): address PR #76 round 2 feedback
deanq Mar 6, 2026
4d7b33c
fix: preserve base image torch by removing Python symlink and targeti…
deanq Mar 7, 2026
f5ef12b
feat: versioned Docker image builds with single pytorch base image
deanq Mar 7, 2026
1dd5723
fix: format dependency_installer.py
deanq Mar 7, 2026
4b5e839
docs: document Python version constraints and base image layout
deanq Mar 8, 2026
3d72bf2
feat: pin GPU images to Python 3.12, default CPU to 3.12
deanq Mar 9, 2026
75ead82
chore: update lockfile
deanq Mar 9, 2026
313aa4a
fix: auto-detect local Python version for CPU image builds
deanq Mar 9, 2026
ca89225
fix(review): address PR #76 feedback -- CI matrix, Dockerfiles, pip s…
deanq Mar 9, 2026
d5de8a8
chore: bump runpod-flash dependency to 1.8.0
deanq Mar 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
153 changes: 99 additions & 54 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ jobs:
platforms: linux/amd64
push: false
tags: flash-cpu:test
build-args: |
PYTHON_VERSION=3.11
cache-from: type=gha
cache-to: type=gha,mode=max
load: true
Expand Down Expand Up @@ -115,6 +117,8 @@ jobs:
platforms: linux/amd64
push: false
tags: flash-lb-cpu:test
build-args: |
PYTHON_VERSION=3.11
cache-from: type=gha
cache-to: type=gha,mode=max
load: true
Expand Down Expand Up @@ -163,6 +167,11 @@ jobs:
runs-on: ubuntu-latest
needs: [release]
if: needs.release.outputs.release_created
strategy:
matrix:
include:
- python-version: "3.12"
is-default: true
steps:
- name: Clear Space
run: |
Expand All @@ -188,14 +197,18 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Extract GPU metadata
id: meta-gpu
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=semver,pattern={{version}},value=${{ needs.release.outputs.tag_name }}
type=raw,value=latest,enable={{is_default_branch}}
- name: Build versioned tags
id: tags
run: |
VERSION="${{ needs.release.outputs.tag_name }}"
VERSION="${VERSION#v}"
PYVER="${{ matrix.python-version }}"
IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}"
TAGS="${IMAGE}:py${PYVER}-${VERSION},${IMAGE}:py${PYVER}-latest"
if [ "${{ matrix.is-default }}" = "true" ]; then
TAGS="${TAGS},${IMAGE}:${VERSION},${IMAGE}:latest"
fi
echo "tags=${TAGS}" >> "$GITHUB_OUTPUT"

- name: Set up uv
uses: astral-sh/setup-uv@v4
Expand All @@ -210,24 +223,29 @@ jobs:
with:
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64
platforms: linux/amd64
push: true
tags: ${{ steps.meta-gpu.outputs.tags }}
labels: ${{ steps.meta-gpu.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
tags: ${{ steps.tags.outputs.tags }}
cache-from: type=gha,scope=gpu
cache-to: type=gha,mode=max,scope=gpu

docker-prod-cpu:
runs-on: ubuntu-latest
needs: [release]
if: needs.release.outputs.release_created
strategy:
matrix:
include:
- python-version: "3.10"
is-default: false
- python-version: "3.11"
is-default: true
- python-version: "3.12"
is-default: false
steps:
- name: Clear Space
run: |
rm -rf /usr/share/dotnet
rm -rf /opt/ghc
rm -rf "/usr/local/share/boost"
rm -rf "$AGENT_TOOLSDIRECTORY"
rm -rf /usr/share/dotnet /opt/ghc /usr/local/share/boost "$AGENT_TOOLSDIRECTORY"
docker system prune -af
df -h

Expand All @@ -249,14 +267,18 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Extract CPU metadata
id: meta-cpu
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-cpu
tags: |
type=semver,pattern={{version}},value=${{ needs.release.outputs.tag_name }}
type=raw,value=latest,enable={{is_default_branch}}
- name: Build versioned tags
id: tags
run: |
VERSION="${{ needs.release.outputs.tag_name }}"
VERSION="${VERSION#v}"
PYVER="${{ matrix.python-version }}"
IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-cpu"
TAGS="${IMAGE}:py${PYVER}-${VERSION},${IMAGE}:py${PYVER}-latest"
if [ "${{ matrix.is-default }}" = "true" ]; then
TAGS="${TAGS},${IMAGE}:${VERSION},${IMAGE}:latest"
fi
echo "tags=${TAGS}" >> "$GITHUB_OUTPUT"

- name: Set up uv
uses: astral-sh/setup-uv@v4
Expand All @@ -273,15 +295,21 @@ jobs:
file: ./Dockerfile-cpu
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta-cpu.outputs.tags }}
labels: ${{ steps.meta-cpu.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
tags: ${{ steps.tags.outputs.tags }}
build-args: |
PYTHON_VERSION=${{ matrix.python-version }}
cache-from: type=gha,scope=cpu-py${{ matrix.python-version }}
cache-to: type=gha,mode=max,scope=cpu-py${{ matrix.python-version }}

docker-prod-lb:
runs-on: ubuntu-latest
needs: [release]
if: needs.release.outputs.release_created
strategy:
matrix:
include:
- python-version: "3.12"
is-default: true
steps:
- name: Clear Space
run: |
Expand All @@ -307,14 +335,18 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Extract Load Balancer metadata
id: meta-lb
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-lb
tags: |
type=semver,pattern={{version}},value=${{ needs.release.outputs.tag_name }}
type=raw,value=latest,enable={{is_default_branch}}
- name: Build versioned tags
id: tags
run: |
VERSION="${{ needs.release.outputs.tag_name }}"
VERSION="${VERSION#v}"
PYVER="${{ matrix.python-version }}"
IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-lb"
TAGS="${IMAGE}:py${PYVER}-${VERSION},${IMAGE}:py${PYVER}-latest"
if [ "${{ matrix.is-default }}" = "true" ]; then
TAGS="${TAGS},${IMAGE}:${VERSION},${IMAGE}:latest"
fi
echo "tags=${TAGS}" >> "$GITHUB_OUTPUT"

- name: Set up uv
uses: astral-sh/setup-uv@v4
Expand All @@ -329,17 +361,25 @@ jobs:
with:
context: .
file: ./Dockerfile-lb
platforms: linux/amd64,linux/arm64
platforms: linux/amd64
push: true
tags: ${{ steps.meta-lb.outputs.tags }}
labels: ${{ steps.meta-lb.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
tags: ${{ steps.tags.outputs.tags }}
cache-from: type=gha,scope=lb
cache-to: type=gha,mode=max,scope=lb

docker-prod-lb-cpu:
runs-on: ubuntu-latest
needs: [release]
if: needs.release.outputs.release_created
strategy:
matrix:
include:
- python-version: "3.10"
is-default: false
- python-version: "3.11"
is-default: true
- python-version: "3.12"
is-default: false
steps:
- name: Clear Space
run: |
Expand All @@ -365,14 +405,18 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Extract CPU Load Balancer metadata
id: meta-lb-cpu
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-lb-cpu
tags: |
type=semver,pattern={{version}},value=${{ needs.release.outputs.tag_name }}
type=raw,value=latest,enable={{is_default_branch}}
- name: Build versioned tags
id: tags
run: |
VERSION="${{ needs.release.outputs.tag_name }}"
VERSION="${VERSION#v}"
PYVER="${{ matrix.python-version }}"
IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-lb-cpu"
TAGS="${IMAGE}:py${PYVER}-${VERSION},${IMAGE}:py${PYVER}-latest"
if [ "${{ matrix.is-default }}" = "true" ]; then
TAGS="${TAGS},${IMAGE}:${VERSION},${IMAGE}:latest"
fi
echo "tags=${TAGS}" >> "$GITHUB_OUTPUT"

- name: Set up uv
uses: astral-sh/setup-uv@v4
Expand All @@ -389,7 +433,8 @@ jobs:
file: ./Dockerfile-lb-cpu
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta-lb-cpu.outputs.tags }}
labels: ${{ steps.meta-lb-cpu.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
tags: ${{ steps.tags.outputs.tags }}
build-args: |
PYTHON_VERSION=${{ matrix.python-version }}
cache-from: type=gha,scope=lb-cpu-py${{ matrix.python-version }}
cache-to: type=gha,mode=max,scope=lb-cpu-py${{ matrix.python-version }}
1 change: 0 additions & 1 deletion .python-version
View file
Open in desktop

This file was deleted.

28 changes: 25 additions & 3 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
FROM pytorch/pytorch:2.9.1-cuda12.8-cudnn9-runtime
# Python 3.12 included in this PyTorch image
# Base image provides Python 3.12 (from runpod/pytorch:1.0.3-cu1281-torch291-ubuntu2204)
FROM runpod/pytorch:1.0.3-cu1281-torch291-ubuntu2204

# Use the base image's Python as-is to preserve pre-installed packages (torch, cuda libs).
# The pytorch base image provides its own Python with torch already installed.
# Symlinking to /usr/bin/python3.X would switch to a bare system Python without torch.
# Validate that the base image provides the expected Python version.
ARG EXPECTED_PYTHON_VERSION=3.12
RUN python --version && \
actual=$(python -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')") && \
if [ "$actual" != "$EXPECTED_PYTHON_VERSION" ]; then \
echo "ERROR: Expected Python $EXPECTED_PYTHON_VERSION but base image provides $actual" && exit 1; \
fi

WORKDIR /app

Expand Down Expand Up @@ -30,9 +41,20 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y --no-ins
&& rm -rf /var/lib/apt/lists/*

# Copy app code and install dependencies
# Use --python to target the base image's Python (preserves torch in its site-packages)
COPY README.md pyproject.toml uv.lock ./
COPY src/ ./
RUN uv export --format requirements-txt --no-dev --no-hashes > requirements.txt \
&& uv pip install --system -r requirements.txt
&& uv pip install --python $(which python) --break-system-packages -r requirements.txt

# Install numpy for the base image's Python version.
# The runpod/pytorch image ships torch but not numpy. Flash build excludes numpy
# from tarballs (BASE_IMAGE_PACKAGES) to save tarball space (~30 MB), so numpy
# must be provided here in the base image.
RUN python -m pip install --no-cache-dir numpy

# Verify torch and numpy are available from the base image
RUN python -c "import torch; print(f'torch {torch.__version__} CUDA {torch.cuda.is_available()}')" \
&& python -c "import numpy; print(f'numpy {numpy.__version__}')"

CMD ["python", "handler.py"]
5 changes: 3 additions & 2 deletions Dockerfile-cpu
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
FROM python:3.12-slim
ARG PYTHON_VERSION=3.12
FROM python:${PYTHON_VERSION}-slim

WORKDIR /app

Expand Down Expand Up @@ -27,6 +28,6 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y --no-ins
COPY README.md pyproject.toml uv.lock ./
COPY src/ ./
RUN uv export --format requirements-txt --no-dev --no-hashes > requirements.txt \
&& uv pip install --system -r requirements.txt
&& uv pip install --system --break-system-packages -r requirements.txt

CMD ["python", "handler.py"]
26 changes: 23 additions & 3 deletions Dockerfile-lb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
FROM pytorch/pytorch:2.9.1-cuda12.8-cudnn9-runtime
# Python 3.12 included in this PyTorch image
# Base image provides Python 3.12 (from runpod/pytorch:1.0.3-cu1281-torch291-ubuntu2204)
FROM runpod/pytorch:1.0.3-cu1281-torch291-ubuntu2204

# Use the base image's Python as-is to preserve pre-installed packages (torch, cuda libs).
# Validate that the base image provides the expected Python version.
ARG EXPECTED_PYTHON_VERSION=3.12
RUN python --version && \
actual=$(python -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')") && \
if [ "$actual" != "$EXPECTED_PYTHON_VERSION" ]; then \
echo "ERROR: Expected Python $EXPECTED_PYTHON_VERSION but base image provides $actual" && exit 1; \
fi

WORKDIR /app

Expand Down Expand Up @@ -30,10 +39,21 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y --no-ins
&& rm -rf /var/lib/apt/lists/*

# Copy app code and install dependencies
# Use --python to target the base image's Python (preserves torch in its site-packages)
COPY README.md pyproject.toml uv.lock ./
COPY src/ ./
RUN uv export --format requirements-txt --no-dev --no-hashes > requirements.txt \
&& uv pip install --system -r requirements.txt
&& uv pip install --python $(which python) --break-system-packages -r requirements.txt

# Install numpy for the base image's Python version.
# The runpod/pytorch image ships torch but not numpy. Flash build excludes numpy
# from tarballs (BASE_IMAGE_PACKAGES) to save tarball space (~30 MB), so numpy
# must be provided here in the base image.
RUN python -m pip install --no-cache-dir numpy

# Verify torch and numpy are available from the base image
RUN python -c "import torch; print(f'torch {torch.__version__} CUDA {torch.cuda.is_available()}')" \
&& python -c "import numpy; print(f'numpy {numpy.__version__}')"

EXPOSE 80

Expand Down
5 changes: 3 additions & 2 deletions Dockerfile-lb-cpu
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
FROM python:3.12-slim
ARG PYTHON_VERSION=3.12
FROM python:${PYTHON_VERSION}-slim

WORKDIR /app

Expand Down Expand Up @@ -27,7 +28,7 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y --no-ins
COPY README.md pyproject.toml uv.lock ./
COPY src/ ./
RUN uv export --format requirements-txt --no-dev --no-hashes > requirements.txt \
&& uv pip install --system -r requirements.txt
&& uv pip install --system --break-system-packages -r requirements.txt

EXPOSE 80

Expand Down
Loading
Loading