feat: Python version matrix for Docker images + error-masking fix#76
feat: Python version matrix for Docker images + error-masking fix#76
Conversation
Parameterize all 4 Dockerfiles with PYTHON_VERSION build arg. GPU images also accept PYTORCH_BASE to select the correct PyTorch base image per Python version. Add versioned Makefile targets: - build-all-versioned: builds 10 images (GPU 3.11/3.12, CPU 3.10-3.12) - build-wip-versioned: multi-platform push with latest alias - smoketest-versioned: verify Python version in each image GPU base image mapping: - Python 3.11: pytorch/pytorch:2.9.1-cuda12.8-cudnn9-runtime - Python 3.12: pytorch/pytorch:2.10.0-cuda12.8-cudnn9-runtime
Previously, _load_generated_handler() silently returned None on any failure (missing file, import error, syntax error), causing deployed endpoints to fall back to the FunctionRequest/Live Serverless handler. This masked real deployment issues like Python version mismatches. Now deployed mode (FLASH_RESOURCE_NAME set) treats handler loading failures as fatal RuntimeError. Live Serverless mode skips the generated handler entirely since it only uses FunctionRequest protocol.
Tests expected None returns but handler.py now raises RuntimeError in deployed mode. Updated all 8 TestLoadGeneratedHandler tests to use pytest.raises(RuntimeError). Also synced uv.lock to pick up latest runpod-flash version.
There was a problem hiding this comment.
Pull request overview
Adds a Python-versioned Docker image matrix (CPU/GPU, QB/LB) to prevent runtime Python mismatches with binary deps, and makes deployed endpoints fail fast when the generated handler can’t be loaded (instead of silently falling back to the Live Serverless protocol).
Changes:
- Parameterizes Docker builds via
PYTHON_VERSION/PYTORCH_BASEbuild args and introduces versioned image tags (e.g.,:py3.11-...). - Adds Makefile targets to build/push/smoketest the multi-version image matrix.
- Updates generated-handler loading behavior to raise in deployed mode, with corresponding unit test updates.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
Makefile |
Adds Python version matrix variables and versioned build/push/smoketest targets. |
Dockerfile |
Switches GPU base image selection to PYTORCH_BASE arg (and adds PYTHON_VERSION arg). |
Dockerfile-lb |
Same as GPU QB Dockerfile, but for LB image. |
Dockerfile-cpu |
Parameterizes CPU base image via python:${PYTHON_VERSION}-slim. |
Dockerfile-lb-cpu |
Parameterizes CPU LB base image via python:${PYTHON_VERSION}-slim. |
src/handler.py |
Makes deployed-mode handler loading failures fatal; keeps Live Serverless path separate. |
tests/unit/test_handler.py |
Updates _load_generated_handler tests to expect RuntimeError instead of None. |
uv.lock |
Updates lockfile with multiple dependency version bumps. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Include platform.python_version() in the worker boot banner for runtime version visibility during E2E testing.
- Add build-time Python version validation to GPU Dockerfiles - Restructure build-all-versioned to run setup once via internal targets - Add version assertion to smoketest-versioned (fail on mismatch)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
tests/unit/test_handler.py:179
test_loads_generated_handler_from_filepatcheshandler.Pathto return atmp_pathfile, but_load_generated_handler()enforceshandler_file.resolve().is_relative_to(Path('/app').resolve()). Sincetmp_pathis not under/app, this test will raise the "resolves outside /app" RuntimeError instead of returning a loaded handler. Adjust the test to bypass the/appsandbox check (e.g., mockresolve()/is_relative_toor patch the/approot used by the function) or refactor_load_generated_handlerto use a patchable constant for the app root.
def test_loads_generated_handler_from_file(self, tmp_path):
"""With valid generated handler file, loads and returns handler function."""
handler_file = tmp_path / "handler_gpu_config.py"
handler_file.write_text(
"async def handler(event):\n"
" return {'result': event.get('input', {}).get('prompt', 'default')}\n"
)
with patch.dict("os.environ", {"FLASH_RESOURCE_NAME": "gpu_config"}):
with patch("handler.Path", return_value=handler_file):
result = _load_generated_handler()
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
- Add default case to pytorch_base() shell function in all Makefile targets - Guard test_handler.py import against FLASH_RESOURCE_NAME env var
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 10 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Summary
PYTHON_VERSIONbuild arg (default 3.11)build-*-versioned,build-wip-versioned,smoketest-versionedtargetsContext
AE-2391: Python version mismatch between user build environment and worker runtime causes silent failures with binary packages (numpy, etc). This PR adds versioned Docker images so the SDK can select the correct runtime for the user's Python version.
Companion PR: runpod/flash (SDK-side version detection and image selection)
Test plan
make build-all-versionedbuilds all 10 images locallymake smoketest-versionedverifies correct Python version in each imagemake testpasses unchanged