Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .github/workflows/security-scan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Security Scan

on:
pull_request:
branches: [main]
push:
branches: [main]
schedule:
# Weekly rescan so newly published CVEs surface without a code change.
- cron: '0 6 * * 1'

permissions:
contents: read
security-events: write

jobs:
fs-scan:
name: Repo filesystem scan (deps + secrets + config)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Trivy filesystem scan
uses: aquasecurity/trivy-action@0.28.0
with:
scan-type: fs
scan-ref: .
severity: CRITICAL,HIGH
exit-code: '1'
ignore-unfixed: true
format: sarif
output: trivy-fs.sarif

- name: Upload SARIF
if: always()
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: trivy-fs.sarif

api-image-scan:
name: API image scan
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Build API image
run: docker build -t openprocessor-api:scan -f Dockerfile .

- name: Trivy image scan
uses: aquasecurity/trivy-action@0.28.0
with:
image-ref: openprocessor-api:scan
severity: CRITICAL,HIGH
exit-code: '1'
ignore-unfixed: true

# The Triton image is ~20 GB (NGC base) — too heavy for hosted runners'
# disk on every PR. It is scanned locally via `make scan-triton` before
# any Docker Hub publish instead.
13 changes: 9 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,11 @@ FROM python:3.13-slim-trixie AS builder

WORKDIR /build

# Install build dependencies (isolated to this stage)
RUN apt-get update && apt-get install -y --no-install-recommends \
# Install build dependencies (isolated to this stage).
# apt-get upgrade pulls Debian point-release security fixes the base tag
# may lag behind.
RUN apt-get update && apt-get upgrade -y \
&& apt-get install -y --no-install-recommends \
build-essential \
gcc \
g++ \
Expand Down Expand Up @@ -48,8 +51,10 @@ LABEL org.opencontainers.image.title="OpenProcessor FastAPI Service" \
org.opencontainers.image.source="https://github.com/davidamacey/OpenProcessor" \
org.opencontainers.image.documentation="https://github.com/davidamacey/OpenProcessor/blob/main/README.md"

# Runtime-only system packages (no build tools)
RUN apt-get update && apt-get install -y --no-install-recommends \
# Runtime-only system packages (no build tools); upgrade first for
# Debian point-release security fixes.
RUN apt-get update && apt-get upgrade -y \
&& apt-get install -y --no-install-recommends \
curl \
jq \
libgl1 \
Expand Down
34 changes: 25 additions & 9 deletions Dockerfile.triton
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
# =============================================================================
# Triton Inference Server with PyTorch for Python Backend (BLS)
# =============================================================================
# Based on NVIDIA Triton 25.10 with PyTorch + TorchVision for Python backend
# models (ocr_pipeline).
# Based on NVIDIA Triton 26.06 (CUDA 13.3, TensorRT 11.0, Ubuntu 24.04) with
# PyTorch + TorchVision for Python backend models (ocr_pipeline).
#
# NOTE: TensorRT engines (.plan) are bound to the TensorRT major version.
# Moving between Triton releases that change TRT versions requires
# re-exporting every engine — see docs/MIGRATION_TRITON_26.md.
# =============================================================================

FROM nvcr.io/nvidia/tritonserver:25.10-py3
FROM nvcr.io/nvidia/tritonserver:26.06-py3

LABEL org.opencontainers.image.title="OpenProcessor Inference Server" \
org.opencontainers.image.description="NVIDIA Triton Inference Server with TensorRT models for visual AI" \
Expand All @@ -15,18 +19,30 @@ LABEL org.opencontainers.image.title="OpenProcessor Inference Server" \
org.opencontainers.image.source="https://github.com/davidamacey/OpenProcessor" \
org.opencontainers.image.documentation="https://github.com/davidamacey/OpenProcessor/blob/main/README.md"

# Install PyTorch and TorchVision (CUDA 12.x compatible)
# Pull in the latest Ubuntu security patches on every build; the NGC base
# lags point releases between monthly tags.
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

# Install PyTorch and TorchVision (CUDA 13.x wheels)
# Required for Python backend models (ROI align, per-box embeddings, OCR pipeline)
# Note: torch/torchvision use the PyTorch CUDA 12.4 wheel index
RUN pip install --no-cache-dir \
torch==2.5.1 \
torchvision==0.20.1 \
--index-url https://download.pytorch.org/whl/cu124 \
&& pip install --no-cache-dir opencv-python-headless==4.10.0.84 \
torch==2.12.1 \
torchvision==0.27.1 \
--index-url https://download.pytorch.org/whl/cu130 \
&& pip install --no-cache-dir opencv-python-headless==4.13.0.92 \
&& python3 -c "import torch; print(f'PyTorch {torch.__version__} installed, CUDA: {torch.cuda.is_available()}')" \
&& python3 -c "import torchvision; print(f'TorchVision {torchvision.__version__} installed')"

HEALTHCHECK --interval=30s --timeout=10s --start-period=120s --retries=3 \
CMD curl -f http://localhost:8000/v2/health/ready || exit 1

# Run as the unprivileged user shipped with the NGC image (uid 1000).
# Triton's ports (8000-8002) are unprivileged and /models is a bind mount,
# so root is unnecessary. If the base image drops this user, the build
# fails loudly here rather than silently reverting to root.
USER triton-server

CMD ["tritonserver", "--model-store=/models"]
2 changes: 1 addition & 1 deletion INSTALLATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -397,7 +397,7 @@ The project includes production-optimized Dockerfiles:
| File | Purpose | Base Image |
|------|---------|------------|
| `Dockerfile` | FastAPI service | `python:3.13-slim-trixie` |
| `Dockerfile.triton` | Triton server | `nvcr.io/nvidia/tritonserver:25.10-py3` |
| `Dockerfile.triton` | Triton server | `nvcr.io/nvidia/tritonserver:26.06-py3` |

### Building Images

Expand Down
16 changes: 16 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -833,6 +833,22 @@ check-all: ## Full system health check (API + Triton + OpenSearch)
@echo ""
@echo "==================================================================================="

# ==================================================================================
# Security Scanning
# ==================================================================================

.PHONY: scan
scan: ## Scan both Docker images for vulnerabilities (trivy/grype/dockle; FAIL_ON_CRITICAL=true)
bash $(SCRIPTS_DIR)/security-scan.sh all

.PHONY: scan-api
scan-api: ## Scan only the API image
bash $(SCRIPTS_DIR)/security-scan.sh api

.PHONY: scan-triton
scan-triton: ## Scan only the Triton image
bash $(SCRIPTS_DIR)/security-scan.sh triton

# ==================================================================================
# Cleanup
# ==================================================================================
Expand Down
34 changes: 20 additions & 14 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ services:
- triton_net

triton-sdk:
image: nvcr.io/nvidia/tritonserver:25.10-py3-sdk
image: nvcr.io/nvidia/tritonserver:26.06-py3-sdk
container_name: triton-sdk
profiles:
- benchmark
Expand All @@ -136,7 +136,7 @@ services:
- triton_net

node-exporter:
image: prom/node-exporter:latest
image: prom/node-exporter:v1.10.2
container_name: triton-node-exporter
restart: always
command:
Expand Down Expand Up @@ -170,7 +170,7 @@ services:
- triton_net

prometheus:
image: prom/prometheus:latest
image: prom/prometheus:v3.12.0
container_name: triton-prometheus
restart: always
ports:
Expand All @@ -191,7 +191,7 @@ services:
- node-exporter

grafana:
image: grafana/grafana:latest
image: grafana/grafana:13.1.0
container_name: triton-grafana
restart: always
ports:
Expand All @@ -216,10 +216,12 @@ services:
- loki

loki:
image: grafana/loki:latest
# Runs as the image's builtin non-root user (uid 10001). Existing
# root-owned loki_data volumes need a one-time chown — see
# docs/MIGRATION_TRITON_26.md.
image: grafana/loki:3.6.12
container_name: triton-loki
restart: always
user: "0" # Run as root to avoid permission issues
ports:
- 4606:3100 # Loki
volumes:
Expand All @@ -229,15 +231,19 @@ services:
networks:
- triton_net

promtail:
image: grafana/promtail:latest
container_name: triton-promtail
# Log shipper: Grafana Alloy (Promtail reached EOL 2026-03-02).
alloy:
image: grafana/alloy:v1.17.1
container_name: triton-alloy
restart: always
volumes:
- ./monitoring/promtail-config.yml:/etc/promtail/config.yml
- /var/run/docker.sock:/var/run/docker.sock
- ./monitoring/alloy-config.alloy:/etc/alloy/config.alloy:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- /var/lib/docker/containers:/var/lib/docker/containers:ro
command: -config.file=/etc/promtail/config.yml
command:
- run
- /etc/alloy/config.alloy
- --storage.path=/tmp/alloy
networks:
- triton_net
depends_on:
Expand All @@ -248,7 +254,7 @@ services:
# OpenSearch 3.x with k-NN plugin for vector similarity search
# ===================================================================
opensearch:
image: opensearchproject/opensearch:3.3.1
image: opensearchproject/opensearch:3.6.0
container_name: triton-opensearch
restart: always
environment:
Expand Down Expand Up @@ -276,7 +282,7 @@ services:
- triton_net

opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:3.3.0
image: opensearchproject/opensearch-dashboards:3.6.0
container_name: triton-opensearch-dashboards
restart: always
environment:
Expand Down
92 changes: 92 additions & 0 deletions docs/MIGRATION_TRITON_26.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Migration Guide: Triton 26.06 / TensorRT 11 Upgrade

This release moves the stack from Triton 25.10 (TensorRT 10.13) to
**Triton 26.06 (CUDA 13.3, TensorRT 11.0)** and pins every container in
`docker-compose.yml`. Existing deployments need the one-time steps below.

## 1. Re-export ALL TensorRT engines (required)

TensorRT serialized engines (`.plan`) are **not portable across TensorRT
major versions**. Every engine built under 25.10 (TRT 10.x) fails to load
on 26.06 (TRT 11.x) with a serialization-version error.

```bash
# Rebuild images first
docker compose build

# Start only what exports need
docker compose up -d triton-server yolo-api

# Re-export every model (runs inside the API container)
docker compose exec yolo-api python /app/export/export_models.py --formats onnx_end2end trt_end2end
docker compose exec yolo-api python /app/export/export_scrfd.py
docker compose exec yolo-api python /app/export/export_face_recognition.py
docker compose exec yolo-api python /app/export/export_mobileclip_image_encoder.py
docker compose exec yolo-api python /app/export/export_mobileclip_text_encoder.py
docker compose exec yolo-api python /app/export/export_paddleocr_det.py
docker compose exec yolo-api python /app/export/export_paddleocr_rec.py

# Restart Triton to load the new engines
docker compose restart triton-server
```

Notes:

- The client-side `tensorrt-cu13==11.0.0.114` pin **must** match the TRT
bundled in the Triton image. Do not bump it independently; upgrade it
together with the Triton base tag.
- GPU requirements: driver R570+ (CUDA 13.x) and compute capability >= 7.5
(Turing or newer). Volta/Pascal are no longer supported by this Triton
release line.

## 2. Loki data volume ownership (one-time)

Loki previously ran as root (`user: "0"`); it now runs as the image's
builtin non-root user (uid 10001). Existing volumes are root-owned and
will fail with permission errors until chowned:

```bash
docker compose stop loki
docker run --rm -v openprocessor_loki_data:/loki alpine chown -R 10001:10001 /loki
docker compose up -d loki
```

Fresh installs need nothing.

## 3. Promtail → Grafana Alloy

Promtail reached end-of-life on 2026-03-02 and has been replaced by
**Grafana Alloy** (`monitoring/alloy-config.alloy`). The shipped pipeline
is equivalent (Docker service discovery for the Triton + API containers).

If you customized `monitoring/promtail-config.yml`, convert it:

```bash
docker run --rm -v $(pwd)/monitoring:/cfg grafana/alloy:v1.17.1 \
convert --source-format=promtail --output=/cfg/alloy-config.alloy /cfg/promtail-config.yml
```

## 4. Health endpoint semantics

- `/live` — process liveness only (new; used by the container HEALTHCHECK).
- `/ready` — actively probes Triton (gRPC `is_server_live`) and OpenSearch;
returns **503 with per-service detail** while any dependency is down.
- `/health` — now an alias of `/ready`. If you previously treated `/health`
as an always-200 liveness signal, point that consumer at `/live`.

## 5. Pinned image matrix

| Service | Image |
|---|---|
| triton-server | `nvcr.io/nvidia/tritonserver:26.06-py3` (via `Dockerfile.triton`) |
| opensearch | `opensearchproject/opensearch:3.6.0` |
| opensearch-dashboards | `opensearchproject/opensearch-dashboards:3.6.0` |
| prometheus | `prom/prometheus:v3.12.0` |
| grafana | `grafana/grafana:13.1.0` |
| loki | `grafana/loki:3.6.12` (non-root) |
| alloy (replaces promtail) | `grafana/alloy:v1.17.1` |
| node-exporter | `prom/node-exporter:v1.10.2` |
| dcgm-exporter | `nvcr.io/nvidia/k8s/dcgm-exporter:3.3.5-3.4.0-ubuntu22.04` |

OpenSearch 3.3 → 3.6 is a same-major upgrade; existing indices roll
forward in place. Take a snapshot first if the data matters.
3 changes: 2 additions & 1 deletion export/export_face_recognition.py
Original file line number Diff line number Diff line change
Expand Up @@ -277,12 +277,13 @@ def convert_to_tensorrt(

try:
import tensorrt as trt
from trt_utils import create_explicit_network

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)

# Create builder
builder = trt.Builder(TRT_LOGGER)
network = builder.create_network(1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))
network = create_explicit_network(builder)
parser = trt.OnnxParser(network, TRT_LOGGER)

# Parse ONNX
Expand Down
Loading
Loading