Bare Metal Proxmox for RAG: NVMe + Postgres + pgvector

Production-tested configs, scripts, and benchmarks for running PostgreSQL with pgvector on bare metal Proxmox with NVMe PCIe passthrough. Everything here came from a real migration off scattered cloud infrastructure onto a single dedicated server.

On reference hardware (32GB VM, 8 cores, NVMe passthrough) we measured 6,611 TPS read-write, 42K TPS read-only, and 1,010 QPS vector search at 7.9ms. See benchmarks/ for full methodology and results.

Architecture

┌──────────────────────────────────────────────────────────┐
│  Bare Metal Host (Debian 12 + Proxmox VE)                │
│                                                          │
│  ┌────────────┐  ┌──────────┐  ┌──────────────────┐     │
│  │ postgres   │  │ dev      │  │ app VMs          │     │
│  │ VM 100     │  │ VM 200   │  │ VM 300+          │     │
│  │ NVMe pthru │  │ sandbox  │  │ your apps        │     │
│  │ 32GB       │  │ 16GB     │  │ sized per app    │     │
│  └─────┬──────┘  └────┬─────┘  └───────┬──────────┘     │
│        └──────────────┴────────────────┘                 │
│                vmbr0 bridge (10.0.0.0/24)                │
└──────────────────────────────────────────────────────────┘

What's Here

Docs

Guide	What it covers
Proxmox Setup Guide	Full architecture, VM creation, network, config tracking
NVMe Passthrough	IOMMU setup, PCIe passthrough step-by-step
Postgres Tuning for NVMe	postgresql.conf for NVMe, pgvector settings, scaling table
Why Proxmox over NixOS	Decision record — we tried NixOS first
Gotchas and Lessons	7 operational war stories (fail2ban, NAT, SSH zombies, etc.)
App VM Best Practices	systemd templates, resource limits, deploy patterns
Deployment Checklist	New app from zero to production
Backup Strategy	pg_dump + restic to B2, restore procedures
TLS Certificates	certbot DNS-01, Caddy auto-TLS

Scripts

Script	What it does
harden-host.sh	SSH hardening, fail2ban with Proxmox jail (systemd backend), auto-updates
harden-vm.sh	Per-VM hardening: fail2ban, sysctl, disable bloat
backup-postgres.sh	Dump all databases, rotate old backups
backup-offsite.sh	Encrypted offsite backup to Backblaze B2 via restic
create-template-vm.sh	Ubuntu cloud image → Proxmox VM template
prepare-prod-template.sh	Harden + clean a VM for cloning
caddy-domain.sh	Add/remove/list domain routes via Caddy admin API

Benchmarks

File	Contents
Methodology	Hardware specs, test setup, caveats
run-pgbench.sh	Reproduce our pgbench benchmarks
run-pgvector.sh	Reproduce our pgvector benchmarks
pgbench results	TPC-B and read-only numbers
pgvector results	HNSW search throughput and index build
Cloud comparison	Side-by-side with Supabase, Alibaba, AWS

Quick Start

If you have a fresh Proxmox box:

1. Enable IOMMU for NVMe passthrough → docs/nvme-passthrough.md
2. Harden the host → sudo ./scripts/harden-host.sh
3. Create a VM template → sudo ./scripts/create-template-vm.sh
4. Set up Postgres VM with NVMe passthrough → docs/proxmox-setup-guide.md
5. Tune Postgres → docs/postgres-tuning-nvme.md
6. Run benchmarks to validate → ./benchmarks/run-pgbench.sh

Configuration

All scripts use environment variables with sensible defaults. Copy .env.example and customize:

cp .env.example .env
# Edit .env with your IPs, credentials, etc.
source .env

Reference Hardware

Our benchmarks ran on: Intel Xeon Gold 6142, 128GB RAM, Samsung PM983 1.92TB NVMe. The Postgres VM gets 32GB RAM and 8 cores — not the full box. Your numbers will scale with your hardware.

License

MIT