Skip to content

Latest commit

 

History

History
100 lines (66 loc) · 4.57 KB

File metadata and controls

100 lines (66 loc) · 4.57 KB

PostgreSQL Tuning for NVMe Storage

NVMe drives change the assumptions baked into PostgreSQL's default configuration. The defaults assume spinning disks where random reads are ~40x slower than sequential. On NVMe, random and sequential reads are nearly identical. These settings tell the query planner and I/O subsystem to take advantage of that.

Core Parameters

For a 32GB RAM Postgres VM with NVMe passthrough:

# postgresql.conf

shared_buffers = 8GB                 # 25% of VM RAM
effective_cache_size = 24GB          # 75% of VM RAM (shared_buffers + OS page cache)
work_mem = 64MB                      # per-sort/hash operation
maintenance_work_mem = 2GB           # VACUUM, CREATE INDEX, ALTER TABLE

# These two are the NVMe-specific wins:
random_page_cost = 1.1               # default is 4.0 (spinning disk). NVMe: random ≈ sequential
effective_io_concurrency = 200       # default is 1. NVMe handles hundreds of concurrent I/O ops

Why Each Matters on NVMe

random_page_cost = 1.1 — This is the most impactful change. The default (4.0) tells the planner that random reads cost 4x sequential, which biases it toward sequential scans. On NVMe, random reads are nearly as fast as sequential. Setting this to 1.1 makes the planner prefer index scans, which is what you want for most workloads.

effective_io_concurrency = 200 — The default (1) assumes a single spinning disk that can only do one I/O at a time. NVMe drives handle massive parallelism — hundreds of concurrent operations with minimal latency increase. This setting controls bitmap heap scan prefetching.

shared_buffers = 25% of RAM — Standard guidance, but critical to get right. Too low wastes NVMe speed on repeated disk reads. Too high (>40%) starves the OS page cache, which PostgreSQL also relies on.

effective_cache_size = 75% of RAM — Not an allocation, just a hint to the planner about how much data is likely cached between shared_buffers and the OS. Helps the planner choose index scans over sequential scans for large tables.

work_mem = 64MB — Per-sort memory. Higher values let sorts and hash joins happen in memory instead of spilling to disk. Be careful: this is per-operation, not per-connection. 20 concurrent queries each doing 3 sorts = 60 × 64MB = 3.8GB.

maintenance_work_mem = 2GB — Used by VACUUM, CREATE INDEX, and ALTER TABLE. Higher values speed up index builds significantly, especially for pgvector HNSW indexes.

pgvector-Specific Settings

For HNSW index builds and vector search:

# Speed up HNSW index construction
max_parallel_maintenance_workers = 4   # parallel index build (PG 15+)
maintenance_work_mem = 2GB             # more memory for index construction

# At query time (can be SET per-session):
# SET hnsw.ef_search = 40;            # trade recall for speed (default varies)

hnsw.ef_search controls the search beam width. Higher values = better recall but slower queries. For most RAG workloads, 40 is a good starting point. You can tune this per-query:

SET hnsw.ef_search = 40;
SELECT * FROM items ORDER BY embedding <=> $1 LIMIT 10;

Scaling by VM Size

VM RAM shared_buffers effective_cache_size work_mem maintenance_work_mem
16 GB 4 GB 12 GB 32 MB 1 GB
32 GB 8 GB 24 GB 64 MB 2 GB
64 GB 16 GB 48 GB 128 MB 4 GB
128 GB 32 GB 96 GB 256 MB 8 GB

random_page_cost and effective_io_concurrency stay the same regardless of RAM — they depend on the storage, not the memory.

Network Access (pg_hba.conf)

If your Postgres VM is on an internal bridge with other VMs:

# /etc/postgresql/15/main/pg_hba.conf

# Local connections
local   all   all                 peer

# Internal bridge subnet (adjust to your network)
host    all   all   10.0.0.0/24   scram-sha-256

And in postgresql.conf:

listen_addresses = '*'    # or specific internal IP like '10.0.0.100'

Validating Your Setup

After applying these settings, restart Postgres and run the benchmark scripts:

# Quick sanity check — should show your new values
psql -c "SHOW shared_buffers; SHOW random_page_cost; SHOW effective_io_concurrency;"

# Run the full benchmark suite
../benchmarks/run-pgbench.sh
../benchmarks/run-pgvector.sh

See benchmarks/ for the full methodology and expected results on reference hardware.