53 lines (39 loc) · 3.26 KB

Network Optimization Explanation

Below is a line-by-line walk-through of the settings, describing what each one changes, and when it helps on a Proxmox host that mainly:

runs VMs and containers
talks to fast local storage or Ceph
moves a lot of data over 10–40 Gb Ethernet
rarely acts as a public-facing web server

1 Socket buffer limits

(affects any program that uses TCP/UDP, including QEMU and Ceph OSDs)

Setting	Meaning (bytes)	Default	When to raise it	Caveat
net.core.rmem_max = 268435456	Max receive buffer a single socket may auto-grow to (256 MiB)	212 KiB	Long-haul or 40/100 GbE links with high bandwidth × latency product	Big cap alone ≠ bigger buffers — apps must ask for them or autotune must ramp up.
net.core.wmem_max = 268435456	Max send buffer (256 MiB)	212 KiB	Bulk senders (NFS server, iSCSI target) over high-speed LAN/WAN	Uses more kernel memory under heavy fan-out loads.
net.ipv4.tcp_rmem = 4096 87380 134217728	Min / default / max recv autotune steps	4 KiB / 85 KiB / 128 MiB	Keep defaults (85 KiB) but allow auto-growth to 128 MiB	Needs net.core.rmem_max ≥ max.
net.ipv4.tcp_wmem = 4096 65536 268435456	Min / default / max send autotune	4 KiB / 64 KiB / 256 MiB	Same logic	As above.

Should you keep them?

Yes if you have 10 Gb + links and move multi-gigabyte files (VM images, Ceph replication).
Otherwise the stock limits (16 MiB max) are fine and use less RAM.

2 Congestion-control & queuing

Setting	What it does	Default	Keep?
net.ipv4.tcp_congestion_control = bbr	Switches to Google BBR congestion algorithm. Gives faster ramp-up and steadier RTT than CUBIC on clean links.	cubic	• Great for WAN or oversubscribed switches.• Safe on LAN.
net.core.default_qdisc = fq	Sets the default queuing discipline to FQ (Fair Queue). Works best with BBR.	pfifo_fast	Yes – low latency, combats buffer-bloat.
net.ipv4.tcp_fastopen = 3	Enables TCP Fast Open for client (1) + server (2) = 3. Skips 1 RTT on connection setup.	1 (client only) or 0	Mostly helps web servers. Harmless but you need app support.

3 Window scaling / SACK / low-latency tweaks

Setting	Default	Meaning & advice
net.ipv4.tcp_window_scaling = 1	1	Always leave enabled; required for windows > 64 KiB.
net.ipv4.tcp_sack = 1	1	Selective ACKs – speeds up recovery; keep 1 unless a buggy firewall disables SACK.
net.ipv4.tcp_no_metrics_save = 1	0	Don’t cache old RTT/cwnd per destination. Can improve first connection after route changes; safe to enable.
net.ipv4.tcp_mtu_probing = 1	0	Actively probe for a working PMTU when ICMP is blocked. Good for WAN, harmless on LAN.
net.ipv4.tcp_low_latency = 1	0	Tells the TCP stack to favor lower latency over throughput in some heuristics. Minor effect; fine to enable.

4 Local port range

Setting	Default	Why bump it
net.ipv4.ip_local_port_range = 10240 65535	32768 60999	Allows ~10 × more concurrent outbound sockets before wrap-around; useful if the host launches many container/VM connections (SSH, Ceph, NFS, web). Safe.