This is a step-by-step walkthrough to turn your desktop computer into a Proxmox VE host with a workstation or gaming VM with GPU and USB controller passthrough.
- CPU: AMD Threadripper 7970x
- GPU: Nvidia RTX 4090 (ASUS)
- Motherboard: Gigabyte TRX50 AI TOP
- Benchmark results listed in 7970X Passthrough Results
- CPU: AMD Ryzen 7970X3D
- GPU: Nvidia RTX 4090 (ASUS)
- Motherboard: ASUS ROG Strix X670E-E
- Benchmark results match a 7800X3D + 7700X CPU (Depending on affinity setting)
- A CPU and motherboard with IOMMU support:
- Intel VT-d or AMD-Vi enabled in BIOS
- For systems with multiple CCDs (7970X/7950X3D), use NPS2/NPS4 respectively.
- Other Bios optimizations listed in the 7970XBiosOptimizations File
- Proxmox VE (tested on 8.x)
- A target VM OS ISO (e.g. Windows 11)
sudoorrootprivileges on the host, I do not recommend this if this is not a dedicated system for this task
- 7970X 3DMark TimeSpy Benchmark Results
- 7970X BIOS Optimizations for Gaming
- Breakdown of CPU Arguments and Topology
- Breakdown of Network Optimizations
- Script for enabling microcode on host system
- Allows you to increase performance of VMs which cannot utilize the amount of cores you have available (experience L3 thrashing with >2 NUMA Nodes)
- Multi-User Single-Computer Gaming with physical display/connections and no remoting
- Quick switching of operating systems without a full reboot
- Live backups of your virtual machines
- You want a proper virtualization host but need an updated kernel
- You have too much time and can't use a computer like a normal person
-
Edit your GRUB command-line:
nano /etc/default/grub
-
Add IOMMU & hugepages options. amd_iommu=on is not necessary and should not be added. On Intel systems, you need to use the equivalent intel_iommu=on appended within the DEFAULT line.
- GRUB_CMDLINE_LINUX_DEFAULT="quiet"
+ GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt hugepagesz=1G default_hugepagesz=1G hugepages=8 kvm.ignore_msrs=1"- Apply:
update-grubThis is necessary to ensure that the vfio drivers can be applied to the passthrough USB/GPU devices. If this step is skipped, the blacklisted drivers will load instead.
- Open (or create)
/etc/initramfs-tools/modulesand add:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd- Rebuild initramfs and reboot:
update-initramfs -u
rebootUnder /etc/modprobe.d/, create the following files.
Ensure your networking is configured properly, this will make your system fully headless. If this is messed up you will need to chroot from recovery to repair the system.
blacklist-usb.conf**
blacklist xhci_hcd
blacklist xhci_pci
blacklist ehci_hcd
blacklist uhci_hcd
blacklist ohci_hcdiommu_unsafe_interrupts.conf**
options vfio_iommu_type1 allow_unsafe_interrupts=1- Run
lspci -vnnand note the IDs of GPU, audio, USB controllers, etc. - For example, grab the two from the following 10de:2684,10de:22ba:
81:00.0 VGA compatible controller [0300]: NVIDIA Corporation AD102 [GeForce RTX 4090] [10de:2684] (rev a1) (prog-if 00 [VGA controller])
Subsystem: ASUSTeK Computer Inc. AD102 [GeForce RTX 4090] [1043:88e2]
Flags: bus master, fast devsel, latency 0, IRQ 91, NUMA node 0, IOMMU group 16
Memory at f0000000 (32-bit, non-prefetchable) [size=16M]
Memory at 10800000000 (64-bit, prefetchable) [size=32G]
Memory at 11000000000 (64-bit, prefetchable) [size=32M]
I/O ports at 3000 [size=128]
Expansion ROM at f1000000 [disabled] [size=512K]
Capabilities: [60] Power Management version 3
Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [78] Express Legacy Endpoint, MSI 00
Capabilities: [b4] Vendor Specific Information: Len=14 <?>
Capabilities: [100] Virtual Channel
Capabilities: [258] L1 PM Substates
Capabilities: [128] Power Budgeting <?>
Capabilities: [420] Advanced Error Reporting
Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
Capabilities: [900] Secondary PCI Express
Capabilities: [bb0] Physical Resizable BAR
Capabilities: [c1c] Physical Layer 16.0 GT/s <?>
Capabilities: [d00] Lane Margining at the Receiver <?>
Capabilities: [e00] Data Link Feature <?>
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau
81:00.1 Audio device [0403]: NVIDIA Corporation AD102 High Definition Audio Controller [10de:22ba] (rev a1)
Subsystem: ASUSTeK Computer Inc. AD102 High Definition Audio Controller [1043:88e2]
Flags: bus master, fast devsel, latency 0, IRQ 43, NUMA node 0, IOMMU group 16
Memory at f1080000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [60] Power Management version 3
Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [78] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [160] Data Link Feature <?>
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel- **
vfio.conf** (replace the IDs below):
options vfio-pci ids=10de:2684,10de:22bablacklist-gpu.conf**
blacklist amdgpu
blacklist nouveau
blacklist nvidiafb
blacklist snd_hda_intelI recommend this as you can then use your passed through device for Bluetooth/connecting to other networks/etc.
blacklist-wifi.conf**
blacklist iwlwifiupdate-initramfs -u
reboot nowOpen the attached Microcode installer and execute. Otherwise, you can also use the Microcode script available at: https://community-scripts.github.io/ProxmoxVE/scripts?id=microcode
chmod +x InstallMicrocode.sh
./InstallMicrocode.sh amd|intelThis ensures that when your VM is powered on, it will properly reload and reset your GPU as if it was baremetal. This is necessary on iGPUs like the 7800X3D/7950X3D.
- Download your GPU’s VBIOS. Grab yours from here: https://www.techpowerup.com/vgabios or extract your ROM directly. Move or wget your ROM into the /usr/share/kvm folder on your PVE host.
wget https://www.techpowerup.com/vgabios/251157/Asus.RTX4090.24576.221014.rom
mv Asus.RTX4090.24576.221014.rom /usr/share/kvm/409024576.rom- Ensure it matches your card’s firmware version.
Tune your network stack for a workstation/gaming targeted environment. Additional information and explanation for these changes can be found in the NetworkOptimizations.md file.
Append to /etc/sysctl.conf:
net.core.rmem_max = 268435456
net.core.wmem_max = 268435456
net.ipv4.tcp_rmem = 4096 87380 134217728
net.ipv4.tcp_wmem = 4096 65536 268435456
net.ipv4.tcp_congestion_control = bbr
net.core.default_qdisc = fq
net.ipv4.tcp_fastopen = 3
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_mtu_probing = 1
net.ipv4.tcp_low_latency = 1
net.ipv4.ip_local_port_range = 10240 65535
Execute changes with:
sysctl -pFor a virtual machine targeted as Windows for gaming, you will need to apply a few tweaks for ensuring EAC/similar tools will not block your device from use. Look at the breakdown of args in the CPUBreakdown file.
Recommendation: Power off the VM before editing its
.conf.
# VM 100: Gaming VM for 7970X/Multi NUMA node CPU (Use Primary cores of first two NUMA nodes, change accordingly)
affinity: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
agent: 1,fstrim_cloned_disks=1
args: -smp '16,cores=16,threads=1,sockets=1,maxcpus=16' -cpu 'host,host-cache-info=on,l3-cache=on,topoext=on,+invtsc,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_crash,hv_ipi,hv_vendor_id=AMDVENDOR,host-phys-bits=true,hypervisor=off,kvm=off'
balloon: 0
bios: ovmf
boot: order=sata0
cpu: host,hidden=1,flags=+pdpe1gb;+hv-tlbflush
hostpci0: 0000:81:00.0,pcie=1,romfile=409024576.rom # x-vga=1 is not necessary. With the ROM added, booting your VM will provide a black screen until your VM loads it's GPU drivers.
hostpci0: 0000:81:00.1,pcie=1
hostpci3: 0000:4b:00.0,pcie=1
# …add other USB controllers as hostpci*
hotplug: disk,network
machine: pc-q35-9.2+pve1
memory: 49152 # Ensure this is the sum of your NUMA 0-X bindings
meta: creation-qemu=9.2.0,ctime=1746507489
name: vmname
net0: e1000e=00:00:5E:14:0A:E1,bridge=vmbr0
numa: 1 # Enable NUMA if necessary by your system
numa0: cpus=0-7,hostnodes=0,memory=24576,policy=bind # Specify the NUMA memory configuration if necessary with your system
numa1: cpus=8-15,hostnodes=1,memory=24576,policy=bind
ostype: win11
sata0: local-lvm:vm-100-disk-0,cache=writeback,discard=on,size=500G,ssd=1 # You can also pass through dedicated hardware as storage (i.e., NVMe hostpci passthrough - best performance)
scsihw: lsi # LSI controller is necessary to bypass EAC, QEMU Storage will be blocked
smbios1: uuid=USE-EXISTING-UUID,manufacturer=R2lnYWJ5dGUgVGVjaG5vbG9neSBDby4sIEx0ZC4=,product=VFJYNTAgQUkgVE9Q,version=RGVmYXVsdCBzdHJpbmctQ0Y=,serial=RGVmYXVsdCBzdHJpbmc=,sku=RGVmYXVsdCBzdHJpbmc=,family=VFJYNTAgTUI=,base64=1 # Fill this information in within your motherboard Options, This converts to Gigabyte Technology Co., Ltd., TRX50 AI TOP, Default string-CF, Default string, Default string, TRX50 MB
sockets: 1
tablet: 0
tpmstate0: local-lvm:vm-100-disk-1,size=4M,version=v2.0
vga: none # Don't confuse the VM by adding a second display adapter.
vmgenid: USE-EXISTING-GENID
vmstatestorage: local-lvmExtract the SMBIOS header from your PVE host using the dmidecode command. This is required to get past EAC. Add this information under VM -> Options -> smbios1:
dmidecode -t 2
# dmidecode 3.4
Getting SMBIOS data from sysfs.
SMBIOS 3.6.0 present.
# SMBIOS implementations newer than version 3.5.0 are not
# fully supported by this version of dmidecode.
Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
Manufacturer: Gigabyte Technology Co., Ltd.
Product Name: TRX50 AI TOP
Version: x.x
Serial Number: Default string
Asset Tag: Default string
Features:
Board is a hosting board
Board is replaceable
Location In Chassis: Default string
Chassis Handle: 0x0003
Type: Motherboard
Contained Object Handles: 0Below are examples for configuring CPU affinity/NUMA(if used by your motherboard/CPUs), more information can be found in the CPUBreakdown file.
On this system (7970X with 4x 32GB of memory), you will want to pass specific CPUs/cores for certain scenarios for the best performance. From testing the best result for gaming was the following - this matches what Ryzen Master does for Core disablement but allows the rest of your CPU to be used:
affinity: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
numa: 1
numa0: cpus=0-7,hostnodes=0,memory=24576,policy=bind
numa1: cpus=8-15,hostnodes=1,memory=24576,policy=bindFor the workstation VM, you can allocate both SMT/HT as well as primary cores. The typical ordering is to use the main core/hyperthread, so on. For Node 0 on the numactl command, you can see that 0-7 are primary cores and 32-39 are HT/SMT, these must be interleaved for the best performance as it is what the guest VM will expect for ordering.
affinity: 0,32,1,33,2,34,3,35,4,36,5,37,6,38,7,39,8,40,9,41,10,42,11,43,12,44,13,45,14,46,15,47,16,48,17,49,18,50,19,51,20,52,21,53,22,54,23,55,24,56,25,57,26,58,27,59,28,60,29,61,30,62,31,63
numa: 1
numa0: cpus=0-15,hostnodes=0,memory=24576,policy=bind
numa1: cpus=16-31,hostnodes=1,memory=24576,policy=bind
numa2: cpus=32-47,hostnodes=2,memory=24576,policy=bind
numa3: cpus=48-63,hostnodes=3,memory=24576,policy=bind- In the PVE GUI: VM → Hardware → Add → PCI Device , then pick your GPU as well as the audio device for your GPU. These are found with the lspci command.
lspci
...
4b:00.0 USB controller: Intel Corporation Thunderbolt 4 NHI [Maple Ridge 4C 2020]
5d:00.0 USB controller: Intel Corporation Thunderbolt 4 USB Controller [Maple Ridge 4C 2020]
6f:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset USB 3.2 Controller (rev 01)
...
81:00.0 VGA compatible controller: NVIDIA Corporation AD102 [GeForce RTX 4090] (rev a1)
81:00.1 Audio device: NVIDIA Corporation AD102 High Definition Audio Controller (rev a1)
...Add these devices accordingly. For the GPU, you will need to edit the .conf file under /etc/pve/qemu-server/. to specify the romfile that you had downloaded/added earlier. Append the following to your hostpci0(GPU) line ,romfile=409024576.rom. Ensure PCI-Express is enabled for all PCI devices. This cannot be done in the PVE GUI.
- hostpci0: 0000:81:00.0,pcie=1
+ hostpci0: 0000:81:00.0,pcie=1,romfile=409024576.rom- In the PVE GUI: VM → Hardware → Add → PCI Device , then pick your USB controller(s).
- Or in the VM’s
.confas additionalhostpci*lines (see above).
- Use e1000e NIC and LSI storage controller to avoid EAC detection.
- Do not install the balloon driver, it will degrade performance. Don't install the serial driver, it's detected by EAC. After installing qemu-guest-agent + drivers, uninstall any
virtio-*drivers in Device Manager. Mostly the virtio serial driver, if you left it. - Change the NICs MAC address to a generic OUI:
00:00:5E:XX:XX:XX. If you leave the BC:24:11 it can be detected as a VM. - Consider disabling Windows mitigations (e.g. Spectre/Meltdown) if it’s a single-tenant gaming host.
- Disable Hardware Accelerated GPU Scheduling for best performance.
dmesg | grep -i vfiofor errors- Verify your IOMMU groups:
find /sys/kernel/iommu_groups/ -type l - Ensure no driver is grabbing the device:
lspci -k -s 81:00.0, you should see that the driver in use is vfio-*** for instance:
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau
If something still doesn’t work, double-check IDs from lspci -vnn, confirm your ROM file, and make sure grub/initramfs changes applied correctly. I am by no means an expert at this, so some of this information could be incorrect. If you see a problem please submit a issue and describe what needs corrected. I mostly am providing this documentation as a means to duplicate this easier in the future.