diff --git a/README.md b/README.md index 79488c76ed7f..563b1d684d32 100644 --- a/README.md +++ b/README.md @@ -1,87 +1,88 @@ +# AMD Secure Encrypted Virtualization (SEV) + +[Overview](#overview) | [Upstream Support](#upstream-support) | [Prepare Host](#prepare-host) | [Build & Install](#build--install) | [Launch Guest](#launch-guest) | [Additional Resources](#resources) | [FAQ](#faq) + ## Overview -This repo will build host/guest kernel, QEMU, and OVMF packages that are known to work in conjunction with the latest development trees for SNP host/hypervisor support. The build scripts will utilize the latest published [development tree for the SNP host kernel](https://github.com/amdese/linux/tree/snp-host-latest), which will generally correspond to the latest patchset posted upstream along with fixes/changes on top resulting from continued development/testing and upstream review. It will also utilize the latest published [development tree for QEMU](https://github.com/amdese/qemu/tree/snp-latest). +This repo will build host/guest kernel, QEMU, and OVMF packages that are known to work in conjunction with the latest development trees for SEV Feature host/hypervisor support. The build scripts will utilize the latest published [development tree for the SNP host kernel](https://github.com/amdese/linux/tree/snp-host-latest), which will generally correspond to the latest patchset posted upstream along with fixes/changes on top resulting from continued development/testing and upstream review. It will also utilize the latest published [development tree for QEMU](https://github.com/amdese/qemu/tree/snp-latest). -Note that SNP hypervisor support is still being actively developed/upstreamed. Branhes of this repository provide early snapshots of new features. Please report any issues with it or any other components built by these scripts via the issue tracker for this repo [here](https://github.com/AMDESE/AMDSEV/issues). +Note that SEV 3.0+ hypervisor support is still being actively developed/upstreamed. Branches of this repository provide early snapshots of new features. Please report any issues with it or any other components built by these scripts via the issue tracker for this repo [here](https://github.com/AMDESE/AMDSEV/issues). -Follow the below steps to build the required components and launch an SEV-SNP guest. These steps are tested primarily in conjunction with Ubuntu 22.04 hosts/guests, but other distros are supported to some degree by contributors to this repo. +### SEV Features -NOTE: If you're building from an existing checkout of this repo and have build issues with edk2, delete the ovmf/ directory prior to starting the build so that it can be re-initialized cleanly. +AMD Secure Encrypted Virtualization (SEV) is a set of extensions to the AMD-V architecture for running encrypted virtual machines (VMs) under KVM. Each AMD EPYC processor family has introduced SEV improvements, sometimes with a milestone feature. -## Upstream support +| SEV Version | EPYC Platform | Description | +|---|---|---| +| **SEV 1.0** | EPYC 7001 | Encrypts guest memory pages so only the guest has access to unencrypted data. Each VM uses a unique encryption key. | +| **SEV 2.0 (ES - Encrypted State)** | EPYC 7002 | Encrypts guest register state during world switches, preventing the hypervisor from directly accessing or modifying registers. Guests are notified before certain world switches via a new exception, allowing selective information sharing. | +| **SEV 3.0 (SNP - Secure Nested Paging)** | EPYC 7003 | Protects guest memory integrity via the Reverse Map Table (RMP), a system-managed structure that tracks ownership and state of each physical page. Prevents hypervisor-based memory remapping attacks by enforcing single-guest page assignment and validating all memory state transitions. | +| **SEV 3.1** | EPYC 9004/8004 | Enhancements to SNP (SVSM, TSME, 1006 ASID keys). | -SEV is an extension to the AMD-V architecture which supports running encrypted -virtual machine (VMs) under the control of KVM. Encrypted VMs have their pages -(code and data) secured such that only the guest itself has access to the -unencrypted version. Each encrypted VM is associated with a unique encryption -key; if its data is accessed by a different entity using a different key, the -encrypted guests data will be incorrectly decrypted, leading to unintelligible -data. +> **Note:** "Legacy SEV" refers to the first generation. This README uses feature acronyms (SEV-ES, SEV-SNP) when referring to specific software interfaces. -SEV support has been accepted in upstream projects. This repository provides -scripts to build various components to enable SEV support until the distros -pick the newer version of components. +This repository provides scripts to build host/guest kernels, QEMU, and OVMF +from AMD's development branches for SEV 3.0+ (SNP). This README covers +host setup, upstream version requirements, and then walks through launching SEV guests. -To enable SEV support we need the following versions. +### Development Features -| Project | Version | -| ------------- |:------------------------------------:| -| kernel | >= 4.16 | -| libvirt | >= 4.5 | -| qemu | >= 2.12 | -| ovmf | >= commit (75b7aa9528bd 2018-07-06 ) | +The `main` branch tracks the +[snp-host-latest](https://github.com/amdese/linux/tree/snp-host-latest) kernel and +[snp-latest](https://github.com/amdese/qemu/tree/snp-latest) QEMU trees, which +carry patches under active upstream review. Current development features include: -SEV-ES is an extension to SEV that protects the guest register state from the -hypervisor. An SEV-ES guest's register state is encrypted during world switches -and cannot be directly accessed or modified by the hypervisor. SEV-ES includes -architectural support for notifying a guest's operating system when certain -types of world switches are about to occur through a new exception. This allows -the guest operating system to selective share information with the hypervisor -when needed for functionality. +| Feature | Component | Description | +|---|---|---| +| guest_memfd hugepage support | Kernel + QEMU | Hugepage-backed private guest memory for improved performance. | +| In-place private/shared memory conversion | QEMU | Converts guest memory between private and shared states without re-allocation. | +| CipherTextHiding | Kernel | Prevents ciphertext side-channel attacks on SNP guests. | +| SNP policy bit publishing | Kernel | Exposes supported SEV-SNP policy bits to userspace via the CCP/PSP driver. | +| SNP_FEATURE_INFO command | Kernel | New firmware command to query SNP feature support. | -SEV-ES support has been submitted and accepted in upstream projects. The -upstream version of the projects should be used. +As these features land upstream, they will be removed from the development +branches and this list. -To enable SEV-ES support we need the following versions. +## Upstream Support -| Project | Version/Tag | -| ------------- |:------------------------------------:| -| kernel | >= 5.11 | -| libvirt | >= 4.5 | -| qemu | >= 6.00 | -| ovmf | >= edk2-stable202102 | +Modern distributions ship all the components needed to run SEV 3.0+ (SNP) +guests without building anything from source. -* SEV support is not available in SeaBIOS. Guest must use OVMF. +For a list of OS distributions that have been tested and certified for SEV +feature support across EPYC platforms, see +[sev-certify](https://github.com/AMDEPYC/sev-certify). -SEV-SNP is an extension to SEV-ES that protects guest memory integrity and -prevents unauthorized memory remapping attacks by the hypervisor. -An SEV-SNP guest's memory is protected by the Reverse Map Table (RMP), -a system-managed data structure that tracks the ownership and state of -each physical page. The RMP enforces that each page can only be assigned to -a single guest at a time and validates all memory state transitions. +### Upstream Version Reference -To enable SEV-SNP support we need the following versions. +| Minimum Version | SEV 1.0 | SEV 2.0 (ES) | SEV 3.0 (SNP) | +|---|---|---|---| +| Kernel (host) | 4.16 | 5.11 | 6.11 | +| Kernel (guest) | 4.16 | 5.11 | 5.19 (6.11 recommended; full attestation and guest_memfd) | +| QEMU | 2.12 | 6.0 | 9.1 | +| OVMF | 75b7aa9528bd | edk2-stable202102 | edk2-stable202405 | +| libvirt | 4.5 | 4.5 | 10.5.0 | +| SEV FW | — | — | 1.51 (0x33) | +| PSP BootLoader | — | — | 00.13.00.70 (AGESA PI 1.0.0.9+) | +| Platform | EPYC 7001+ | EPYC 7002+ | EPYC 7003+ | -| Project | Version/Tag | -| ------------- |:------------------------------------:| -| kernel | >= 6.11 | -| libvirt | >= (FIXME) | -| qemu | >= 10.00 (FIXME) | -| ovmf | >= (FIXME) | +SEV 3.1 (EPYC 9004/8004) uses the same software stack as SEV 3.0 +with additional hardware capabilities. -## Build +## Prepare Host -The following command builds the host and guest Linux kernel, qemu and ovmf bios used for launching SEV-SNP guest. +These requirements apply regardless of whether you use distribution +packages or build from source. Optionally install +[snphost](https://github.com/virtee/snphost) and run `snphost ok` +for a comprehensive host readiness check of all requirements below +(CPU/BIOS/FW). -```` -# git clone https://github.com/AMDESE/AMDSEV.git -# git checkout snp-latest -# ./build.sh --package -# sudo cp kvm.conf /etc/modprobe.d/ -```` -On succesful build, the binaries will be available in `snp-release-`. +### CPU Requirements -## Prepare Host +SEV 3.0 (SNP) requires an AMD EPYC processor with Zen 3 or newer +microarchitecture. See the [SEV Features](#sev-features) table for platform +requirements by SEV version. + +### BIOS Settings Verify that the following BIOS settings are enabled. The setting may vary based on the vendor BIOS. The menu options below are from an AMD BIOS. @@ -95,18 +96,37 @@ Advanced → NBIO Common Options → IOMMU/Security SEV-SNP Support → Enable ``` -Run the following command to install the Linux kernel on the host machine. +### SEV Firmware + +Minimum firmware versions are listed in the +[Upstream Version Reference](#upstream-version-reference) table above. +The latest firmware is available at [developer.amd.com/sev](https://developer.amd.com/sev/) and via +the `linux-firmware` package. + +#### Update Firmware +To manually update firmware (example for EPYC 7003 Series): ``` -# cd snp-release- -# ./install.sh +# wget https://download.amd.com/developer/eula/sev/amd_sev_fam19h_model0xh_1.54.01.zip +# unzip amd_sev_fam19h_model0xh_1.54.01.zip +# sudo mkdir -p /lib/firmware/amd +# sudo cp amd_sev_fam19h_model0xh_1.54.01.sbin /lib/firmware/amd/amd_sev_fam19h_model0xh.sbin ``` -Reboot the machine and choose SNP Host kernel from the grub menu. +Then either reboot the host, or reload the ccp driver to complete the firmware upgrade process: + +``` +sudo rmmod kvm_amd +sudo rmmod ccp +sudo modprobe ccp +sudo modprobe kvm_amd +``` + +#### Verify Run the following commands to verify that SNP is enabled in the host. -```` +``` # dmesg | grep -e SEV-SNP -e RMP SEV-SNP: Segmented RMP base table physical range [0x000000009ba00000 - 0x000000009ba05000] SEV-SNP: Reserving start/end of RMP table on a 2MB boundary [0x0000000055a00000] @@ -124,100 +144,95 @@ Y # cat /sys/module/kvm_amd/parameters/sev_snp Y -```` - -*NOTE: If your SEV-SNP firmware is older than 1.54, see the "Upgrade SEV firmware" section to upgrade the firmware* -## Prepare Guest - -Note: SNP requires OVMF be used as the guest BIOS in order to boot. This implies that the guest must have been initially installed using OVMF so that a UEFI partition is present. - -If you do not already have an installed guest, you can use the launch-qemu.sh script to create it (be sure to append "console=ttyS0,115200n8" to the installation kernel command line in order to perform the installation via the serial console): - -```` -# ./launch-qemu.sh -hda -cdrom -```` - -Boot up a guest (tested with Ubuntu 22.04 and 24.04, but any standard *.deb or *.rpm-based distro should work) and install the guest kernel packages built in the previous step. The guest kernel packages are available in 'snp-release-/linux/guest' directory. - -## Launch SNP Guest - -To launch the SNP guest use the launch-qemu.sh script provided in this repository - -```` -# ./launch-qemu.sh -hda -sev-snp -```` +``` -To launch SNP disabled guest, simply remove the "-sev-snp" from the above command line. +**Notes:** +- EPYC 7003 Series systems require the minimum PSP BootLoader version listed in the [Upstream Version Reference](#upstream-version-reference) table. Attempting to update firmware with an older BootLoader will fail with `SEV: failed to INIT error 0x1, rc -5`. Update the system BIOS if you see this error. +- For EPYC 9004 Series systems, SEV firmware updates are delivered through system BIOS updates. -Once the guest is booted, run the following command inside the guest VM to verify that SNP is enabled: +## Build & Install -```` -$ dmesg | grep -i snp -AMD Memory Encryption Features active: SEV SEV-ES SEV-SNP -```` +You may skip this step if you do not require development patches not yet available upstream. -## Upgrade SEV firmware +The `main` branch targets SEV 3.0 (SNP) development and builds host/guest kernels, QEMU, and OVMF from AMD development trees. -The SEV-SNP support requires firmware version >= 1.51:1 (or 1.33 in hexadecimal). The latest SEV-SNP firmware is available on https://developer.amd.com/sev and via the linux-firmware project. +### Build -The steps below document the firmware upgrade process for the latest SEV-SNP firmware available on https://developer.amd.com/sev at the time this was written. Currently, these steps only apply for Milan systems. A similar procedure can be used for newer firmwares as well: +The following command builds the host and guest Linux kernel, QEMU, and OVMF used for launching an SEV-SNP guest. +Target repositories and branches are determined by [`stable-commits`](stable-commits). ``` -# wget https://download.amd.com/developer/eula/sev/amd_sev_fam19h_model0xh_1.54.01.zip -# unzip amd_sev_fam19h_model0xh_1.54.01.zip -# sudo mkdir -p /lib/firmware/amd -# sudo cp amd_sev_fam19h_model0xh_1.54.01.sbin /lib/firmware/amd/amd_sev_fam19h_model0xh.sbin +# git clone https://github.com/AMDESE/AMDSEV.git +# ./build.sh --package +# sudo cp kvm.conf /etc/modprobe.d/ ``` +On successful build, the binaries will be available in `snp-release-`. -Then either reboot the host, or reload the ccp driver to complete the firmware upgrade process: +NOTE: edk2 build issues are often caused by stale submodule state. +Delete the `ovmf/` directory and rebuild to force a fresh clone. -``` -sudo rmmod kvm_amd -sudo rmmod ccp -sudo modprobe ccp -sudo modprobe kvm_amd -``` +### Install -Current Milan SEV/SNP FW requires a PSP BootLoader version of 00.13.00.70 or greater. Milan AGESA PI 1.0.0.9 included a sufficient PSP BootLoader. Attempting to update to current SEV FW with an older BootLoader will fail. If the following error appears after updating the firmware manually, update the system to the latest available BIOS: +Run the following command to install the Linux kernel on the host machine. ``` -$ sudo dmesg | grep -i sev -[ 4.364896] ccp 0000:47:00.1: SEV: failed to INIT error 0x1, rc -5 +# cd snp-release- +# ./install.sh ``` -For Genoa firmware updates, the system BIOS has to be updated to get the latest sev firmware. - - -# Additional Resources +Reboot the machine and choose SNP Host kernel from the grub menu. -[AMD SEV developer portal](https://developer.amd.com/sev/) +## Launch Guest -[SME/SEV white paper](http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf) +### Prepare Guest -[SEV API Spec](https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf) +Note: SNP requires OVMF be used as the guest BIOS in order to boot. This implies that the guest must have been initially installed using OVMF so that a UEFI partition is present. -[APM Section 15.34](http://support.amd.com/TechDocs/24593.pdf) +If you do not already have an installed guest, you can use the launch-qemu.sh script to create it (be sure to append "console=ttyS0,115200n8" to the installation kernel command line in order to perform the installation via the serial console): -[KVM forum slides](http://www.linux-kvm.org/images/7/74/02x08A-Thomas_Lendacky-AMDs_Virtualizatoin_Memory_Encryption_Technology.pdf) +``` +# ./launch-qemu.sh -hda -cdrom +``` -[KVM forum videos](https://www.youtube.com/watch?v=RcvQ1xN55Ew) +Boot up a guest (tested with Ubuntu 22.04 and 24.04, but any standard \*.deb or \*.rpm-based distro should work) and install the guest kernel packages built in the previous step. The guest kernel packages are available in 'snp-release-/linux/guest' directory. -[Linux kernel](https://elixir.bootlin.com/linux/latest/source/Documentation/virtual/kvm/amd-memory-encryption.rst) +### Launch Guest -[Linux kernel](https://elixir.bootlin.com/linux/latest/source/Documentation/x86/amd-memory-encryption.txt) +To launch the SNP guest use the launch-qemu.sh script provided in this repository -[Libvirt LaunchSecurity tag](https://libvirt.org/formatdomain.html#sev) +``` +# ./launch-qemu.sh -hda -sev-snp +``` -[Libvirt SEV](https://libvirt.org/kbase/launch_security_sev.html) +To launch SNP disabled guest, simply remove the "-sev-snp" from the above command line. -[Libvirt SEV domainCap](https://libvirt.org/formatdomaincaps.html#elementsSEV) +Once the guest is booted, run the following command inside the guest VM to verify that SNP is enabled: -[Qemu doc](https://git.qemu.org/?p=qemu.git;a=blob;f=docs/amd-memory-encryption.txt;h=f483795eaafed8409b1e96806ca743354338c9dc;hb=HEAD) +``` +$ dmesg | grep -i snp +AMD Memory Encryption Features active: SEV SEV-ES SEV-SNP +``` -[guest_memfd (a.k.a. "gmem", or "Unmapped Private Memory")](https://lore.kernel.org/kvm/20230914015531.1419405-1-seanjc@google.com/) + +## Additional Resources + +- [AMD SEV Developer Portal](https://developer.amd.com/sev/) +- [AMD SME/SEV White Paper (PDF)](http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf) +- [SEV Secure Key Management API Specification (PDF)](https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf) +- [AMD64 Architecture Programmer's Manual, Volume 2 -- Section 15.34: SEV (PDF)](http://support.amd.com/TechDocs/24593.pdf) +- [KVM Forum Presentation Slides -- AMD Virtualization Memory Encryption (PDF)](http://www.linux-kvm.org/images/7/74/02x08A-Thomas_Lendacky-AMDs_Virtualizatoin_Memory_Encryption_Technology.pdf) +- [KVM Forum Presentation Video -- AMD Memory Encryption (YouTube)](https://www.youtube.com/watch?v=RcvQ1xN55Ew) +- [Linux Kernel Documentation -- KVM AMD Memory Encryption (RST)](https://elixir.bootlin.com/linux/latest/source/Documentation/virtual/kvm/amd-memory-encryption.rst) +- [Linux Kernel Documentation -- x86 AMD Memory Encryption (TXT)](https://elixir.bootlin.com/linux/latest/source/Documentation/x86/amd-memory-encryption.txt) +- [Libvirt Domain XML -- LaunchSecurity SEV](https://libvirt.org/formatdomain.html#sev) +- [Libvirt Knowledge Base -- Launch Security with SEV](https://libvirt.org/kbase/launch_security_sev.html) +- [Libvirt Domain Capabilities -- SEV](https://libvirt.org/formatdomaincaps.html#elementsSEV) +- [QEMU Documentation -- AMD Memory Encryption](https://git.qemu.org/?p=qemu.git;a=blob;f=docs/amd-memory-encryption.txt;h=f483795eaafed8409b1e96806ca743354338c9dc;hb=HEAD) +- [guest_memfd (a.k.a. "gmem") -- Unmapped Private Memory (lore.kernel.org)](https://lore.kernel.org/kvm/20230914015531.1419405-1-seanjc@google.com/) +- [AMD SEV Enablement Guide (PDF)](https://docs.amd.com/v/u/en-US/58207-using-sev-with-amd-epyc-processors) -# FAQ +## FAQ * How do I know if hypervisor supports SEV feature ? @@ -264,7 +279,7 @@ for additional information. * How to increase SWIOTLB limit ? - When SEV is enabled, all the DMA operations inside the guest are performed on the shared memory. Linux kernel uses SWIOTLB bounce buffer for DMA operations inside SEV guest. A guest panic will occur if kernel runs out of the SWIOTLB pool. Linux kernel default to 64MB SWIOTLB pool. It is recommended to increase the swiotlb pool size to 512MB. The swiotlb pool size can be increased in guest by appending the following in the grub.cfg file + When SEV is enabled, all the DMA operations inside the guest are performed on the shared memory. Linux kernel uses SWIOTLB bounce buffer for DMA operations inside SEV guest. A guest panic will occur if kernel runs out of the SWIOTLB pool. Linux kernel default to 64MB SWIOTLB pool. It is recommended to increase the swiotlb pool size to 512MB. The swiotlb pool size can be increased in guest by appending the following in the grub.cfg file Append the following in /etc/defaults/grub @@ -277,7 +292,7 @@ And regenerate the grub.cfg. * SWIOTLB allocation failure causing kernel panic - SWIOTLB size, when not specifically specified, is automatically calculated based on the amount of guest memory, up to 1GB maximum. However, the guest may not have enough contiguous memory below 4GB to satisify the SWIOTLB allocation requirement, in which case the kernel will panic: + SWIOTLB size, when not specifically specified, is automatically calculated based on the amount of guest memory, up to 1GB maximum. However, the guest may not have enough contiguous memory below 4GB to satisfy the SWIOTLB allocation requirement, in which case the kernel will panic: 
  [    0.004318] software IO TLB: SWIOTLB bounce buffer size adjusted to 965MB
@@ -285,7 +300,7 @@ And regenerate the grub.cfg.
  [    1.015953] Kernel panic - not syncing: Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer
- In this situation, please specify the SWIOTLB size, as shown in [ How to increase SWIOTLB limit](#faq-4), to a value that allows the guest to boot. + In this situation, please specify the SWIOTLB size, as shown in [How to increase SWIOTLB limit](#faq-4), to a value that allows the guest to boot. * virtio-blk device runs out-of-dma-buffer error @@ -295,7 +310,7 @@ And regenerate the grub.cfg. * SEV_INIT fails with error 0x13 - The error 0x13 is a defined as HWERROR_PLATFORM in the SEV specification. The error indicates that memory encryption support is not enabled in the host BIOS. Look for the SMEE setting in your BIOS menu and explicitly enable it. You can verify that SMEE is enabled on your machine by running the below command + The error 0x13 is a defined as HWERROR_PLATFORM in the SEV specification. The error indicates that memory encryption support is not enabled in the host BIOS. Look for the SMEE setting in your BIOS menu and explicitly enable it. You can verify that SMEE is enabled on your machine by running the below command ``` $ sudo modprobe msr $ sudo rdmsr 0xc0010010