AEM for UEFI v0.5.0-rc2 doesn't boot on Intel

[krystian-hebel]

Affected component(s) or functionality
GRUB/Xen

Brief summary
Attempting to boot fails with ACM error 0xC00008F1 - class 0xF, major 2, minor 0. Intel stopped including error code description with its ACM package, but according to https://kib.kiev.ua/x86docs/Intel/TXT/315168-013.pdf this would be Invalid MLE GUID.

Version
v0.5.0-rc2

To Reproduce

1. Install AEM packages under Qubes OS on VP4670.
2. Do the AEM installation itself.
3. Try to boot.

Expected behavior
System boots.

Actual behavior
Platform reboots, reporting TXT error on next attempt.

Screenshots

Additional context
This may be caused by unaccounted shift between base and load address. It was used to update the entry point, but not os_sinit_data->mle_hdr_base - it still assumes that the image begins 2 MiB further than it does.

[SergiiDmytruk]

Adding 2 MiB to the value of os_sinit_data->mle_hdr_base didn't do anything and I would be surprised if it did because the code worked on OptiPlex with legacy and UEFI. There was some unclear issue with SeaBIOS on VP4670, but I think GRUB didn't get to starting DRTM in that case, weird that different issues happen on the same model.

[SergiiDmytruk]

Reverting TrenchBoot/grub@1a75073 ("loader/slaunch/x86_efi_linux.c: compute DLME size from MLE header for non-Linux") also did nothing (tried it just in case).

Intel stopped including error code description with its ACM package

The hell is wrong with Intel... At least in 630744_003.zip (which is from 2023 while that documentation is from 2016) class 0xF, major 2 means ERR_BIOS_PROPERTIES_NO_BIOS_MODULES. Maybe something is wrong with the firmware? GRUB doesn't load SINIT because the one in the firmware is up-to-date. There is also some BIOS ACM involved, maybe that one is missing/inappropriate.

[SergiiDmytruk]

TBOOT works, so not a firmware issue. Trying to boot Xen with MultiBoot2 results in the same TXT.ERRORCODE and v0.4.1 does get to at least start booting Xen, so something is broken in TB GRUB for both MB2 and EFI.

[SergiiDmytruk]

v0.4.1 rebased onto the same GRUB commit as v0.5.0 RCs seems to start DRTM. Comparing that to v0.4.1 didn't suggest what might be the cause, so I decided to check that it's not some upstream changes and rebased part of v0.5.0 onto 2.06 (dropped commits with conflicts): MB2 boot fails just like with v0.5.0 with 0xc0001c21 error (it's different from EFI boot, but likely to be related, MB2 shouldn't really fail and there was no EFI in v0.4.1 to compare good/bad code). 0xc0001c21 is supposedly about MTRR mask, but that part didn't really change.

[krystian-hebel]

but that part didn't really change.

You mean in GRUB or in coreboot?

There is MAXPHYADDR bit in MLE header capabilities in Xen, I recall that we have changed it at some point, but I don't remember why, when, which way and on which branch. Maybe it was lost during one of many rebases.

[SergiiDmytruk]

but that part didn't really change.

You mean in GRUB or in coreboot?

In GRUB.

There is MAXPHYADDR bit in MLE header capabilities in Xen, I recall that we have changed it at some point, but I don't remember why, when, which way and on which branch. Maybe it was lost during one of many rebases.

Well, if GRUB v0.4.1 can initiate TXT, then I guess Xen is fine. That 8th bit isn't set in Xen's header, TXT SDG says that SINIT always sets it to "1" and GRUB seems to always treat it as "0", but if there is less then 64 GiB of RAM, this shouldn't even matter if I get its meaning right.

[SergiiDmytruk]

Looks like 0xC00008F1 does actually mean invalid UUID, although the reason for that is different: the difference between file and virtual offsets. Now I'm not sure what worked on OptiPlex and how, although maybe its ACM and BIOS handle things differently.

Making sure offset to the header is correct results in a different error: 0xc00014f1, which according to the linked PDF means "Invalid MLE size" in MLE header.

Adding mle_hdr->mle_end = slparams->mle_size; results in yet another error: 0xc0001c21 (the one about MTRRs). Multiboot2 path didn't help in understanding what causes this one.

[SergiiDmytruk]

There is MAXPHYADDR bit in MLE header capabilities in Xen, I recall that we have changed it at some point, but I don't remember why, when, which way and on which branch. Maybe it was lost during one of many rebases.

Seems relevant after all. Handling masks by taking MAXPHYADDR value from CPUID turns 0xc0001c21 into 0x80000000 ("legacy shutdown" which seems to mean a reboot in MLE). Looks like TB GRUB was never tested on ACMs that require this.

Xen does resume execution, but then fails on memory accesses. My memory must has failed me and I confused UEFI on OptiPlex with UEFI on SuperMicro, I did try it on OptiPlex, but it couldn't have worked there.

There is an issue with high PMR, which GRUB sets as:

vtd_pmr_lo_base: 0x0 vtd_pmr_lo_size:0x99400000 vtd_pmr_hi_base: 0x100200000 vtd_pmr_hi_size: 0x35e600000

but Xen's target address is 0xffff82d040000000 which is beyond RAM anyway, so I guess the check shouldn't be performed on EFI. Commenting it out results in a hang somewhere in map_slaunch_mem_regions() called from __start_xen().

[SergiiDmytruk]

At last, it boots and even works (openQA tests pass). The boot is somewhat slower than without DRTM (likely related to TrenchBoot/xen#16, but TrenchBoot/xen#17 is included, so the delay is not too long, maybe 10 seconds and then slower than normal boot for a while). Cleaning up changes to send PRs after making openQA do some runs.