HOBS should be self-describing structures

[rminnich]

HOBS should not be binary blobs with no way to determine what is in them.

Protobufs might be overkill but some form of self-describing structure is essential.

[mauricema]

Hi, Ron,
Thank you for providing feedback on this. I think it is a very good topic to have some further discussion.
The reason why we currently proposed HOBs as the data structure is because it is an existing standard defined in UEFI PI, and also relatively straightforward to produce and consume. Current Intel FSP solution used it to communicate the platform information to bootloader. Similarly UEFI payload also used it to provide UEFI services on top of other bootloaders such as Slim Bootloader and coreboot. We want to reuse some of the existing data structures if possible, but also open to any other options.
On the other side, we understand HOB has its own limitations. For example, the 64KB data size limit for each HOB, not very flexible on data structure definitions, etc. As you suggested, something like self-describing structure might be preferable at the interface layer to make data structure easier to exchange and process. However, we are a little bit concerned about the overhead that bootloader and payload might have to deal with. You mentioned Protobufs, and I know it has been widely used in the machine learning infrastructure to describe the different network layers. For bootloaders, I agree with you it might be too heavy. So do you have any good candidate that will fit well for firmware world ? We'd love to hear the recommendation if you have any.

[gdong1]

For the parameters from the bootloader to the payload, it involves parameter format and parameter content.
This issue is about the parameter format. There are several options here:

Hand Off Block (HOB) -- Currently proposed by the spec.

   HOB is used by EDKII BIOS from PEI phase to DXE phase. and HOB is also used by Firmware Support Package (FSP) and Slim Bootloader. HOB uses HobType and HobLength to identify a HOB structure. HOB is defined by existing Platform Initialization specification and flexible to extend. 

Compact/Flat Data Structure

Using a simple header before each actual data could save memory size. For example coreboot uses 'tag' and 'size' for coreboot data structure.

Self-describing Data Structure

   Proposed by this issue, but a little concern on the overhead on both the bootloader and the payload. Need more details for this option.

The current specification selects to use HOB format since it is well defined by an existing specification and used by some firmware components.  but specification is open to other options. Let's get more feedbacks on this topic.

[gdong1]

Based on some feedbacks and investigation, we would like to use Concise Binary Object Representation (CBOR) to replace HOB list as the data format to pass to payload from bootloader. The details on CBOR could be found from
https://intel.github.io/tinycbor/current/ and https://github.com/intel/tinycbor
Comparing with BSON and Protol Buffer/Nanopb, CBOR is simple (e.g. no callback needed to encode/decode) and good enough for bootloader data.
While waiting for more comments on this topic, we would do a POC with CBOR, then will update the spec to finalize it. If there is other comment, please comment on it ASAP.

[ArthurHeymans]

Based on some feedbacks and investigation, we would like to use Concise Binary Object Representation (CBOR) to replace HOB list as the data format to pass to payload from bootloader. The details on CBOR could be found from https://intel.github.io/tinycbor/current/ and https://github.com/intel/tinycbor Comparing with BSON and Protol Buffer/Nanopb, CBOR is simple (e.g. no callback needed to encode/decode) and good enough for bootloader data. While waiting for more comments on this topic, we would do a POC with CBOR, then will update the spec to finalize it. If there is other comment, please comment on it ASAP.

I think what Ron meant was that the interface from and to FSP should be using self describing structures and CBOR seems like a really nice fit for instance to replace FSP UPDs or output HOBs.

To pass data to the payload there is likely no value to reinvent the wheel. Most payloads for instance already support the coreboot tables. Converting all of them to a new format is likely much work with little gain. Also CBOR is likely to be too complicated for some payloads. For instance coreboot does not know how to load x86 Linux kernels. This is done by prepending a assembly written position independent trampoline (146 LOC) that parses the coreboot tables, fills in Linux Zero page and boot to it. I highly doubt that this is possible with more complicated structures or does CBOR have a x86 assembly parser?

[gdong1]

When this issue was filed on Nov 3 2020, only universal payload is available in this website so I don't believe Ron meant the interface for FSP.
HOB or CBMEM is very similar for payload. and I agree payload parser would be very simple to parse them to retrieve data. That's why HOB is initially selected in the universal payload specification (HOB is also defined for PI spec).