Enable C LTO for firmware

[benma]

Compile and link the cross-compiled C firmware objects with GCC LTO. This lets the linker optimize across C object and static-library boundaries while keeping the existing section garbage collection and stack protector.

The ARM toolchain must use the GCC LTO-aware archive utilities; plain arm-none-eabi-ar/ranlib can archive LTO objects without the plugin and produces an invalid tiny image where live objects are not extracted. Select arm-none-eabi-gcc-ar, arm-none-eabi-gcc-nm, and arm-none-eabi-gcc-ranlib in the ARM toolchain file.

With LTO, bootloader stack-protector references are generated after the normal symbol liveness pass. Mark the local __stack_chk_fail and __stack_chk_guard definitions as used/externally visible so they are kept for those late-generated references. __stack_chk_fail is also marked no_stack_protector to avoid protecting the failure handler itself.

Measured with make -j firmware.
Saves 12040 bytes of firmware ROM.

Verification:

• make -j firmware
• make -j bootloader

[NickeZ]

Did you try the bootloader? I remember having to fix it because we have assembly instructions that don't name the variables they use.

https://github.com/NickeZ/bitbox02-firmware/commits/enable-lto/

[NickeZ]

Let's only do it for firmware, not bootloader nor factory setup. And you can remove -ffucntion-sections and -fdata-sections and --gc-sections, because they only apply without LTO.

[benma]

Let's only do it for firmware, not bootloader nor factory setup.

Added a change to limit to firmware, and also to exclude asf4 drivers etc. Having the drivers do lto in firmware but not in bootloader/factorysetup was too involved, and we get most of the gains in the firmware already without doing lto on the drivers.

And you can remove -ffucntion-sections and -fdata-sections and --gc-sections, because they only apply without LTO.

It seems they still apply, codex:

---

No, that is too broad.

--gc-sections still applies with LTO. LTO may remove dead code earlier at IR level, so -ffunction-sections / -fdata-sections are often less important for LTO-compiled objects, but the linker still performs
section GC on the final linked image.

In our scoped setup this matters even more because firmware still links non-LTO code:

• ASF4 drivers remain non-LTO.
• samd51a-ds and embedded-swd remain non-LTO.
• libc/libgcc/vendor blobs/Rust archives may still benefit from section GC.

For those inputs, -ffunction-sections -fdata-sections plus --gc-sections is still the normal mechanism that lets the linker drop unused functions/data at fine granularity.

So the accurate version is: for code fully optimized by LTO, these flags may be partially redundant; for mixed LTO/non-LTO firmware, they are still useful and should not be removed without measuring.

[cedwies]

I tried verifying codex, seems correct. After checking GCC -flto doc, GCC section flags and GNU ld --gc-sections, nothing says "disabled when LTO is enabled". GCC docs actually explicitly say LTO links can mix objects with LTO bytecode and normal final object code.

A current concern of me: cryptoauthlib and optiga get compiled with LTO notes, while firmware-setup gets compiled without LTO notes.

But in src/CMakeLists.txt (line 495) every target links the same libraries. If we want firmware-setup to not do LTO link (as comment in line 487 suggests), shouldn't we add -fno-lto for factory setup?

Like:

if(firmware STREQUAL "factory-setup")
  target_compile_options(${elf} PRIVATE -fno-lto)
  target_link_libraries(${elf} PRIVATE -fno-lto)
else()
  target_compile_options(${elf} PRIVATE -flto -ffat-lto-objects)
  target_link_libraries(${elf} PRIVATE -flto)
endif()

[benma]

I tried verifying codex, seems correct. After checking GCC -flto doc, GCC section flags and GNU ld --gc-sections, nothing says "disabled when LTO is enabled". GCC docs actually explicitly say LTO links can mix objects with LTO bytecode and normal final object code.

A current concern of me: cryptoauthlib and optiga get compiled with LTO notes, while firmware-setup gets compiled without LTO notes.

But in src/CMakeLists.txt (line 495) every target links the same libraries. If we want firmware-setup to not do LTO link (as comment in line 487 suggests), shouldn't we add -fno-lto for factory setup?

Like:

if(firmware STREQUAL "factory-setup")
  target_compile_options(${elf} PRIVATE -fno-lto)
  target_link_libraries(${elf} PRIVATE -fno-lto)
else()
  target_compile_options(${elf} PRIVATE -flto -ffat-lto-objects)
  target_link_libraries(${elf} PRIVATE -flto)
endif()

-flto was not enabled, so one does not need to disable it explicitly. I disabled it though for consistency with the compile option and for clarity.

[cedwies]

-flto was not enabled, so one does not need to disable it explicitly. I disabled it though for consistency with the compile option and for clarity.

My main concern was the following part of the GCC docs:

GCC automatically performs link-time optimization if any of the objects involved were compiled with the -flto command-line option. You can always override the automatic decision to do link-time optimization by passing -fno-lto to the link command.

Thanks for specifying -fno-lto :)

[cedwies]

tACK

saves:

firmware: 8.856 bytes
firmware-btc: 9.008 bytes
factory-setup: 0 bytes