Skip to content

bootc generic-iso using ostree rootfs#2414

Open
brlane-rht wants to merge 12 commits into
osbuild:mainfrom
brlane-rht:HMS-10627-bootc-iso
Open

bootc generic-iso using ostree rootfs#2414
brlane-rht wants to merge 12 commits into
osbuild:mainfrom
brlane-rht:HMS-10627-bootc-iso

Conversation

@brlane-rht

@brlane-rht brlane-rht commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

This changes the bootc generic-iso so that the rootfs is the same as the bootc PXE image type.

Note that this currently doesn't actually work right. Something is wrong with how dracut or ostree is setting up the booted filesystem, it doesn't mount things correctly so on boot a number of services fail:
FIXED -- ends up dmsquash-live treats LiveOS/rootfs.img differently, switched to using LiveOS/squashfs.img and now it boots the same as bootc PXE tar.

This also renames it to generic-iso, and it drops the support for embedding a container payload since this isn't an installer iso and is not a raw filesystem (I think the user can do the same thing when setting up their bootc container, but have not explored that).

@brlane-rht brlane-rht requested a review from supakeen June 11, 2026 23:27
@brlane-rht brlane-rht force-pushed the HMS-10627-bootc-iso branch 2 times, most recently from 4f1b580 to 5a6c3bd Compare June 12, 2026 19:12
@supakeen

supakeen commented Jun 13, 2026

Copy link
Copy Markdown
Member

Will need some careful review; at least the name change will immediately break things for users and the non-embedding of containers (while I agree this can probably be done inside the bootable container itself and it's likely better there) is the same.

FIXED -- ends up dmsquash-live treats LiveOS/rootfs.img differently, switched to using LiveOS/squashfs.img and now it boots the same as bootc PXE tar.

In what way? I'm interested what the behavioral difference is.

I'm also tagging in @ondrejbudai since he's done a bunch with the ISOs for bootc too 🙂.

@supakeen supakeen requested a review from ondrejbudai June 13, 2026 15:39
@brlane-rht

Copy link
Copy Markdown
Contributor Author

Will need some careful review; at least the name change will immediately break things for users and the non-embedding of containers

Not sure what to do about that.

FIXED -- ends up dmsquash-live treats LiveOS/rootfs.img differently, switched to using LiveOS/squashfs.img and now it boots the same as bootc PXE tar.

In what way? I'm interested what the behavioral difference is.

When it is named squashfs.img it uses overlayfs and when it is rootfs.img it uses device-mapper:

https://github.com/dracut-ng/dracut/blob/main/modules.d/70dmsquash-live/dmsquash-live-root.sh#L330

is the section. With device-mapper the ostree root setup doesn't properly mount things and then a whole mess of services don't start (it actually still boots, which is surprising).

@brlane-rht brlane-rht force-pushed the HMS-10627-bootc-iso branch from d2983aa to 56545de Compare June 16, 2026 23:30
@brlane-rht brlane-rht force-pushed the HMS-10627-bootc-iso branch from 56545de to cb45918 Compare June 25, 2026 21:18
@brlane-rht brlane-rht marked this pull request as ready for review June 25, 2026 21:25
@brlane-rht brlane-rht requested review from a team, achilleas-k and thozza as code owners June 25, 2026 21:25
@brlane-rht

Copy link
Copy Markdown
Contributor Author

I've fixed some issues I was having with fstab mounts (have to pass fstab=no to the kernel/systemd), and added a README that is read from the file storage and written to the ISO, similar to the one in the PXE tar.

I've also experimented with running podman pull ... in the Containerfile and my simple tests work inside the booted system so you can still embed other containers.

@schutzbot

Copy link
Copy Markdown
Collaborator

This PR changes the image-builder API or behaviour causing integration failures with osbuild-composer. The next update of the image-builder dependency in osbuild-composer will need work to adapt to these changes.

This is simply a notice. It will not block this PR from being merged.

@brlane-rht

Copy link
Copy Markdown
Contributor Author

Not sure what's up with the manifest check. Running ./tools/gen-manifest-checksums.sh works fine locally (and I've updated the checksums).

@lzap

lzap commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Not sure what's up with the manifest check. Running ./tools/gen-manifest-checksums.sh works fine locally (and I've updated the checksums).

There are some compilation errors, thus manifests are not generated correctly?!

Error: pkg/image/container_based_iso.go:106:56: cannot use osTreePipeline (variable of type *"github.com/osbuild/image-builder/pkg/manifest".OSFromContainer) as "github.com/osbuild/image-builder/pkg/manifest".TreePipeline value in argument to manifest.NewISOTree: *"github.com/osbuild/image-builder/pkg/manifest".OSFromContainer does not implement "github.com/osbuild/image-builder/pkg/manifest".TreePipeline (missing method Platform)
Error: pkg/image/container_based_iso.go:106:72: cannot use bootTreePipeline (variable of type *"github.com/osbuild/image-builder/pkg/manifest".EFIBootTree) as []"github.com/osbuild/image-builder/pkg/manifest".ISOBootloader value in argument to manifest.NewISOTree
Error: pkg/image/container_based_iso.go:111:18: isoTreePipeline.RootfsCompression undefined (type *"github.com/osbuild/image-builder/pkg/manifest".ISOTree has no field or method RootfsCompression)
Error: pkg/image/container_based_iso.go:115:18: isoTreePipeline.KernelOpts undefined (type *"github.com/osbuild/image-builder/pkg/manifest".ISOTree has no field or method KernelOpts

I think I know what is happening: unit and build tests do pass because they only test the tip of the branch. However, manifest generation is performed on every single commit individually. One of your commits does not compile correctly.

@brlane-rht

brlane-rht commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Ah, right, I'd forgotten it tried on every commit. Problem is I don't want to make every commit compile, they're easier to read as distinct changes that then converge.
Ends up compile failures are skipped, this was failing because the checksums needed to be updated :)

@brlane-rht brlane-rht force-pushed the HMS-10627-bootc-iso branch from cb45918 to 1301d1e Compare June 26, 2026 15:44
Comment thread data/files/iso/generic/README Outdated
Comment thread data/distrodefs/bootc-generic/imagetypes.yaml Outdated
Comment thread data/files/iso/generic/README Outdated

@supakeen supakeen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor bits; currently building this PR and trying it out in a few different scenarios.

@supakeen

supakeen commented Jun 29, 2026

Copy link
Copy Markdown
Member

Tried some bits mostly seems to work OK aside from bootc itself being really unhappy about it all.

Poking @jbtrystram / @joelcapitao to take a look as well (RPMs are available in this PR, so is the docs) as they initially requested this feature.

Would also still really like @ondrejbudai's input on this since it changes the contract for ISOs a bit and perhaps we need to go onto composefs if that's at all possible on ISOs (followup).

Remove the rootfs compression code, it is now created by BootcRootFS and
is included in the treePipeline.

Use the ISOBootloader list and common bootloader code from
iso_bootloaders. This allows the iso to support other arches and
bootloader configurations.

Also adds optional support for setting the ostree= value in the grub.cfg
file when SetOSTREE is true. KernelOpts must already contain
'ostree=@ostree@' if this option is used. It will add the
org.osbuild.ostree.grub stage which mounts the rootfs.img, examines the
ostree filesystem to extract the boot path and then substitute it for
@ostree@ in the grub.cfg file.

Related: HMS-10627
dmsquash-live-root treats squashfs.img and rootfs.img differently, it
uses the same codepath as http root for squashfs.img which is where the
root filesystem detection (looking for /usr or /ostree) is triggered.

If rootfs.img is used dracut sets up the root using device-mapper
instead of overlay, and this does not work with ostree.

Related: HMS-10627
The partition is needed for installing bootc to a partitioned disk image,
and the erofs settings are used for the rootfs of the iso.

Related: HMS-10627
This changes to using the bootc container directly instead of the
deployed filesystem. It builds on top of BootcRootFS (also used by the
bootc PXE tar image type). It uses the iso and installer customizations
from the bootc distro YAML for setting up the rootfs compression and
optional kernel arguments.

It also uses the common bootloaders functions which should make it
easier to eventually add support for other platforms.

Related: HMS-10627
This starts by making sure the required modules have been included in
the container's initrd. It also hooks up the blueprint customizations,
and uses the YAML for the iso configuration (erofs or squashfs rootfs
type).

Related: HMS-10627
It's redundant and makes the ibcli image names longer than they need to
be. Keep bootc-generic-iso as an alias.

Related: HMS-10627
This adds generation of the manifest checksums.

Related: HMS-10627
Set the menu timeout if it is non-zero. 0 means use the stage default,
so it is always safe to set it.

Related: HMS-10627
@brlane-rht brlane-rht force-pushed the HMS-10627-bootc-iso branch from 1301d1e to 26ffe9e Compare June 30, 2026 21:48

# Include the dmsquash-live module in the initramfs
RUN cat <<EOF > /usr/lib/dracut/dracut.conf.d/40-iso.conf
compress="xz"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now default to zstd. Why would we use xz here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because that's what the boot.iso uses.

@supakeen supakeen Jul 1, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea the ISOs still tend to use xz/lzma for their initramfs compression as they mostly care about size and (usually) only get built once, same reason most of the disk images in Koji get xz compressed and not zstd; only the size matters since they're only built once.

We could switch that but I think that would be a separate thing to consider, it would make some form of sense for containers when iterating on an ISO 🙂

RUN dnf -y install grub2-efi-*-cdboot xorriso isomd5sum dracut-live erofs-utils shim && dnf clean all
RUN mkdir /boot/efi && cp -r /usr/lib/efi/shim/*/EFI /boot/efi && cp -r /usr/lib/efi/grub2/*/EFI/* /boot/efi/EFI/

# Override using composefs for ostree (it is incompatible with the erofs rootfs)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds weird. Not sure why that would not work. I think we do that with the CoreOS LiveISO.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be related to how dmsquash-live sets up the root, it does that first, then the ostree module takes over. Without this it fails to mount.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants