lib: Add experimental unified storage support for install #1601

jmarrero · 2025-09-09T17:28:09Z

lib: Add experimental unified storage support for install

Add an experimental --experimental-unified-storage flag to bootc install
that uses bootc's container storage (/usr/lib/bootc/storage) to pull
images first, then imports from there. This is the same approach used
for logically bound images (LBIs).

Background:
The unified storage approach allows bootc to share container images with
podman's storage, reducing disk space and enabling further integration
with the Podman ecosystem.

Changes:

Add --experimental-unified-storage CLI flag to install subcommands
Add sysroot_path parameter to prepare_for_pull_unified() and pull_unified()
to handle the different mount points during install vs upgrade/switch
Handle localhost images specially by exporting from ostree to container
storage first (these can't be pulled from a registry)
Skip pull in prepare_for_pull_unified() if image already exists in
bootc storage
Add TMT test for install with unified storage flag
Add TMT test for switching to unified storage on running system

The sysroot_path fix is needed because during install the target disk
is mounted at a specific path (e.g., /var/mnt), not /sysroot. Skopeo
needs the actual filesystem path to find the bootc storage.

Relates: #20

Assisted-by: Claude Code (Sonnet 4.5 & Opus 4.5)

[core@cosa-devsh ~]$ sudo podman --storage-opt=additionalimagestore=/usr/lib/bootc/storage images
REPOSITORY                       TAG         IMAGE ID      CREATED       SIZE
quay.io/jmarrero_rh/soft-reboot  1           97f84fcb062e  25 hours ago  1.83 GB

cgwalters

Thanks for working on this!

cgwalters · 2025-09-09T18:20:03Z

crates/lib/src/cli.rs

+    /// Use podman/skopeo to pull image to additionalimagestore, then read from container storage.
+    /// This provides a unified approach that leverages existing container tooling.
+    #[clap(long)]
+    pub(crate) unified: bool,


I don't think this should be a CLI flag that operates just once; it should be something like bootc image --set-unified or so and act persistently.

Also we should support setting this at install time so that it happens from the very start.

(hmm my bad I thought we discussed this but I failed to update the issue #20 or something maybe?)

Ahh man yeah, we talked about saving the config on the origin file IIRC, I forgot.

cgwalters · 2025-09-09T18:20:41Z

crates/lib/src/podman.rs

+    use bootc_utils::CommandRunExt;
+
+    // Use podman pull with additionalimagestore pointing to bootc storage
+    let bootc_storage_path = "/usr/lib/bootc/storage";


Use podstorage.rs instead please

To elaborate on this, an important aspect here is currently the GC of the bootc/storage instance is rooted in the set of LBIs. That will need to be extended to include the host image.

Another big task related to this - we'll eventually really want to use the podman HTTP APIs so we can properly monitor progress. Right now we are very crudely relying on doing this synchronously to the existing tty and reusing the podman CLI tty output. But that can't really work if we ever need to render our own progress APIs.

https://lib.rs/crates/bollard might be a good choice for this?

crates/lib/src/deploy.rs

crates/lib/src/cli.rs

jlebon · 2025-10-07T13:31:09Z

Do we get reflinks when importing from the alternate storage into the ostree repo?

cgwalters · 2025-10-07T14:36:03Z

Do we get reflinks when importing from the alternate storage into the ostree repo?

#20 links to containers/container-libs#144

jlebon · 2025-10-07T15:42:36Z

Do we get reflinks when importing from the alternate storage into the ostree repo?

#20 links to containers/container-libs#144

Hmm, that's about copying between two container storages, but I was asking about the ostree import. But #20 does say:

From there, we do a reflink/hardlink copy into the ostree store (i.e. This would duplicate metadata, but not data on disk. Reflinks should always work.

Which kinda answers the question.

Basically without reflinks into the ostree storage, IMO this makes it a lot less obvious that this new option is objectively better and it instead becomes a discussion of trade-offs.

jmarrero · 2025-12-02T20:06:50Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces experimental support for unified storage, allowing bootc to pull images into its own container storage before importing them. This is a significant feature that touches installation, upgrade, and switch paths. The changes include new CLI options, a new image set-unified command, and the core logic for the unified pull mechanism.

The implementation is comprehensive and includes new integration tests. My review has identified a few areas for improvement:

A bug in the image reference formatting for unified storage checks during installation.
Inconsistent error handling when checking for images in the unified store.
A potential panic due to an .unwrap() call on a Result.

Addressing these points will improve the robustness and maintainability of this new feature.

crates/lib/src/image.rs

crates/lib/src/install.rs

crates/lib/src/deploy.rs

cgwalters · 2025-12-02T20:51:58Z

On the progress status side maybe we can use https://lib.rs/crates/bollard to talk to podman and I think it should help us get data out of the events?

crates/lib/src/image.rs

gemini-code-assist

Code Review

This pull request introduces experimental unified storage support for bootc install and related commands. The changes are extensive and well-thought-out, adding a new --experimental-unified-storage flag and the bootc image set-unified command. The implementation correctly handles different scenarios, such as install versus upgrade, and the special case of images originating from containers-storage. The addition of comprehensive integration and TMT tests is commendable. I've identified one high-severity logical issue in the implementation of bootc image set-unified where it could operate on a staged image instead of the booted one. I've also noted a correct and important bug fix in the image pruning logic. Overall, this is a solid contribution that adds a valuable feature.

crates/lib/src/image.rs

cgwalters

Thanks! A first pass review here, this is looking quite plausible!

crates/lib/src/cli.rs

cgwalters · 2025-12-10T19:18:03Z

crates/lib/src/deploy.rs

+pub(crate) async fn should_use_unified_storage(
+    store: &Storage,
+    imgref: &ImageReference,
+    explicit_flag: Option<bool>,


This seems strange to me; nothing AFAICS passes Some(false). And if Some(true) is passed the caller will just get back true. So why even call this?

crates/lib/src/image.rs

crates/lib/src/utils.rs

cgwalters · 2025-12-10T20:07:41Z

crates/lib/src/image.rs

+        // Check if the image already exists in the default containers-storage.
+        // This can happen if someone did a local build (e.g., podman build) and
+        // we don't want to overwrite it with an export from ostree.
+        ensure_floating_c_storage_initialized();


I think we can put this global init somewhere farther up?

crates/lib/src/image.rs

crates/system-reinstall-bootc/src/main.rs

crates/lib/src/image.rs

jmarrero · 2025-12-11T03:45:06Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces experimental support for unified storage in bootc install, which allows sharing container images with podman's storage to reduce disk usage and enable offline installations. The changes include a new --experimental-unified-storage flag, modifications to upgrade and switch to auto-detect and use unified storage, and special handling for localhost images. New integration and TMT tests are also added to cover these new code paths.

My review focuses on correctness and maintainability. I've identified a couple of areas for improvement:

An inconsistency in the image pulling strategy for unified storage, which could lead to unnecessary network operations.
A small opportunity to simplify code by removing a redundant check, improving readability and reflecting the guarantees provided by other functions.

Overall, the changes are well-structured and the addition of tests is great. Addressing these points will improve the robustness and maintainability of the new feature.

crates/lib/src/deploy.rs

crates/lib/src/image.rs

cgwalters · 2025-12-11T19:45:24Z

Hmm the failure of test-install is weird; I'm not seeing the job fail in e.g. #1846 or other PRs.

crates/lib/src/deploy.rs

crates/lib/src/install.rs

tmt/tests/booted/test-switch-to-unified.nu

crates/lib/src/install.rs

cgwalters

Marking as requested changes per incompatible change to install flow

jmarrero · 2025-12-12T12:50:32Z

Failures on rawhide are the same we saw in another PR yesterday: #1849

cgwalters

Cool, just two nits

docs/src/man/bootc-install-to-disk.8.md

crates/lib/src/spec.rs

Add an experimental --experimental-unified-storage flag to bootc install that uses bootc's container storage (/usr/lib/bootc/storage) to pull images first, then imports from there. This is the same approach used for logically bound images (LBIs). Background: The unified storage approach allows bootc to share container images with podman's storage, reducing disk space and enabling better integration with podman. Changes: - Add --experimental-unified-storage CLI flag to install subcommands - Add sysroot_path parameter to prepare_for_pull_unified() and pull_unified() to handle the different mount points during install vs upgrade/switch - Handle container-storage transport - Skip pull in prepare_for_pull_unified() if image already exists in bootc storage - Add TMT test for install with unified storage flag - Add TMT test for switching to unified storage on running system The sysroot_path fix is needed because during install the target disk is mounted at a specific path (e.g., /var/mnt), not /sysroot. Skopeo needs the actual filesystem path to find the bootc storage. Relates: bootc-dev#20 Assisted-by: Claude Code (Sonnet 4.5) Signed-off-by: Joseph Marrero Corchado <jmarrero@redhat.com>

Refactor the unified storage code to get the sysroot path from the Storage struct instead of passing it as a parameter through multiple function calls. Why this change is needed: The previous implementation passed sysroot_path as an Option parameter to prepare_for_pull_unified() and pull_unified(). This was awkward because: - Callers had to know whether to pass None (for booted systems) or Some(path) (for install scenarios) - The path is inherently tied to the Storage instance's lifecycle - It required threading parameters through multiple layers This follows the existing pattern where Storage already encapsulates storage-related state, and addresses review feedback to "get this stuff from the Storage". Assisted-by: Claude Code (Opus 4.5) Signed-off-by: Joseph Marrero Corchado <jmarrero@redhat.com>

cgwalters · 2025-12-14T14:41:50Z

crates/lib/src/deploy.rs

-    let sysroot_base = sysroot_path
-        .map(|p| p.to_string())
-        .unwrap_or_else(|| "/sysroot".to_string());
+    // Use the physical path to bootc storage from the Storage struct


This is OK but I've been trying to ensure that bootc uses file descriptors and not file paths as much as possible. The problem here is the case where we have a fd but need to pass that state down to a subprocess which expects a file path.

podstorage.rs goes to quite some effort to do this for the primary storage, see new_podman_cmd_in - I think we could use a similar technique here.

bootc-bot bot requested a review from cgwalters September 9, 2025 17:28

cgwalters reviewed Sep 9, 2025

View reviewed changes

jmarrero force-pushed the unified-storage-wip branch from 1baf1e8 to 9c3ef81 Compare October 9, 2025 18:15

jmarrero changed the title ~~WIP: Add --unified to use our container-storage for host images~~ WIP: Use our container-storage for host images if the refspec exists on it Oct 9, 2025

jmarrero changed the title ~~WIP: Use our container-storage for host images if the refspec exists on it~~ WIP: Use our container-storage for host images if the refspec exists Oct 9, 2025

jmarrero force-pushed the unified-storage-wip branch 2 times, most recently from e891e6c to be75055 Compare October 15, 2025 17:57

github-actions bot added the area/install Issues related to `bootc install` label Oct 15, 2025

jmarrero force-pushed the unified-storage-wip branch from 291fbd7 to fe10204 Compare November 11, 2025 19:53

jmarrero force-pushed the unified-storage-wip branch 3 times, most recently from 06be8fb to b801a17 Compare November 21, 2025 20:50

github-actions bot added the area/system-reinstall-bootc Issues related to system-reinstall-botoc label Nov 21, 2025

jmarrero force-pushed the unified-storage-wip branch from b801a17 to 68b1ba6 Compare November 21, 2025 20:56

jmarrero force-pushed the unified-storage-wip branch 3 times, most recently from ee300e3 to 50c4f70 Compare December 2, 2025 19:28

jmarrero changed the title ~~WIP: Use our container-storage for host images if the refspec exists~~ lib: Add experimental unified storage support for install Dec 2, 2025

gemini-code-assist bot reviewed Dec 2, 2025

View reviewed changes

crates/lib/src/image.rs Outdated Show resolved Hide resolved

crates/lib/src/install.rs Outdated Show resolved Hide resolved

crates/lib/src/deploy.rs Outdated Show resolved Hide resolved

cgwalters reviewed Dec 2, 2025

View reviewed changes

crates/lib/src/image.rs Outdated Show resolved Hide resolved

cgwalters reviewed Dec 4, 2025

View reviewed changes

crates/lib/src/image.rs Outdated Show resolved Hide resolved

jmarrero force-pushed the unified-storage-wip branch 4 times, most recently from 3803e4c to bb9883e Compare December 10, 2025 15:41

gemini-code-assist bot reviewed Dec 10, 2025

View reviewed changes

crates/lib/src/image.rs Outdated Show resolved Hide resolved

cgwalters requested changes Dec 10, 2025

View reviewed changes

jmarrero force-pushed the unified-storage-wip branch from 7651032 to ddbba51 Compare December 11, 2025 03:41

gemini-code-assist bot reviewed Dec 11, 2025

View reviewed changes

crates/lib/src/deploy.rs Show resolved Hide resolved

crates/lib/src/image.rs Show resolved Hide resolved

jmarrero force-pushed the unified-storage-wip branch 3 times, most recently from ba8e4ef to 7f11db8 Compare December 11, 2025 10:08

cgwalters mentioned this pull request Dec 11, 2025

Store container manifest #1830

Merged

cgwalters approved these changes Dec 11, 2025

View reviewed changes

cgwalters reviewed Dec 11, 2025

View reviewed changes

crates/lib/src/deploy.rs Outdated Show resolved Hide resolved

cgwalters reviewed Dec 11, 2025

View reviewed changes

crates/lib/src/install.rs Outdated Show resolved Hide resolved

crates/lib/src/install.rs Outdated Show resolved Hide resolved

cgwalters reviewed Dec 11, 2025

View reviewed changes

tmt/tests/booted/test-switch-to-unified.nu Show resolved Hide resolved

crates/lib/src/install.rs Outdated Show resolved Hide resolved

cgwalters requested changes Dec 11, 2025

View reviewed changes

jmarrero force-pushed the unified-storage-wip branch from 7f11db8 to a3aea8e Compare December 12, 2025 03:25

github-actions bot added the area/documentation Updates to the documentation label Dec 12, 2025

jmarrero force-pushed the unified-storage-wip branch 3 times, most recently from 62d48ab to 52f5e25 Compare December 12, 2025 04:53

jmarrero mentioned this pull request Dec 12, 2025

rawhide tests "failing with error: Status: loading sysroot: Unexpected state: /run/ostree-booted found, but no /boot/loader directory" #1849

Open

jmarrero requested a review from cgwalters December 12, 2025 12:48

cgwalters approved these changes Dec 12, 2025

View reviewed changes

docs/src/man/bootc-install-to-disk.8.md Outdated Show resolved Hide resolved

crates/lib/src/spec.rs Outdated Show resolved Hide resolved

cgwalters mentioned this pull request Dec 12, 2025

Fix signature handling with additionalimagestore bootc-dev/bcvk#166

Merged

jmarrero added 2 commits December 13, 2025 12:44

jmarrero force-pushed the unified-storage-wip branch from 52f5e25 to 39ace6a Compare December 13, 2025 18:03

cgwalters approved these changes Dec 14, 2025

View reviewed changes

cgwalters merged commit 68d5cea into bootc-dev:main Dec 14, 2025
44 of 46 checks passed

lib: Add experimental unified storage support for install #1601

lib: Add experimental unified storage support for install #1601

Uh oh!

Conversation

jmarrero commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jlebon commented Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cgwalters commented Oct 7, 2025

Uh oh!

jlebon commented Oct 7, 2025

Uh oh!

jmarrero commented Dec 2, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cgwalters commented Dec 2, 2025

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jmarrero commented Dec 11, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

cgwalters commented Dec 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cgwalters left a comment

jmarrero commented Sep 9, 2025 •

edited

Loading

jlebon commented Oct 7, 2025 •

edited

Loading