oci: Add OCI-native manifest/config storage and image management#216
oci: Add OCI-native manifest/config storage and image management#216jeckersb merged 1 commit intocomposefs:mainfrom
Conversation
cgwalters
force-pushed
the
oci-native-layer
branch
2 times, most recently
from
0ece424 to
10bb53f
Compare
|
Something I want to highlight in this PR that I think is quite cool and important is that this project is now more directly/obviously supporting non-container OCI types i.e. OCI artifacts. Often when people say "OCI" they think of course about the classic tar layer container images, but that's not the only use case for OCI. And it makes an equal amount of sense to store e.g. AI model training weights or disk images in a composefs repo as it does OCI. So again "OCI" != "OCI container images" in general. |
|
I looked briefly. This seems like pretty cool/useful stuff... There's a bit of tension here with #218. What's the eventual plan? Import via containers-image-proxy-rs, or skip skopeo and use the cstor-rs stuff (via jsonrpc) directly? |
The two PRs are 100% compatible AFAICS and complementary. This is just extending things so we store the manifest too etc., it's orthogonal to how we fetch the data.
I think for bootc we will likely always fetch via podman/skopeo for the near term future by default because of the "containers/image" part of container-libs (talking to registries, registries.conf auth file handling etc.) into a containers-storage, and then reflink from there into composefs (and then reflink from there into ostree if enabled). |
|
With this (unlike containers-storage) we store the transport. So I think with this we wouldn't need an "origin" in bootc - it would just map from e.g. UKI -> "image".
ref https://docs.podman.io/en/latest/markdown/podman-artifact.1.html how stores things separately. |
cgwalters
force-pushed
the
oci-native-layer
branch
from
10bb53f to
6abc58b
Compare
cgwalters
force-pushed
the
oci-native-layer
branch
2 times, most recently
from
df741d1 to
ae68d2e
Compare
Switch how we store OCI images to also include and identify images by having explicit `oci/` tags which point to a manifest, which points to a config. We continue to generate splitstream for manifest and config, but we now always store the original context as "external" objects. A big rationale for this is it will align with a future proposal for composefs OCI sealing where we include fsverity signatures for the manifest/config as a detached object. With this for example, bootc can stop storing the manifest on its own. Other big features: - Full general support for OCI artifacts as well - Support for OCI referrers as well (to store sigstore and future composefs signatures) Finally we have an initial sketch for storing multi-arch images. Assisted-by: OpenCode (Opus 4.6) Signed-off-by: Colin Walters <walters@verbum.org>
cgwalters
force-pushed
the
oci-native-layer
branch
from
ae68d2e to
5b54c79
Compare
cgwalters
changed the title
| stream.add_named_stream_ref(name, value) | ||
| } | ||
| stream.write_inline(json_bytes); | ||
| stream.write_external(json_bytes)?; |
There was a problem hiding this comment.
jeckersb
left a comment
jeckersb
left a comment
There was a problem hiding this comment.
👍 we went over this in a live session earlier
3 participants
Switch how we store OCI images to also include and identify
images by having explicit
oci/tags which point to a manifest,which points to a config.
We continue to generate splitstream for manifest and config, but
we now always store the original context as "external" objects.
A big rationale for this is it will align with a future proposal
for composefs OCI sealing where we include fsverity signatures
for the manifest/config as a detached object.
With this for example, bootc can stop storing the manifest on its own.
See-also: bootc-dev/bootc#1830
Other big features:
composefs signatures)
Finally we have an initial sketch for storing multi-arch images.
Assisted-by: OpenCode (Opus 4.5)