expanding rootfs size does not seem to work

[travisghansen]

Asking around on slack and looking at various PR/code there seems to be some ability to increase the rootfs size from 20GB. I was directed to create an issue on this repo.

Working with containerd there are several PRs that reference this (linked below) each describing containerd ns labels. We have applied those labels and they seem to have no effect.

# containerd code base
egrep -i -R 'rootfs' ./plugins | grep -E -i "label"
egrep: warning: egrep is obsolescent; using grep -E
./plugins/snapshots/lcow/lcow.go:	rootfsSizeLabel           = "containerd.io/snapshot/io.microsoft.container.storage.rootfs.size-gb"
./plugins/snapshots/lcow/lcow.go:	rootfsLocLabel            = "containerd.io/snapshot/io.microsoft.container.storage.rootfs.location"
./plugins/snapshots/lcow/lcow.go:				if sizeGBstr, ok := snapshotInfo.Labels[rootfsSizeLabel]; ok {
./plugins/snapshots/lcow/lcow.go:				scratchLocation := snapshotInfo.Labels[rootfsLocLabel]
./plugins/snapshots/windows/common.go:	if sizeGBstr, ok := snapshotInfo.Labels[rootfsSizeInGBLabel]; ok {
./plugins/snapshots/windows/common.go:		log.G(ctx).Warnf("%q label is deprecated, please use %q instead.", rootfsSizeInGBLabel, rootfsSizeInBytesLabel)
./plugins/snapshots/windows/common.go:			return 0, fmt.Errorf("failed to parse label %q=%q: %w", rootfsSizeInGBLabel, sizeGBstr, err)
./plugins/snapshots/windows/common.go:	if sizeBytesStr, ok := snapshotInfo.Labels[rootfsSizeInBytesLabel]; ok {
./plugins/snapshots/windows/common.go:			return 0, fmt.Errorf("failed to parse label %q=%q: %w", rootfsSizeInBytesLabel, sizeBytesStr, err)https://github.com/containerd/containerd/issues/6694
./plugins/snapshots/windows/windows.go:	// Deprecated: use rootfsSizeInBytesLabel
./plugins/snapshots/windows/windows.go:	rootfsSizeInGBLabel = "containerd.io/snapshot/io.microsoft.container.storage.rootfs.size-gb"
./plugins/snapshots/windows/windows.go:	// rootfsSizeInBytesLabel is a label to control a Windows containers scratch space
./plugins/snapshots/windows/windows.go:	rootfsSizeInBytesLabel = "containerd.io/snapshot/windows/rootfs.sizebytes"

For example we have tried setting both containerd.io/snapshot/windows/rootfs.sizebytes and containerd.io/snapshot/io.microsoft.container.storage.rootfs.size-gb and neither seem to impact anything.

Are we doing something wrong or is the simply not implemented end-to-end?

Thanks!

• Windows snapshotter touch ups and new functionality containerd/containerd#6918
• Support rootfs WindowsContainerResources.rootfs_size_in_bytes in CRI API  containerd/containerd#7199
• Windows rootfs.size-gb label is not expanding volume to specified size containerd/containerd#6694

[mendsley]

I can confirm that if the snapshot (not the image or container) has the containerd.io/snapshot/windows/rootfs.sizebytes label, it'll create the writable VHD layer of the requested size when a container is started. So from the containerd standpoint it should work end-to-end, but the various other tooling for this is somewhat spotty

ctr container create ... has a --snapshotter-label that can do this. I don't know for certain if nerdctl create/run passes labels to the snapshotter or not.

I do know that the docker commands with containerd do not by default. We work around this locally with the PR: moby/moby#49906

[travisghansen]

Thanks for the response. I have since also confirmed the behavior and have patched ctr (no PR yet) and have it working.

diff --git a/cmd/ctr/commands/images/mount.go b/cmd/ctr/commands/images/mount.go
index c97954267..63c5a7746 100644
--- a/cmd/ctr/commands/images/mount.go
+++ b/cmd/ctr/commands/images/mount.go
@@ -25,6 +25,7 @@ import (
 	"github.com/containerd/containerd/v2/cmd/ctr/commands"
 	"github.com/containerd/containerd/v2/core/leases"
 	"github.com/containerd/containerd/v2/core/mount"
+	"github.com/containerd/containerd/v2/core/snapshots"
 	"github.com/containerd/containerd/v2/defaults"
 	"github.com/containerd/errdefs"
 	"github.com/containerd/platforms"
@@ -114,11 +115,16 @@ When you are done, use the unmount command.
 
 		s := client.SnapshotService(snapshotter)
 
+		labels := commands.LabelArgs(cliContext.StringSlice("label"))
+		opts := []snapshots.Opt{
+			snapshots.WithLabels(labels),
+		}
+
 		var mounts []mount.Mount
 		if cliContext.Bool("rw") {
-			mounts, err = s.Prepare(ctx, target, chainID)
+			mounts, err = s.Prepare(ctx, target, chainID, opts...)
 		} else {
-			mounts, err = s.View(ctx, target, chainID)
+			mounts, err = s.View(ctx, target, chainID, opts...)
 		}
 		if err != nil {
 			if errdefs.IsAlreadyExists(err) {

Ideally some annotation/label on the pod would be read and push that down to containerd.

• support ephemeral rw image volume mounts kubernetes/kubernetes#131557

Given no movement on the above issue and barring some ability to set that on pod spec for the running image/container my intent is to build a csi inline ephemeral driver (I already have it coded up) to interface with containerd and mount the given image (using ctr mount image behind the scenes with the patch from above to force the scratch size higher). Once I have the driver built and deployed and tested I will report back here.

Either way being able to increase the scratch size of the running image/container is a desirable behavior.