Skip to content

feat: add dind-box family — Docker-in-Docker for every box variant (#80)#81

Merged
konard merged 8 commits into
mainfrom
issue-80-bded956c66f7
Apr 28, 2026
Merged

feat: add dind-box family — Docker-in-Docker for every box variant (#80)#81
konard merged 8 commits into
mainfrom
issue-80-bded956c66f7

Conversation

@konard
Copy link
Copy Markdown
Member

@konard konard commented Apr 28, 2026

Summary

Implements issue #80. Adds a dind-box family — a Docker-in-Docker sibling for every existing box image, plus the case-study deliverable that the issue explicitly asks for.

The user's instruction in pull/81#issuecomment-4335908257 was: "name it dind-box, and support alternative version for all box types we have (for each language and each arch and so on). Let's fully implement it." — that's exactly what this PR does.

What you can pull after this lands

For every existing image, there is now a <image>-dind sibling on both Docker Hub and ghcr.io, multi-arch (linux/amd64 + linux/arm64):

Source image New dind sibling
konard/box konard/box-dind
konard/box-essentials konard/box-essentials-dind
konard/box-js konard/box-js-dind
konard/box-python konard/box-python-dind
konard/box-go konard/box-go-dind
konard/box-rust konard/box-rust-dind
konard/box-java konard/box-java-dind
konard/box-kotlin konard/box-kotlin-dind
konard/box-ruby konard/box-ruby-dind
konard/box-php konard/box-php-dind
konard/box-perl konard/box-perl-dind
konard/box-swift konard/box-swift-dind
konard/box-lean konard/box-lean-dind
konard/box-rocq konard/box-rocq-dind

Each dind variant ships the Docker Engine, Docker CLI, containerd, Buildx, and Compose v2, plus an entrypoint that starts the inner dockerd and drops to the box user.

How it's structured

Just one generic recipe under ubuntu/24.04/dind/ rather than 14 duplicated Dockerfiles:

File Purpose
ubuntu/24.04/dind/Dockerfile FROM ${BASE_IMAGE} then runs install.sh. Used by the workflow with --build-arg BASE_IMAGE=... for every variant.
ubuntu/24.04/dind/install.sh Adds Docker apt repo, installs docker-ce, docker-ce-cli, containerd.io, docker-buildx-plugin, docker-compose-plugin, fuse-overlayfs. Adds box user to the docker group.
ubuntu/24.04/dind/dind-entrypoint.sh Starts the inner dockerd (auto-picks overlay2 / fuse-overlayfs / vfs), waits up to DIND_WAIT_SECONDS for the socket, then runusers into box and chains to the existing /usr/local/bin/entrypoint.sh (or the bare command if the base image doesn't have one — js box). Has env-var overrides for storage driver, data root, log file, and a DIND_SKIP_DAEMON=1 for Sysbox-only / DooD use cases.

The release workflow runs this recipe in a 14 × 2 matrix (variant × arch), then assembles per-variant multi-arch manifests, mirroring the existing per-image cadence used for js / essentials / languages / full. Change-detection now also has a dind-changed output so unrelated PRs don't trigger dind rebuilds.

Recommended invocation (security model)

⚠️ dind-box variants ship a Docker daemon and require elevated privileges to run.

Full design and threat model: docs/case-studies/issue-80/CASE-STUDY.md. Original case study from earlier in this PR is preserved verbatim and now has a 2026-04-28 implementation update at the top documenting the final naming and scope decision.

Files changed

ubuntu/24.04/dind/Dockerfile                  (new)
ubuntu/24.04/dind/install.sh                  (new)
ubuntu/24.04/dind/dind-entrypoint.sh          (new)
.github/workflows/release.yml                 (3 new jobs + dind sections in release notes + dind change detection + PR smoke test; release-notes step split into two to stay under GitHub's 21000-char per-step expression limit)
README.md                                     (dind-box image tables for both registries + security banner + architecture diagram update)
docs/case-studies/issue-80/CASE-STUDY.md      (implementation update header noting final dind-box naming)
.changeset/issue-80-dind-box.md               (new: minor bump describing the feature)

Test plan

  • Workflow YAML validates with actionlint (zero issues) and Python YAML parser.
  • All shell scripts pass bash -n syntax check.
  • PR-time CI smoke test added in docker-build-test: builds box-dind-test from the freshly-built box-test and verifies docker --version, dockerd --version, docker buildx version, docker compose version, and /etc/box/variant are all present and correct.
  • Multi-arch manifest jobs follow the same pattern as the existing js/essentials/languages/docker manifests.
  • Change-detection: a PR that only touches ubuntu/24.04/dind/** will rebuild only the dind matrix (dind-changed output).
  • Cannot test from inside a dind-box without a host that allows --privileged or --runtime=sysbox-runc. GitHub-hosted runners disallow --privileged, which is why the PR-time smoke test verifies CLI binaries only and leaves the live-dockerd test for users on hosts that allow it.

What I learned writing this

While iterating on the workflow I tripped on GitHub Actions' "Exceeded max expression length 21000" limit — the existing release-notes heredoc was already ~17 KB, and adding the dind-box tables pushed the single run: step over the per-step expression budget. GitHub's failure was opaque (zero jobs, zero billable time, run name = path string) until I scraped the run page HTML for the annotation. Fixed by splitting the dind tables into a separate appended step that builds them via a bash for-loop, keeping the GitHub-side expression interpolation small.

How this addresses each requirement from the case study

Req Addressed by
FR-80.1 superset of konard/box konard/box-dind and per-language *-dind variants are built FROM their non-dind sibling.
FR-80.2 docker / dockerd / containerd / buildx / compose install.sh installs all five from Docker's apt repo.
FR-80.3 auto-start dockerd dind-entrypoint.sh starts dockerd on container boot and waits for the socket.
FR-80.4 multi-arch Workflow has both build-dind-amd64 (ubuntu-24.04) and build-dind-arm64 (ubuntu-24.04-arm).
FR-80.5 both registries Workflow tags both ${DOCKERHUB_IMAGE_NAME}-... and ${GHCR_REGISTRY}/${GHCR_IMAGE_NAME}-....
FR-80.6 docker ps scoping Nested DinD: each container owns its own dockerd → naturally scoped.
FR-80.7 host stays "somewhat safe" Sysbox is documented as the recommended runtime. DooD with bind-mounted socket is rejected as a default.
FR-80.8 / DOC-80.2 README docs New "dind-box" section + security model banner.
NFR-80.1 non-root by default Entrypoint starts dockerd as root just long enough, then runusers into box.
NFR-80.2 modular pipeline Single Dockerfile + --build-arg BASE_IMAGE=...; the workflow matrix targets every base.
NFR-80.3 per-image change detection dind=true/false output in detect-changes.
DOC-80.1 case study Already in docs/case-studies/issue-80/; updated with implementation header.

Fixes #80

Adding .gitkeep for PR creation (default mode).
This file will be removed when the task is complete.

Issue: #80
@konard konard self-assigned this Apr 28, 2026
…n-Box)

- Compile issue text, requirements list, and solution plans in docs/case-studies/issue-80/CASE-STUDY.md
- Collect upstream evidence (Docker DinD/DooD/Sysbox/Tecnativa/devcontainers/OWASP/Quarkslab) in research.md
- Preserve original issue text in issue.md
- Recommend nested DinD as the default solution: it satisfies "docker ps only shows my containers"
  by construction, and document Sysbox as the recommended secure runtime
- Rejected DooD with host /var/run/docker.sock as a default for host-safety reasons

Removes the .gitkeep marker file used during PR bootstrap (per its own commit message).
@konard konard changed the title [WIP] Add super-box image, that does all the same, but also built up on docker with elevated permissions, that allows to control docker from inside dockers docs: case study for issue #80 (super-box / Docker-in-Box) Apr 28, 2026
@konard konard marked this pull request as ready for review April 28, 2026 13:35
@konard
Copy link
Copy Markdown
Member Author

konard commented Apr 28, 2026

Solution summary

Done. The PR is ready for review.

Summary of what I did

The issue body explicitly asked for a case study (compile data into docs/case-studies/issue-{id}/, list every requirement, propose solutions, check existing components) — not yet for an implementation. So this PR is documentation‑only.

Three new files under docs/case-studies/issue-80/:

  • issue.md — original issue text preserved.
  • research.md — upstream research (Docker DinD/DooD/rootless, Sysbox, Tecnativa proxy, FoxxMD docker-proxy-filter, devcontainers Features, OWASP, Quarkslab, jpetazzo's classic warning, Ubuntu 24.04 AppArmor caveat).
  • CASE-STUDY.md — extracts 11 requirements (FR‑80.1…8, NFR‑80.1…3, DOC‑80.1…2) from the short issue body, walks the solution space per requirement, and recommends a default plan: nested DinD on top of box-essentials (recipe template: cruizba/ubuntu-dind), which satisfies the host‑safety stretch goal "docker ps only shows my containers" by construction (each box owns its own daemon). DooD with host /var/run/docker.sock is rejected as a default. Sysbox is documented as the recommended secure runtime. A rootless variant is noted as a follow‑up tag.

The bootstrap .gitkeep is removed (its commit message said it would be when the task is complete). PR 81 title/body updated, marked Ready for review: #81

Want me to /schedule a follow-up agent in ~1 week to open the implementation PR (ubuntu/24.04/super-box/ Dockerfile + release-workflow job + README rows) once this case study is approved?


This summary was automatically extracted from the AI working session output.

@konard
Copy link
Copy Markdown
Member Author

konard commented Apr 28, 2026

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Public pricing estimate: $3.275113
  • Calculated by Anthropic: $3.455112
  • Difference: $0.180000 (+5.50%)

📊 Context and tokens usage:

Claude Opus 4.7:

  • 97.4K / 1M (10%) input tokens, 28.3K / 128K (22%) output tokens

Total: (178.7K + 2.1M cached) input tokens, 28.3K output tokens, $2.868548 cost

Claude Haiku 4.5:

  • 9.5K / 64K (15%) output tokens

Total: 295.9K input tokens, 9.5K output tokens, $0.406565 cost

🤖 Models used:

  • Tool: Anthropic Claude Code
  • Requested: opus
  • Main model: Claude Opus 4.7 (claude-opus-4-7)
  • Additional models:
    • Claude Haiku 4.5 (claude-haiku-4-5-20251001)

📎 Log file uploaded as Gist (1515KB)


Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard
Copy link
Copy Markdown
Member Author

konard commented Apr 28, 2026

✅ Ready to merge

This pull request is now ready to be merged:

  • All CI checks have passed
  • No merge conflicts
  • No pending changes

Monitored by hive-mind with --auto-restart-until-mergeable flag

@konard
Copy link
Copy Markdown
Member Author

konard commented Apr 28, 2026

Ok, name it dind-box, and support alternative verson for all box types we have (for each language and each arch and so on).

Let's fully implement it.

@konard konard marked this pull request as draft April 28, 2026 14:01
@konard
Copy link
Copy Markdown
Member Author

konard commented Apr 28, 2026

🤖 AI Work Session Started

Starting automated work session at 2026-04-28T14:01:09.324Z

The PR has been converted to draft mode while work is in progress.

This comment marks the beginning of an AI work session. Please wait for the session to finish, and provide your feedback.

Adds a generic dind recipe at ubuntu/24.04/dind/ that takes any base box image
as --build-arg BASE_IMAGE=... and produces a "<base>-dind" sibling. Each
running dind-box has its own inner dockerd, so docker ps -a inside lists only
that box's children — the host-isolation goal in issue #80.

Wires the recipe into the release workflow as a 14×2 (variant × arch) matrix
so dind variants are published alongside every existing image (full,
essentials, js, and every language box) on Docker Hub and ghcr.io with the
standard latest/version/-amd64/-arm64 tag set and per-variant multi-arch
manifests.

README + case study updated with image tables and the security model
(default --privileged, recommended Sysbox runtime, do-not-mount-host-socket
warning, per-box docker ps scoping).
@konard konard changed the title docs: case study for issue #80 (super-box / Docker-in-Box) feat: add dind-box family — Docker-in-Docker for every box variant (#80) Apr 28, 2026
@konard konard marked this pull request as ready for review April 28, 2026 14:11
konard added 5 commits April 28, 2026 14:14
Hyphen-prefixed values like '-js' inside YAML flow-style mappings caused
GitHub Actions' workflow parser to reject the file (no jobs at all in the
last run). Switching to explicit block-style with quoted strings makes the
intent unambiguous and matches the style used elsewhere in this workflow.
GitHub Actions accepts object-typed matrix entries, but actionlint flagged
'object should not be evaluated in template with ${{ }}' for steps that
referenced ${{ matrix.variant }} without a sub-key. The previous push to
main with object-typed matrices failed at workflow load (zero jobs, zero
billable time, run name showed file path instead of workflow name) — this
matches GitHub's behaviour when it can't materialize the matrix.

Switch to a flat list of strings and derive base_image_suffix / dind_suffix
inline in each step. Same matrix size (14 entries × 2 archs).
… matrix)

Push to feature branch keeps producing 0-second 'failure' runs that record
no jobs, no billable time, and report the workflow file path as the run
name. Local YAML and actionlint both pass. The pre-existing language matrix
uses a flow-style inline list 'language: [a, b, c]' on one line — switch
the dind matrix to the same form so it matches the proven pattern.
Just in case GitHub's stricter parser dislikes em-dashes inside new run/comment
blocks. Existing em-dashes in the architecture diagram heredoc are left alone
since they were already passing on prior runs.
GitHub Actions rejected the workflow with 'Exceeded max expression length 21000'
on line 2466 (the create-release step's run block). Adding the dind-box tables
to the existing heredoc pushed the step's expression over the per-step limit.

Move the dind-box tables to a new 'Append dind-box tables to release notes'
step that uses a bash for-loop to generate rows programmatically, keeping the
GitHub-side expression interpolation small. Move 'gh release create/edit' to
its own step so each step stays well under the limit.
@konard
Copy link
Copy Markdown
Member Author

konard commented Apr 28, 2026

Solution summary

The polling background task is still waiting. It will notify me when complete. Let me summarize the work done:

Summary

I implemented the dind-box family per the user's directive in pull/81#issuecomment-4335908257"name it dind-box, and support alternative version for all box types we have (for each language and each arch and so on). Let's fully implement it."

What's in the PR (#81)

New files (ubuntu/24.04/dind/):

  • Dockerfile — generic FROM ${BASE_IMAGE} recipe
  • install.sh — Docker apt install (CE, CLI, containerd, Buildx, Compose, fuse-overlayfs)
  • dind-entrypoint.sh — starts inner dockerd, waits for socket, drops to box user via runuser

Workflow (.github/workflows/release.yml):

  • dind-changed change detection
  • 14×2 matrix (build-dind-amd64/-arm64) covering js/essentials/full + 11 languages
  • dind-manifest job assembling per-variant multi-arch manifests
  • PR-time smoke test that verifies docker/dockerd/buildx/compose CLIs

Docs: README image tables for both registries + security banner; case study updated with implementation note; changeset bumping minor version.

Issues hit and fixed (visible in commit history)

  1. YAML flow-style with hyphen-prefixed strings confused parsers — fixed by switching to inline [a, b, c] form mirroring the existing language matrix.
  2. Em-dashes in run blocks — replaced with hyphens (just to be safe).
  3. GitHub Actions 21000-char per-step expression limit — the existing release-notes heredoc was already ~17 KB; adding the dind tables pushed it over. The failure surfaced as name: ".github/workflows/release.yml", zero jobs, zero billable time. Found the actual error by scraping the run-page HTML for Invalid workflow file ... Exceeded max expression length 21000. Fixed by splitting the dind tables into a separate Append dind-box tables to release notes step that builds rows via a bash for-loop (keeps GitHub-side interpolation small).

CI status

The pull_request event run on commit a6c99bb is currently in progress (docker-build-test building all images including dind). I've started a background poller; will be notified when it finishes.

PR is ready for review (isDraft: false, mergeable: MERGEABLE, mergeStateStatus: UNSTABLE while CI runs).


This summary was automatically extracted from the AI working session output.

@konard
Copy link
Copy Markdown
Member Author

konard commented Apr 28, 2026

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost: $31.363773

📊 Context and tokens usage:

  • 324.7K / 1M (32%) input tokens, 113.1K / 128K (88%) output tokens

Total: (334.2K + 52.9M cached) input tokens, 113.1K output tokens, $31.363773 cost

🤖 Models used:

  • Tool: Anthropic Claude Code
  • Requested: opus
  • Model: Claude Opus 4.7 (claude-opus-4-7)

📎 Log file uploaded as Gist (9097KB)


Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard
Copy link
Copy Markdown
Member Author

konard commented Apr 28, 2026

✅ Ready to merge

This pull request is now ready to be merged:

  • All CI checks have passed
  • No merge conflicts
  • No pending changes

Monitored by hive-mind with --auto-restart-until-mergeable flag

@konard konard merged commit da0c1f3 into main Apr 28, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add super-box image, that does all the same, but also built up on docker with elevated permissions, that allows to control docker from inside dockers

1 participant