Skip to content

Commit 873fa70

Browse files
wdconincwdconinc
committed
Reapply "feat: move from two-track to three-track container build (#115)" (#126)
This reverts commit 161a8ec.
1 parent b0dc3d0 commit 873fa70

3 files changed

Lines changed: 44 additions & 60 deletions

File tree

‎.github/workflows/build-push.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -406,7 +406,7 @@ jobs:
406406
arch: amd64
407407
runner: ubuntu-latest
408408
PLATFORM: linux/amd64
409-
target: builder_concretization_default
409+
target: builder_concretization_custom
410410
- BUILD_IMAGE: eic_
411411
BUILD_TYPE: default
412412
BUILDER_IMAGE: cuda_devel
@@ -415,7 +415,7 @@ jobs:
415415
arch: amd64
416416
runner: ubuntu-latest
417417
PLATFORM: linux/amd64
418-
target: builder_concretization_default
418+
target: builder_concretization_custom
419419
fail-fast: false
420420
steps:
421421
- name: Free Disk Space (Ubuntu)

‎containers/eic/Dockerfile‎

Lines changed: 27 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@ ARG RUNTIME_IMAGE="debian_stable_base"
66
ARG INTERNAL_TAG="master"
77

88
##
9-
## This docker build follows two tracks, in order to ensure that we build all packages
9+
## This docker build follows three tracks, in order to ensure that we build all packages
1010
## in a builder image, but install them in a runtime image, while at the same time
11-
## avoiding a expensive filesystem copy operation at the end that breaks layering.
11+
## avoiding an expensive filesystem copy operation at the end that breaks layering.
1212
##
1313
## The build is split in an infrequently-changing default environment, upon which
1414
## an environment with custom versions (e.g. individual commits) is layered. The
@@ -18,15 +18,20 @@ ARG INTERNAL_TAG="master"
1818
## The separation in a builder and runtime image is particularly relevant to end up with
1919
## lightweight images for expensive build dependencies, such as for example CUDA.
2020
##
21-
## builder track runtime track
22-
## ----------------------------------------------------------------------
21+
## builder track: runtime track:
22+
## concretization: installation: concretization/installation:
23+
## ---------------------------------------------------------------------------------------
2324
## builder_image runtime_image
2425
## builder_concretization_default
25-
## builder_installation_default -> runtime_concretization_default (copy spack.lock)
26-
## \-> runtime_installation_default (from buildcache)
27-
## builder_concretization_custom
28-
## builder_installation_custom -> runtime_concretization_custom (copy spack.lock)
29-
## \-> runtime_installation_custom (from buildcache)
26+
## \-> builder_installation_default
27+
## runtime_default
28+
## (copy spack.lock from builder_installation_default)
29+
## (install via buildcache)
30+
## \-> builder_concretization_custom
31+
## \-> builder_installation_custom
32+
## \-> runtime_custom
33+
## (copy spack.lock from builder_installation_custom)
34+
## (install via buildcache)
3035
##
3136

3237

@@ -103,14 +108,14 @@ EOF
103108

104109

105110
## ========================================================================================
106-
## runtime_concretization_default
107-
## - runtime base with concretization of default versions (taken from equivalent builder)
111+
## runtime_default
112+
## - runtime base with installation of default versions (buildcache populated by builder)
108113
## ========================================================================================
109-
FROM ${DOCKER_REGISTRY}${RUNTIME_IMAGE}:${INTERNAL_TAG} AS runtime_concretization_default
114+
FROM ${DOCKER_REGISTRY}${RUNTIME_IMAGE}:${INTERNAL_TAG} AS runtime_default
110115
ARG TARGETPLATFORM
111116

112117
# Open Container Initiative labels
113-
LABEL org.opencontainers.image.title="Electron-Ion Collider runtime concretization image (default configuration, $TARGETPLATFORM)"
118+
LABEL org.opencontainers.image.title="Electron-Ion Collider runtime image (default configuration, $TARGETPLATFORM)"
114119

115120
## Copy our default environment
116121
COPY --from=spack-environment . /opt/spack-environment/
@@ -124,17 +129,6 @@ COPY --from=builder_installation_default \
124129
/opt/spack-environment/${ENV}/spack.* \
125130
/opt/spack-environment/${ENV}/
126131

127-
128-
## ========================================================================================
129-
## runtime_installation_default
130-
## - runtime base with installation of default versions (buildcache populated by builder)
131-
## ========================================================================================
132-
FROM runtime_concretization_default AS runtime_installation_default
133-
ARG TARGETPLATFORM
134-
135-
# Open Container Initiative labels
136-
LABEL org.opencontainers.image.title="Electron-Ion Collider runtime installation image (default configuration, $TARGETPLATFORM)"
137-
138132
# Installation (default environment, from buildcache)
139133
RUN --mount=type=cache,target=/var/cache/spack \
140134
--mount=type=secret,id=mirrors,target=/opt/spack/etc/spack/mirrors.yaml \
@@ -153,7 +147,7 @@ EOF
153147
## builder_concretization_custom
154148
## - builder base with concretization of custom versions
155149
## ========================================================================================
156-
FROM builder_installation_default AS builder_concretization_custom
150+
FROM builder_concretization_default AS builder_concretization_custom
157151
ARG TARGETPLATFORM
158152

159153
# Open Container Initiative labels
@@ -239,14 +233,14 @@ EOF
239233

240234

241235
## ========================================================================================
242-
## runtime_concretization_custom
243-
## - runtime base with concretization of custom versions (taken from equivalent builder)
236+
## runtime_custom
237+
## - runtime base with installation of custom versions (buildcache populated by builder)
244238
## ========================================================================================
245-
FROM runtime_installation_default AS runtime_concretization_custom
239+
FROM runtime_default AS runtime_custom
246240
ARG TARGETPLATFORM
247241

248242
# Open Container Initiative labels
249-
LABEL org.opencontainers.image.title="Electron-Ion Collider runtime concretization image (custom configuration, $TARGETPLATFORM)"
243+
LABEL org.opencontainers.image.title="Electron-Ion Collider runtime image (custom configuration, $TARGETPLATFORM)"
250244

251245
# Set spack environment directory
252246
ENV SPACK_ENV=/opt/spack-environment/${ENV}/epic
@@ -261,18 +255,7 @@ COPY --from=builder_installation_custom \
261255
/opt/spack-environment/packages.yaml \
262256
/opt/spack-environment/
263257

264-
265-
## ========================================================================================
266-
## runtime_installation_custom
267-
## - runtime base with installation of custom versions (buildcache populated by builder)
268-
## ========================================================================================
269-
FROM runtime_concretization_custom AS runtime_installation_custom
270-
ARG TARGETPLATFORM
271-
272-
# Open Container Initiative labels
273-
LABEL org.opencontainers.image.title="Electron-Ion Collider runtime installation image (custom configuration, $TARGETPLATFORM)"
274-
275-
# Installation (default environment, from buildcache)
258+
# Installation (custom environment, from buildcache)
276259
RUN --mount=type=cache,target=/var/cache/spack \
277260
--mount=type=secret,id=mirrors,target=/opt/spack/etc/spack/mirrors.yaml \
278261
<<EOF
@@ -284,13 +267,13 @@ EOF
284267

285268

286269
## ========================================================================================
287-
## final image, based on runtime_installation_custom
270+
## final image, based on runtime_custom
288271
## ========================================================================================
289-
FROM runtime_installation_custom AS final
272+
FROM runtime_custom AS final
290273
ARG TARGETPLATFORM
291274

292275
# Open Container Initiative labels
293-
LABEL org.opencontainers.image.title="Electron-Ion Collider runtime installation image (custom configuration, $TARGETPLATFORM)"
276+
LABEL org.opencontainers.image.title="Electron-Ion Collider runtime image (custom configuration, $TARGETPLATFORM)"
294277

295278
## Ensure views directories, not symlinks
296279
RUN <<EOF

‎docs/architecture.md‎

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -4,29 +4,30 @@ The EIC container infrastructure uses a multi-stage build approach with separate
44

55
## Build Strategy
66

7-
The container build follows a two-track approach:
7+
The container build follows a three-track approach:
88

99
```mermaid
1010
flowchart TB
11-
subgraph "Builder Track"
11+
subgraph "Builder Concretization Track"
1212
A[builder_image<br/>debian_stable_base] --> B[builder_concretization_default<br/>Concretize spack environment]
13-
B --> C[builder_installation_default<br/>Build packages]
14-
C --> D[builder_concretization_custom<br/>Concretize custom versions]
15-
D --> E[builder_installation_custom<br/>Build custom packages]
13+
B --> C[builder_concretization_custom<br/>Concretize custom versions]
14+
end
15+
16+
subgraph "Builder Installation Track"
17+
B --> D[builder_installation_default<br/>Build packages]
18+
C --> E[builder_installation_custom<br/>Build custom packages]
1619
end
1720
1821
subgraph "Runtime Track"
19-
F[runtime_image<br/>debian_stable_base] --> G[runtime_concretization_default<br/>Copy spack.lock from builder]
20-
G --> H[runtime_installation_default<br/>Install from buildcache]
21-
H --> I[runtime_concretization_custom<br/>Copy custom spack.lock]
22-
I --> J[runtime_installation_custom<br/>Install custom from buildcache]
23-
J --> K[Final Image<br/>eic_ci / eic_xl]
22+
F[runtime_image<br/>debian_stable_base] --> G[runtime_default<br/>Copy spack.lock, install from buildcache]
23+
G --> H[runtime_custom<br/>Copy custom spack.lock, install from buildcache]
24+
H --> K[Final Image<br/>eic_ci / eic_xl]
2425
end
2526
26-
C -.->|spack.lock| G
27-
C -.->|buildcache| H
28-
E -.->|spack.lock| I
29-
E -.->|buildcache| J
27+
D -.->|spack.lock| G
28+
D -.->|buildcache| G
29+
E -.->|spack.lock| H
30+
E -.->|buildcache| H
3031
```
3132

3233
## Multi-Architecture Support

0 commit comments

Comments
 (0)