Skip to content

NO-JIRA: hack/build.sh: parallelize the build if MAKEFLAGS is set #10364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jhixson74 wants to merge 1 commit into from
Closed

NO-JIRA: hack/build.sh: parallelize the build if MAKEFLAGS is set #10364

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cluster-api/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ $(GO_BUILD_TARGETS): go-build.%: bin/$(TARGET_OS_ARCH)/cluster-api-provider-%
$(PROVIDER_TARGETS): bin/$(TARGET_OS_ARCH)/cluster-api-provider-%: providers/%/go.mod
cd providers/$*; \
if [ -f main.go ]; then path="."; else path=./vendor/`grep _ tools.go|awk '{ print $$2 }'|sed 's|"||g'`; fi; \
go build -gcflags $(GCFLAGS) -ldflags $(LDFLAGS) -o ../../bin/$(TARGET_OS_ARCH)/cluster-api-provider-$* "$$path";
go build $(GOBUILDFLAGS) -gcflags $(GCFLAGS) -ldflags $(LDFLAGS) -o ../../bin/$(TARGET_OS_ARCH)/cluster-api-provider-$* "$$path";
Copy link
Copy Markdown
Contributor

@patrickdillon patrickdillon Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, outside of your changes, but I notice that we're apparently not ${GOFLAGS} in the capi builds... I wonder if CI is using those 👀

Copy link
Copy Markdown
Contributor

@patrickdillon patrickdillon Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the build logs, it's drop -mod=vendor

2026-03-06T18:25:27.160016001Z go build  -gcflags "" -ldflags "-s -w" -o ../../bin/linux_amd64/cluster-api-provider-openstack "$path";

vs

26-03-06T18:27:51.399947790Z + go build -mod=vendor -gcflags '' -ldflags ' -X github.com/openshift/installer/pkg/version.Raw=v1.4.21-pre-246-g170afd6c6ccd7679f4c55d91808988eeedbfe1fb-dirty -X github.com/openshift/installer/pkg/version.Commit=170afd6c6ccd7679f4c55d91808988eeedbfe1fb -X github.com/openshift/installer/

🤔

Copy link
Copy Markdown
Contributor

@patrickdillon patrickdillon Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to not be an issue 😅

        -mod mode
                module download mode to use: readonly, vendor, or mod.
                By default, if a vendor directory is present and the go version in go.mod
                is 1.14 or higher, the go command acts as if -mod=vendor were set.

But we should probably fix that in case art changes anything in the future

Copy link
Copy Markdown
Member

@tthvo tthvo Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aha, #10364 (comment) makes sense. I was wondering the same thing :D

Copy link
Copy Markdown
Member Author

@jhixson74 jhixson74 Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the build logs, it's drop -mod=vendor

2026-03-06T18:25:27.160016001Z go build  -gcflags "" -ldflags "-s -w" -o ../../bin/linux_amd64/cluster-api-provider-openstack "$path";

vs

26-03-06T18:27:51.399947790Z + go build -mod=vendor -gcflags '' -ldflags ' -X github.com/openshift/installer/pkg/version.Raw=v1.4.21-pre-246-g170afd6c6ccd7679f4c55d91808988eeedbfe1fb-dirty -X github.com/openshift/installer/pkg/version.Commit=170afd6c6ccd7679f4c55d91808988eeedbfe1fb -X github.com/openshift/installer/

🤔

I haven't done anything with GOFLAGS here. I intentionally left it alone.


.PHONY: go-build-cluster-api
go-build-cluster-api: bin/$(TARGET_OS_ARCH)/cluster-api
Expand All @@ -41,7 +41,7 @@ go-mod-tidy-vendor-cluster-api:

bin/$(TARGET_OS_ARCH)/cluster-api: cluster-api/go.mod
cd cluster-api; \
go build -gcflags $(GCFLAGS) -ldflags $(LDFLAGS) -o ../bin/$(TARGET_OS_ARCH)/cluster-api ./vendor/sigs.k8s.io/cluster-api
go build $(GOBUILDFLAGS) -gcflags $(GCFLAGS) -ldflags $(LDFLAGS) -o ../bin/$(TARGET_OS_ARCH)/cluster-api ./vendor/sigs.k8s.io/cluster-api

.PHONY: go-clean
go-clean: go-clean-providers go-clean-cluster-api
Expand Down
22 changes: 19 additions & 3 deletions hack/build-cluster-api.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,34 @@ ENVTEST_K8S_VERSION="1.32.0"
ENVTEST_ARCH=$(go env GOOS)-$(go env GOARCH)

copy_cluster_api_to_mirror() {
ZIPFLAGS="-j1"

mkdir -p "${CLUSTER_API_BIN_DIR}"
mkdir -p "${CLUSTER_API_MIRROR_DIR}"

# Clean the mirror, but preserve the README file.
rm -rf "${CLUSTER_API_MIRROR_DIR:?}/*.zip"
# If the archive exists, just update it with any changes
if [ -f "${CLUSTER_API_MIRROR_DIR}/cluster-api.zip" ]; then
ZIPFLAGS="${ZIPFLAGS} -u"
fi

if test "${SKIP_ENVTEST}" != y; then
sync_envtest
fi

set +e

# Zip every binary in the folder into a single zip file.
zip -j1 "${CLUSTER_API_MIRROR_DIR}/cluster-api.zip" "${CLUSTER_API_BIN_DIR}"/*
# shellcheck disable=SC2086
zip ${ZIPFLAGS} "${CLUSTER_API_MIRROR_DIR}/cluster-api.zip" "${CLUSTER_API_BIN_DIR}"/*
RET=$?

# Nothing to do
if [ "${RET}" = "12" ]; then
return 0
fi

set -e
return ${RET}
}

sync_envtest() {
Expand Down
16 changes: 14 additions & 2 deletions hack/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,20 @@ fi
export CGO_ENABLED=0
MODE="${MODE:-release}"

# if -j [::numeric::] is in MAKEFLAGS, parse it out and pass it along to go build
MAKEJOBS="$(echo "'${MAKEFLAGS}'" | sed -nE 's|.*-j\s+?([1-9]+).*|\1|p')"
# shellcheck disable=SC2086
if [ -n "${MAKEJOBS}" ] && [ ${MAKEJOBS} -gt 0 ]; then
GOBUILDFLAGS="${GOBUILDFLAGS} -p ${MAKEJOBS}"
fi

# pass along to any forked processes (sub make files and go build processes)
export GOBUILDFLAGS
export MAKEFLAGS

# build cluster-api binaries
make -C cluster-api all
# shellcheck disable=SC2086
make ${MAKEFLAGS} -C cluster-api all
copy_cluster_api_to_mirror

GIT_COMMIT="${SOURCE_GIT_COMMIT:-$(git rev-parse --verify 'HEAD^{commit}')}"
Expand Down Expand Up @@ -60,4 +72,4 @@ fi
echo "building openshift-install"

# shellcheck disable=SC2086
go build ${GOFLAGS} -gcflags "${GCFLAGS}" -ldflags "${LDFLAGS}" -tags "${TAGS}" -o "${OUTPUT}" ./cmd/openshift-install
go build ${GOBUILDFLAGS} ${GOFLAGS} -gcflags "${GCFLAGS}" -ldflags "${LDFLAGS}" -tags "${TAGS}" -o "${OUTPUT}" ./cmd/openshift-install