NO-JIRA: hack/build.sh: parallelize the build if MAKEFLAGS is set by jhixson74 · Pull Request #10364 · openshift/installer

jhixson74 · 2026-03-05T23:09:12Z

Parallelize build when setting MAKEFLAGS. Keep default behavior otherwise.

To test:
env MAKEFLAGS="-j $(nproc)" ./hack/build.sh

This will pass along the output of nproc, or if you manually specify a number. make will get -j , and go build will get -p . Nothing changes if not setting MAKEFLAGS with a -j option.

Also updated the way we create the cluster-api.zip file. Instead of deleting it and zipping all the providers everytime hack/build.sh is invoked, we just update specific providers when changed and only update that provider in cluster-api.zip.

openshift-ci-robot · 2026-03-05T23:09:17Z

@jhixson74: This pull request explicitly references no jira issue.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2026-03-05T23:09:41Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign barbacbd for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jhixson74 · 2026-03-05T23:14:52Z

/cc @patrickdillon

tthvo · 2026-03-06T02:37:00Z

hack/build.sh


 # shellcheck disable=SC2086
-go build ${GOFLAGS} -gcflags "${GCFLAGS}" -ldflags "${LDFLAGS}" -tags "${TAGS}" -o "${OUTPUT}" ./cmd/openshift-install
+go build ${GOBUILDJOBS} ${GOFLAGS} -gcflags "${GCFLAGS}" -ldflags "${LDFLAGS}" -tags "${TAGS}" -o "${OUTPUT}" ./cmd/openshift-install


💡I notice we currently have the GOFLAGS. Should we use that instead of defining a new one? For example, we run:

GOFLAGS='-p=16 --mod=vendor' ./hack/build.sh

With this, I guess we just need to adjust cluster-api/Makefile so that go build considers GOFLAGS and hack/build.sh to pass down the env var?

This breaks when go generate runs. That's why I did it this way.

ahh whoops. Just curious, may I know how or why it failed (i.e. error message)?

I guess I was asking because I haven't seen go generate failed in my local setup. Maybe, I am missing something 😓 Also, latest change might have broken the build in ci/prow/e2e-aws-ovn and ci/prow/images:

+ make '' -C cluster-api all make: *** empty string invalid as file name. Stop.

Bleh. Yeah, the shell linter wants variables quoted. But for cases like this, you intentionally don't quote them ;-) I will update.

As for go generate, it honors GOFLAGS. -p is not a valid option to go generate

Ah right, sorry GOFLAGS is a built-in Golang env var, which applies to all go commands🤦 I have one more suggestion if it makes it easier:

We can define a variable GOBUILDFLAGS for specificially for building and pass it directly to go build:

go build ${GOBUILDFLAGS} ${GOFLAGS} -gcflags "${GCFLAGS}" -ldflags "${LDFLAGS}" -tags "${TAGS}" -o "${OUTPUT}" ./cmd/openshift-install

Then we can run build with any supported go flags instead of parsing the number out:

GOBUILDFLAGS="-p $(nproc)" ./hack/build.sh

While that is a nice feature to have, it's outside of the scope here. This particular PR passes along the number of jobs to both make, and go build. For this to work, we now need to set GOBUILDFLAGS and MAKEFLAGS in the environment to keep both consistent. For now, I think this works fine as is.

I met you half way. I do like the name GOBUILDFLAGS better. I also modified it here so that can be set in the environment. If MAKEFLAGS is set with -j, it will automagically set GOBUILDFLAGS.

Ahh right, we need it for both make and go. The changes look good to me, thanks!

Sorry for the long thread :(

Could you help me understand the reason to also set -p for go build? I thought we just need to allocate >= 1 make job slots via -j <n>, right? On main, we can already do MAKEFLAGS="-j 16" ./hack/build.sh, which does parallelize the capi provider build.

For go, go help build shows 👇, which is, in most cases, already building packages in parallel?

-p n the number of programs, such as build commands or test binaries, that can be run in parallel. The default is GOMAXPROCS, normally the number of CPUs available.

tthvo · 2026-03-06T04:26:13Z

hack/build.sh

 release)
 	LDFLAGS="${LDFLAGS} -s -w"
-	TAGS="${TAGS} release"
+	TAGS="${TAGS} release"


There is a small empty character after this line (i.e. reported as ␦):

$ cat hack/build.sh | grep -A3 -- 'release)' release) LDFLAGS="${LDFLAGS} -s -w" TAGS="${TAGS} release"␦ ;;

Do you see it? Maybe we should remove it...

My VSCode also reported the same thing:

Blah. That explains why there is a difference showing. I'm not sure what I did here, but I'll nuke it.

Nice, I thought i was seeing things :D

tthvo · 2026-03-06T04:35:48Z

hack/build.sh


 # shellcheck disable=SC2086
-go build ${GOFLAGS} -gcflags "${GCFLAGS}" -ldflags "${LDFLAGS}" -tags "${TAGS}" -o "${OUTPUT}" ./cmd/openshift-install
+go build ${GOBUILDJOBS} ${GOFLAGS} -gcflags "${GCFLAGS}" -ldflags "${LDFLAGS}" -tags "${TAGS}" -o "${OUTPUT}" ./cmd/openshift-install


As for go generate, it honors GOFLAGS. -p is not a valid option to go generate

Ah right, sorry GOFLAGS is a built-in Golang env var, which applies to all go commands🤦 I have one more suggestion if it makes it easier:

We can define a variable GOBUILDFLAGS for specificially for building and pass it directly to go build:

go build ${GOBUILDFLAGS} ${GOFLAGS} -gcflags "${GCFLAGS}" -ldflags "${LDFLAGS}" -tags "${TAGS}" -o "${OUTPUT}" ./cmd/openshift-install

Then we can run build with any supported go flags instead of parsing the number out:

GOBUILDFLAGS="-p $(nproc)" ./hack/build.sh

tthvo · 2026-03-06T06:11:27Z

hack/build.sh

+if [ ${MAKEJOBS} -gt 0 ]; then
+        GOBUILDFLAGS="${GOBUILDFLAGS} -p ${MAKEJOBS}"
+fi


Suggested change

if [ ${MAKEJOBS} -gt 0 ]; then

GOBUILDFLAGS="${GOBUILDFLAGS} -p ${MAKEJOBS}"

fi

if [ -n "${MAKEJOBS}" ] && [ "${MAKEJOBS}" -gt 0 ]; then

GOBUILDFLAGS="${GOBUILDFLAGS} -p ${MAKEJOBS}"

fi

We should check for non-empty first before comparing the integer. If not, there is actually a silent failure:

+ '[' -gt 0 ']' ./hack/build.sh: line 26: [: -gt: unary operator expected

Reference: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_installer/10364/pull-ci-openshift-installer-main-artifacts-images/2029785055106174976/artifacts/build-logs/installer-artifacts-amd64.log

Updated the regex to be more bulletproof. Now we won't get a non-digit. We either get null, or the number of jobs passed to -j.

tthvo · 2026-03-06T17:11:51Z

/test golint

tthvo

Looks good to me!

@jhixson74 May I get help with #10364 (comment) before tagging?

jhixson74 · 2026-03-06T17:51:57Z

Looks good to me!

@jhixson74 May I get help with #10364 (comment) before tagging?

I made one more optimization to building the capi providers. Please review ;-) go build -p works like make -j. It will create the number of threads specified. As you specified, if not, GOMAXPROCS is used, which is the number of cores.

jhixson74 · 2026-03-06T17:55:57Z

/hold

It appears go changes aren't being noticed.

jhixson74 · 2026-03-06T18:01:48Z

/hold cancel

zip exits with code 12 if there is nothing to do, so added check for that.

jhixson74 · 2026-03-06T18:17:26Z

/test golint

tthvo

/lgtm

The build is pretty fast now :D I guess e2e is passing the build means it's good :D

It will create the number of threads specified. As you specified, if not, GOMAXPROCS is used, which is the number of cores.

Oh thanks, I guess it's for making it consistent between make and go, but not a hard requirement to build in parallel...

patrickdillon · 2026-03-06T20:25:26Z

Do you see improvements when using this? I'm curious about the results.

go build already uses all cores; the make jobs are running in serial, but each job is a go build so is parallelized across all cores (right?)

when I have tested (along time ago) I didn't see a lot of spare cpu space, except with the compression/zipping part

It's not really clear to me why we should link the number of make jobs to the go -p flag.

patrickdillon · 2026-03-06T20:27:40Z

cluster-api/Makefile

 	cd providers/$*; \
 	if [ -f main.go ]; then path="."; else path=./vendor/`grep _ tools.go|awk '{ print $$2 }'|sed 's|"||g'`; fi; \
-	go build -gcflags $(GCFLAGS) -ldflags $(LDFLAGS) -o ../../bin/$(TARGET_OS_ARCH)/cluster-api-provider-$* "$$path";
+	go build $(GOBUILDFLAGS) -gcflags $(GCFLAGS) -ldflags $(LDFLAGS) -o ../../bin/$(TARGET_OS_ARCH)/cluster-api-provider-$* "$$path";


hm, outside of your changes, but I notice that we're apparently not ${GOFLAGS} in the capi builds... I wonder if CI is using those 👀

Looking at the build logs, it's drop -mod=vendor

2026-03-06T18:25:27.160016001Z go build -gcflags "" -ldflags "-s -w" -o ../../bin/linux_amd64/cluster-api-provider-openstack "$path";

vs

26-03-06T18:27:51.399947790Z + go build -mod=vendor -gcflags '' -ldflags ' -X github.com/openshift/installer/pkg/version.Raw=v1.4.21-pre-246-g170afd6c6ccd7679f4c55d91808988eeedbfe1fb-dirty -X github.com/openshift/installer/pkg/version.Commit=170afd6c6ccd7679f4c55d91808988eeedbfe1fb -X github.com/openshift/installer/

🤔

Seems to not be an issue 😅

-mod mode module download mode to use: readonly, vendor, or mod. By default, if a vendor directory is present and the go version in go.mod is 1.14 or higher, the go command acts as if -mod=vendor were set.

But we should probably fix that in case art changes anything in the future

Aha, #10364 (comment) makes sense. I was wondering the same thing :D

Looking at the build logs, it's drop -mod=vendor

2026-03-06T18:25:27.160016001Z go build -gcflags "" -ldflags "-s -w" -o ../../bin/linux_amd64/cluster-api-provider-openstack "$path";

vs

26-03-06T18:27:51.399947790Z + go build -mod=vendor -gcflags '' -ldflags ' -X github.com/openshift/installer/pkg/version.Raw=v1.4.21-pre-246-g170afd6c6ccd7679f4c55d91808988eeedbfe1fb-dirty -X github.com/openshift/installer/pkg/version.Commit=170afd6c6ccd7679f4c55d91808988eeedbfe1fb -X github.com/openshift/installer/

🤔

I haven't done anything with GOFLAGS here. I intentionally left it alone.

tthvo · 2026-03-06T20:55:06Z

Do you see improvements when using this? I'm curious about the results.
go build already uses all cores; the make jobs are running in serial, but each job is a go build so is parallelized across all cores (right?)

For me, running time MAKEFLAGS="-j $(nproc)" ./hack/build.sh, I noticed the capi providers are built concurrently instead of one at a time. This seems a bit faster against a -j 1 or default ./hack/build.sh.

$ time ./hack/build.sh
# providers are built one at a time
real	0m49.057s
user	0m55.159s
sys	0m15.100s

$ time MAKEFLAGS="-j $(nproc)" ./hack/build.sh
# providers are built concurrently
real	0m31.646s
user	1m4.316s
sys	0m15.624s

It's not really clear to me why we should link the number of make jobs to the go -p flag.

🤔 Right, I had the same question in #10364 (comment). I assumed it was syncing the make and go allowed threads. Thinking about it again, I am now unsure too :(

jhixson74 · 2026-03-06T22:03:45Z

Do you see improvements when using this? I'm curious about the results.

go build already uses all cores; the make jobs are running in serial, but each job is a go build so is parallelized across all cores (right?)

when I have tested (along time ago) I didn't see a lot of spare cpu space, except with the compression/zipping part

It's not really clear to me why we should link the number of make jobs to the go -p flag.

We can of course enable different knobs. None of this is on by default though, and is pretty much only useful to those of us working on it. Go uses all cores by default. Make does not. We can optimize this even more if we could get rid of build*.sh files and turn them into makefiles. What if I don't want all cores used? What if go is taking up too many resources on my laptop (hint: it does... all the time, and brings it to a screeching halt). That's why I've kept this little hack around for years. I'm attempting to (once again) bring it in ;-)

Would you like me to decouple the number of make jobs from go build jobs? I'm fine with that. Or should I just close this out?

openshift-ci · 2026-03-06T22:15:26Z

@jhixson74: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 5, 2026

jhixson74 force-pushed the main-parallelize-build branch from b58ef2e to 190b870 Compare March 5, 2026 23:09

openshift-ci bot requested review from bfournie and tthvo March 5, 2026 23:10

jhixson74 mentioned this pull request Mar 5, 2026

CORS-3897: Dual stack support for cluster manifest & VNet #10319

Open

openshift-ci bot requested a review from patrickdillon March 5, 2026 23:14

jhixson74 force-pushed the main-parallelize-build branch 2 times, most recently from cc295b9 to 0e8c398 Compare March 6, 2026 02:15