Publish individual bins for Linux-arm64 with regression test for cross-build #10606
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -265,6 +265,8 @@ jobs: | |||||
| build_makefile: | ||||||
| name: Build/Makefile | ||||||
| needs: [ min_version, deps ] | ||||||
| permissions: | ||||||
| contents: write # Publish individual arm binaries instead of discarding | ||||||
| runs-on: ${{ matrix.job.os }} | ||||||
| env: | ||||||
| SCCACHE_GHA_ENABLED: "true" | ||||||
|
|
@@ -297,10 +299,6 @@ jobs: | |||||
| if [[ -d target ]]; then | ||||||
| mv -T target target.cache | ||||||
| fi | ||||||
| # build (host) | ||||||
| make build | ||||||
| echo "Check that target directory will be ignored by backup tools" | ||||||
|
|
@@ -322,37 +320,60 @@ jobs: | |||||
| disable_search: true | ||||||
| flags: makefile,${{ matrix.job.os }} | ||||||
| fail_ci_if_error: false | ||||||
| - name: "`make install PROG_PREFIX=uu- CARGO_BUILD_TARGET=aarch64-unknown-linux-gnu PROFILE=release COMPLETIONS=n MANPAGES=n LOCALES=n`" | ||||||
| shell: bash | ||||||
| run: | | ||||||
| set -x | ||||||
| sudo apt-get install -y gcc-aarch64-linux-gnu | ||||||
| RUSTFLAGS="${RUSTFLAGS} -C strip=symbols" | ||||||
|
Collaborator
There was a problem hiding this comment.
Suggested change
Contributor
Author
There was a problem hiding this comment. shellcheck complains if we don't split |
||||||
| export RUSTFLAGS | ||||||
|
Collaborator
There was a problem hiding this comment.
Suggested change
|
||||||
| # Verify uudoc isn't cross-compiled by checking man pages were generated successfully | ||||||
| # (cross-compiled uudoc couldn't execute on host, so man pages would be missing) | ||||||
| env CARGO_BUILD_TARGET=aarch64-unknown-linux-gnu make install-manpages PREFIX=/tmp/usr UTILS=true | ||||||
| test -f /tmp/usr/share/man/man1/true.1 | ||||||
| DESTDIR=/tmp/ make install CARGO_BUILD_TARGET=aarch64-unknown-linux-gnu PROG_PREFIX=uu- PROFILE=release COMPLETIONS=n MANPAGES=n LOCALES=n | ||||||
| # We don't build coreutils without MULTICALL=y | ||||||
| ! test -e target/aarch64-unknown-linux-gnu/release/coreutils | ||||||
| # Check that the progs are missing prefix | ||||||
| test -f /tmp/usr/local/bin/tty | ||||||
| # Check that the manpage is not present | ||||||
| ! test -f /tmp/usr/local/share/man/man1/whoami.1 | ||||||
| # Check that the completion is not present | ||||||
| ! test -f /tmp/usr/local/share/zsh/site-functions/_install | ||||||
| ! test -f /tmp/usr/local/share/bash-completion/completions/head.bash | ||||||
| ! test -f /tmp/usr/local/share/fish/vendor_completions.d/cat.fish | ||||||
| # We publish them instead of discarding | ||||||
|
Collaborator
There was a problem hiding this comment. I think that in order to change what gets released it would be good to have approval from @sylvestre . Can you please open an issue regarding the addition of individual arm binaries to the releases to discuss this?
Contributor
Author
There was a problem hiding this comment. Actually, I just don't want to discard bins with release build (not interested in arm).
Collaborator
There was a problem hiding this comment. As far as I know, 99,999% of users will get uutils coreutils using their package-manager, and the distros like freebsd will use the source code of uutils coreutils and compile it themselves and not use the binary packages. So I am not sure it is useful to publish individual binaries for arm or freebsd? What is the use case? That is why I suggested to create an issue to discuss it.
Contributor
Author
There was a problem hiding this comment. Package manager does not provide binaries from main branch and most distributor provides single binary. |
||||||
| # This is a hack to reuse CI resources, but we should cleanup this job someday | ||||||
| mv /tmp/usr/local/libexec/coreutils/libstdbuf.* -t /tmp/usr/local/bin | ||||||
| ZSTD_CLEVEL=19 tar --zstd -caf individual-aarch64-unknown-linux-gnu.tar.zst -C /tmp/usr/local bin | ||||||
| env: | ||||||
| RUST_BACKTRACE: "1" | ||||||
| - name: Publish | ||||||
| uses: softprops/action-gh-release@v2 | ||||||
| if: github.event_name == 'push' && github.ref == 'refs/heads/main' | ||||||
| with: | ||||||
| tag_name: latest-commit | ||||||
| draft: false | ||||||
| prerelease: true | ||||||
| files: | | ||||||
| individual-aarch64-unknown-linux-gnu.tar.zst | ||||||
| env: | ||||||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||||||
| - name: "`make PROFILE=release-small PROG_PREFIX=uu- install`" | ||||||
| shell: bash | ||||||
| run: | | ||||||
| set -x | ||||||
| DESTDIR=/tmp/ make PROFILE=release-small PROG_PREFIX=uu- install | ||||||
| # Check PROFILE | ||||||
| target/release-small/uu-true | ||||||
| # Check that the utils are prefixed | ||||||
| test -f /tmp/usr/local/bin/uu-tty | ||||||
| # Check that the manpage is present | ||||||
| test -f /tmp/usr/local/share/man/man1/uu-md5sum.1 | ||||||
| # Check that the completion is present | ||||||
| test -f /tmp/usr/local/share/zsh/site-functions/_uu-b2sum | ||||||
| test -f /tmp/usr/local/share/bash-completion/completions/uu-head.bash | ||||||
| test -f /tmp/usr/local/share/fish/vendor_completions.d/uu-cat.fish | ||||||
| env: | ||||||
| RUST_BACKTRACE: "1" | ||||||
| - name: "`make uninstall`" | ||||||
|
|
@@ -361,18 +382,19 @@ jobs: | |||||
| set -x | ||||||
| DESTDIR=/tmp/ make uninstall | ||||||
| # Check that the utils are not present | ||||||
| ! test -f /tmp/usr/local/bin/u_tty | ||||||
| # Check that the manpage is not present | ||||||
| ! test -f /tmp/usr/local/share/man/man1/u_whoami.1 | ||||||
| # Check that the completion is not present | ||||||
| ! test -f /tmp/usr/local/share/zsh/site-functions/_u_install | ||||||
| ! test -f /tmp/usr/local/share/bash-completion/completions/u_head.bash | ||||||
| ! test -f /tmp/usr/local/share/fish/vendor_completions.d/u_cat.fish | ||||||
| - name: "`make install PROFILE=release-small MULTICALL=n`" | ||||||
| shell: bash | ||||||
| run: | | ||||||
| set -x | ||||||
| # cargo should not rebuild with same profile | ||||||
| timeout 30 DESTDIR=/tmp/ make PROFILE=release-small MULTICALL=n install | ||||||
| # Check that *sum are present | ||||||
| for s in {md5,b2,sha1,sha224,sha256,sha384,sha512}sum | ||||||
| do test -e /tmp/usr/local/bin/${s} | ||||||
|
|
@@ -381,7 +403,9 @@ jobs: | |||||
| shell: bash | ||||||
| run: | | ||||||
| set -x | ||||||
| DESTDIR=/tmp/ make MULTICALL=y LN="ln -svf" install | ||||||
| # release should be default profile | ||||||
| test -e target/release/coreutils | ||||||
| # Check that symlinks of *sum are present | ||||||
| for s in {md5,b2,sha1,sha224,sha256,sha384,sha512}sum | ||||||
| do test $(readlink /tmp/usr/local/bin/${s}) = coreutils | ||||||
|
|
||||||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As far as I understand the "build_makefile" jobs are test jobs. The "build" job at line 627 already has "permissions: contents: write" and already has one line for aarch64-unknown-linux-gnu.
Are you trying to publish files from the build_makefile job instead of from the build job so that you can get libstdbuf.so compiled with feat_external_stdbuf? Or why not use the build job?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should it not rather be something like this in the publish job around line 534?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
x64 individual bins are published by reusing
compute_size:.libstdbuf.sois also one of a reason to publish from here.Ofcause I'm considering to merge jobs for publishing (and deduplicating release build). But currently, I can't.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(We should stop/merge duplicated build for
compute_size:)