Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions .github/workflows/commit-convention.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: commit-convention

# Guard the version-bump markers the vendor workflow reads to pick the next
# release's semver bump. Markers are namespaced [bump:patch|minor|major]; a
# typo like [bump:pacth] would otherwise silently ship as a default minor, so
# reject anything malformed before merge. Make this a required status check on
# master so it actually blocks the merge.
on:
pull_request:
types: [opened, edited, reopened, synchronize]

permissions:
contents: read

jobs:
markers:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Validate [bump:*] markers
env:
PR_TITLE: ${{ github.event.pull_request.title }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
# Scan the PR title plus each commit's SUBJECT line (not the body) —
# the marker lives in the subject, like [skip ci], so prose in a body
# that merely documents the markers never trips the check. The
# namespaced bump:<level> form also won't collide with ordinary
# bracket text ([skip ci], markdown links, …).
commits="$(git log --format=%s "${BASE_SHA}..${HEAD_SHA}" 2>/dev/null || true)"
text="$PR_TITLE
$commits"

fail=false

# A candidate marker is a single token [bump:<word>]. Prose/notation
# like [bump:patch|minor|major] or [bump:*] (pipes, stars, spaces)
# isn't a marker attempt — only a malformed SINGLE level is a typo.
bad="$(printf '%s' "$text" \
| grep -oiE '\[bump:[a-z0-9.]+\]' \
| grep -viE '\[bump:(patch|minor|major)\]' || true)"
if [ -n "$bad" ]; then
echo "::error::Unrecognised bump marker(s): $(printf '%s' "$bad" | tr '\n' ' ')"
echo "Valid markers: [bump:patch], [bump:minor], [bump:major]."
fail=true
fi

if $fail; then
echo "See the commit convention in README.md."
exit 1
fi

# Advisory: report the level this PR is asking for.
if printf '%s' "$text" | grep -qi '\[bump:major\]'; then lvl="major"
elif printf '%s' "$text" | grep -qi '\[bump:patch\]'; then lvl="patch (only if every commit in the release range is [bump:patch])"
else lvl="minor (default)"; fi
echo "✓ bump markers valid — this PR requests a **${lvl}** bump" >> "$GITHUB_STEP_SUMMARY"
95 changes: 80 additions & 15 deletions .github/workflows/vendor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,11 @@ name: vendor-toolchain
# when its inputs changed, otherwise its binary is reused from that release.
# Tick `rebuild_all` on a manual run to force a full from-source rebuild.
#
# Run manually from the Actions tab, or by changing the build recipe.
# Run manually from the Actions tab (pick the branch), or by changing the build
# recipe on master or a release/* maintenance branch. A push to release/* cuts a
# back-patch: the version is computed from the highest tag reachable on THAT
# branch, so a fix on release/0.6 (forked at v0.6.0) publishes v0.6.1 even while
# master is already on v0.8 — see "Compute next toolchain version" below.

on:
workflow_dispatch:
Expand All @@ -22,8 +26,14 @@ on:
description: 'Rebuild every tool from source (ignore change detection)'
type: boolean
default: false
flavor:
description: 'Optional variant suffix (e.g. asan, debug, musl). Publishes vX.Y.Z-<flavor> off the latest clean release with NO version bump. Leave empty for a normal release.'
type: string
default: ''
push:
branches: [master]
branches:
- master
- 'release/*'
paths:
- 'build/**'
- '.github/workflows/vendor.yml'
Expand All @@ -32,7 +42,9 @@ permissions:
contents: write

concurrency:
group: vendor-toolchain
# Per-ref: a release/* back-patch never queues behind a master build — each
# cuts its own line's tag, so they can publish in parallel.
group: vendor-toolchain-${{ github.ref }}
cancel-in-progress: false

jobs:
Expand All @@ -56,9 +68,14 @@ jobs:
FORCE: ${{ github.event.inputs.rebuild_all }}
run: |
set -euo pipefail
# Previous published release (highest vX.Y tag), if any.
prev="$(git tag -l 'v[0-9]*.[0-9]*' | sed 's/^v//' \
| sort -t. -k1,1n -k2,2n | tail -1)"
# Previous published release on THIS line: the highest semver tag
# reachable from HEAD. On master that's the global highest; on a
# release/* branch it's that line's latest, so a back-patch reuses the
# right baseline and compares fingerprints against it. Match only
# clean vX.Y[.Z] tags — any suffixed tag (vX.Y.Z-<flavor>) is an
# unordered variant that must never serve as a baseline.
prev="$(git tag -l 'v[0-9]*' --merged HEAD \
| grep -E '^v[0-9]+\.[0-9]+(\.[0-9]+)?$' | sed 's/^v//' | sort -V | tail -1)"
prevtag=""
[ -n "$prev" ] && prevtag="v$prev"
echo "prevtag=$prevtag" >> "$GITHUB_OUTPUT"
Expand Down Expand Up @@ -258,17 +275,62 @@ jobs:
done

- name: Compute next toolchain version
env:
FLAVOR: ${{ github.event.inputs.flavor }}
run: |
set -euo pipefail
# Highest existing vX.Y tag + 0.1 (tenths to avoid float drift);
# first publish is v0.1. Each publish is an immutable release.
latest="$(git tag -l 'v[0-9]*.[0-9]*' | sed 's/^v//' \
| sort -t. -k1,1n -k2,2n | tail -1)"
if [ -z "$latest" ]; then
next="0.1"
# Releases are semver vMAJOR.MINOR.PATCH, one immutable release each.
# The bump is the HIGHEST level any commit since the previous release
# on THIS line (latest..HEAD) asks for, via a marker in its SUBJECT:
# * [bump:major] -> major (X+1.0.0)
# * [bump:minor] -> minor (X.Y+1.0)
# * [bump:patch] -> patch (X.Y.Z+1)
# An UNMARKED commit takes the branch default: minor on master, patch
# on a release/* maintenance branch (where you're almost always
# back-patching, and a minor bump could collide with a mainline tag).
# "latest" is the highest tag REACHABLE from HEAD, so a back-patch on
# release/0.6 (forked at v0.6.0) bumps the 0.6 line, not master's. The
# PR check (commit-convention.yml) rejects malformed markers before
# merge. Pre-semver tags (v0.1..v0.5) read as 0.MINOR.0; the first
# publish ever is v0.1.0.
case "${GITHUB_REF_NAME:-}" in
release/*) default_lvl=1 ;; # maintenance line: unmarked -> patch
*) default_lvl=2 ;; # mainline: unmarked -> minor
esac
# Clean vX.Y[.Z] releases only; suffixed tags (vX.Y.Z-<flavor>) are
# unordered variants and never advance the line.
latest="$(git tag -l 'v[0-9]*' --merged HEAD \
| grep -E '^v[0-9]+\.[0-9]+(\.[0-9]+)?$' | sed 's/^v//' | sort -V | tail -1)"
flavor="${FLAVOR:-}"
if [ -n "$flavor" ]; then
# A flavor build is an unordered VARIANT of an existing release: it
# pins to the latest clean release reachable and appends the suffix,
# WITHOUT bumping the line (so it never becomes a baseline above).
# Dispatch-only; consumers opt in by pinning vX.Y.Z-<flavor>.
echo "$flavor" | grep -qE '^[a-z0-9][a-z0-9.]*$' \
|| { echo "::error::flavor must match ^[a-z0-9][a-z0-9.]*$ (got '$flavor')"; exit 1; }
[ -n "$latest" ] || { echo "::error::no clean vX.Y.Z release to flavor yet"; exit 1; }
next="${latest}-${flavor}"
elif [ -z "$latest" ]; then
next="0.1.0"
else
tenths=$(( $(echo "$latest" | awk -F. '{print $1*10 + $2}') + 1 ))
next="$(( tenths / 10 )).$(( tenths % 10 ))"
maj=$(echo "$latest" | awk -F. '{print $1+0}')
min=$(echo "$latest" | awk -F. '{print $2+0}')
pat=$(echo "$latest" | awk -F. '{print $3+0}') # missing patch -> 0
rank=0 # highest level seen: 1=patch 2=minor 3=major
while IFS= read -r sha; do
msg="$(git log -1 --format=%s "$sha")" # subject line only
if printf '%s' "$msg" | grep -qi '\[bump:major\]'; then lvl=3
elif printf '%s' "$msg" | grep -qi '\[bump:minor\]'; then lvl=2
elif printf '%s' "$msg" | grep -qi '\[bump:patch\]'; then lvl=1
else lvl=$default_lvl; fi
[ "$lvl" -gt "$rank" ] && rank=$lvl
done < <(git rev-list "v${latest}..HEAD")
case "$rank" in
3) next="$((maj + 1)).0.0" ;;
1) next="${maj}.${min}.$((pat + 1))" ;;
*) next="${maj}.$((min + 1)).0" ;; # rank 2, or 0 (empty range)
esac
fi
echo "TC=$next" >> "$GITHUB_ENV"
sed -i "s|^TOOLCHAIN_VERSION=.*|TOOLCHAIN_VERSION=${next}|" build/versions.env
Expand Down Expand Up @@ -332,8 +394,11 @@ jobs:
cp libbpf-headers.tar.gz dist/libbpf-headers.tar.gz
# qemu for the optional kernel-matrix runner (fetched on demand, not by `make`).
for a in x86_64 aarch64; do cp "qemu-$a.tar.gz" "dist/qemu-$a.tar.gz"; done
# Flavored tags (vX.Y.Z-<flavor>) are opt-in variants — mark them
# prerelease so GitHub never surfaces one as "Latest".
prerelease=""; case "$tag" in *-*) prerelease="--prerelease" ;; esac
gh release view "$tag" >/dev/null 2>&1 \
|| gh release create "$tag" --title "$tag" \
|| gh release create "$tag" --title "$tag" $prerelease \
--notes "Static build toolchain $tag — clang/make/git built from source; bpftool/esbuild/libbpf headers re-hosted; qemu for the kernel-matrix runner."
# --clobber so a same-version re-run refreshes assets.
gh release upload "$tag" dist/* --clobber
56 changes: 52 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,24 @@ into a shared per-machine cache.
## What it ships

Each tool is a fully-static binary (no shared-library deps), built or fetched
per arch (`x86_64`, `aarch64`) and published on an immutable, version-tagged
release **`v0.1`, `v0.2`, …**. The tag carries the version, so the assets are
plain-named (`clang-x86_64`, `make-aarch64`, …); a consumer pins one version.
per arch (`x86_64`, `aarch64`) and published on an immutable, semver-tagged
release **`vMAJOR.MINOR.PATCH`** (`v0.6.0`, `v0.6.1`, …). The tag carries the
version, so the assets are plain-named (`clang-x86_64`, `make-aarch64`, …); a
consumer pins one version.

CI picks the bump from the commits since the last release, via a `[bump:LEVEL]`
marker in the commit **subject** (the body is free prose — put the marker on the
subject line, like `[skip ci]`; for squash merges that's the PR title):

| Marker | Bump | Notes |
| --- | --- | --- |
| *(none)* or `[bump:minor]` | **minor** `X.Y+1.0` | the default — a normal push |
| `[bump:patch]` | **patch** `X.Y.Z+1` | only if **every** commit in the release range is `[bump:patch]` |
| `[bump:major]` | **major** `X+1.0.0` | one such commit makes the whole release major |

The release takes the highest level any commit asks for. A PR check
(`commit-convention.yml`) rejects malformed markers (e.g. `[bump:pacth]`)
before merge, so a typo can't silently mis-version a release.

| tool | for | source |
|-----------|----------------------------------------------------|--------|
Expand Down Expand Up @@ -67,11 +82,44 @@ Change a tool pin in [`build/versions.env`](build/versions.env) and push — the
[`vendor-toolchain`](.github/workflows/vendor.yml) workflow rebuilds clang/make/
git (and the test-runner qemu) on native x86_64 and arm64 runners, re-hosts
bpftool/veristat/lvh/esbuild/headers,
**computes the next `vX.Y`** (highest existing + 0.1), publishes all assets to
**computes the next semver tag** (highest existing, bumped by the level the
commit messages ask for — minor by default), publishes all assets to
that immutable release, and records the version + checksums into `versions.env`.
The version bumps only when this repo changes, so consumers re-fetch only on a
real toolchain change.

### Back-patching an older line

To ship a fix on an older release while `master` has moved on, branch the line
from its tag and push fixes there:

```
git branch release/v0.6 v0.6.0 && git push origin release/v0.6
# PR the fix into release/v0.6, then merge
```

A push to `release/*` runs the same workflow, but the version is computed from
the highest tag **reachable on that branch** — so a fix on `release/v0.6`
(forked at `v0.6.0`) publishes `v0.6.1`, independent of whatever `master` is on.
On `release/*` an unmarked commit defaults to a **patch** bump (not minor), so a
hotfix can't accidentally collide with a mainline minor tag; use `[bump:minor]`
/`[bump:major]` to override. The branch name is convention only — `release/vX.Y`
(the minor line) is the natural granularity since patches walk it forward.

### Flavored variants

A **flavor** is an opt-in variant of an existing release, tagged
`vX.Y.Z-<flavor>` (e.g. `v0.6.0-asan`). Run the workflow manually from the
Actions tab with the **`flavor`** input set: it pins to the latest clean
release reachable, appends the suffix, and publishes a separate (prerelease)
release **without bumping the version line**. Flavored tags are unordered — the
version computation matches only clean `vX.Y[.Z]` tags, so a flavor never
becomes a baseline or shifts mainline. Consumers stay on clean versions by
default and opt in by pinning `TOOLCHAIN_VERSION=X.Y.Z-<flavor>`.

(The suffix is a label only — producing genuinely different binaries for a
flavor, e.g. via extra build-args, is wired up per flavor when defined.)

To avoid burning an hour rebuilding clang on every push, the workflow only
rebuilds a from-source tool when its inputs changed: a `detect` step
fingerprints each tool's Dockerfile plus the version vars it consumes against
Expand Down
Loading