artifact-uboot: opt-in cache-bust via UBOOT_HASH_EXTRA for prebuilt blobs#9995
Conversation
📝 WalkthroughWalkthroughIntroduces ChangesU-Boot artifact cache-bust with UBOOT_HASH_EXTRA
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
lib/functions/artifacts/artifact-uboot.sh (1)
133-135: 💤 Low valueConsider deduplicating file paths before hashing.
The tokenization splits
UBOOT_TARGET_MAPon both;and:, treating all tokens equally. If the same file path appears in multiple roles (e.g., source and destination inblob.bin:blob.bin), it will be added to the array multiple times and hashed multiple times. This makes the hash depend on multiplicity rather than just content.While this still achieves cache invalidation when content changes, it could cause unnecessary rebuilds if the target map is refactored to remove duplicate references. In practice, prebuilt blobs are likely referenced once, so the impact is minimal.
For robustness, consider deduplicating the array:
Optional improvement for deduplication
declare _tok +declare -A seen_files=() for _tok in ${UBOOT_TARGET_MAP//[;:]/ }; do - [[ -f "${_tok}" ]] && uboot_blob_files+=("${_tok}") + if [[ -f "${_tok}" && -z "${seen_files[${_tok}]}" ]]; then + uboot_blob_files+=("${_tok}") + seen_files["${_tok}"]=1 + fi done🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/functions/artifacts/artifact-uboot.sh` around lines 133 - 135, The loop that processes UBOOT_TARGET_MAP and populates the uboot_blob_files array does not deduplicate file paths, so identical files appearing multiple times in the target map (e.g., "blob.bin:blob.bin") will be added to the array multiple times and hashed multiple times. To fix this, deduplicate the uboot_blob_files array after the loop completes by using a bash associative array or by checking if each file path already exists in the array before adding it. This ensures each unique file path is only hashed once, preventing unnecessary rebuilds when the target map is refactored to remove duplicate references.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@lib/functions/artifacts/artifact-uboot.sh`:
- Around line 133-135: The loop that processes UBOOT_TARGET_MAP and populates
the uboot_blob_files array does not deduplicate file paths, so identical files
appearing multiple times in the target map (e.g., "blob.bin:blob.bin") will be
added to the array multiple times and hashed multiple times. To fix this,
deduplicate the uboot_blob_files array after the loop completes by using a bash
associative array or by checking if each file path already exists in the array
before adding it. This ensures each unique file path is only hashed once,
preventing unnecessary rebuilds when the target map is refactored to remove
duplicate references.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 638c82df-da10-4dfd-8304-50907de40967
📒 Files selected for processing (1)
lib/functions/artifacts/artifact-uboot.sh
|
@rpardini Do we have a better way? |
I'd do it "out of band". Maybe introduce a new array var, eg |
|
Also: this would never work in a json/matrix prepare -- the blobs simply won't be there at that stage. (maybe for this vim1/vim2 case, those blobs are in armbian/build, it might work by accident. but won't work for eg rkbin etc). |
…lobs The u-boot artifact version hashed the UBOOT_TARGET_MAP string but not the identity of any prebuilt blob it points at. Bumping a blob in place (same path) left the version unchanged, so the build reused the cached .deb and the new blob never got packaged (the khadas-vim1/vim2 0.17.3 bump, #9994, needs this). Parsing UBOOT_TARGET_MAP for filenames is fragile (it can't tell build inputs from outputs), and content-hashing the file doesn't work in general: a u-boot blob may be fetched at build time and not live in this repo, so it doesn't exist at matrix-prep when prepare_version runs — reading it would fail or be non-deterministic. Instead add an opt-in string UBOOT_HASH_EXTRA, declared + documented centrally in main-config.sh. A board/family that ships a prebuilt or external blob sets it (typically to the blob's upstream version) and bumps it on every blob change; it folds into the artifact version's -V hash. Appended ONLY when set, so boards that don't use it keep their exact previous version (no churn). khadas-vim1/vim2 opt in. Direction per @rpardini review: handle it out-of-band, don't parse UBOOT_TARGET_MAP. Signed-off-by: Igor Pecovnik <igor@armbian.com>
45d8786 to
e38f9dc
Compare
|
Then perhaps rather this way? |
Yes, that can work. |
|
✅ This PR has been reviewed and approved — all set for merge! |
Summary
The u-boot artifact version hashed the
UBOOT_TARGET_MAPstring but not the identity of any prebuilt blob it points at. Bumping a blob in place (same path) left the artifact version unchanged, so the build reused the cached.deband the new blob never got packaged.This adds an opt-in version string
UBOOT_HASH_EXTRAthat a board/family sets when it ships a prebuilt or externally-fetched u-boot blob, and bumps whenever the blob changes. It folds into the artifact version's-Vhash, forcing a repackage.Why a string, not a file content-hash
The first cut parsed
UBOOT_TARGET_MAPfor filenames and hashed the bytes. Two problems killed that (per review):prepare_versionruns. Reading it would fail or be non-deterministic across hosts.A string sidesteps both: no filesystem access at matrix-prep, and it works for any blob regardless of where it lives.
Behaviour
lib/functions/configuration/main-config.sh—UBOOT_HASH_EXTRAdeclared + documented centrally (with the other u-boot defaults);:-so it never clobbers a board value.lib/functions/artifacts/artifact-uboot.sh— appended tovars_to_hashonly when set.config/sources/families/meson-gxl.conf— khadas-vim1 / khadas-vim2 opt in.No churn: with
UBOOT_HASH_EXTRAunset,-Vis byte-identical to the pre-change baseline, so every other board's u-boot artifact is untouched. Set → distinct hash; bumping0.11→0.17.3→ distinct again → repackage.Note for the blob bump (#9994)
This is the manual-but-universal trade-off: the blob-bump PR (#9994) must also bump
UBOOT_HASH_EXTRA(→khadas-uboot-0.17.3) when it swaps the blob, or the cache won't invalidate.Split out from #9994, which depends on this to actually repackage the new blobs.