flowey: add --use-local-deps option to build-igvm, add local request variant to download_openvmm_deps#2388
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for using locally-downloaded dependencies in the flowey build system as a step toward reproducible builds. The main change renames download_openvmm_deps to resolve_openvmm_deps and adds a LocalPath request variant alongside the existing Version variant. A new --use-local-deps CLI option is added to the build-igvm pipeline for local testing.
Key changes:
- Renamed
download_openvmm_depsmodule toresolve_openvmm_depsto reflect its dual purpose (download or resolve local paths) - Added
LocalPathrequest variant with logic to resolve dependencies from local x64/aarch64 subdirectories - Added
--use-local-depsand--custom-openvmm-depsCLI options to build-igvm pipeline for testing with local dependencies
Reviewed Changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| flowey/flowey_lib_hvlite/src/resolve_openvmm_deps.rs | Adds LocalPath request variant and implementation to resolve dependencies from local filesystem paths with x64/aarch64 subdirectory structure |
| flowey/flowey_lib_hvlite/src/lib.rs | Updates module declaration from download_openvmm_deps to resolve_openvmm_deps |
| flowey/flowey_lib_hvlite/src/init_vmm_tests_env.rs | Updates import and request references to use resolve_openvmm_deps |
| flowey/flowey_lib_hvlite/src/init_openvmm_magicpath_openhcl_sysroot.rs | Updates import and request references to use resolve_openvmm_deps |
| flowey/flowey_lib_hvlite/src/init_openvmm_magicpath_linux_test_kernel.rs | Updates import and request references to use resolve_openvmm_deps |
| flowey/flowey_lib_hvlite/src/build_openhcl_initrd.rs | Updates import and request references to use resolve_openvmm_deps |
| flowey/flowey_lib_hvlite/src/build_openhcl_igvm_from_recipe.rs | Updates import and request references to use resolve_openvmm_deps |
| flowey/flowey_lib_hvlite/src/_jobs/cfg_versions.rs | Adds LocalDependencyRequest struct and Local/Download request variants; routes local dependency requests to resolve_openvmm_deps |
| flowey/flowey_lib_hvlite/src/_jobs/cfg_common.rs | Updates import to use resolve_openvmm_deps |
| flowey/flowey_hvlite/src/pipelines/vmm_tests.rs | Changes cfg_versions request from struct to Download enum variant |
| flowey/flowey_hvlite/src/pipelines/restore_packages.rs | Changes cfg_versions request from struct to Download enum variant |
| flowey/flowey_hvlite/src/pipelines/mod.rs | Updates clippy lint attribute from cfg_attr to unconditional expect |
| flowey/flowey_hvlite/src/pipelines/custom_vmfirmwareigvm_dll.rs | Changes cfg_versions request from struct to Download enum variant |
| flowey/flowey_hvlite/src/pipelines/checkin_gates.rs | Changes cfg_versions request from struct to Download enum variant |
| flowey/flowey_hvlite/src/pipelines/build_igvm.rs | Adds --use-local-deps and --custom-openvmm-deps CLI parameters; conditionally uses Local or Download cfg_versions request |
| flowey/flowey_hvlite/src/pipelines/build_docs.rs | Changes cfg_versions request from struct to Download enum variant |
| .github/workflows/openvmm-pr.yaml | Updates generated workflow step names from download_openvmm_deps to resolve_openvmm_deps |
| .github/workflows/openvmm-pr-release.yaml | Updates generated workflow step names from download_openvmm_deps to resolve_openvmm_deps |
| .github/workflows/openvmm-ci.yaml | Updates generated workflow step names from download_openvmm_deps to resolve_openvmm_deps |
Comments suppressed due to low confidence (2)
flowey/flowey_lib_hvlite/src/resolve_openvmm_deps.rs:71
- The error message refers to "Path requests" but the actual request variant is named
LocalPath. Consider updating the error message for consistency: "Cannot specify both Version and LocalPath requests".
flowey/flowey_lib_hvlite/src/resolve_openvmm_deps.rs:75 - The error message refers to "Path request" but the actual request variant is named
LocalPath. Consider updating the error message for consistency: "Must specify a Version or LocalPath request".
88bce71 to
ab5ffa1
Compare
3f8b4a8 to
2c67f00
Compare
2c67f00 to
ac020bd
Compare
c6f14fd to
1a3d0b1
Compare
| pub const PROTOC: &str = "27.1"; | ||
|
|
||
| #[derive(Serialize, Deserialize, Clone, Debug, PartialEq, Eq, PartialOrd, Ord)] | ||
| pub struct LocalDependencyRequest { |
There was a problem hiding this comment.
Would it be easier/cleaner for this type to take a CommonArch and a PathBuf, instead of these two options? Then you just check that for every arch that shows up the path is always the same, but you wouldn't have to do the option dance in build_igvm.
There was a problem hiding this comment.
Or maybe an OpenvmmDepsArch instead of a CommonArch
| }, | ||
| job = if use_local_deps { | ||
| let openvmm_deps_path = custom_openvmm_deps | ||
| .clone() |
There was a problem hiding this comment.
nit: i don't think you need to clone here, and the expect can just be an unwrap since clap guarantees this'll be here
1a3d0b1 to
3ab8c8c
Compare
3ab8c8c to
9a7234f
Compare
9a7234f to
6075c72
Compare
6075c72 to
edce9f0
Compare
Similar to #2388, adds the ability to override the protoc path with a local path. This should work independently of the other arguments but also with them. Tested all combinations of arguments specified and all work as expected. This is another step towards reproducible builds where we're adding the capability to use "local" versions of everything that will eventually be supplied by nix.
In order to get to reproducible builds, we're going to need all nodes that currently download something at build time to allow for Nix to download them instead and then specify the path at which they've been downloaded. There are several nodes that don't currently have Local override request variants. I've started with
download_openvmm_deps(renamed toresolve_openvmm_deps, but have also plumbed a--use-local-depscommand line parameter to build-igvm for testing locally.