build-ci v3: Update Infra For spack v1

[CodeGat]

References issue ACCESS-NRI/build-ci#231 and PR ACCESS-NRI/build-ci#253
References project Spack `0.22` to `1.0` Migration

Important

This PR is a major update to the infrastructure. See below for the prerequisites for this repository to be able to merge this PR.

Important

This major version change marks the end of major infrastructure updates for CI using spack < 1.0. They will still get bug fixes and non-breaking features if required.
If you want to deploy to instances of spack < 1.0, you must use build-ci v2.
If you want to deploy to instances of spack >= 1.0, you must use build-ci v3.

Background

We are looking to transition build-ci to v3! This is due to a migration to spack v1.X, from v0.X, which offers many bug fixes, features, and optimisations.

One of these changes is the splitting of spacks core codebase from the builtin spack-packages repository, which means that we have another lever to tweak for our builds: the version of spack, spack-config, our spack-packages and the (new!) builtin spack-packages.

Since we want to leave open the possibility of forking the builtin spack-packages, we have also decided to rename ACCESS-NRI/spack-packages to ACCESS-NRI/access-spack-packages. See ACCESS-NRI/access-spack-packages#295 for more info.

Therefore this is a major version update, as there are the following changes to the inputs (and analogous changes to the workflow outputs):

• Deleted the optional spack-packages-ref input, which defaulted to main (in future, api-v1).
• Added the optional access-spack-packages-ref input, which defaults to the api-v2 branch.
• Added the optional builtin-spack-packages-ref input, which defaults to the main branch.

This means that:

• If you want to use container-image-version: :rocky-0.22-*, or spack-ref < releases/1.0, you must stay on v2 due to the input changes.
• If you want to use container-image-version: :rocky-1.*-*, or spack-ref >= releases/1.0, you must migrate to v3 due to the input changes.

It is possible to test across these versions of spack if required, via a mix of workflow versions. Ask @CodeGat if you need this functionality.

Features

The main new features include:

• Proper Spack v1 Support: Users now can tweak both the builtin spack-packages repository, as well as our own access-spack-packages repository. All of the bugfixes, features and optimisations added since 0.22 are available to us for future inclusion into build-ci.

Prerequisites for Merging

• Update build-ci entrypoints to v3 (this PR!)
• Determine if a specific builtin-spack-packages-ref is required
• Determine if the added access-spack-packages-ref is appropriate
• Validate if maintainers want to move to spack v1 over spack v0.22

[dougiesquire]

The reason this is failing is because it is concretizing to build_system=makefile which doesn't support GCC (because Intel flags are just hardcoded in the SPR):

==> Concretized 1 spec:
 -   ph4oyj4  access-mocsy@git.450839c0ebf57e912c25c0cf7c63432d36fec552=stable build_system=makefile commit=450839c0ebf57e912c25c0cf7c63432d36fec552 platform=linux os=rocky8 target=x86_64 %c,fortran=gcc@13.2.0

But I don't understand why it is trying to use build_system=makefile here. The default is build_system=cmake and that is being used correctly with Spack v1 elsewhere (e.g. see this access-generic-tracers build from today). It also used to be used correctly by the build-ci here before this update.

If I try locally with my own Spack v1.1 instance and a copy of the failing spack.yaml, it concretizes to build_system=cmake as expected. I.e.:

spack:
  specs:
  # *_compiler is defined in the .github/build-ci/data/standard.json file
  - access-mocsy@git.450839c0ebf57e912c25c0cf7c63432d36fec552=stable
  packages:
    gcc:
      require:
      - '@13.2.0'
    all:
      require:
      - '%access_gcc'
      - 'target=x86_64'
  concretizer:
    unify: false
  view: false

concretizes as:

==> Starting concretization
==> Concretized 1 spec:
 -   7rql3jw  access-mocsy@git.450839c0ebf57e912c25c0cf7c63432d36fec552=stable~ipo~shared build_system=cmake build_type=RelWithDebInfo commit=450839c0ebf57e912c25c0cf7c63432d36fec552 generator=make precision=2 platform=linux os=rocky8 target=x86_64 %c,fortran=gcc@13.2.0
[e]  c6epq3u      ^cmake@3.31.6~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release platform=linux os=rocky8 target=x86_64
 -   spro2dg      ^compiler-wrapper@1.0 build_system=generic platform=linux os=rocky8 target=x86_64
[e]  4tmoi6d      ^gcc@13.2.0+binutils+bootstrap~graphite+libsanitizer~mold~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' platform=linux os=rocky8 target=x86_64
 -   kx2mogc      ^gcc-runtime@13.2.0 build_system=generic platform=linux os=rocky8 target=x86_64
[e]  vuczjrb      ^glibc@2.28 build_system=autotools platform=linux os=rocky8 target=x86_64
 -   bfssup2      ^gmake@4.4.1~guile build_system=generic platform=linux os=rocky8 target=x86_64 %c=gcc@13.2.0
[e]  q3qnxmz      ^openmpi@4.1.7+atomics~cuda~cxx~cxx_exceptions~debug+fortran~gpfs~internal-hwloc~internal-libevent~internal-pmix~ipv6~java~lustre~memchecker~openshmem~orterunprefix~rocm+romio+rsh~static~two_level_namespace+vt+wrapper-rpath build_system=autotools fabrics:=none romio-filesystem:=none schedulers:=none platform=linux os=rocky8 target=x86_64

What am I missing?

[dougiesquire]

Note the intel and oneapi builds are also incorrectly using build_system=makefile, they just don't fail because the hardcoded flags are valid for these compilers