ScaFFold Benchmark and spack-pip Package Manager

[michaelmckinsey1]

Description

This PR is blocked by:

• Application needs to be public https://github.com/LBANN/ScaFFold
• Depends on Adds Python bindings to Adiak Adiak#38
  • upstream the adiak package adiak: Update Package for v0.5.0 Release spack/spack-packages#2300
• (not mandatory) Needing to set MPICH_GPU_SUPPORT_ENABLED=0 on coral2 otherwise GTL linker error
• (ignore using roctx for now) -> (not mandatory) ScaFFold+rocm caliper=rocm hangs (for rocprof)
  • Test spack package with rocprof on another benchmark (we were using roctracer before so maybe it is this)
  • run debugger

Summary:

• Add spack-pip package manager
  • requires updating ramble tag to dev once Add support for multiple package-manager experiments GoogleCloudPlatform/ramble#1248 merged
  • Add infrastructure for new spack-pip mgr to benchpark
• Add py-scaffold python benchmark, which has Caliper annotations via python bindings.
• supersedes [WIP] ScaFFold Benchmark #687
• Update spack-packages tag to get updated version of pybind11 package.

Reproducers:

# Tuolumne profile rocm kernels and mpi functions
$ rm -rf tuolumne/ && benchpark system init --dest tuolumne llnl-elcapitan cluster=tuolumne && rm -rf py-scaffold-sp && benchpark experiment init --dest py-scaffold-sp      --system tuolumne py-scaffold+rocm scaffold_path=/usr/WS1/mckinsey/ScaFFold-spack/      package_manager=spack-pip caliper=mpi,time,rocm
$ rm -rf wkp && benchpark setup py-scaffold-sp wkp && . wkp/setup.sh && . wkp/.latest-experiment.sh 

# Matrix profile cuda kernels and mpi functions
$ rm -rf matrix/ && benchpark system init --dest matrix llnl-matrix && rm -rf py-scaffold-sp-mat/ && benchpark experiment init --dest py-scaffold-sp-mat      --system matrix py-scaffold+cuda scaffold_path=/usr/WS1/mckinsey/ScaFFold-spack/      package_manager=spack-pip caliper=mpi,time,cuda
$ rm -rf wkp2 && benchpark setup py-scaffold-sp-mat wkp2 && . wkp2/setup.sh  && . wkp2/.latest-experiment.sh 

Adding/modifying a system (docs: Adding a System)

• Update llnl-elcapitan and llnl-matrix

Adding/modifying a benchmark (docs: Adding a Benchmark)

• Add experiment, application, and package for py-scaffold

Adding/modifying core functionality, CI, or documentation:

• (optional) Gitlab tests - Cannot add unit tests for this benchmark until it is made public