hms-build-image-workflows

Table of contents

• hms-build-image-workflows
  • Build and release image workflow
    • Workflow Inputs
    • Workflow secrets
    • Build and release job
    • Update PR with comment job
    • Full example
      • Workflow to build image with standard Dockerfile name and build context.
      • Workflow to build image with non-standard Dockerfile name
      • Workflow to build image with non-standard build context
  • Run unit test workflow
    • Workflow inputs
    • Workflow secrets
    • Full example
  • Run integration test workflow
    • Workflow inputs
    • Workflow secrets
    • Full example
  • Run ct test workflow
    • Workflow inputs
    • Workflow secrets
    • Full example
  • Release model

The hms-build-image-workflows repository contains Github Action workflows to facilitate the process of building container images and running unit and integration tests from HMS repositories.

Build and release image workflow

The build and release image workflow located at .github/workflows/build_and_release_image.yaml in this repository will build container image from a Dockerfile and publish and sign the artifact in Artifactory. Stable artifacts are produced when a commit is tagged. For all other pushes or PRs an unstable artifact is produced.

The workflow is composed to two jobs.

1. The Build and release job publishes and signs artifacts in Artifactory.
2. The Update PR with comment job will run if the workflow was triggered from PR event, and will create or update a comment on the PR with links to artifacts produced by the workflow run.

Repository requirements:

• The application Dockerfile is located in the root directory of the repository.
• If enable-pr-comment is set to True, then it is expected that PR comment template is located at .github/build-image-comment-template.md within the repo.

Workflow Inputs

Name	Data Type	Required Field	Default value	Description
runs-on	string	Optional	ubuntu-latest	The type of machine to run the job on.
image-name	string	Required		Container image name. For example, cray-firmware-action
artifactory-repo	string	Optional	csm-docker	Artifactory repository to publish to.
docker-registry	string	Optional	artifactory.algol60.net	Registry to publish container images to.
docker-build-context	string	Optional	.	Build's context is the set of files located in the specified PATH.
docker-build-file	string	Optional	""	Path to the Dockerfile. If set to the empty string it will default to {docker-build-context}/Dockerfile.
enable-latest-tag	string	Optional	False	Enable the latest tag for stable builds. Choose from true or false
snyk-severity	string	Optional	high	Only report vulnerabilities of provided level or higher. Choose from: low, medium, high, or critical
trivy-enable	string	Optional	False	Enable or disable the Trivy Vulnerability scanner. Choose from true or false
trivy-exit-code	string	Optional	0	Exit code when vulnerabilities were found
trivy-severity	string	Optional	CRITICAL,HIGH	Severities of vulnerabilities to be displayed
enable-pr-comment	string	Optional	True	Control whether the update-pr-with-artifacts job runs on PR builds. Choose from true or false

Workflow secrets

Name	Required Field	Description
SNYK_TOKEN	Required	Snyk authorization token.
ARTIFACTORY_ALGOL60_USERNAME	Required	Artifactory username used to pull and push container images.
ARTIFACTORY_ALGOL60_TOKEN	Required	Artifactory token used to pull and push container images.
COSIGN_GCP_PROJECT_ID	Required	Project ID in GCP for cosign
COSIGN_GCP_SA_KEY	Required	Service account key in GCP for cosign
COSIGN_KEY	Required	Cosign key

Build and release job

The build and release job performs the following high level actions:

1. Uses Docker to build the Dockerfile
2. Run Snyk (and Trivy if enabled) against the built container image to find vulnerabilities.
3. Push container image into correct location within Artifactory (either to stable or unstable)
4. Create Software Bill of Materials (SBOM) using Syft.

The build and release job is composed of various 3rd party and internally developed Github Actions.

• Internally developed Github Actions:

  • Cray-HPE/hms-build-metadata-action/generate-build-metadata@v1
  • Cray-HPE/.github/actions/csm-sign-image@v2.0-csm-sign-image
  • Cray-HPE/.github/actions/csm-generate-attach-sign-sbom@v1.0-csm-generate-attach-sign-sbom

• 3rd party Github Actions:

  • actions/checkout@v3
  • aquasecurity/trivy-action@master
  • docker/build-push-action@v3
  • docker/login-action@v2
  • docker/metadata-action@v4
  • docker/setup-buildx-action@v2
  • docker/setup-qemu-action@v2
  • snyk/actions/docker@master

Update PR with comment job

The update PR with comment job only runs if the build and release workflow was triggered from a PR (include draft PRs), otherwise it is skipped. The purpose of this job is to make it easy to easily identify artifacts that were produced from the PR build run. With information to easily:

• Identify the image location with Artifactory.
• Steps to load the container images easily onto development systems.
• Retrieve the generated SBOM in SPDX format of the built container image.

The PR comment Markdown template is expected to exist at .github/build-image-comment-template.md inside of the Git repository calling this workflow. This template uses the Golang templating language (note the hmtl/template package is being used to render the template instead of text/template but the documentation for text/template is still applicable).

The following table shows what template variables are currently exposed by this job when a comment template is rendered:

Variable	Example
artifactoryRepo	csm-docker
image	artifactory.algol60.net/csm-docker/unstable/cray-power-control:0.0.1-20220204191303.a2980b2
imageTag	0.0.1-20220204191303.a2980b2
imageRepository	artifactory.algol60.net/csm-docker/stable/cray-power-control
imageName	cray-power-control
stableString	stable or unstable
PRHeadSha	a2980b2

The update PR with comment job is composed of mostly 3rd part Github Actions

• 3rd party Github Actions:
  • actions/checkout@v2
  • chuhlomin/render-template@v1.2
  • peter-evans/find-comment@v1
  • peter-evans/create-or-update-comment@v1

Full example

Workflow to build image with standard Dockerfile name and build context.

Sample build and publish docker image workflow (.github/workflows/build_and_release_image.yaml) in use by the hms-power-control repository. This assumes the Dockerfile is in the root of the repository, with the name Dockerfile.

IMPORTANT when creating or migrating a repository to use this workflow make sure that the value for .jobs.build_and_release.with.image-name is set to the desired image name for the container registry.

name: Build and Publish Docker Image # Consider changing the workflow name to make it distinguishable from other flows.
on:
  - push # Perform a build of the contents from the branch
  - pull_request # Perform a build after merging with the target branch
  - workflow_dispatch
jobs:
  build_and_release:
    uses: Cray-HPE/hms-build-image-workflows/.github/workflows/build_and_release_image.yaml@v1
    with:
      image-name: cray-power-control # This is the only field that needs to be changed
      enable-pr-comment: true
    secrets: inherit

Workflow to build image with non-standard Dockerfile name

Sample build and publish docker image workflow to build a Dockerfile with a non-standard name. The example below will use the Dockerfile named Dockerfile.hms-pytest inside of the root of the repository.

IMPORTANT when creating or migrating a repository to use this workflow make sure that the value for .jobs.build_and_release.with.image-name is set to the desired image name for the container registry.

name: Build and Publish Docker Image # Consider changing the workflow name to make it distinguishable from other flows.
on:
  - push # Perform a build of the contents from the branch
  - pull_request # Perform a build after merging with the target branch
  - workflow_dispatch
jobs:
  build_and_release:
    uses: Cray-HPE/hms-build-image-workflows/.github/workflows/build_and_release_image.yaml@v1
    with:
      image-name: hms-pytest # Adjust this to match the container image 
      docker-build-file: Dockerfile.hms-pytest # Adjust this to match the desired Dockerfile name
      enable-pr-comment: true
    secrets: inherit

Workflow to build image with non-standard build context

Sample build and publish docker image workflow to build a Dockerfile with a non-standard build context. The example below will use the Dockerfile named Dockerfile inside of the directory tests/ct.

IMPORTANT when creating or migrating a repository to use this workflow make sure that the value for .jobs.build_and_release.with.image-name is set to the desired image name for the container registry.

name: Build and Publish Docker Image # Consider changing the workflow name to make it distinguishable from other flows.
on:
  - push # Perform a build of the contents from the branch
  - pull_request # Perform a build after merging with the target branch
  - workflow_dispatch
jobs:
  build_and_release:
    uses: Cray-HPE/hms-build-image-workflows/.github/workflows/build_and_release_image.yaml@v1
    with:
      image-name: cray-power-control-test # Adjust this to match the container image 
      docker-build-context: tests/ct # Adjust this to match the desired Docker build context
      enable-pr-comment: true
    secrets: inherit

Run unit test workflow

The run unit test workflow .github/workflows/run_unit_test.yaml in this repository is designed to execute unit tests within a Github repository. This effectively runs make unittest in the root of the repository.

Requirements:

• Makefile is present in the root of the repository.
• The Makefile has the unittest target defined to run the repositories unit tests.

The update PR with comment job is composed of mostly 3rd party Github Actions

• 3rd party Github Actions:
  • actions/checkout@v3

Workflow inputs

Name	Data Type	Required Field	Default value	Description
runs-on	string	Optional	ubuntu-latest	The type of machine to run the job on.

Workflow secrets

Name	Required Field	Description
ARTIFACTORY_ALGOL60_READONLY_USERNAME	Required	Artifactory readonly username used to download helm charts. Note these credentials are not used to upload artifacts to artifactory.
ARTIFACTORY_ALGOL60_READONLY_TOKEN	Required	Artifactory readonly token for the given user to download helm charts. Note these credentials are not used to upload artifacts to artifactory.

Full example

Sample run unit tests workflow (.github/workflows/run_unit_test.yaml) in use by the hms-power-control repository.

name: Run Unit Tests
on: [push, pull_request, workflow_dispatch]
jobs:
  run_unit_test:
    uses: Cray-HPE/hms-build-image-workflows/.github/workflows/run_unit_test.yaml@v1
    with:
      runs-on: ubuntu-latest

Sample Makefile target in use by the hms-power-control repository. The hms-power-control repository uses the unittest target to run the runUnitTest.sh script.

unittest:
	./runUnitTest.sh

Run integration test workflow

The run unit test workflow .github/workflows/run_integration_test.yaml in this repository is designed to execute unit tests within a Github repository. This effectively runs make integration in the root of the repository.

Requirements:

• Makefile is present in the root of the repository.
• The Makefile has the integration target defined to run the repositories unit tests.

The update PR with comment job is composed of mostly 3rd part Github Actions

• 3rd party Github Actions:
  • actions/checkout@v3

Workflow inputs

Name	Data Type	Required Field	Default value	Description
runs-on	string	Optional	ubuntu-latest	The type of machine to run the job on.

Workflow secrets

Name	Required Field	Description
ARTIFACTORY_ALGOL60_READONLY_USERNAME	Required	Artifactory readonly username used to download helm charts. Note these credentials are not used to upload artifacts to artifactory.
ARTIFACTORY_ALGOL60_READONLY_TOKEN	Required	Artifactory readonly token for the given user to download helm charts. Note these credentials are not used to upload artifacts to artifactory.

Full example

Sample run unit tests workflow (.github/workflows/run_integration_test.yaml) in use by the hms-power-control repository.

name: Run Integration Tests
on: [pull_request, workflow_dispatch]
jobs:
  run_integration_test:
    uses: Cray-HPE/hms-build-image-workflows/.github/workflows/run_integration_test.yaml@v1
    with:
      runs-on: ubuntu-latest
    secrets: inherit

Sample Makefile target in use by the hms-power-control repository. The hms-power-control repository uses the integration target to run the runIntegration.sh script.

integration:
	./runIntegration.sh

Run ct test workflow

The run ct test workflow .github/workflows/run_ct_test.yaml in this repository is designed to execute unit tests within a Github repository. This effectively runs make ct in the root of the repository.

Requirements:

• Makefile is present in the root of the repository.
• The Makefile has the ct target defined to run the repositories unit tests.

The update PR with comment job is composed of mostly 3rd party Github Actions

• 3rd party Github Actions:
  • actions/checkout@v3

Workflow inputs

Name	Data Type	Required Field	Default value	Description
runs-on	string	Optional	ubuntu-latest	The type of machine to run the job on.

Workflow secrets

Name	Required Field	Description
ARTIFACTORY_ALGOL60_READONLY_USERNAME	Required	Artifactory readonly username used to download helm charts. Note these credentials are not used to upload artifacts to artifactory.
ARTIFACTORY_ALGOL60_READONLY_TOKEN	Required	Artifactory readonly token for the given user to download helm charts. Note these credentials are not used to upload artifacts to artifactory.

Full example

Sample run unit tests workflow (.github/workflows/run_unit_test.yaml) in use by the hms-firmware-action repository.

name: Run CT Tests
on: [push, pull_request, workflow_dispatch]
jobs:
  run_ct_test:
    uses: Cray-HPE/hms-build-image-workflows/.github/workflows/run_ct_test.yaml@v1
    with:
      runs-on: ubuntu-latest
    secrets: inherit

Sample Makefile target in use by the hms-firmware-action repository. The hms-firmware-action repository uses the ct target to run the runCT.sh script.

ct:
	./runCT.sh

Release model

When you make changes you should tag the code branch with an vX.Y.Z semver and move/create the vX tag.

the vX tag (eg v1) is used by the 'invoking' workflows. The contract is that vX(n) MUST be backwards compatible.
the vX.Y.Z tag is used to distinguish code changes as a release.