Skip to content

refactor!: drop GitHub Deployments, rename annotations for Swarmia (CI-1229) #136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
0x46616c6b wants to merge 1 commit into
base: main
Choose a base branch
from
Open

refactor!: drop GitHub Deployments, rename annotations for Swarmia (CI-1229) #136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 8 additions & 45 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,51 +106,17 @@ jobs:
clusters/customization/prod/mothership/diablo-redbook/diablo-redbook-helm.yaml spec.template.spec.containers.redbook.image
```

### Build, Push, Deploy and Track Deployment
### Deployment tracking annotations

When `create-deployment` is set to `true`, the action will:
1. Create a GitHub Deployment on the source repository for each target environment
2. Set the deployment status to `in_progress`
3. Write deployment tracking annotations (`deploy.staffbase.com/repo`, `deploy.staffbase.com/sha`, `deploy.staffbase.com/deployment-id`) to the Application CR in the mops overlay
Whenever the action updates a GitOps file, it stamps the following annotations onto the manifest's `metadata.annotations`:

The environment name is derived from the mops file path (e.g. `kubernetes/namespaces/<service>/prod/de1/...` becomes `prod-de1`).
| Annotation | Value |
|------------|-------|
| `deploy.staffbase.com/repositoryFullName` | The source repository in `owner/repo` form (`$GITHUB_REPOSITORY`) |
| `deploy.staffbase.com/commitSha` | The commit SHA being deployed (`$GITHUB_SHA`) |
| `deploy.staffbase.com/version` | The deployed image tag — `dev-<short-sha>` on `dev`, `main-<short-sha>` on `main`/`master`, the tag name on tag pushes |

The calling workflow must grant the `deployments: write` permission:

```yaml
name: CD

on: [ push ]

permissions:
deployments: write

jobs:
ci-cd:
name: Build, Push and Deploy

runs-on: ubuntu-24.04

steps:
- name: Checkout
uses: actions/checkout@v6

- name: GitOps (build, push, deploy and track)
uses: Staffbase/gitops-github-action@v7.1
with:
docker-username: ${{ vars.HARBOR_USERNAME }}
docker-password: ${{ secrets.HARBOR_PASSWORD }}
docker-image: private/diablo-redbook
gitops-token: ${{ secrets.GITOPS_TOKEN }}
create-deployment: true
github-token: ${{ github.token }}
gitops-dev: |-
clusters/customization/dev/mothership/diablo-redbook/diablo-redbook-helm.yaml spec.template.spec.containers.redbook.image
gitops-stage: |-
clusters/customization/stage/mothership/diablo-redbook/diablo-redbook-helm.yaml spec.template.spec.containers.redbook.image
gitops-prod: |-
clusters/customization/prod/mothership/diablo-redbook/diablo-redbook-helm.yaml spec.template.spec.containers.redbook.image
```
These keys mirror the [Swarmia Deployment API](https://help.swarmia.com/settings/organization/configuring-deployments-in-swarmia) field names and are read by `flux-deployment-reporter` to report deployments to Swarmia once Flux finishes reconciling.

## Inputs

Expand Down Expand Up @@ -179,16 +145,13 @@ jobs:
| `gitops-stage` | Files which should be updated by the GitHub Action for STAGE, must be relative to the root of the GitOps repository | |
| `gitops-prod` | Files which should be updated by the GitHub Action for PROD, must be relative to the root of the GitOps repository | |
| `working-directory` | The directory in which the GitOps action should be executed. The docker-file variable should be relative to working directory. | `.` |
| `create-deployment` | Create GitHub Deployments on the source repository and write tracking annotations to the GitOps CRs | `false` |
| `github-token` | GitHub Token for creating deployments (requires `deployments: write` permission). Required when `create-deployment` is `true`. | |

## Outputs

| Name | Description |
|-----------------|---------------------|
| `docker-digest` | Digest of the image |
| `docker-tag` | Tag of the image |
| `deployment-id` | JSON map of environment to GitHub Deployment ID (set when `create-deployment` is `true`) |

## Contributing

Expand Down
115 changes: 0 additions & 115 deletions action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,13 +95,6 @@ inputs:
description: 'The path relative to the repo root dir in which the GitOps action should be executed.'
required: false
default: '.'
create-deployment:
description: 'Create GitHub Deployments on the source repository and write tracking annotations to the GitOps CRs'
required: false
default: 'false'
github-token:
description: 'GitHub Token for creating deployments (requires deployments: write permission). Required when create-deployment is true.'
required: false

outputs:
docker-tag:
Expand All @@ -110,9 +103,6 @@ outputs:
docker-digest:
description: 'Docker digest'
value: ${{ steps.docker_build.outputs.digest || steps.docker_retag.outputs.digest }}
deployment-id:
description: 'JSON map of environment to GitHub Deployment ID (only set when create-deployment is true)'
value: ${{ steps.create_deployments.outputs.deployment_ids || steps.update_image.outputs.deployment_ids }}

runs:
using: "composite"
Expand Down Expand Up @@ -187,109 +177,6 @@ runs:
token: ${{ inputs.gitops-token }}
path: .github/${{ inputs.gitops-repository }}

- name: Create GitHub Deployments
id: create_deployments
if: inputs.create-deployment == 'true' && inputs.github-token != '' && inputs.gitops-token != ''
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
INPUT_GITOPS_DEV: ${{ inputs.gitops-dev }}
INPUT_GITOPS_STAGE: ${{ inputs.gitops-stage }}
INPUT_GITOPS_PROD: ${{ inputs.gitops-prod }}
DOCKER_IMAGE: ${{ inputs.docker-registry }}/${{ inputs.docker-image }}
DOCKER_TAG: ${{ steps.preparation.outputs.tag }}
with:
github-token: ${{ inputs.github-token }}
script: |
const ref = process.env.GITHUB_REF;
const image = process.env.DOCKER_IMAGE;
const tag = process.env.DOCKER_TAG;

let fileList = '';
if ((ref === 'refs/heads/master' || ref === 'refs/heads/main') && process.env.INPUT_GITOPS_STAGE) {
fileList = process.env.INPUT_GITOPS_STAGE;
} else if (ref === 'refs/heads/dev' && process.env.INPUT_GITOPS_DEV) {
fileList = process.env.INPUT_GITOPS_DEV;
} else if (ref.startsWith('refs/tags/') && process.env.INPUT_GITOPS_PROD) {
fileList = process.env.INPUT_GITOPS_PROD;
}

if (!fileList) {
core.setOutput('deployment_ids', '{}');
return;
}

// Derive unique environments from file paths
// Path format: kubernetes/namespaces/<service>/<env>/<cluster>/<file>.yaml
const environments = [...new Set(
fileList.split('\n')
.map(line => line.trim())
.filter(line => line.length > 0)
.map(line => {
const filePath = line.split(/\s+/)[0];
const parts = filePath.split('/');
return `${parts[3]}-${parts[4]}`;
})
)];

// Deactivate stale in-progress/queued/pending deployments for an environment
async function deactivateStaleDeployments(environment) {
const { data: deployments } = await github.rest.repos.listDeployments({
...context.repo,
environment,
per_page: 30
});

for (const dep of deployments) {
const { data: statuses } = await github.rest.repos.listDeploymentStatuses({
...context.repo,
deployment_id: dep.id,
per_page: 1
});

const latestState = statuses[0]?.state;
if (['in_progress', 'queued', 'pending'].includes(latestState)) {
core.info(`Marking stale deployment ${dep.id} (${latestState}) as inactive`);
await github.rest.repos.createDeploymentStatus({
...context.repo,
deployment_id: dep.id,
state: 'inactive',
description: 'Superseded by newer deployment'
});
}
}
}

const ids = {};
for (const env of environments) {
try {
await deactivateStaleDeployments(env);

const { data: deployment } = await github.rest.repos.createDeployment({
...context.repo,
ref: context.sha,
environment: env,
auto_merge: false,
required_contexts: [],
payload: { image, tag },
description: `Deploy ${image}:${tag} to ${env}`
});

await github.rest.repos.createDeploymentStatus({
...context.repo,
deployment_id: deployment.id,
state: 'in_progress',
description: 'Updating GitOps repository'
});

core.info(`Created deployment ${deployment.id} for environment ${env}`);
ids[env] = String(deployment.id);
} catch (error) {
core.warning(`Failed to create GitHub Deployment for ${env}: ${error.message}`);
}
}

core.setOutput('deployment_ids', JSON.stringify(ids));

- name: Update Docker Image in Repository
id: update_image
if: inputs.gitops-token != ''
Expand All @@ -300,8 +187,6 @@ runs:
INPUT_DOCKER_IMAGE: ${{ inputs.docker-image }}
INPUT_TAG: ${{ steps.preparation.outputs.tag }}
INPUT_PUSH: ${{ steps.preparation.outputs.push }}
INPUT_CREATE_DEPLOYMENT: ${{ inputs.create-deployment }}
INPUT_DEPLOYMENT_IDS: ${{ steps.create_deployments.outputs.deployment_ids }}
INPUT_GITOPS_USER: ${{ inputs.gitops-user }}
INPUT_GITOPS_EMAIL: ${{ inputs.gitops-email }}
INPUT_GITOPS_TOKEN: ${{ inputs.gitops-token }}
Expand Down
33 changes: 4 additions & 29 deletions scripts/lib/gitops-functions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
#
# Expected env vars (set by caller):
# INPUT_DOCKER_REGISTRY, INPUT_DOCKER_IMAGE, INPUT_TAG, INPUT_PUSH,
# INPUT_CREATE_DEPLOYMENT, INPUT_DEPLOYMENT_IDS,
# INPUT_GITOPS_USER, INPUT_GITOPS_TOKEN,
# INPUT_GITOPS_ORGANIZATION, INPUT_GITOPS_REPOSITORY,
# GITHUB_REPOSITORY, GITHUB_SHA, IMAGE
Expand All @@ -29,16 +28,6 @@ commit_changes() {
fi
}

# Derives the environment identifier from a mops file path.
# Expected path format: kubernetes/namespaces/<service>/<env>/<cluster>/<file>.yaml
derive_environment() {
local file_path="$1"
local env cluster
env=$(echo "$file_path" | cut -d'/' -f4)
cluster=$(echo "$file_path" | cut -d'/' -f5)
echo "${env}-${cluster}"
}

update_file() {
local file="$1"
local field="$2"
Expand All @@ -49,24 +38,10 @@ update_file() {
echo "Run update ${file} ${field} ${image}"
yq -i ."${field}"=\""${image}"\" "${file}"

if [[ "${INPUT_CREATE_DEPLOYMENT}" == "true" ]]; then
local deploy_env
deploy_env=$(derive_environment "${file}")

echo "Writing deployment annotations to ${file}"
yq -i '.metadata.annotations["deploy.staffbase.com/repo"] = "'"${GITHUB_REPOSITORY}"'"' "${file}"
yq -i '.metadata.annotations["deploy.staffbase.com/sha"] = "'"${GITHUB_SHA}"'"' "${file}"

# Write deployment-id annotation if available from the create_deployments step
local deploy_id=""
if [[ -n "${INPUT_DEPLOYMENT_IDS:-}" && "${INPUT_DEPLOYMENT_IDS}" != "{}" ]]; then
deploy_id=$(echo "${INPUT_DEPLOYMENT_IDS}" | jq -r --arg env "$deploy_env" '.[$env] // empty')
fi

if [[ -n "$deploy_id" ]]; then
yq -i '.metadata.annotations["deploy.staffbase.com/deployment-id"] = "'"${deploy_id}"'"' "${file}"
fi
fi
echo "Writing deployment annotations to ${file}"
yq -i '.metadata.annotations["deploy.staffbase.com/repositoryFullName"] = "'"${GITHUB_REPOSITORY}"'"' "${file}"
yq -i '.metadata.annotations["deploy.staffbase.com/commitSha"] = "'"${GITHUB_SHA}"'"' "${file}"
yq -i '.metadata.annotations["deploy.staffbase.com/version"] = "'"${INPUT_TAG}"'"' "${file}"
}

process_file_updates() {
Expand Down
5 changes: 2 additions & 3 deletions scripts/update-gitops.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,9 @@
#
# Required env vars: GITHUB_REF, GITHUB_SHA, GITHUB_REPOSITORY,
# INPUT_DOCKER_REGISTRY, INPUT_DOCKER_IMAGE, INPUT_TAG, INPUT_PUSH,
# INPUT_CREATE_DEPLOYMENT, INPUT_GITOPS_USER, INPUT_GITOPS_EMAIL,
# INPUT_GITOPS_USER, INPUT_GITOPS_EMAIL,
# INPUT_GITOPS_TOKEN, INPUT_GITOPS_ORGANIZATION, INPUT_GITOPS_REPOSITORY
# Optional env vars: INPUT_GITOPS_DEV, INPUT_GITOPS_STAGE, INPUT_GITOPS_PROD,
# INPUT_DEPLOYMENT_IDS
# Optional env vars: INPUT_GITOPS_DEV, INPUT_GITOPS_STAGE, INPUT_GITOPS_PROD

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=lib/common.sh
Expand Down
68 changes: 15 additions & 53 deletions tests/lib-gitops-functions.bats
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ setup() {
export INPUT_DOCKER_IMAGE="my-service"
export INPUT_TAG="main-abcdef12"
export INPUT_PUSH="true"
export INPUT_CREATE_DEPLOYMENT="false"
export INPUT_DEPLOYMENT_IDS="{}"
export INPUT_GITOPS_USER="Staffbot"
export INPUT_GITOPS_TOKEN="fake-token"
export INPUT_GITOPS_ORGANIZATION="Staffbase"
Expand Down Expand Up @@ -44,47 +42,13 @@ esac
GIT_MOCK
chmod +x "${TEST_TEMP_DIR}/mocks/git"

# Create mock jq that passes through
cat > "${TEST_TEMP_DIR}/mocks/jq" << 'JQ_MOCK'
#!/usr/bin/env bash
# Use real jq if available, otherwise simple passthrough
if command -v /usr/bin/jq &>/dev/null; then
/usr/bin/jq "$@"
elif command -v /opt/homebrew/bin/jq &>/dev/null; then
/opt/homebrew/bin/jq "$@"
else
cat
fi
JQ_MOCK
chmod +x "${TEST_TEMP_DIR}/mocks/jq"

export PATH="${TEST_TEMP_DIR}/mocks:$PATH"
}

teardown() {
teardown_common
}

# --- derive_environment ---

@test "derive_environment extracts env and cluster from standard mops path" {
run derive_environment "kubernetes/namespaces/my-service/prod/de1/deployment.yaml"
assert_success
assert_output "prod-de1"
}

@test "derive_environment handles stage environment" {
run derive_environment "kubernetes/namespaces/my-service/stage/us1/deployment.yaml"
assert_success
assert_output "stage-us1"
}

@test "derive_environment handles dev environment" {
run derive_environment "kubernetes/namespaces/my-service/dev/de1/deployment.yaml"
assert_success
assert_output "dev-de1"
}

# --- update_file ---

@test "update_file calls yq to check and update field" {
Expand All @@ -94,33 +58,31 @@ teardown() {
grep -q 'yq -i' "${TEST_TEMP_DIR}/yq_calls.log"
}

@test "update_file writes deployment annotations when create-deployment is true" {
export INPUT_CREATE_DEPLOYMENT="true"
update_file "kubernetes/namespaces/svc/prod/de1/deployment.yaml" "spec.image" "$IMAGE"
grep -q 'deploy.staffbase.com/repo' "${TEST_TEMP_DIR}/yq_calls.log"
grep -q 'deploy.staffbase.com/sha' "${TEST_TEMP_DIR}/yq_calls.log"
@test "update_file always writes deploy.staffbase.com/repositoryFullName annotation" {
update_file "deployment.yaml" "spec.image" "$IMAGE"
grep -q 'deploy.staffbase.com/repositoryFullName' "${TEST_TEMP_DIR}/yq_calls.log"
}

@test "update_file skips annotations when create-deployment is false" {
export INPUT_CREATE_DEPLOYMENT="false"
@test "update_file always writes deploy.staffbase.com/commitSha annotation" {
update_file "deployment.yaml" "spec.image" "$IMAGE"
! grep -q 'deploy.staffbase.com' "${TEST_TEMP_DIR}/yq_calls.log" 2>/dev/null || true
grep -q 'deploy.staffbase.com/commitSha' "${TEST_TEMP_DIR}/yq_calls.log"
}

@test "update_file writes deployment-id annotation when deployment ID is available" {
export INPUT_CREATE_DEPLOYMENT="true"
export INPUT_DEPLOYMENT_IDS='{"prod-de1":"12345"}'
update_file "kubernetes/namespaces/svc/prod/de1/deployment.yaml" "spec.image" "$IMAGE"
grep -q 'deploy.staffbase.com/deployment-id' "${TEST_TEMP_DIR}/yq_calls.log"
@test "update_file writes deploy.staffbase.com/version annotation with INPUT_TAG" {
update_file "deployment.yaml" "spec.image" "$IMAGE"
grep -q "deploy.staffbase.com/version.*${INPUT_TAG}" "${TEST_TEMP_DIR}/yq_calls.log"
}

@test "update_file skips deployment-id annotation when no matching deployment ID" {
export INPUT_CREATE_DEPLOYMENT="true"
export INPUT_DEPLOYMENT_IDS='{"stage-us1":"99999"}'
update_file "kubernetes/namespaces/svc/prod/de1/deployment.yaml" "spec.image" "$IMAGE"
@test "update_file does not write deploy.staffbase.com/deployment-id annotation" {
update_file "deployment.yaml" "spec.image" "$IMAGE"
! grep -q 'deployment-id' "${TEST_TEMP_DIR}/yq_calls.log"
}

@test "update_file does not write legacy /repo or /sha annotation keys" {
update_file "deployment.yaml" "spec.image" "$IMAGE"
! grep -qE 'deploy\.staffbase\.com/(repo|sha)"' "${TEST_TEMP_DIR}/yq_calls.log"
}

# --- commit_changes ---

@test "commit_changes commits and pushes when push is true" {
Expand Down
Loading
Loading