|
| 1 | +--- |
| 2 | +title: Migrating Armory CD from Operator to Kustomize Deployment |
| 3 | +linkTitle: Migrating from Operator to Kustomize Deployment |
| 4 | +aliases: [] |
| 5 | +description: > |
| 6 | + Learn how to migrate Armory CD from Operator to Kustomize Deployment. |
| 7 | +--- |
| 8 | + |
| 9 | +## Migrating Armory CD from Operator to Kustomize Deployment |
| 10 | + |
| 11 | +### Introduction |
| 12 | + |
| 13 | +This document provides step-by-step instructions for migrating your Spinnaker installation from using the Operator deployment method to a native Kubernetes deployment using Kustomize. This approach gives you more direct control over your Spinnaker resources and removes the dependency on the Operator. |
| 14 | + |
| 15 | +{{% alert color="warning" title="Important" %}} |
| 16 | +Please thoroughly test this migration in a non-production environment before deploying to production. |
| 17 | +{{% /alert %}} |
| 18 | + |
| 19 | + |
| 20 | +#### Prerequisites |
| 21 | + |
| 22 | +- Kubectl command-line tool installed and configured to access your cluster |
| 23 | +- Basic understanding of Kubernetes resources (deployments, services, configmaps) |
| 24 | +- Access to the current Spinnaker namespace |
| 25 | + |
| 26 | +#### Migration Process Overview |
| 27 | + |
| 28 | +1. Download current configuration files and Kubernetes resources |
| 29 | +2. Set up Kustomize structure for native deployment |
| 30 | +3. Remove Operator ownership from services |
| 31 | +4. Scale down the Operator |
| 32 | +5. Deploy using Kustomize |
| 33 | +6. Validate the deployment |
| 34 | +7. Remove Operator and CRDs (after confirming stability) |
| 35 | + |
| 36 | +##### Step 1: Download Current Configuration |
| 37 | + |
| 38 | +The script provided below will download: |
| 39 | +- All configuration files located in /opt/spinnaker/config from each service |
| 40 | +- All deployment, service, and statefulset YAML files for each service |
| 41 | + |
| 42 | + |
| 43 | +###### How to Use the Download Script |
| 44 | + |
| 45 | +1. Save the script at the bottom of this document to a file named `download_spinnaker_configs.sh` |
| 46 | +2. Make the script executable: |
| 47 | +`chmod +x download_spinnaker_configs.sh` |
| 48 | +3. Run the script with your Spinnaker namespace: |
| 49 | +`./download_spinnaker_configs.sh your-spinnaker-namespace` |
| 50 | +4. The script will create an operator-migration directory containing all needed files |
| 51 | + |
| 52 | +###### What Gets Downloaded |
| 53 | + |
| 54 | +- **Deployments:** YAML files for each service deployment |
| 55 | +- **Services:** YAML files for all Spinnaker services |
| 56 | +- **StatefulSets:** YAML files for any statefulsets (like front50) |
| 57 | +- **Configuration Files:** All files from /opt/spinnaker/config in each pod |
| 58 | + |
| 59 | +##### Step 2: Set Up Kustomize Structure |
| 60 | + |
| 61 | +Create a Kustomize directory structure for your Spinnaker deployment: |
| 62 | +1. Move the downloaded deployments and services to their respective directories |
| 63 | +2. Create configmaps from the downloaded configuration files |
| 64 | +3. Set up the kustomization.yaml files |
| 65 | + |
| 66 | +{{% alert color="warning" title="Tip" %}} |
| 67 | +You can use the GitHub - spinnaker/spinnaker-kustomize: Spinnaker installation via kustomize as a reference for Kustomize structure |
| 68 | +{{% /alert %}} |
| 69 | + |
| 70 | +##### Step 3: Remove Operator Ownership from Services |
| 71 | + |
| 72 | +This step detaches the Operator's control while keeping services running. |
| 73 | +1. Identify resources owned by the Operator: |
| 74 | +`kubectl get all -n your-spinnaker-namespace -o json | jq '.items[] | select(.metadata.ownerReferences[]? | .apiVersion=="spinnaker.armory.io/v1alpha2" and .kind=="SpinnakerService") | {name: .metadata.name, kind: .kind}'` |
| 75 | +2. Remove ownership references using patch commands: |
| 76 | +`kubectl patch deployment spin-deck -n your-spinnaker-namespace --type json -p='[{"op": "remove", "path": "/metadata/ownerReferences"}]'` |
| 77 | +3. Repeat for all resources with Operator ownership |
| 78 | +{{% alert color="warning" title="Note" %}} |
| 79 | +This breaks the connection between the Operator and services but keeps everything running |
| 80 | +{{% /alert %}} |
| 81 | + |
| 82 | +##### Step 4: Verify Ownership Removal |
| 83 | + |
| 84 | +Confirm that no resources are still owned by the Operator: |
| 85 | +`kubectl get all -n your-spinnaker-namespace -o json | jq '.items[] | select(.metadata.ownerReferences[]? | .apiVersion=="spinnaker.armory.io/v1alpha2" and .kind=="SpinnakerService") | {name: .metadata.name, kind: .kind}'` |
| 86 | +The command should return empty if all ownership references have been removed. |
| 87 | + |
| 88 | +##### Step 5: Extra Precautions Before Deployment |
| 89 | +Compare current resources with your Kustomize configurations: |
| 90 | +`kubectl diff -f <(kustomize build ./overlays/prod)` |
| 91 | + |
| 92 | +Review the differences carefully. Look for: |
| 93 | +- Immutable field changes (might require special handling) |
| 94 | +- Configuration changes that could affect service behavior |
| 95 | +- Missing resources that should be included |
| 96 | + |
| 97 | +###### Perform a Dry Run |
| 98 | + |
| 99 | +Test your deployment without actually applying changes: |
| 100 | +`kubectl apply --dry-run=client -f <(kustomize build ./overlays/prod)` |
| 101 | + |
| 102 | +##### Step 6: Scale Down the Operator |
| 103 | +Prevent the Operator from interfering with your deployment: |
| 104 | +`kubectl scale deployment spinnaker-operator -n your-spinnaker-namespace --replicas=0` |
| 105 | + |
| 106 | +##### Step 7: Deploy Using Kustomize |
| 107 | + |
| 108 | +Apply your Kustomize configurations: |
| 109 | + |
| 110 | +`kubectl apply -f <(kustomize build ./overlays/prod)` |
| 111 | + |
| 112 | +##### Step 8: Validate and Monitor |
| 113 | +1. Check that all pods are running: |
| 114 | +`kubectl get pods -n your-spinnaker-namespace` |
| 115 | +2. Verify Spinnaker services are accessible: |
| 116 | + - Access the Spinnaker UI |
| 117 | + - Test a simple pipeline |
| 118 | + - Check integrations are working |
| 119 | +3. Monitor the environment for stability over the next few days |
| 120 | + |
| 121 | +##### Step 9: Remove Operator and CRDs |
| 122 | + |
| 123 | +Once stability is confirmed (at least 24 hours later): |
| 124 | +1. Remove the Operator CRDs: |
| 125 | +`kubectl delete crd spinnakerservices.spinnaker.armory.io` |
| 126 | +2. Remove the Operator deployment if still present: |
| 127 | +`kubectl delete deployment spinnaker-operator -n your-spinnaker-namespace` |
| 128 | + |
| 129 | + |
| 130 | +##### Rollback Plan |
| 131 | +If issues arise during migration: |
| 132 | +1. Scale up the Operator: |
| 133 | +`kubectl scale deployment spinnaker-operator -n your-spinnaker-namespace --replicas=1` |
| 134 | +2. Reapply the previous SpinnakerService resource: |
| 135 | +`kubectl apply -f original-spinnakerservice.yaml` |
| 136 | +3. Allow the Operator to reconcile and restore the previous state |
| 137 | + |
| 138 | +#### Download Script |
| 139 | +```bash |
| 140 | +#!/bin/bash |
| 141 | + |
| 142 | +# Script to download ONLY files from /opt/spinnaker/config in Spinnaker pods |
| 143 | + |
| 144 | +set -e |
| 145 | + |
| 146 | +if [ -z "$1" ]; then |
| 147 | + echo "Please provide a namespace" |
| 148 | + echo "Usage: $0 <namespace>" |
| 149 | + exit 1 |
| 150 | +fi |
| 151 | + |
| 152 | +NAMESPACE=$1 |
| 153 | +OUTPUT_DIR="operator-migration" |
| 154 | + |
| 155 | +echo "Creating output directory: $OUTPUT_DIR" |
| 156 | +rm -rf "$OUTPUT_DIR" |
| 157 | +mkdir -p "$OUTPUT_DIR" |
| 158 | + |
| 159 | +# 1. First download deployments and services |
| 160 | +echo "Downloading Kubernetes deployments and services..." |
| 161 | + |
| 162 | +# Download Deployments |
| 163 | +echo "Downloading deployments..." |
| 164 | +mkdir -p "$OUTPUT_DIR/deployments" |
| 165 | +kubectl get deployments -n "$NAMESPACE" -o name | while read -r deployment; do |
| 166 | + deployment_name=$(echo "$deployment" | cut -d/ -f2) |
| 167 | + echo "Downloading deployment: $deployment_name" |
| 168 | + kubectl get deployment "$deployment_name" -n "$NAMESPACE" -o yaml > "$OUTPUT_DIR/deployments/$deployment_name.yaml" |
| 169 | +done |
| 170 | + |
| 171 | +# Download Services |
| 172 | +echo "Downloading services..." |
| 173 | +mkdir -p "$OUTPUT_DIR/services" |
| 174 | +kubectl get services -n "$NAMESPACE" -o name | while read -r service; do |
| 175 | + service_name=$(echo "$service" | cut -d/ -f2) |
| 176 | + echo "Downloading service: $service_name" |
| 177 | + kubectl get service "$service_name" -n "$NAMESPACE" -o yaml > "$OUTPUT_DIR/services/$service_name.yaml" |
| 178 | +done |
| 179 | + |
| 180 | +# Download StatefulSets if any |
| 181 | +echo "Checking for statefulsets..." |
| 182 | +if kubectl get statefulsets -n "$NAMESPACE" 2>/dev/null | grep -q .; then |
| 183 | + mkdir -p "$OUTPUT_DIR/statefulsets" |
| 184 | + kubectl get statefulsets -n "$NAMESPACE" -o name | while read -r statefulset; do |
| 185 | + statefulset_name=$(echo "$statefulset" | cut -d/ -f2) |
| 186 | + echo "Downloading statefulset: $statefulset_name" |
| 187 | + kubectl get statefulset "$statefulset_name" -n "$NAMESPACE" -o yaml > "$OUTPUT_DIR/statefulsets/$statefulset_name.yaml" |
| 188 | + done |
| 189 | +fi |
| 190 | + |
| 191 | +# 2. Now download files from /opt/spinnaker/config |
| 192 | +echo "Downloading files ONLY from /opt/spinnaker/config..." |
| 193 | + |
| 194 | +# Get all pods |
| 195 | +PODS=$(kubectl get pods -n "$NAMESPACE" -o name | cut -d/ -f2) |
| 196 | +for POD in $PODS; do |
| 197 | + SERVICE=$(echo "$POD" | sed -E 's/([a-z-]+)-[0-9a-z-]+.*/\1/') |
| 198 | + |
| 199 | + echo "Processing pod: $POD (service: $SERVICE)" |
| 200 | + mkdir -p "$OUTPUT_DIR/$SERVICE" |
| 201 | + |
| 202 | + # Check ONLY for /opt/spinnaker/config |
| 203 | + if kubectl exec -n "$NAMESPACE" "$POD" -- ls -la /opt/spinnaker/config &>/dev/null; then |
| 204 | + echo "Found /opt/spinnaker/config directory in $POD" |
| 205 | + |
| 206 | + # List all files first |
| 207 | + CONFIG_FILES=$(kubectl exec -n "$NAMESPACE" "$POD" -- find /opt/spinnaker/config -type f 2>/dev/null) |
| 208 | + if [ -z "$CONFIG_FILES" ]; then |
| 209 | + echo "No files found in /opt/spinnaker/config for $POD" |
| 210 | + continue |
| 211 | + fi |
| 212 | + |
| 213 | + echo "Found $(echo "$CONFIG_FILES" | wc -l | tr -d ' ') files in /opt/spinnaker/config for $POD" |
| 214 | + |
| 215 | + # Download each file |
| 216 | + for FILE in $CONFIG_FILES; do |
| 217 | + FILENAME=$(basename "$FILE") |
| 218 | + echo "Downloading $FILENAME from $POD" |
| 219 | + |
| 220 | + FILE_CONTENT=$(kubectl exec -n "$NAMESPACE" "$POD" -- cat "$FILE" 2>/dev/null) |
| 221 | + if [ $? -eq 0 ] && [ -n "$FILE_CONTENT" ]; then |
| 222 | + echo "$FILE_CONTENT" > "$OUTPUT_DIR/$SERVICE/$FILENAME" |
| 223 | + echo "Saved $FILENAME to $OUTPUT_DIR/$SERVICE/$FILENAME" |
| 224 | + else |
| 225 | + echo "Failed to download $FILENAME or file is empty" |
| 226 | + fi |
| 227 | + done |
| 228 | + |
| 229 | + # Check if we downloaded any files |
| 230 | + if [ -z "$(ls -A "$OUTPUT_DIR/$SERVICE" 2>/dev/null)" ]; then |
| 231 | + echo "No files were successfully downloaded from $POD" |
| 232 | + else |
| 233 | + echo "Successfully downloaded $(ls -1 "$OUTPUT_DIR/$SERVICE" | wc -l | tr -d ' ') files from $POD" |
| 234 | + fi |
| 235 | + else |
| 236 | + echo "No /opt/spinnaker/config directory found in $POD" |
| 237 | + fi |
| 238 | +done |
| 239 | + |
| 240 | +echo "======================================" |
| 241 | +echo "Download completed. Files saved to: $OUTPUT_DIR" |
| 242 | +echo "Summary of downloaded files by service:" |
| 243 | + |
| 244 | +# Generate summary |
| 245 | +for DIR in $(find "$OUTPUT_DIR" -mindepth 1 -maxdepth 1 -type d | sort); do |
| 246 | + SERVICE=$(basename "$DIR") |
| 247 | + FILE_COUNT=$(find "$DIR" -type f | wc -l) |
| 248 | + |
| 249 | + echo "- $SERVICE: $FILE_COUNT files" |
| 250 | + if [ "$FILE_COUNT" -gt 0 ]; then |
| 251 | + ls -1 "$DIR" | sort | while read -r file; do |
| 252 | + echo " - $file" |
| 253 | + done |
| 254 | + fi |
| 255 | +done |
| 256 | + |
| 257 | +echo "Script execution complete!" |
| 258 | + |
| 259 | +``` |
| 260 | +### Conclusion |
| 261 | +By following these steps, you'll successfully migrate from the Spinnaker Operator to a native Kubernetes deployment using Kustomize. This approach gives you more direct control over your Spinnaker resources and eliminates dependency on the Operator. |
| 262 | +If you encounter any issues during the migration process, please submit a support ticket for assistance. |
0 commit comments