Skip to content

GitHubWorkflows.md

Latest commit

 

History

History
1495 lines (1230 loc) · 33.1 KB

GitHubWorkflows.md

File metadata and controls

1495 lines (1230 loc) · 33.1 KB

GitHub Workflows: Automating CI/CD and More with GitHub Actions

This comprehensive guide covers GitHub Actions from basics to advanced automation. Learn to create robust CI/CD pipelines, automate releases, and streamline your development workflow with powerful GitHub Actions features.

📋 Table of Contents

1. Introduction to GitHub Actions

What are GitHub Actions?

GitHub Actions is a CI/CD platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository, or deploy merged pull requests to production.

Key Concepts

  • Workflows: Automated processes defined in YAML files
  • Events: Triggers that start workflows (push, PR, schedule, etc.)
  • Jobs: Set of steps that execute on the same runner
  • Steps: Individual tasks within a job
  • Actions: Reusable units of code
  • Runners: Machines that execute workflows

Common Use Cases

  • Continuous Integration (CI): Build and test code
  • Continuous Deployment (CD): Deploy to staging/production
  • Code Quality: Linting, formatting, security scanning
  • Automation: Issue management, releases, notifications
  • Testing: Unit tests, integration tests, end-to-end tests

Getting Started

  1. Create .github/workflows/ directory
  2. Add workflow YAML files
  3. Push to trigger workflows
  4. Monitor execution in Actions tab

2. Anatomy of a Workflow File

Basic Structure

name: CI Pipeline

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Run tests
        run: npm test

Complete Workflow Anatomy

name: 'Complete CI/CD Pipeline'
description: 'Build, test, and deploy application'

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]
  schedule:
    - cron: '0 0 * * 0'  # Weekly on Sundays

env:
  NODE_VERSION: '18'

jobs:
  lint:
    name: 'Lint Code'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ env.NODE_VERSION }}
      - run: npm ci
      - run: npm run lint

  test:
    name: 'Run Tests'
    runs-on: ubuntu-latest
    needs: lint
    strategy:
      matrix:
        node-version: [16, 18, 20]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm ci
      - run: npm test -- --coverage
      - uses: codecov/codecov-action@v3

  build:
    name: 'Build Application'
    runs-on: ubuntu-latest
    needs: test
    if: github.ref == 'refs/heads/main'
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ env.NODE_VERSION }}
      - run: npm ci
      - run: npm run build
      - uses: actions/upload-artifact@v3
        with:
          name: build-files
          path: dist/

3. Example: CI Workflow

Simple Node.js CI

name: Node.js CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  build-and-test:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [14.x, 16.x, 18.x]

    steps:
      - uses: actions/checkout@v4

      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: 'npm'

      - run: npm ci
      - run: npm run build --if-present
      - run: npm test

Python CI/CD

name: Python CI/CD

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.8", "3.9", "3.10"]

    steps:
      - uses: actions/checkout@v4
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run tests
        run: |
          python -m pytest tests/ -v --cov=src --cov-report=xml

      - name: Upload coverage
        uses: codecov/codecov-action@v3
        with:
          file: ./coverage.xml

4. Workflow Triggers

Event Types

  • push: Trigger on pushes to specified branches/tags
  • pull_request: Trigger on PR events (opened, synchronized, reopened)
  • schedule: Trigger on cron schedule
  • workflow_dispatch: Manual trigger via UI/CLI
  • release: Trigger on release creation
  • issues: Trigger on issue events
  • workflow_run: Trigger when another workflow completes

Advanced Trigger Configuration

on:
  push:
    branches: [main, 'feature/**']
    tags: ['v*']
    paths:
      - 'src/**'
      - '!src/docs/**'

  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]
    paths-ignore:
      - 'docs/**'
      - '*.md'

  schedule:
    - cron: '0 0 * * 1'  # Every Monday at midnight

  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to deploy to'
        required: true
        default: 'staging'
        type: choice
        options:
          - staging
          - production

Conditional Triggers

on:
  pull_request:
    types: [opened, synchronize]
    branches: [main]

jobs:
  test:
    if: github.event.pull_request.draft == false
    runs-on: ubuntu-latest
    steps:
      - run: echo "Only runs on ready PRs"

5. Advanced Job Features

Job Strategies

Matrix Strategy:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [14, 16, 18]
        os: [ubuntu-latest, windows-latest]
        exclude:
          - os: windows-latest
            node-version: 14
        include:
          - os: ubuntu-latest
            node-version: 18
            experimental: true
    steps:
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}

Job Dependencies

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      build-id: ${{ steps.build.outputs.build-id }}
    steps:
      - id: build
        run: echo "build-id=123" >> $GITHUB_OUTPUT

  test:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - run: echo "Build ID is ${{ needs.build.outputs.build-id }}"

  deploy:
    needs: [build, test]
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
      - run: echo "Deploying build ${{ needs.build.outputs.build-id }}"

Environment and Secrets

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - name: Deploy to production
        run: |
          echo "Deploying to ${{ vars.ENVIRONMENT }}"
          echo "Using secret: ${{ secrets.DEPLOY_KEY }}"
        env:
          API_KEY: ${{ secrets.API_KEY }}

6. Reusable Workflows & Composite Actions

Reusable Workflows

Create reusable workflow:

# .github/workflows/reusable-deploy.yml
name: Reusable Deploy

on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
    secrets:
      deploy-key:
        required: true

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment }}
    steps:
      - uses: actions/checkout@v4
      - run: echo "Deploying to ${{ inputs.environment }}"

Use reusable workflow:

# .github/workflows/ci.yml
name: CI

on: [push]

jobs:
  test:
    uses: ./.github/workflows/reusable-test.yml

  deploy-staging:
    needs: test
    uses: ./.github/workflows/reusable-deploy.yml
    with:
      environment: staging
    secrets:
      deploy-key: ${{ secrets.STAGING_DEPLOY_KEY }}

Composite Actions

Create composite action:

# .github/actions/setup-node/action.yml
name: 'Setup Node'
description: 'Setup Node.js with caching'

inputs:
  node-version:
    description: 'Node.js version'
    required: true

runs:
  using: 'composite'
  steps:
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ inputs.node-version }}
        cache: 'npm'
    - run: npm ci
      shell: bash

Use composite action:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/setup-node
        with:
          node-version: '18'
      - run: npm test

7. Automating Releases

Automated Releases with Semantic Versioning

name: Release

on:
  push:
    branches: [main]

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '18'
          cache: 'npm'

      - run: npm ci

      - name: Run tests
        run: npm test

      - name: Build
        run: npm run build

      - name: Release
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: npx semantic-release

Manual Release Creation

name: Create Release

on:
  workflow_dispatch:
    inputs:
      version:
        description: 'Release version'
        required: true
        type: string

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Create Release
        uses: actions/create-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          tag_name: ${{ github.event.inputs.version }}
          release_name: Release ${{ github.event.inputs.version }}
          body: |
            ## Changes
            - Feature 1
            - Bug fix 2
          draft: false
          prerelease: false

GitHub Releases with Assets

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build binaries
        run: |
          # Build your application
          make build

      - name: Create Release
        id: create_release
        uses: actions/create-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          tag_name: ${{ github.ref }}
          release_name: Release ${{ github.ref }}
          draft: false
          prerelease: false

      - name: Upload Release Asset
        uses: actions/upload-release-asset@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          upload_url: ${{ steps.create_release.outputs.upload_url }}
          asset_path: ./bin/myapp
          asset_name: myapp-linux-amd64
          asset_content_type: application/octet-stream

8. Pull Request Checks and Auto Merge

Branch Protection Rules

Configure in repository settings:

  • Require PR reviews
  • Require status checks
  • Require branches to be up-to-date
  • Include administrators

Status Checks

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm test

  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm run lint

  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: github/super-linter@v5

Auto Merge

Enable auto-merge for approved PRs:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - run: echo "Tests passed"

  auto-merge:
    needs: test
    runs-on: ubuntu-latest
    if: github.event.pull_request.merged != true
    steps:
      - uses: peter-evans/enable-pull-request-automerge@v3
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          merge-method: squash

9. Monitoring and Debugging

Workflow Logs and Artifacts

  • View logs in Actions tab
  • Download artifacts from workflow runs
  • Use actions/upload-artifact to save files
- uses: actions/upload-artifact@v3
  with:
    name: test-results
    path: test-results/
    retention-days: 30

Debugging Techniques

Enable debug logging:

jobs:
  debug:
    runs-on: ubuntu-latest
    steps:
      - name: Enable debug
        run: echo "ACTIONS_RUNNER_DEBUG=true" >> $GITHUB_ENV
      - run: echo "Debug mode enabled"

SSH into runner:

- name: Setup tmate session
  uses: mxschmitt/action-tmate@v3
  if: failure()

Workflow Insights

  • Workflow analytics in repository insights
  • Billing and usage in organization settings
  • Performance metrics for optimization

10. Best Practices

Workflow Design

  • Keep workflows simple and focused
  • Use descriptive names for jobs and steps
  • Pin action versions (avoid @latest)
  • Use caching to speed up builds
  • Handle failures gracefully

Security

  • Never log secrets in workflow output
  • Use least privilege for tokens
  • Regularly rotate secrets
  • Audit workflow permissions

Performance

  • Use appropriate runner sizes
  • Cache dependencies between runs
  • Parallelize independent jobs
  • Use matrix builds for testing

Maintenance

  • Regularly update actions
  • Remove unused workflows
  • Document complex workflows
  • Monitor for deprecated features

11. Advanced Patterns

Monorepo Workflows

name: Monorepo CI

on: [push, pull_request]

jobs:
  changes:
    runs-on: ubuntu-latest
    outputs:
      api: ${{ steps.filter.outputs.api }}
      web: ${{ steps.filter.outputs.web }}
    steps:
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v2
        id: filter
        with:
          filters: |
            api: 'api/**'
            web: 'web/**'

  api:
    needs: changes
    if: needs.changes.outputs.api == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: cd api && npm test

  web:
    needs: changes
    if: needs.changes.outputs.web == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: cd web && npm test

Container-Based Workflows

jobs:
  test:
    runs-on: ubuntu-latest
    container:
      image: node:18-alpine
      env:
        NODE_ENV: test
    services:
      postgres:
        image: postgres:13
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test

Self-Hosted Runners

jobs:
  deploy:
    runs-on: [self-hosted, linux, gpu]
    steps:
      - uses: actions/checkout@v4
      - run: ./deploy.sh

12. Security and Compliance

Security Scanning

jobs:
  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: 'fs'
          scan-ref: '.'
          format: 'sarif'
          output: 'trivy-results.sarif'

      - name: Upload Trivy scan results
        uses: github/codeql-action/upload-sarif@v2
        if: always()
        with:
          sarif_file: 'trivy-results.sarif'

Dependency Scanning

- uses: github/super-linter@v5
- uses: actions/dependency-review-action@v3
- uses: securecodewarrior/github-actions-gosec@master

Secret Management

  • Repository secrets for environment-specific values
  • Organization secrets for shared values
  • Environment secrets for deployment secrets
  • Dependabot secrets for automated updates

13. Performance Optimization

Caching Strategies

- uses: actions/cache@v3
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-

- uses: actions/cache@v3
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}

Parallel Execution

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        test-group: [1, 2, 3, 4]
    steps:
      - run: npm test -- --group ${{ matrix.test-group }}

Resource Optimization

  • Use larger runners for CPU-intensive tasks
  • Use spot instances for non-critical jobs
  • Implement job timeouts
  • Monitor usage and costs

14. Custom Runners and Self-Hosted Infrastructure

Self-Hosted Runner Setup

Requirements:

  • Supported OS: Linux, Windows, macOS
  • Minimum specs: 2-core CPU, 8GB RAM
  • Network: Outbound HTTPS to GitHub
  • Storage: 10GB+ free space

Installation:

# Download runner
curl -o actions-runner-linux-x64-2.311.0.tar.gz \
  -L https://github.com/actions/runner/releases/download/v2.311.0/actions-runner-linux-x64-2.311.0.tar.gz

# Extract
tar xzf ./actions-runner-linux-x64-2.311.0.tar.gz

# Configure
./config.sh --url https://github.com/owner/repo --token TOKEN

# Run
./run.sh

As a service:

# Create service user
sudo useradd -m -s /bin/bash github

# Install as service
sudo ./svc.sh install github
sudo ./svc.sh start

Runner Groups and Labels

Organize runners:

# Repository settings
runs-on:
  group: 'gpu-runners'
  labels: 'gpu, nvidia'

Runner configuration:

# Add labels during setup
./config.sh --labels gpu,nvidia,cuda-11.8

Auto-scaling Runners

AWS EC2 auto-scaling:

# .github/workflows/auto-scale.yml
name: Auto Scale Runners

on:
  workflow_run:
    workflows: ["CI"]
    types: [requested]

jobs:
  scale-up:
    runs-on: ubuntu-latest
    steps:
      - name: Scale up EC2 runners
        run: |
          aws ec2 run-instances \
            --image-id ami-12345678 \
            --count 2 \
            --instance-type c5.large \
            --user-data file://runner-setup.sh

15. Advanced Workflow Patterns

Workflow Templates and Reusability

Workflow templates:

# .github/workflow-templates/ci-template.yml
name: CI Template

description: 'Reusable CI workflow template'

inputs:
  node-version:
    description: 'Node.js version'
    required: true
    default: '18'

runs:
  using: 'composite'
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ inputs.node-version }}
    - run: npm ci
      shell: bash
    - run: npm test
      shell: bash

Using templates:

# .github/workflows/ci.yml
name: CI

on: [push, pull_request]

jobs:
  test:
    uses: ./.github/workflow-templates/ci-template.yml
    with:
      node-version: '20'

Dynamic Workflow Generation

Generate workflows programmatically:

# .github/workflows/generate.yml
name: Generate Workflows

on:
  push:
    paths:
      - 'services/**'

jobs:
  generate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Generate service workflows
        run: |
          for service in services/*; do
            if [ -d "$service" ]; then
              ./scripts/generate-workflow.sh "$service"
            fi
          done
      - name: Commit generated workflows
        run: |
          git add .github/workflows/
          git commit -m "chore: update generated workflows" || true
          git push

Workflow Composition

Composite actions for complex workflows:

# .github/actions/deploy/action.yml
name: 'Deploy Application'
description: 'Deploy to specified environment'

inputs:
  environment:
    description: 'Deployment environment'
    required: true

runs:
  using: 'composite'
  steps:
    - name: Setup deployment
      run: |
        echo "Deploying to ${{ inputs.environment }}"
      shell: bash

    - name: Deploy
      uses: ./.github/actions/deploy-${{ inputs.environment }}

16. Workflow Orchestration and Coordination

Cross-Repository Workflows

Repository dispatch events:

# Repository A: Trigger external workflow
jobs:
  trigger:
    runs-on: ubuntu-latest
    steps:
      - name: Trigger downstream build
        run: |
          curl -X POST \
            -H "Authorization: token ${{ secrets.REPO_B_TOKEN }}" \
            -H "Accept: application/vnd.github.v3+json" \
            https://api.github.com/repos/owner/repo-b/dispatches \
            -d '{"event_type":"build-triggered", "client_payload":{"ref":"${{ github.ref }}"}}'

Receiving repository:

# Repository B: Handle dispatch
on:
  repository_dispatch:
    types: [build-triggered]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - run: echo "Building from ${{ github.event.client_payload.ref }}"

Workflow Dependencies Across Repositories

Using GitHub Apps for coordination:

// GitHub App for cross-repo coordination
app.on('workflow_run.completed', async (context) => {
  const { workflow_run } = context.payload;

  if (workflow_run.name === 'Release') {
    // Trigger dependent repository builds
    await context.github.repos.createDispatchEvent({
      owner: 'dependent-org',
      repo: 'dependent-repo',
      event_type: 'upstream-release',
      client_payload: {
        version: workflow_run.head_sha,
        artifacts: workflow_run.artifacts_url
      }
    });
  }
});

Monorepo Orchestration

Service dependency management:

# .github/workflows/monorepo-deploy.yml
name: Monorepo Deploy

on:
  push:
    branches: [main]

jobs:
  detect-changes:
    runs-on: ubuntu-latest
    outputs:
      services: ${{ steps.changes.outputs.services }}
    steps:
      - uses: actions/checkout@v4
      - name: Detect changed services
        id: changes
        run: |
          # Logic to detect which services changed
          echo "services=$(./scripts/detect-changes.sh)" >> $GITHUB_OUTPUT

  deploy-services:
    needs: detect-changes
    runs-on: ubuntu-latest
    strategy:
      matrix:
        service: ${{ fromJson(needs.detect-changes.outputs.services) }}
    steps:
      - name: Deploy ${{ matrix.service }}
        run: ./scripts/deploy-service.sh ${{ matrix.service }}

17. Workflow Analytics and Insights

Performance Monitoring

Workflow metrics collection:

jobs:
  metrics:
    runs-on: ubuntu-latest
    if: always()  # Run even if previous jobs fail
    steps:
      - name: Collect workflow metrics
        run: |
          echo "Workflow: ${{ github.workflow }}"
          echo "Run ID: ${{ github.run_id }}"
          echo "Duration: $((SECONDS - START_TIME)) seconds"
          echo "Status: ${{ job.status }}"
        env:
          START_TIME: ${{ github.event.workflow_run.created_at }}

      - name: Send metrics to monitoring
        run: |
          curl -X POST ${{ secrets.METRICS_WEBHOOK }} \
            -H 'Content-Type: application/json' \
            -d "{
              \"workflow\": \"${{ github.workflow }}\",
              \"duration\": $((SECONDS - START_TIME)),
              \"status\": \"${{ job.status }}\",
              \"repository\": \"${{ github.repository }}\"
            }"

Cost Optimization

Usage tracking:

jobs:
  cost-analysis:
    runs-on: ubuntu-latest
    steps:
      - name: Calculate workflow costs
        run: |
          # Estimate costs based on runtime and resources
          RUNTIME_MINUTES=$((SECONDS / 60))
          COST_PER_MINUTE=0.008  # Ubuntu runner cost
          TOTAL_COST=$(echo "scale=2; $RUNTIME_MINUTES * $COST_PER_MINUTE" | bc)

          echo "Runtime: ${RUNTIME_MINUTES} minutes"
          echo "Estimated cost: $${TOTAL_COST}"

      - name: Log cost data
        run: |
          # Send to cost monitoring system
          echo "Workflow cost logged"

Success Rate Analytics

Workflow success tracking:

jobs:
  analytics:
    runs-on: ubuntu-latest
    if: always()
    steps:
      - name: Update success metrics
        run: |
          if [ "${{ job.status }}" = "success" ]; then
            echo "✅ Workflow succeeded"
            # Increment success counter
          else
            echo "❌ Workflow failed"
            # Log failure reasons
          fi

      - name: Generate reports
        run: |
          # Generate weekly/monthly reports
          # Identify failure patterns
          # Suggest optimizations

18. Future of GitHub Actions

Emerging Features

Actions improvements:

  • Composite actions v2 with better reusability
  • Reusable workflows v2 with enhanced features
  • Runner improvements with better performance
  • Security enhancements with runtime protection

Platform evolution:

  • GitHub Codespaces integration for cloud development
  • Copilot integration for workflow generation
  • Advanced caching with distributed storage
  • Real-time collaboration features

Industry Trends

Infrastructure as Code:

# Infrastructure workflows
jobs:
  infrastructure:
    runs-on: ubuntu-latest
    steps:
      - uses: hashicorp/terraform-github-actions@master
        with:
          tf_api_token: ${{ secrets.TF_API_TOKEN }}
          tf_host: app.terraform.io
          tf_workspace: ${{ github.event.repository.name }}

AI/ML Pipelines:

jobs:
  ml-pipeline:
    runs-on: [self-hosted, gpu]
    steps:
      - uses: actions/checkout@v4
      - name: Train model
        run: |
          python train.py --data ${{ github.workspace }}/data
      - name: Evaluate model
        run: |
          python evaluate.py --model model.pkl
      - name: Deploy model
        uses: ./.github/actions/model-deploy
        with:
          model-path: model.pkl

Serverless and Edge Computing

Edge deployment workflows:

jobs:
  edge-deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build for edge
        run: |
          npm run build:edge
      - name: Deploy to edge locations
        run: |
          # Deploy to Cloudflare Workers, Vercel Edge, etc.
          npx wrangler publish
      - name: Update edge configuration
        run: |
          # Update CDN configurations
          # Update DNS records

19. Workflow Philosophy and Culture

Building a CI/CD Culture

Team practices:

  • Test-driven development encouragement
  • Code review culture establishment
  • Continuous learning promotion
  • Failure as learning opportunities

Leadership role:

  • Lead by example with quality PRs
  • Mentor team members in best practices
  • Celebrate successes and improvements
  • Address bottlenecks proactively

Sustainability and Well-being

Avoiding burnout:

  • Reasonable SLAs for PR reviews
  • Work-life balance respect
  • Automation over manual processes
  • Regular process reviews

Inclusive practices:

  • Timezone consideration in schedules
  • Multiple communication channels
  • Cultural sensitivity awareness
  • Accessibility considerations

Measuring Success

Key metrics:

  • Deployment frequency
  • Lead time for changes
  • Change failure rate
  • Time to recovery

Qualitative measures:

  • Team satisfaction
  • Code quality perception
  • Learning and growth
  • Community engagement

20. Troubleshooting Common Issues

Workflow Fails to Trigger

Common causes:

  • Incorrect event configuration: Check on: section matches your trigger
  • Branch protection rules: Ensure workflows run before branch protection
  • Path filters: Verify file paths match your filters
  • Permissions: Check repository/organization permissions

Debugging steps:

# Add debug logging
jobs:
  debug:
    runs-on: ubuntu-latest
    steps:
      - name: Debug trigger
        run: |
          echo "Event: ${{ github.event_name }}"
          echo "Ref: ${{ github.ref }}"
          echo "SHA: ${{ github.sha }}"
          echo "Actor: ${{ github.actor }}"

Action Version Issues

Problems with @latest:

# ❌ Avoid this
- uses: actions/checkout@latest

# ✅ Pin versions
- uses: actions/checkout@v4

Finding compatible versions:

# Check action releases
curl -s https://api.github.com/repos/actions/checkout/releases/latest | jq -r .tag_name

Runner Resource Issues

Common problems:

  • Out of disk space: Clean up workspace
  • Memory limits: Use larger runners or optimize memory usage
  • Timeout errors: Increase timeout or optimize workflow

Solutions:

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10  # Increase timeout
    steps:
      - name: Clean workspace
        run: |
          df -h
          sudo rm -rf /usr/local/lib/android  # Free space
          df -h

Secret and Environment Issues

Debugging secrets:

# Never log secrets directly
- name: Debug environment
  run: |
    echo "HAS_SECRET: ${{ secrets.MY_SECRET != '' }}"
    echo "ENV_VAR: ${{ vars.MY_VAR }}"

Common issues:

  • Secret not available: Check repository/organization/environment settings
  • Permission denied: Verify token scopes
  • Environment not set: Check environment configuration

Dependency and Cache Issues

Cache problems:

- uses: actions/cache@v3
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-

Debugging cache:

- name: Cache debug
  run: |
    echo "Cache hit: ${{ steps.cache.outputs.cache-hit }}"
    ls -la ~/.npm || true

Matrix Build Failures

Debugging matrix jobs:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node: [16, 18, 20]
      fail-fast: false  # Continue on failure
    steps:
      - name: Debug matrix
        run: |
          echo "Node version: ${{ matrix.node }}"
          echo "Runner OS: ${{ runner.os }}"

Network and API Issues

Handling API rate limits:

- name: API call with retry
  uses: nick-invision/retry@v2
  with:
    timeout_minutes: 10
    max_attempts: 3
    command: curl -f https://api.github.com/repos/${{ github.repository }}/issues

Network timeouts:

- name: Network request
  run: |
    timeout 30 curl -f --retry 3 --retry-delay 5 https://api.example.com

Artifact and Upload Issues

Debugging uploads:

- uses: actions/upload-artifact@v3
  if: always()  # Upload even on failure
  with:
    name: debug-logs
    path: |
      **/*.log
      test-results/
    retention-days: 7

Permission and Security Issues

Common permission errors:

  • Resource not accessible: Check GITHUB_TOKEN permissions
  • Action requires permissions: Add required permissions to workflow

Fix permissions:

permissions:
  contents: read
  issues: write
  pull-requests: write
  id-token: write  # For OIDC

jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      deployments: write
    steps:
      - run: echo "Deploy with specific permissions"

Workflow Performance Issues

Identifying bottlenecks:

jobs:
  performance-test:
    runs-on: ubuntu-latest
    steps:
      - name: Time operation
        run: |
          START=$(date +%s)
          # Your slow operation here
          npm test
          END=$(date +%s)
          echo "Duration: $((END - START)) seconds"

Optimization strategies:

  • Parallel jobs: Split work across multiple jobs
  • Caching: Cache dependencies and build artifacts
  • Conditional execution: Skip unnecessary steps
  • Resource optimization: Use appropriate runner sizes

Summary

GitHub Actions provides powerful automation capabilities for modern software development. From simple CI pipelines to complex deployment workflows, Actions can streamline your entire development lifecycle. Focus on writing maintainable workflows, implementing proper security measures, and continuously optimizing performance.

Additional Resources