Skip to content

ci-cd.md

Latest commit

 

History

History
328 lines (251 loc) · 8.66 KB

ci-cd.md

File metadata and controls

328 lines (251 loc) · 8.66 KB

CI/CD Pipeline

Charon uses GitHub Actions for continuous integration and deployment. The CI/CD system is designed around pre-commit hooks as the single source of truth for code quality, with streamlined workflows that avoid duplication.

Overview

The CI/CD pipeline consists of three main jobs:

  1. Lint & Format: Runs all pre-commit hooks for code quality
  2. Tests: Executes pytest test suite with proper reporting
  3. Security: Scans Terraform code with Checkov for security issues

CI Workflow (.github/workflows/ci.yml)

Design Philosophy

Pre-commit hooks are the single source of truth for all linting and formatting rules. The CI workflow simply runs the same hooks that developers run locally, ensuring consistency between local development and CI.

This approach:

  • Eliminates duplicate configuration between CI and pre-commit
  • Makes CI failures locally reproducible
  • Reduces maintenance burden (update one place, not two)
  • Ensures developers see the same errors locally as in CI

Lint & Format Job

Runs all pre-commit hooks defined in .pre-commit-config.yaml:

lint:
  name: Lint & Format (pre-commit)
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: "3.11"
    - name: Setup Terraform
      uses: hashicorp/setup-terraform@v3
      with:
        terraform_version: "1.5"
    - name: Install dependencies
      run: |
        pip install pre-commit
        pip install pytest pytest-mock pytest-cov requests kubernetes python-dotenv
    - name: Cache pre-commit hooks
      uses: actions/cache@v4
      with:
        path: ~/.cache/pre-commit
        key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
    - name: Initialize Terraform (for validation hooks)
      run: terraform -chdir=terraform init -backend=false
    - name: Run pre-commit
      run: pre-commit run --all-files

What it does:

  • Sets up Python 3.11 and Terraform 1.5
  • Installs pre-commit and test dependencies
  • Caches pre-commit hooks for faster runs
  • Initializes Terraform (required for validation hooks)
  • Runs ALL pre-commit hooks on all files

Pre-commit hooks include:

  • Python: ruff (linting/formatting), flake8, mypy, pyupgrade
  • Terraform: fmt, validate, docs, checkov
  • YAML: yamlfmt, yamllint
  • Markdown: markdownlint, format-markdown
  • Shell: shellcheck
  • Git: gitlint (commit messages)
  • GitHub: check-github-workflows

Test Job

Runs pytest test suite with proper test reporting:

test:
  name: Tests
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: "3.11"
    - name: Install dependencies
      run: |
        pip install pytest pytest-mock pytest-cov requests kubernetes python-dotenv
    - name: Run tests
      run: pytest tests/ -v --tb=short --no-cov

What it does:

  • Sets up Python 3.11
  • Installs test dependencies
  • Runs pytest with verbose output
  • Disables coverage in CI (enabled in local development via pyproject.toml)

Why separate from pre-commit?

  • Provides dedicated test reporting in GitHub Actions UI
  • Allows test results to be tracked independently
  • Enables future test result caching and parallelization

Security Job

Scans Terraform code for security issues using Checkov:

security:
  name: Security Scan (Checkov)
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - name: Run Checkov
      uses: bridgecrewio/checkov-action@v12
      with:
        directory: terraform/
        quiet: true
        compact: true
        soft_fail: true

What it does:

  • Runs Bridgecrew's Checkov security scanner
  • Scans Terraform code for security issues
  • Uses soft_fail to warn without blocking (also runs in pre-commit)

Why separate from pre-commit?

  • Uses external GitHub Action with specialized features
  • Provides security-specific reporting in GitHub UI
  • Allows security scans to be tracked and audited separately

Configuration Files

pyproject.toml

Central configuration file for all Python tools:

[tool.ruff]
line-length = 120

[tool.ruff.lint]
select = ["E", "F", "W", "Q", "I"]  # Error, Flake8, Warning, Quotes, Import sorting

[tool.flake8]
max-line-length = 120
extend-ignore = ["E203", "E302", "W503"]

[tool.mypy]
ignore_missing_imports = true
disallow_untyped_defs = true

[tool.pytest.ini_options]
addopts = ["--durations=0", "--last-failed", "--new-first", "-vvv"]
testpaths = ["tests"]

[tool.coverage.run]
branch = true
source = ["scripts"]

[tool.isort]
profile = "black"
line_length = 120

[tool.bandit]
exclude_dirs = ["tests"]
skips = ["B603", "B607", "B404", "B105", "B101"]

[tool.yamllint]
extends = "relaxed"

Consolidation benefits:

  • All Python tool configs in one place
  • No more scattered .bandit, pytest.ini, etc.
  • Tools read from pyproject.toml automatically
  • Easier to maintain consistent settings

package.json

Configuration for Node.js tools (markdownlint, prettier):

{
  "devDependencies": {
    "markdownlint": "^0.39.0",
    "markdownlint-cli": "^0.46.0",
    "prettier": "^3.4.0"
  },
  "prettier": {
    "trailingComma": "es5",
    "tabWidth": 2,
    "semi": false,
    "singleQuote": true
  },
  "markdownlint": {
    "default": true,
    "MD007": false,
    "MD013": { "line_length": 120 },
    "MD030": false
  }
}

Consolidation benefits:

  • Moved from .prettierrc and .markdownlint.json
  • All Node.js tool configs in one file
  • Standard practice for Node.js projects

.pre-commit-config.yaml

Defines all code quality hooks (see Pre-commit Hooks for details).

Trigger Conditions

The CI workflow runs on:

on:
  push:
    branches: [main, master]
  pull_request:
    branches: [main, master]

When it runs:

  • Every push to main or master branches
  • Every pull request targeting main or master

Other Workflows

Charon includes several other GitHub Actions workflows:

PR Review (pr-review.yml)

Automated code review for pull requests using AI:

  • Reviews code changes for quality and best practices
  • Checks for common issues and anti-patterns
  • Provides suggestions for improvements

Issue Triage (issue-triage.yml)

Automatic labeling and triage of GitHub issues:

  • Adds labels based on issue content
  • Assigns to appropriate team members
  • Prioritizes based on keywords

Release (release.yml)

Semantic versioning and release automation:

  • Creates releases based on conventional commits
  • Generates changelogs
  • Tags releases with proper versions

Docker Publish (docker-publish.yml)

Container image publishing:

  • Builds Docker images for services
  • Publishes to GitHub Container Registry
  • Tags images with version and latest

Local Development

To run the same checks locally that CI runs:

# Install pre-commit
pip install pre-commit

# Install hooks (runs automatically on commit)
pre-commit install

# Run all hooks manually
pre-commit run --all-files

# Run specific hook
pre-commit run ruff-check
pre-commit run terraform-fmt

# Run tests
pytest tests/ -v

See Pre-commit Hooks for detailed usage.

Troubleshooting

CI fails but local pre-commit passes

  1. Ensure pre-commit is up to date: pre-commit autoupdate
  2. Clear cache and re-run: pre-commit clean && pre-commit run --all-files
  3. Check if you're using correct Python version: python --version (should be 3.11)
  4. Ensure Terraform is initialized: terraform -chdir=terraform init -backend=false

Tests pass locally but fail in CI

  1. Check test environment variables in tests/conftest.py
  2. Ensure no local .env is affecting tests
  3. Run tests in clean environment: python -m pytest tests/ -v --tb=short --no-cov

Checkov security scan fails

  1. Review Checkov errors in CI logs
  2. Run locally: pre-commit run checkov
  3. Add exceptions in pyproject.toml if needed (with justification)
  4. Checkov is set to soft_fail: true, so it warns without blocking

Best Practices

  1. Run pre-commit before pushing: pre-commit run --all-files
  2. Fix issues locally: Don't rely on CI to catch formatting issues
  3. Test in clean environment: Use conda/venv to match CI environment
  4. Check CI logs: If CI fails, read the full logs for context
  5. Keep configs synchronized: Don't bypass pre-commit hooks

Related Documentation

Navigation: Documentation Index | Home