Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .docker/ci-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/bash
set -e

# Install system dependencies quietly
apt-get update -qq
apt-get install -y -qq make > /dev/null 2>&1

# Install Python dependencies quietly
pip install -q -e . 2>/dev/null
pip install -q mypy pytest pytest-asyncio pytest-cov black ruff 2>/dev/null

# Execute the passed command
exec "$@"
102 changes: 102 additions & 0 deletions .github/AUTH_SETUP.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# Pattern Stack Authentication Setup

This document explains how to set up authentication for Pattern Stack repositories to access private dependencies.

## Current Setup: Service Account + PAT

### 1. Create Service Account (One-time per organization)

1. **Create GitHub Account**
- Create a new GitHub account: `pattern-stack-ci`
- Use an email like `ci@pattern-stack.com`

2. **Add to Organization**
- Invite `pattern-stack-ci` to the `pattern-stack` organization
- Grant `Read` access to repositories that need to be accessed by CI
- Specifically ensure access to:
- `pattern-stack/geography-patterns`
- `pattern-stack/backend-patterns`

### 2. Generate Personal Access Token

1. **Login as Service Account**
- Login to GitHub as `pattern-stack-ci`

2. **Create PAT**
- Go to Settings → Developer settings → Personal access tokens → Tokens (classic)
- Click "Generate new token (classic)"
- **Name**: `Pattern Stack CI Access`
- **Expiration**: 1 year (set calendar reminder to rotate)
- **Scopes**:
- ✅ `repo` (Full control of private repositories)
- Generate and copy the token

### 3. Add to Repository Secrets

For each repository that needs access:

1. Go to repository Settings → Secrets and variables → Actions
2. Click "New repository secret"
3. **Name**: `PATTERN_STACK_TOKEN`
4. **Value**: The PAT from step 2
5. Save

### 4. Verify Setup

The workflows should now:
- Use `PATTERN_STACK_TOKEN` for checkout and git configuration
- Successfully install dependencies from private repositories
- Pass all CI checks

## Auth Pattern Used in Workflows

All workflows use this consistent pattern:

```yaml
steps:
- uses: actions/checkout@v4
with:
token: ${{ secrets.PATTERN_STACK_TOKEN }}

- name: Configure git for private repo access
run: |
git config --global url."https://x-access-token:${{ secrets.PATTERN_STACK_TOKEN }}@github.com/".insteadOf "https://github.com/"

- name: Install dependencies
run: |
uv sync --frozen
# Dependencies from private repos now work
```

## Future Migration: GitHub App

When scaling to multiple repositories, we'll migrate to a GitHub App approach:

1. **Benefits**: Better security, automatic token rotation, granular permissions
2. **Implementation**: Pattern Stack tooling will automate the creation and installation
3. **Migration**: Seamless - workflows use same `PATTERN_STACK_TOKEN` interface

## Troubleshooting

### Common Issues

1. **"fatal: could not read Username"**
- Verify `PATTERN_STACK_TOKEN` secret exists in repository
- Check service account has access to target repositories
- Verify PAT has `repo` scope

2. **PAT Expired**
- Generate new PAT with same scopes
- Update `PATTERN_STACK_TOKEN` secret in all repositories
- Set calendar reminder for next rotation

3. **403 Forbidden**
- Service account needs to be added to private repositories
- Check organization membership and repository access

### Security Notes

- PAT has broad access - rotate regularly (annually)
- Only add to repositories that need private dependency access
- Consider GitHub App migration for better security posture
- Monitor usage in organization audit logs
84 changes: 84 additions & 0 deletions .github/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# GitHub Actions Workflows

This directory contains CI/CD workflows for the geography-patterns monorepo.

## Workflow Structure

### Per-Project Testing
- **`test-wof-explorer.yml`** - Tests for WOF Explorer package
- **`test-geo-platform.yml`** - Tests for Geo Platform package

### Quality Checks
- **`quality-checks.yml`** - Linting, type checking, and formatting checks across both packages

### Orchestration
- **`ci.yml`** - Main CI workflow that runs all checks together

## Workflow Details

### Test WOF Explorer (`test-wof-explorer.yml`)
- **Triggers**: Changes to `wof-explorer/` directory, workflow file, or dependencies
- **Python versions**: 3.11, 3.12, 3.13
- **Test database**: Downloads Barbados WOF database for integration tests
- **Commands**:
- `uv run pytest tests/ -v`
- `uv run pytest tests/test_examples.py -v`

### Test Geo Platform (`test-geo-platform.yml`)
- **Triggers**: Changes to `geo-platform/` directory, workflow file, or dependencies
- **Python versions**: 3.11, 3.12, 3.13
- **Services**: PostgreSQL with PostGIS extension
- **Commands**:
- `uv run pytest __tests__/unit/ -v`
- `uv run pytest __tests__/integration/ -v`
- `uv run pytest __tests__/ -v`

### Quality Checks (`quality-checks.yml`)
- **Triggers**: All pushes and PRs
- **Matrix**: Runs for both `wof-explorer` and `geo-platform`
- **Jobs**:
- **Lint**: `uv run ruff check .`
- **Typecheck**: `uv run mypy src/`
- **Format Check**: `uv run ruff format --check .` (+ black for WOF Explorer)

### Main CI (`ci.yml`)
- **Triggers**: Pushes to main/develop branches, all PRs
- **Strategy**: Orchestrates all other workflows
- **Final check**: Ensures all sub-workflows pass before marking CI as successful

## Quality Standards

### Expected Results
- **Geo Platform**: All checks should pass (0 linting issues, 0 type issues)
- **WOF Explorer**: Known issues documented (41 linting issues, 343 type issues)

### Failure Handling
- Geo Platform failures block CI
- WOF Explorer quality issues are documented but don't block CI (`continue-on-error: true`)
- Test failures always block CI for both packages

## Local Development

Run the same checks locally using Make commands:

```bash
# Run all checks
make check

# Per-package testing
make test-wof
make test-geo

# Quality checks
make lint
make typecheck
make format
```

## Path-Based Triggers

Workflows are optimized to only run when relevant files change:

- Package-specific workflows only trigger on changes to their respective directories
- Quality checks run on all changes
- Dependencies changes (pyproject.toml, uv.lock) trigger relevant workflows
16 changes: 16 additions & 0 deletions .github/actions/setup/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Setup Environment
description: Setup cli-patterns environment

inputs:
python-version:
description: Python version
default: '3.9'

runs:
using: composite
steps:
# In future, this would be: pattern-stack/actions/setup@v1
# For now, use local pattern-stack standard
- uses: ./.github/workflows/pattern-stack/setup
with:
python-version: ${{ inputs.python-version }}
85 changes: 85 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
name: CI

on:
push:
branches: [main, develop]
pull_request:
workflow_dispatch:
inputs:
use_docker:
description: 'Run tests in Docker'
type: boolean
default: false

env:
PYTHONPATH: src

jobs:
quality:
name: Quality Checks
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup
- run: make quality

test:
name: Test - ${{ matrix.suite }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
suite: [unit, integration, parser, executor, design]

steps:
- uses: actions/checkout@v4

# Native path (default)
- if: ${{ !inputs.use_docker }}
uses: ./.github/actions/setup
- if: ${{ !inputs.use_docker }}
run: make test-${{ matrix.suite }}

# Docker path (on demand)
- if: ${{ inputs.use_docker }}
run: |
docker compose -f docker-compose.ci.yml run \
ci make test-${{ matrix.suite }}

test-fast:
name: Fast Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup
- run: make test-fast

# Python compatibility check (on main branch)
compatibility:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
name: Python ${{ matrix.python }}
runs-on: ubuntu-latest
strategy:
matrix:
python: ["3.9", "3.11", "3.13"]

steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup
with:
python-version: ${{ matrix.python }}
- run: make test-fast

# Future: Performance benchmarks
benchmark:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
name: Performance Benchmark
runs-on: ubuntu-latest
continue-on-error: true # Don't fail CI if benchmarks regress (for now)

steps:
- uses: actions/checkout@v4
- name: Run benchmarks in consistent environment
run: |
docker compose -f docker-compose.ci.yml run \
benchmark make benchmark || echo "No benchmarks yet"
26 changes: 26 additions & 0 deletions .github/workflows/pattern-stack/setup/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Pattern Stack Environment Setup
description: Standard environment setup for Pattern Stack projects

inputs:
python-version:
description: Python version to use
default: '3.9'

runs:
using: composite
steps:
- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true

- name: Setup Python
shell: bash
run: uv python install ${{ inputs.python-version }}

- name: Install dependencies
shell: bash
run: |
uv sync --frozen --all-extras
uv pip install mypy pytest pytest-asyncio pytest-cov black ruff
echo "✅ Pattern Stack environment ready (Python ${{ inputs.python-version }})"
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -176,3 +176,8 @@ Desktop.ini
*.swp
*.swo
*~.cli_patterns_history

# CI artifacts
/tmp/
benchmark_results.json
# .docker/ # Commented out to allow CI entrypoint script
Loading
Loading