Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 43 additions & 3 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ The orchestrator for the entire release pipeline. It sequences:
- `client_tests`
- `cli_tests`
- `submodule_tests` (auto-enabled for non-RC `pypi` releases even if left `false`)
- `extras_integration_tests` (runs with `test_type='full'` covering 10 platform/Python combinations)

When invoked via `workflow_call`, two optional overrides are also honored:
- `deploy_testpypi`
Expand Down Expand Up @@ -81,7 +82,9 @@ Primary CI workflow; it runs on PRs (`latest`, `develop`, `pre/*`), merge queue
- **Code quality**: `ruff format`, `ruff check`, `mypy`, `zizmor`, schema regeneration, commit/branch linting.
- **Local tests**: Self-hosted Slurm runners on Python 3.10 and 3.13 (coverage enforced, diff-coverage comments for 3.13).
- **Remote tests**: GitHub-hosted matrix across Windows, Linux, and macOS for Python 3.10–3.13.
- **Optional suites**: CLI tests, version consistency checks, and submodule validation (non-RC release tags only) can be toggled via inputs.
- **Optional suites**: CLI tests, version consistency checks, submodule validation (non-RC release tags only), and `tidy3d-extras` integration tests can be toggled via inputs.
- **Extras integration tests**: When enabled on merge_group, runs basic smoke tests (4 configurations). When called from release workflow, runs full tests (10 configurations covering all architectures and Python 3.10/3.13).
- **Test type control**: `test_type` input ("basic" or "full") can override automatic selection for extras integration tests.
- **Dynamic scope**: Determines which jobs to run based on the event (draft PRs, approvals, merge queue, manual overrides).
- **Outputs**: `workflow_success` summarizes whether every required job succeeded; the release workflow uses this to decide if deployment can continue.

Expand All @@ -91,17 +94,53 @@ Primary CI workflow; it runs on PRs (`latest`, `develop`, `pre/*`), merge queue

Reusable workflow that runs the develop-CLI integration tests. It is usually invoked by the main tests workflow when `cli_tests` is requested but can also be triggered directly.

### `tidy3d-extras-python-client-tests-integration.yml`

Dedicated integration test workflow for `tidy3d-extras` package. Tests the optional extras functionality with two test modes: basic (smoke tests) and full (comprehensive coverage).

**Test Modes:**
- **Basic tests** (default for PRs): 4 test configurations for fast smoke testing
- Linux x86_64 (`ubuntu-latest`) - Python 3.10
- macOS arm64 (`macos-latest`) - Python 3.10
- Windows x64 (`windows-latest`) - Python 3.10
- Windows x64 (`windows-latest`) - Python 3.13
- **Full tests** (default for releases): 10 test configurations covering all architectures and Python versions
- Linux x86_64 (`ubuntu-latest`) - Python 3.10, 3.13
- Linux aarch64 (`linux-arm64`) - Python 3.10, 3.13
- macOS x86_64 (`macos-15-intel`) - Python 3.10, 3.13
- macOS arm64 (`macos-latest`) - Python 3.10, 3.13
- Windows x64 (`windows-latest`) - Python 3.10, 3.13

**Key Features:**
- **Test type control**: `test_type` input ("basic" or "full") determines scope
- **Automatic selection**: Basic for merge_group (PR merges), full for release workflow
- **Architecture coverage**: Full mode tests all runner architectures where wheels are built (x86_64, aarch64, arm64)
- **Python version coverage**: Full mode tests both minimum supported (3.10) and latest (3.13) Python versions
- **AWS CodeArtifact integration**: Authenticates with CodeArtifact to access private dependencies
- **Comprehensive test coverage**: Includes doctests, extras license verification, and full test suite with coverage reporting
- **Release tag support**: Can test against a specific release tag via the `release_tag` input
- **Invocation**: Called from `tidy3d-python-client-tests.yml` when `extras_integration_tests` is enabled, or run manually via `workflow_dispatch`
- **Outputs**: `workflow_success` indicates whether all integration tests passed

The workflow ensures that the `tidy3d-extras` package installs and functions correctly across all supported platforms and architectures before releases.

## Maintenance Workflows

### `tidy3d-python-client-daily.yml`

Scheduled at 05:00 UTC and also manually runnable. It fans out to:
- `tidy3d-python-client-update-lockfile.yml` – keeps dependencies fresh.
- The submodule smoke-test workflow – ensures docs/notebooks submodules stay aligned (same helper the release tests call).
- `tidy3d-python-client-release.yml` – runs a daily draft release (`daily-0.0.0`) with client and CLI tests enabled to catch breaking changes early. This validates that the package can be built and tested against the latest develop branch without actually publishing artifacts.

### `tidy3d-python-client-update-lockfile.yml`

Manual or called workflow that updates `poetry.lock`, authenticates against AWS CodeArtifact, and opens a PR on `develop` with the refreshed lockfile (`daily-chore/update-poetry-lock`). Requires `AWS_CODEARTIFACT_ACCESS_KEY` and `AWS_CODEARTIFACT_ACCESS_SECRET`.
Manual or called workflow that updates `poetry.lock`, authenticates against AWS CodeArtifact, and opens a PR with the refreshed lockfile. Requires `AWS_CODEARTIFACT_ACCESS_KEY` and `AWS_CODEARTIFACT_ACCESS_SECRET`.

**Key inputs:**
- `source_branch` – branch to checkout and update lockfile for (defaults to `develop`). Useful for updating lockfiles on feature branches or release branches.
- `run_workflow` – boolean to enable/disable the workflow execution.

The workflow creates a PR with branch name `chore/update-poetry-lock-{source_branch}` targeting the specified source branch.

## Documentation Workflows

Expand Down Expand Up @@ -212,4 +251,5 @@ Private dependencies are sourced through AWS CodeArtifact:

- Release workflow details: `docs/development/release/version.rst`
- Development guidelines: `AGENTS.md`
- Docker development environment: `docs/development/docker.rst` – comprehensive guide for setting up and using the Docker-based development environment
- General repository info: `README.md`
Loading
Loading