Skip to content

Pgpkgmigrations#437

Merged
bitner merged 9 commits into
mainfrom
pgpkgmigrations
May 12, 2026
Merged

Pgpkgmigrations#437
bitner merged 9 commits into
mainfrom
pgpkgmigrations

Conversation

@bitner
Copy link
Copy Markdown
Collaborator

@bitner bitner commented May 11, 2026

Technical Context

  • Related Issues: N/A
  • Breaking Changes: Yes — migration filenames are renamed from pgstac.X.Y.Z.sql / pgstac.X.Y.Z-A.B.C.sql to pgstac--X.Y.Z.sql / pgstac--X.Y.Z--A.B.C.sql. Any tooling or scripts that reference migration files by the old dotted naming convention will need to be updated. The pypgstac migrate CLI and Python API continue to work without changes.

Description

This PR introduces pgstac-migrate, a new lightweight standalone Python package that owns all of PgSTAC's runtime migration logic, and backs stageversion / makemigration with pgpkg for reproducible SQL assembly and diff-based migration generation.

Migration filename canonicalization

All migration files are renamed to use the double-dash separator:
pgstac.0.9.10.sql → pgstac--0.9.10.sql
pgstac.0.9.10-0.9.11.sql → pgstac--0.9.10--0.9.11.sql

This matches the standard PostgreSQL extension file naming convention (<extension>--<version>.sql, <extension>--<from>--<to>.sql), removes ambiguity between version separators and the dots in version numbers, and improves interoperability with extension tooling like pgpkg.

New pgstac-migrate package

src/pgstac-migrate/ is a focused package that can be installed independently of pypgstac:

  • CLI: pgstac-migrate migrate [--to VERSION] [--dry-run], plus info, versions, plan, build-artifact
  • Python API: pgstac_migrate.api.migrate(target, conninfo, ...) — usable directly without the full pypgstac stack
  • Baked artifact: migration SQL is assembled into a single migrations.tar.zst bundle at publish time. This ships at 74 KB versus the 8.1 MB of raw SQL that pypgstac previously bundled via a symlink — roughly a 110× reduction in migration payload. Those bytes move entirely out of pypgstac and into pgstac-migrate.

pypgstac migrate is maintained as a compatibility wrapper — with planned deprecation

pypgstac migrate continues to work exactly as before; Migrate.run_migration() now delegates to pgstac_migrate.api.migrate() internally. No changes to calling code are required. This compatibility surface will be maintained for the near term but is planned for deprecation in a future minor release. Users should begin migrating to pgstac-migrate CLI or API directly.

pgpkg-backed stageversion and makemigration

The in-container scripts now use pgpkg for SQL assembly and migration diffing, replacing the previous approach that used migra against two live databases. The optional PGPKG_LOCAL_REPO_DIR / PGPKG_REPO_DIR environment variable allows testing against a local pgpkg checkout; default behavior resolves pgpkg from PyPI with no local setup required.

Reviewers: please focus on src/pgstac-migrate/ (new package structure and API surface), src/pypgstac/src/pypgstac/migrate.py (delegation wiring), and the cross-surface parity test at src/pgstac-migrate/tests/test_parity.py which asserts both tools produce identical ordered SQL for all migration paths.


Checklist

  • Linting: Ruff lint and format pass; ty type checks pass; all pre-commit hooks pass
  • Tests: All existing tests pass. New tests added:
    • src/pgstac-migrate/tests/ — CLI delegation, API, version source, and cross-surface plan parity (7 parametrized cases asserting pgstac-migrate catalog and pypgstac MigrationPath produce identical SQL file sequences)
    • src/pypgstac/tests/test_migrate_wrapper.py — delegation and compat layer coverage
    • scripts/container-scripts/test wired to run pgstac-migrate tests in CI
  • Edge Cases: Parity test covers fresh install, single-hop, and multi-hop upgrade paths; dry-run mode verified; delegation wiring tested with monkeypatching to confirm correct kwargs forwarded
  • Documentation: CHANGELOG.md and docs/src/release-notes.md updated; CONTRIBUTING.md updated to reflect two PyPI packages and new release workflow; CLAUDE.md and AGENTS.md updated with pgstac-migrate architecture and migration-engineer workflow
  • Accountability: I can explain every line submitted

AI tool usage

Copilot assisted with test scaffolding (test_parity.py, delegation tests), changelog entries, and pre-commit fix diagnosis. All generated content was reviewed, run, and validated before inclusion.

@bitner bitner requested review from gadomski and hrodmn May 11, 2026 18:10
Comment thread .github/workflows/release.yml Outdated
Comment thread .github/workflows/release.yml Outdated
Comment thread scripts/container-scripts/test Outdated
Comment thread scripts/makemigration Outdated
Comment thread src/pgstac-migrate/README.md
Comment thread src/pypgstac/src/pypgstac/migrate.py
Comment thread src/pypgstac/pyproject.toml Outdated
bitner and others added 3 commits May 12, 2026 09:15
Co-authored-by: Pete Gadomski <pete.gadomski@gmail.com>
Co-authored-by: Pete Gadomski <pete.gadomski@gmail.com>
- expand pgstac-migrate README with full CLI/API/env var docs and troubleshooting
- make psycopg[binary] mandatory in pgstac-migrate and pypgstac
- make psycopg-pool mandatory in pypgstac
- remove redundant psycopg optional/group wiring and update test script flags
- remove pgstac-migrate upper bound in pypgstac dependency
- update release workflow paths and uv setup/build step
- refresh docs/changelog references for pgpkg>=0.1.1
- regenerate uv lockfiles
@bitner bitner requested a review from gadomski May 12, 2026 17:44
@bitner bitner merged commit a911d64 into main May 12, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants