Skip to content

Commit 6292eee

Browse files
committed
feat(postgres): derive version from cloudsync.h, ship upgrade scripts
Before this change, default_version in the control file was frozen at '1.0' across all 16 releases. Every user's pg_extension.extversion reported '1.0' regardless of which .so they ran, no per-release upgrade scripts existed, and no CI check flagged missing ones — so a version bump could silently leave users unable to reach the new release via `ALTER EXTENSION cloudsync UPDATE` and force them into a destructive DROP EXTENSION ... CASCADE; CREATE EXTENSION ... workaround. Single source of truth: the PG extension version is now derived from CLOUDSYNC_VERSION in src/cloudsync.h (what the rest of the repo already tracks). docker/Makefile.postgresql reads it and generates cloudsync.control and cloudsync--<version>.sql at build time from new .in templates; the generated files are gitignored. Per-release upgrade scripts live under src/postgresql/migrations/ as cloudsync--<from>--<to>.sql and are picked up by postgres-install, postgres-package, and all three release Dockerfiles via wildcard. Bootstrap: cloudsync--1.0--1.0.17.sql (comment-only, no SQL surface changes) lets existing extversion='1.0' deployments upgrade cleanly. CI gate: scripts/check-postgres-migration.sh + a postgres-check-migration Make target + a new postgres-migration-check workflow job (both postgres-test and postgres-build now block on it) resolve the previous release's extversion from the most recent semver tag's control file (or CLOUDSYNC_VERSION at that tag for new-scheme tags) and fail the build if cloudsync--<prev>--<curr>.sql is missing. Sub-second; runs on every PR. Also: bumped CLOUDSYNC_VERSION to 1.0.17; taught Dockerfile.supabase to accept CLOUDSYNC_VERSION as a build arg for the image label; swept version-hardcoded filenames out of docker/README.md, docs/internal/supabase-flyio.md, and docs/postgresql/quickstarts/postgres.md. Verified end-to-end: rebuilt the local debug container with a persistent volume at installed_version='1.0', ran `ALTER EXTENSION cloudsync UPDATE;`, confirmed installed_version moved to '1.0.17'.
1 parent 1214933 commit 6292eee

16 files changed

Lines changed: 265 additions & 43 deletions

File tree

.github/workflows/main.yml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,8 +239,24 @@ jobs:
239239
path: dist/${{ matrix.name == 'apple-xcframework' && 'CloudSync.*' || 'cloudsync.*'}}
240240
if-no-files-found: error
241241

242+
postgres-migration-check:
243+
if: ${{ !contains(github.event.head_commit.message, '[auto-update]') }}
244+
runs-on: ubuntu-22.04
245+
name: postgresql migration script check
246+
timeout-minutes: 2
247+
steps:
248+
- uses: actions/checkout@v4.2.2
249+
with:
250+
# Need full history + tags so `git describe` can find the previous
251+
# release tag that the check script compares against.
252+
fetch-depth: 0
253+
254+
- name: verify migration script for current CLOUDSYNC_VERSION
255+
run: make postgres-check-migration
256+
242257
postgres-test:
243258
if: ${{ !contains(github.event.head_commit.message, '[auto-update]') }}
259+
needs: [postgres-migration-check]
244260
runs-on: ubuntu-22.04
245261
name: postgresql ${{ matrix.postgres_tag }} build + test
246262
timeout-minutes: 10
@@ -284,6 +300,7 @@ jobs:
284300
285301
postgres-build:
286302
if: ${{ !contains(github.event.head_commit.message, '[auto-update]') }}
303+
needs: [postgres-migration-check]
287304
runs-on: ${{ matrix.os }}
288305
name: postgresql${{ matrix.postgres_version }}-${{ matrix.name }}-${{ matrix.arch }} build
289306
timeout-minutes: 15

.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,11 @@ dist/
77
/curl/src
88
openssl/
99

10+
# Generated PostgreSQL extension files (produced from .in templates by
11+
# docker/Makefile.postgresql; version is derived from src/cloudsync.h)
12+
/docker/postgresql/cloudsync.control
13+
/src/postgresql/cloudsync--*.sql
14+
1015
# Test artifacts
1116
/coverage
1217
unittest

docker/Makefile.postgresql

Lines changed: 48 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,12 @@ PG_INCLUDEDIR := $(shell $(PG_CONFIG) --includedir-server 2>/dev/null)
1212

1313
# Extension metadata
1414
EXTENSION = cloudsync
15-
EXTVERSION = 1.0
15+
# Read the extension version from src/cloudsync.h (CLOUDSYNC_VERSION) so the
16+
# PG extension version stays in sync with the rest of the repo on every release.
17+
EXTVERSION := $(shell sed -n 's/^\#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' src/cloudsync.h)
18+
ifeq ($(strip $(EXTVERSION)),)
19+
$(error Could not read CLOUDSYNC_VERSION from src/cloudsync.h)
20+
endif
1621

1722
# Detect OS for platform-specific settings
1823
ifneq ($(OS),Windows_NT)
@@ -82,17 +87,32 @@ endif
8287
PG_EXTENSION_SQL = src/postgresql/$(EXTENSION)--$(EXTVERSION).sql
8388
PG_EXTENSION_CONTROL = docker/postgresql/$(EXTENSION).control
8489

90+
# Input templates (tracked). @EXTVERSION@ is substituted at build time.
91+
PG_EXTENSION_SQL_IN = src/postgresql/$(EXTENSION).sql.in
92+
PG_EXTENSION_CONTROL_IN = docker/postgresql/$(EXTENSION).control.in
93+
94+
# Upgrade scripts (cloudsync--<from>--<to>.sql) are hand-written per release.
95+
PG_MIGRATIONS_DIR = src/postgresql/migrations
96+
PG_MIGRATION_SQLS = $(wildcard $(PG_MIGRATIONS_DIR)/$(EXTENSION)--*--*.sql)
97+
8598
# ============================================================================
8699
# PostgreSQL Build Targets
87100
# ============================================================================
88101

89-
.PHONY: postgres-check postgres-build postgres-install postgres-package postgres-clean postgres-test \
102+
.PHONY: postgres-check postgres-check-migration postgres-build postgres-install postgres-package postgres-clean postgres-test \
90103
postgres-docker-build postgres-docker-build-asan postgres-docker-run postgres-docker-run-asan postgres-docker-stop postgres-docker-rebuild \
91104
postgres-docker-debug-build postgres-docker-debug-run postgres-docker-debug-rebuild \
92105
postgres-docker-shell postgres-dev-rebuild postgres-help unittest-pg \
93106
postgres-supabase-build postgres-supabase-rebuild postgres-supabase-run-smoke-test \
94107
postgres-docker-run-smoke-test
95108

109+
# Verify that a cloudsync--<prev>--<curr>.sql upgrade script exists for the
110+
# current CLOUDSYNC_VERSION in src/cloudsync.h. Release-blocking: a missing
111+
# upgrade script silently breaks ALTER EXTENSION cloudsync UPDATE for every
112+
# existing deployment. Runs in <1s; safe to call on every PR.
113+
postgres-check-migration:
114+
@scripts/check-postgres-migration.sh
115+
96116
# Check if PostgreSQL is available
97117
postgres-check:
98118
@echo "Checking PostgreSQL installation..."
@@ -102,9 +122,19 @@ postgres-check:
102122
@echo "Share directory: $(PG_SHAREDIR)"
103123
@echo "Include directory: $(PG_INCLUDEDIR)"
104124

125+
# Generate the versioned install script from the template
126+
$(PG_EXTENSION_SQL): $(PG_EXTENSION_SQL_IN) src/cloudsync.h
127+
@echo "Generating $@ (version $(EXTVERSION))"
128+
@sed 's/@EXTVERSION@/$(EXTVERSION)/g' $(PG_EXTENSION_SQL_IN) > $@
129+
130+
# Generate the control file from the template
131+
$(PG_EXTENSION_CONTROL): $(PG_EXTENSION_CONTROL_IN) src/cloudsync.h
132+
@echo "Generating $@ (version $(EXTVERSION))"
133+
@sed 's/@EXTVERSION@/$(EXTVERSION)/g' $(PG_EXTENSION_CONTROL_IN) > $@
134+
105135
# Build PostgreSQL extension
106-
postgres-build: postgres-check
107-
@echo "Building PostgreSQL extension..."
136+
postgres-build: postgres-check $(PG_EXTENSION_SQL) $(PG_EXTENSION_CONTROL)
137+
@echo "Building PostgreSQL extension (version $(EXTVERSION))..."
108138
@echo "Compiling source files..."
109139
@for src in $(PG_ALL_SRC); do \
110140
echo " CC $$src"; \
@@ -125,26 +155,37 @@ postgres-install: postgres-build
125155
install -m 644 $(PG_EXTENSION_SQL) $(PG_SHAREDIR)/extension/
126156
@echo "Installing control file to $(PG_SHAREDIR)/extension/"
127157
install -m 644 $(PG_EXTENSION_CONTROL) $(PG_SHAREDIR)/extension/
158+
@if [ -n "$(PG_MIGRATION_SQLS)" ]; then \
159+
echo "Installing $(words $(PG_MIGRATION_SQLS)) migration script(s) to $(PG_SHAREDIR)/extension/"; \
160+
install -m 644 $(PG_MIGRATION_SQLS) $(PG_SHAREDIR)/extension/; \
161+
fi
128162
@echo ""
129163
@echo "Installation complete!"
130164
@echo "To use the extension, run in psql:"
131165
@echo " CREATE EXTENSION $(EXTENSION);"
166+
@echo "To upgrade an existing installation, run in psql:"
167+
@echo " ALTER EXTENSION $(EXTENSION) UPDATE;"
132168

133169
# Package extension files for distribution
134170
PG_DIST_DIR = dist/postgresql
135171

136172
postgres-package: postgres-build
137-
@echo "Packaging PostgreSQL extension..."
173+
@echo "Packaging PostgreSQL extension (version $(EXTVERSION))..."
138174
@mkdir -p $(PG_DIST_DIR)
139175
cp $(PG_EXTENSION_LIB) $(PG_DIST_DIR)/
140176
cp $(PG_EXTENSION_SQL) $(PG_DIST_DIR)/
141177
cp $(PG_EXTENSION_CONTROL) $(PG_DIST_DIR)/
178+
@if [ -n "$(PG_MIGRATION_SQLS)" ]; then \
179+
echo "Including $(words $(PG_MIGRATION_SQLS)) migration script(s)"; \
180+
cp $(PG_MIGRATION_SQLS) $(PG_DIST_DIR)/; \
181+
fi
142182
@echo "Package ready in $(PG_DIST_DIR)/"
143183

144184
# Clean PostgreSQL build artifacts
145185
postgres-clean:
146186
@echo "Cleaning PostgreSQL build artifacts..."
147187
rm -f $(PG_OBJS) $(PG_EXTENSION_LIB)
188+
rm -f $(PG_EXTENSION_SQL) $(PG_EXTENSION_CONTROL)
148189
@echo "Clean complete"
149190

150191
# Test extension (requires running PostgreSQL)
@@ -314,7 +355,7 @@ postgres-supabase-build:
314355
echo "Using base image: $$supabase_cli_image"; \
315356
echo "Pulling fresh base image to avoid layer accumulation..."; \
316357
docker pull "$$supabase_cli_image" 2>/dev/null || true; \
317-
docker build --build-arg SUPABASE_POSTGRES_TAG="$(SUPABASE_POSTGRES_TAG)" -f "$$tmp_dockerfile" -t "$$supabase_cli_image" .; \
358+
docker build --build-arg SUPABASE_POSTGRES_TAG="$(SUPABASE_POSTGRES_TAG)" --build-arg CLOUDSYNC_VERSION="$(EXTVERSION)" -f "$$tmp_dockerfile" -t "$$supabase_cli_image" .; \
318359
rm -f "$$tmp_dockerfile"; \
319360
echo "Build complete: $$supabase_cli_image"
320361

@@ -361,6 +402,7 @@ postgres-help:
361402
@echo ""
362403
@echo "Build & Install:"
363404
@echo " postgres-check - Verify PostgreSQL installation"
405+
@echo " postgres-check-migration - Verify a migration script exists for the current version"
364406
@echo " postgres-build - Build extension (.so file)"
365407
@echo " postgres-install - Install extension to PostgreSQL"
366408
@echo " postgres-clean - Clean build artifacts"

docker/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ docker/
1010
│ ├── Dockerfile # Custom PostgreSQL image
1111
│ ├── docker-compose.yml
1212
│ ├── init.sql # CloudSync metadata tables
13-
│ └── cloudsync.control
13+
│ └── cloudsync.control.in # template; cloudsync.control is generated at build time
1414
```
1515

1616
## Option 1: Standalone PostgreSQL

docker/postgresql/Dockerfile.release

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ RUN case "${TARGETARCH}" in \
3131
mkdir -p /tmp/cloudsync && \
3232
tar -xzf /tmp/cloudsync.tar.gz -C /tmp/cloudsync && \
3333
install -m 755 /tmp/cloudsync/cloudsync.so "$(pg_config --pkglibdir)/" && \
34-
install -m 644 /tmp/cloudsync/cloudsync--1.0.sql "$(pg_config --sharedir)/extension/" && \
34+
install -m 644 /tmp/cloudsync/cloudsync--*.sql "$(pg_config --sharedir)/extension/" && \
3535
install -m 644 /tmp/cloudsync/cloudsync.control "$(pg_config --sharedir)/extension/" && \
3636
rm -rf /tmp/cloudsync /tmp/cloudsync.tar.gz && \
3737
apt-get purge -y curl && apt-get autoremove -y && rm -rf /var/lib/apt/lists/*

docker/postgresql/Dockerfile.supabase

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,13 +30,21 @@ RUN if [ ! -x "$CLOUDSYNC_PG_CONFIG" ]; then \
3030
# Collect build artifacts (avoid installing into the Nix store)
3131
RUN mkdir -p /tmp/cloudsync-artifacts/lib /tmp/cloudsync-artifacts/extension && \
3232
cp /tmp/cloudsync/cloudsync.so /tmp/cloudsync-artifacts/lib/ && \
33-
cp /tmp/cloudsync/src/postgresql/cloudsync--1.0.sql /tmp/cloudsync-artifacts/extension/ && \
34-
cp /tmp/cloudsync/docker/postgresql/cloudsync.control /tmp/cloudsync-artifacts/extension/
33+
cp /tmp/cloudsync/src/postgresql/cloudsync--*.sql /tmp/cloudsync-artifacts/extension/ && \
34+
cp /tmp/cloudsync/docker/postgresql/cloudsync.control /tmp/cloudsync-artifacts/extension/ && \
35+
# Include per-release upgrade scripts so ALTER EXTENSION ... UPDATE works
36+
if ls /tmp/cloudsync/src/postgresql/migrations/cloudsync--*--*.sql 1>/dev/null 2>&1; then \
37+
cp /tmp/cloudsync/src/postgresql/migrations/cloudsync--*--*.sql /tmp/cloudsync-artifacts/extension/; \
38+
fi
3539

3640
# Runtime image based on Supabase Postgres
3741
ARG SUPABASE_POSTGRES_TAG=17.6.1.071
3842
FROM public.ecr.aws/supabase/postgres:${SUPABASE_POSTGRES_TAG}
3943

44+
# Extension version (derived from src/cloudsync.h by the Makefile and passed in
45+
# as a build arg); used only for the image label.
46+
ARG CLOUDSYNC_VERSION=unknown
47+
4048
# Match builder pg_config path
4149
ENV CLOUDSYNC_PG_CONFIG=/root/.nix-profile/bin/pg_config
4250

@@ -85,5 +93,5 @@ EXPOSE 5432
8593
WORKDIR /
8694

8795
# Add label with extension version
88-
LABEL org.sqliteai.cloudsync.version="1.0" \
96+
LABEL org.sqliteai.cloudsync.version="${CLOUDSYNC_VERSION}" \
8997
org.sqliteai.cloudsync.description="PostgreSQL with CloudSync CRDT extension"

docker/postgresql/Dockerfile.supabase.release

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,10 +42,10 @@ RUN case "${TARGETARCH}" in \
4242
SHAREDIR_STD="/usr/share/postgresql" && \
4343
install -d "$PKGLIBDIR" "$SHAREDIR_PGCONFIG/extension" && \
4444
install -m 755 /tmp/cloudsync/cloudsync.so "$PKGLIBDIR/" && \
45-
install -m 644 /tmp/cloudsync/cloudsync--1.0.sql /tmp/cloudsync/cloudsync.control "$SHAREDIR_PGCONFIG/extension/" && \
45+
install -m 644 /tmp/cloudsync/cloudsync--*.sql /tmp/cloudsync/cloudsync.control "$SHAREDIR_PGCONFIG/extension/" && \
4646
if [ "$SHAREDIR_STD" != "$SHAREDIR_PGCONFIG" ]; then \
4747
install -d "$SHAREDIR_STD/extension" && \
48-
install -m 644 /tmp/cloudsync/cloudsync--1.0.sql /tmp/cloudsync/cloudsync.control "$SHAREDIR_STD/extension/"; \
48+
install -m 644 /tmp/cloudsync/cloudsync--*.sql /tmp/cloudsync/cloudsync.control "$SHAREDIR_STD/extension/"; \
4949
fi && \
5050
rm -rf /tmp/cloudsync /tmp/cloudsync.tar.gz && \
5151
apt-get purge -y curl && apt-get autoremove -y && rm -rf /var/lib/apt/lists/*

docker/postgresql/cloudsync.control

Lines changed: 0 additions & 22 deletions
This file was deleted.
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# CloudSync PostgreSQL Extension Control File
2+
#
3+
# Generated from cloudsync.control.in by docker/Makefile.postgresql.
4+
# Do not edit the generated file; edit the .in template instead.
5+
# The version below is read from CLOUDSYNC_VERSION in src/cloudsync.h.
6+
7+
comment = 'CloudSync - CRDT-based multi-master database synchronization'
8+
default_version = '@EXTVERSION@'
9+
relocatable = true
10+
requires = ''
11+
superuser = false
12+
module_pathname = '$libdir/cloudsync'
13+
trusted = true

docs/internal/supabase-flyio.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -86,10 +86,10 @@ The `make postgres-supabase-build` command does the following:
8686
1. **Pulls the official Supabase Postgres base image** (e.g., `public.ecr.aws/supabase/postgres:15.8.1.085`) — this is Supabase's standard PostgreSQL image that ships with ~30 extensions pre-installed (PostGIS, pgvector, etc.)
8787
2. **Runs a multi-stage Docker build** using `docker/postgresql/Dockerfile.supabase`:
8888
- **Stage 1 (builder)**: Installs C build tools (`gcc`, `make`), copies the CloudSync source code (`src/`, `modules/`), and compiles `cloudsync.so` against Supabase's `pg_config`
89-
- **Stage 2 (runtime)**: Starts from a clean Supabase Postgres image and copies in just three files:
89+
- **Stage 2 (runtime)**: Starts from a clean Supabase Postgres image and copies in just three kinds of file:
9090
- `cloudsync.so` — the compiled extension binary
91-
- `cloudsync.control` — tells PostgreSQL the extension's name and version
92-
- `cloudsync--1.0.sql` — the SQL that defines all CloudSync functions
91+
- `cloudsync.control` — tells PostgreSQL the extension's name and default version (generated at build time from `cloudsync.control.in`, with the version read from `src/cloudsync.h`)
92+
- `cloudsync--<version>.sql` — the SQL that defines all CloudSync functions for the current release (e.g. `cloudsync--1.0.16.sql`), plus any `cloudsync--<from>--<to>.sql` upgrade scripts shipped under `src/postgresql/migrations/`
9393
3. **Tags the result** with the same name as the base image, so it's a drop-in replacement
9494

9595
To find the correct tag, clone the Supabase repo and check:
@@ -118,7 +118,8 @@ Verify CloudSync is installed inside the image:
118118
```bash
119119
docker run --rm <your-dockerhub-username>/supabase-postgres-cloudsync:15.8.1.085 \
120120
find / -name "cloudsync*" -type f 2>/dev/null
121-
# Should list cloudsync.so, cloudsync.control, and cloudsync--1.0.sql
121+
# Should list cloudsync.so, cloudsync.control, and cloudsync--<version>.sql
122+
# (plus any cloudsync--<from>--<to>.sql upgrade scripts)
122123
# in /nix/store/...-postgresql-and-plugins-15.8/ paths
123124
```
124125

0 commit comments

Comments
 (0)