Adjusted CI to use the hardened demo smoke test so GitHub stops failing after SIGKILL:

.github/workflows/ci.yml now runs demo/postgres-survive-kill9/ci-smoke.sh instead of the inline snippet. The script already force-recreates the Postgres service after a kill and waits longer for readiness/pgbench tables—this is what fixed the local failures.
Added a brief note in the demo README about mirroring the script in CI if needed.
./ci-smoke.sh passes locally; this should resolve the GitHub “Postgres did not restart” failure.

Author: Shane Wall

.github/workflows/ci.yml

@@ -26,46 +26,4 @@ jobs:
         run: |
           set -euo pipefail
           cd demo/postgres-survive-kill9
-          docker compose up -d
-
-          echo "Waiting for Postgres to be ready..."
-          for i in {1..30}; do
-            if docker compose exec -T postgres pg_isready -U postgres >/dev/null 2>&1; then
-              ready=1
-              break
-            fi
-            sleep 2
-          done
-          if [ "${ready:-0}" != "1" ]; then
-            echo "Postgres did not become ready in time" >&2
-            exit 1
-          fi
-
-          baseline=$(docker compose exec -T postgres psql -U postgres -d bench -t -A -c "SELECT count(*) FROM pgbench_history;" 2>/dev/null || echo "0")
-          echo "Baseline transactions: ${baseline}"
-
-          docker kill -s KILL "$(docker compose ps -q postgres)"
-
-          echo "Waiting for Postgres to restart..."
-          unset ready
-          for i in {1..30}; do
-            sleep 2
-            if docker compose exec -T postgres pg_isready -U postgres >/dev/null 2>&1; then
-              ready=1
-              break
-            fi
-          done
-          if [ "${ready:-0}" != "1" ]; then
-            echo "Postgres did not restart" >&2
-            exit 1
-          fi
-
-          after=$(docker compose exec -T postgres psql -U postgres -d bench -t -A -c "SELECT count(*) FROM pgbench_history;" 2>/dev/null || echo "0")
-          echo "Post-restart transactions: ${after}"
-          if [ "${after}" -lt "${baseline}" ]; then
-            echo "Transaction count decreased after kill -9" >&2
-            exit 1
-          fi
-
-          curl -sf http://localhost:9911/metrics | grep diffkeeper_recovery_total
-          docker compose down -v
+          ./ci-smoke.sh

demo/postgres-survive-kill9/README.md

@@ -36,6 +36,17 @@ For the scripted check that kills Postgres once and asserts the counter never dr
 ./ci-smoke.sh
 ```
 
+If you need an inline CI snippet, mirror what `ci-smoke.sh` does (restart after SIGKILL and wait for pgbench tables):
+
+```bash
+docker compose up -d
+./ci-smoke.sh  # simplest
+# or inline:
+# docker compose kill -s KILL postgres
+# docker compose up -d --force-recreate postgres
+# ...wait for pg_isready and pgbench_history to exist...
+```
+
 You should see the transaction count increase continuously, even while the container is being killed.
 
 ## Metrics

demo/postgres-survive-kill9/ci-smoke.sh

@@ -12,7 +12,7 @@ trap cleanup EXIT
 wait_pg() {
   echo "Waiting for Postgres to be ready..."
   local ready=0
-  for _ in {1..60}; do
+  for _ in {1..120}; do
     if docker compose exec -T postgres pg_isready -U postgres >/dev/null 2>&1; then
       ready=1
       break
@@ -27,7 +27,7 @@ wait_pg() {
 
 wait_pgbench_tables() {
   echo "Waiting for pgbench tables..."
-  for _ in {1..60}; do
+  for _ in {1..120}; do
     if docker compose exec -T postgres psql -U postgres -d bench -t -A -c "SELECT 1 FROM pg_tables WHERE tablename='pgbench_history';" 2>/dev/null | grep -q 1; then
       return
     fi
@@ -47,7 +47,7 @@ echo "Baseline transactions: ${baseline}"
 docker compose kill -s KILL postgres
 
 echo "Waiting for Postgres to restart..."
-docker compose up -d postgres >/dev/null 2>&1
+docker compose up -d --force-recreate postgres >/dev/null 2>&1
 wait_pg
 wait_pgbench_tables