Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/MainDistributionPipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
name: Build extension binaries
uses: duckdb/extension-ci-tools/.github/workflows/_extension_distribution.yml@v1.5-variegata
with:
duckdb_version: v1.5.2
duckdb_version: v1.5.3
ci_tools_version: v1.5-variegata
extension_name: quackscale
extra_toolchains: 'go'
Expand All @@ -27,7 +27,7 @@ jobs:
name: Code Quality Check
uses: duckdb/extension-ci-tools/.github/workflows/_extension_code_quality.yml@v1.5-variegata
with:
duckdb_version: v1.5.2
duckdb_version: v1.5.3
ci_tools_version: v1.5-variegata
extension_name: quackscale
format_checks: 'format;tidy'
156 changes: 146 additions & 10 deletions .github/workflows/Release.yml
Original file line number Diff line number Diff line change
@@ -1,28 +1,164 @@
# Build a Linux DuckDB binary with quackscale embedded and attach it to the GitHub release.
# E2e workflows download this artifact instead of compiling from source.
# Publish unsigned quackscale loadable binaries to GitHub Pages (INSTALL FROM …)
# and attach a linux QuackTail tarball (duckdb + extension) to GitHub Releases.
#
# One-time setup: repo Settings → Pages → Build and deployment → Source: GitHub Actions.
#
# Layout after deploy:
# https://quackscience.github.io/duckdb-quackscale/v1.5.3/linux_amd64/quackscale.duckdb_extension.gz
#
# Users:
# SET allow_unsigned_extensions = true;
# INSTALL quackscale FROM 'https://quackscience.github.io/duckdb-quackscale';
# LOAD quackscale;
#
# Note: deploy-pages replaces the whole site each run (one DuckDB version hosted).
name: Release

on:
release:
types: [published]
workflow_dispatch:
inputs:
release_tag:
description: 'Git tag for QuackTail linux tarball (manual runs; optional)'
required: false
type: string
skip_pages:
description: 'Skip GitHub Pages packaging and deploy'
required: false
default: false
type: boolean
skip_tarball:
description: 'Skip QuackTail linux tarball upload'
required: false
default: false
type: boolean

permissions:
contents: write
concurrency:
group: pages
cancel-in-progress: false

env:
EXT_NAME: quackscale
DUCKDB_VERSION: v1.5.3
RELEASE_ASSET: quacktail-linux-amd64
PAGES_BASE_URL: https://quackscience.github.io/duckdb-quackscale

jobs:
build-linux:
name: Build QuackTail linux amd64
build:
name: Build extension binaries
uses: duckdb/extension-ci-tools/.github/workflows/_extension_distribution.yml@v1.5-variegata
with:
duckdb_version: v1.5.3
ci_tools_version: v1.5-variegata
extension_name: quackscale
extra_toolchains: go
exclude_archs: 'windows_amd64;windows_arm64;windows_amd64_mingw;wasm_mvp;wasm_eh;wasm_threads;osx_amd64'

package-pages:
name: Package extension repository
runs-on: ubuntu-latest
needs: build
if: github.event_name == 'release' || inputs.skip_pages != true
steps:
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
path: artifacts
pattern: ${{ env.EXT_NAME }}-${{ env.DUCKDB_VERSION }}-extension-*

- name: Assemble GitHub Pages tree
shell: bash
run: |
set -euo pipefail
shopt -s nullglob
prefix="${EXT_NAME}-${DUCKDB_VERSION}-extension-"
found=0
mkdir -p _site
for dir in artifacts/${prefix}*/; do
arch="$(basename "$dir")"
arch="${arch#${prefix}}"
src="${dir}${EXT_NAME}.duckdb_extension"
if [[ ! -f "$src" ]]; then
echo "::warning::no ${EXT_NAME}.duckdb_extension in ${dir}, skipping ${arch}"
continue
fi
out="_site/${DUCKDB_VERSION}/${arch}"
mkdir -p "$out"
cp "$src" work.bin
truncate -s -256 work.bin
dd if=/dev/zero bs=256 count=1 status=none >> work.bin
gzip -n < work.bin > "${out}/${EXT_NAME}.duckdb_extension.gz"
rm -f work.bin
echo "packaged ${arch}"
found=$((found + 1))
done
if [[ "$found" -eq 0 ]]; then
echo "::error::no extension binaries found to package"
exit 1
fi
echo "Packaged ${found} platform(s) for ${DUCKDB_VERSION}"

- name: Write landing page
shell: bash
env:
PAGES_BASE_URL: ${{ env.PAGES_BASE_URL }}
EXT_NAME: ${{ env.EXT_NAME }}
DUCKDB_VERSION: ${{ env.DUCKDB_VERSION }}
run: |
set -euo pipefail
{
echo '<!DOCTYPE html>'
echo '<html lang="en"><head><meta charset="utf-8"><title>quackscale extension repository</title></head><body>'
echo '<h1>quackscale</h1>'
echo "<p>Unsigned DuckDB extension repository. Requires DuckDB ${DUCKDB_VERSION}.</p>"
echo '<pre>SET allow_unsigned_extensions = true;'
echo "INSTALL ${EXT_NAME} FROM '${PAGES_BASE_URL}';"
echo "LOAD ${EXT_NAME};</pre>"
echo '<h2>Available binaries</h2><ul>'
while IFS= read -r rel; do
echo " <li><a href=\"${rel}\">${rel}</a></li>"
done < <(cd _site && find . -name '*.duckdb_extension.gz' | sort | sed 's|^\./||')
echo '</ul></body></html>'
} > _site/index.html

- name: Upload Pages artifact
uses: actions/upload-pages-artifact@v3
with:
path: _site

deploy-pages:
name: Deploy to GitHub Pages
runs-on: ubuntu-latest
needs: package-pages
if: github.event_name == 'release' || inputs.skip_pages != true
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4

build-quacktail-bundle:
name: Build QuackTail linux amd64 tarball
runs-on: ubuntu-latest
timeout-minutes: 90
if: >-
(github.event_name == 'release') ||
(github.event_name == 'workflow_dispatch' && inputs.skip_tarball != true && inputs.release_tag != '')
permissions:
contents: write
env:
RELEASE_TAG: ${{ github.event.release.tag_name || inputs.release_tag }}

steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.release.tag_name }}
ref: ${{ github.event.release.tag_name || inputs.release_tag || github.ref }}
submodules: recursive

- uses: actions/setup-go@v5
Expand Down Expand Up @@ -51,15 +187,15 @@ jobs:
cp build/release/extension/quackscale/quackscale.duckdb_extension \
"dist/${RELEASE_ASSET}/extension/quackscale/"
{
echo "tag=${{ github.event.release.tag_name }}"
echo "tag=${RELEASE_TAG}"
echo "duckdb=${{ env.DUCKDB_VERSION }}"
echo "commit=${{ github.sha }}"
} > "dist/${RELEASE_ASSET}/VERSION"
tar -czf "dist/${RELEASE_ASSET}-${{ github.event.release.tag_name }}.tar.gz" -C dist "${RELEASE_ASSET}"
tar -czf "dist/${RELEASE_ASSET}-${RELEASE_TAG}.tar.gz" -C dist "${RELEASE_ASSET}"
ls -lh dist/

- name: Upload release assets
uses: softprops/action-gh-release@v2
with:
tag_name: ${{ github.event.release.tag_name }}
tag_name: ${{ env.RELEASE_TAG }}
files: dist/*.tar.gz
28 changes: 28 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,34 @@ Do **not** copy the random `auth_token` from each `CALL quack_serve`. Use a flee

---

## Install

QuackScale ships in two forms: a **loadable extension** (stock DuckDB + `INSTALL`) and a **QuackTail release bundle** (prebuilt `duckdb` binary for Linux CI and compose).

### Loadable extension (GitHub Pages)

Published on each [release](https://github.com/quackscience/duckdb-quackscale/releases) to a DuckDB custom extension repository. Requires **DuckDB v1.5.3** and unsigned extension support:

```sql
SET allow_unsigned_extensions = true;
INSTALL quackscale FROM 'https://quackscience.github.io/duckdb-quackscale';
LOAD quackscale;
```

CLI equivalent: `duckdb -unsigned` then `LOAD quackscale;`

Install [Quack](https://duckdb.org/docs/current/quack/overview) from core separately (`INSTALL quack FROM core; LOAD quack;`) for `quack_serve`, `ATTACH`, and `quack_query`.

### QuackTail release bundle (GitHub Releases)

Linux amd64 tarball with `duckdb` and `extension/quackscale/quackscale.duckdb_extension` — used by [Headscale e2e CI](.github/workflows/headscale-e2e.yml) and `examples/` with `BUILD_FROM_SOURCE=0`. Download from [Releases](https://github.com/quackscience/duckdb-quackscale/releases) (`quacktail-linux-amd64-*.tar.gz`).

### Build from source

See [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for submodules, Go/libtailscale, and `make release`.

---

## Quick start

### Server
Expand Down
20 changes: 18 additions & 2 deletions docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,12 +72,28 @@ When bumping the DuckDB target:
|----------|---------|---------|
| [headscale-e2e.yml](../.github/workflows/headscale-e2e.yml) | **Manual only** | Release-binary two-node e2e (no source build) |
| [headscale-integration.yml](../.github/workflows/headscale-integration.yml) | PR | Source build + Headscale smoke |
| [Release.yml](../.github/workflows/Release.yml) | Release published | Build linux release tarball |
| [Release.yml](../.github/workflows/Release.yml) | Release published / manual | Extension repo → GitHub Pages; linux QuackTail tarball → Releases |
| [libtailscale-integration.yml](../.github/workflows/libtailscale-integration.yml) | PR | libtailscale `go test` |
| [MainDistributionPipeline.yml](../.github/workflows/MainDistributionPipeline.yml) | PR | Extension distribution CI |

**E2e never runs on push/PR** and never compiles DuckDB in CI — use `workflow_dispatch` on `headscale-e2e` with a release tag. Full DuckLake compose demo is local dev only (`scripts/ci_compose_e2e.sh`).

### Release and GitHub Pages

On **Release published** (or manual **Release** workflow):

1. **build** — extension-ci-tools matrix (`quackscale` per platform, same exclusions as MainDistributionPipeline).
2. **package-pages** + **deploy-pages** — unsigned `.duckdb_extension.gz` under `v1.5.3/{arch}/` at `https://quackscience.github.io/duckdb-quackscale`.
3. **build-quacktail-bundle** — linux amd64 `quacktail-linux-amd64-{tag}.tar.gz` attached to the GitHub Release.

**One-time repo setup:** Settings → Pages → Build and deployment → **Source: GitHub Actions**.

**Manual Pages-only test:** Actions → Release → Run workflow → leave `release_tag` empty, uncheck skip_pages.

**Manual tarball test:** provide an existing git tag in `release_tag`, uncheck skip_tarball.

Each Pages deploy replaces the whole site (one DuckDB version hosted). To host multiple DuckDB versions, accumulate version directories in a follow-up change.

## Roadmap (selected)

| Item | Status |
Expand All @@ -88,7 +104,7 @@ When bumping the DuckDB target:
| `ATTACH … TYPE quacktail_lake` (Tier 3 native catalog) | Planned |
| `ducklake_discover()` enriched discovery | Planned |
| `quackscale_serve()` one-call server bootstrap | Planned |
| Community extension descriptor publish | Planned |
| Community extension descriptor publish | Done (GitHub Pages on release) |

## Risks

Expand Down
Loading