Skip to content

Ducklake support#1

Merged
lmangani merged 25 commits into
mainfrom
ducklake
May 30, 2026
Merged

Ducklake support#1
lmangani merged 25 commits into
mainfrom
ducklake

Conversation

@lmangani
Copy link
Copy Markdown
Contributor

No description provided.

lmangani and others added 25 commits May 30, 2026 07:40
Server bootstrap attaches a local DuckLake catalog (inventory table + Parquet
under /work/lake), then quack_serve. Client e2e queries remote.lake.inventory
through tailscale_quack_forward. Install ducklake in compose images alongside quack.

Co-authored-by: Cursor <cursoragent@cursor.com>
Mount /var/lib/ducklake on quacktail-server only; seed inventory on first
boot and attach-only on restart so stop/start keeps Parquet and metadata.

Co-authored-by: Cursor <cursoragent@cursor.com>
Use $'...' for lake inventory statements so DuckDB gets real newlines, and
stop the client retry loop on SIGINT/SIGTERM instead of treating interrupts
as transient failures.

Co-authored-by: Cursor <cursoragent@cursor.com>
Plain quack ATTACH only exposes the primary catalog; switch the client
to quack_discover plus ducklake:quack with shared Parquet DATA_PATH.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ake.

Avoid background pipe deadlock; discover and query lake tables via quack_query
on the server instead of client ducklake:quack attach with mismatched paths.

Co-authored-by: Cursor <cursoragent@cursor.com>
Probe lake via duckdb_databases() and run all quack_query lake checks
before ATTACH to avoid nested remote session deadlocks.

Co-authored-by: Cursor <cursoragent@cursor.com>
quack_query(FROM quack_discover()) deadlocks on the server; use forward
quack_uri for discovery. Write DuckDB stdout directly to client.out and run
lake quack_query before ATTACH remote.

Co-authored-by: Cursor <cursoragent@cursor.com>
Stop forcing bootstrap refresh every run (wrong stale check), restore live
tee output, fail fast on parser errors, fix DISCOVERED SQL quoting, and
lower demo timeout/retries to 60s / 3 attempts.

Co-authored-by: Cursor <cursoragent@cursor.com>
One-shot clients hung forever because tsnet and quack_forward background
threads kept DuckDB alive. Add CALL tailscale_down(), CLIENT_DEMO_DONE
marker, and entrypoint watchdog to SIGINT the session after teardown.

Co-authored-by: Cursor <cursoragent@cursor.com>
Always emit CALL tailscale_down in client SQL; wait for teardown then
SIGTERM/KILL the DuckDB process instead of leaving tsnet threads running.

Co-authored-by: Cursor <cursoragent@cursor.com>
…tries.

Probe for tailscale_down before emitting client SQL, run teardown before
CLIENT_DEMO_DONE, merge stderr into client.out, and verify the function at
image build time when compiling from source.

Co-authored-by: Cursor <cursoragent@cursor.com>
Write DuckDB results to client.out during the background session, then
print the filtered transcript before the success banner (same as before).

Co-authored-by: Cursor <cursoragent@cursor.com>
Discover server-side lake tables via quack_query and create local views
so clients query lake.* without hand-written wrappers or client DATA_PATH.
Compose bootstrap prefers attach_ducklake when the function is available.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add QUACKTAIL_ROLE=bootstrap to refresh /work without running DuckDB;
detect missing attach_ducklake/tailscale_down in volume SQL; verify
attach_ducklake at image build time.

Co-authored-by: Cursor <cursoragent@cursor.com>
Stop the server from writing stale client_session.sql to the shared volume;
always refresh client SQL on the client container; print quackscale caps so
BUILD_FROM_SOURCE=0 / v1.0.2 release gaps are obvious.

Co-authored-by: Cursor <cursoragent@cursor.com>
Install quack/ducklake before copying quackscale to the nested extension
path, verify at image build time, stamp build-info, and fail the client
demo when release binaries (v1.0.2) lack attach_ducklake.

Co-authored-by: Cursor <cursoragent@cursor.com>
Exclude host build/ from context, install patch and build-essential for
libtailscale, and use a dedicated builder script with clearer errors.

Co-authored-by: Cursor <cursoragent@cursor.com>
Emit CLIENT_DEMO_DONE before teardown so the compose watchdog can exit,
and make tailscale_close asynchronous so CALL tailscale_down() returns.

Co-authored-by: Cursor <cursoragent@cursor.com>
Drop runtime capability logging, unused client-tsnet.log handling, and
duplicate README build steps while keeping build-time image verify.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Replace nine overlapping docs with GUIDE (use cases and patterns),
AUTHENTICATION (tailnet + Quack credentials), and DEVELOPMENT
(contributor build/CI). Update README and examples cross-links.

Co-authored-by: Cursor <cursoragent@cursor.com>
Highlight WireGuard security, identity-based access, and in-process
tailnet management; reorganize functions by category with doc links.

Co-authored-by: Cursor <cursoragent@cursor.com>
Replace release-binary headscale-e2e with scripts/ci_compose_e2e.sh
(examples/docker-compose.yml). Add patch to headscale-integration build
deps; document workflow split in test/e2e and DEVELOPMENT docs.

Co-authored-by: Cursor <cursoragent@cursor.com>
Remove pull_request trigger added by mistake; PR CI uses docker-compose-build.

Co-authored-by: Cursor <cursoragent@cursor.com>
headscale-e2e is workflow_dispatch only and uses ci_headscale_e2e.sh
with pre-built release duckdb. Delete docker-compose-build workflow;
keep ci_compose_e2e.sh for local source-build demo only.

Co-authored-by: Cursor <cursoragent@cursor.com>
@lmangani lmangani merged commit 6a9c365 into main May 30, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant