feat: transparent tailnet http routing via httputil#3
Merged
Conversation
Wrap DuckDB's global HTTPUtil so HTTP to tailnet hosts (100.64.0.0/10, *.ts.net) is dialed over the embedded tsnet node, letting clients ATTACH 'quack:100.x:9494' directly with no tailscale_quack_forward. Auto-installed on tailscale_up/login (opt out: http_route => false); coexists with the forwarder (disjoint by host). The keep-alive client holds one connection per peer, frames responses by Content-Length or chunked transfer-encoding, redials once on a stale pooled connection, and pools idle clients. quack_uri()/quack_discover() now prefer the routable 100.x IPv4. Ported from smithclay/duckdb-tailscale, adding keep-alive and forwarder coexistence. Verified on real Tailscale (direct ATTACH, no forwarder).
|
|
Contributor
Pre-release v1.0.4 builds with this PR for testing in CI https://github.com/Query-farm/quackscale/releases/tag/v1.0.4 |
Address PR review findings on the transparent router: - ToHTTPResponse sets success/request_error from the status, so 4xx/5xx (and unparseable status lines) are no longer reported as successful fetches. - RoundTrip only retries idempotent requests (no body); a POST/PUT body is never resent on a reused-connection failure. - IsTailnetHost no longer intercepts https:// URLs — those go to httpfs (real TLS) instead of being mis-sent as cleartext to port 80. - Strict Content-Length / chunk-size parsing (reject non-digits), and distinguish a mid-body socket error from a clean EOF so truncated bodies aren't accepted. - Close a kept-alive connection if the peer over-sends past Content-Length. - e2e capability gate fails loudly if quackscale can't load (vs silently skipping the router probe); probe enable/skip is now logged in bootstrap and entrypoint. - Comment/doc fixes (RegisterTailscaleHTTPUtil call sites, response framing, DialTCP concurrency rationale, RoutableTailnetIP, bootstrap probe rationale). Verified on real Tailscale (direct ATTACH still passes) + offline tests 24/24.
Contributor
|
Fantastic @smithclay 🎉 does the README need to be updated too perhaps? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
After
tailscale_up, QuackScale wraps DuckDB's globalHTTPUtilso any HTTP request to a tailnet host (100.64.0.0/10,*.ts.net) is dialed over the embedded tsnet node. Clients canATTACH 'quack:100.x.x.x:9494'(or hit anyread_csv/httpfsURL on the tailnet) directly — notailscale_quack_forward.Auto-installed on
tailscale_up/tailscale_login; opt out withhttp_route => false. It coexists with the existing forwarder (disjoint by host:IsTailnetHostmatches only100.64/10+*.ts.net, never the forwarder's127.0.0.1).Based on duckdb-tailscale
The HTTPUtil-wrapping approach is ported from smithclay/duckdb-tailscale (
src/tailscale_http.cpp), rebased onto QuackScale'sTailscaleBridge. This PR extends it with:Content-Lengthand chunked transfer-encoding framing, redial-once on a stale pooled connection, bounded idle-client pool (vs duckdb-tailscale's one-request-per-connectionConnection: close).tailscale_quack_forwardrun side by side; the forwarder stays for MagicDNS short names, pinned loopback ports, and non-HTTP clients.Changes
src/tailscale_http.{hpp,cpp}—TailscaleHTTPUtil(wraps the previous util, delegates non-tailnet traffic to httpfs) + keep-aliveTailscaleHTTPClient.TailscaleBridge::DialTCP— tsnet dial that releases the handle lock before dialing, so httpfs's parallel range reads aren't serialized.quackscale_extension.cpp—http_routeparameter (defaulttrue); router installed on up/login.quack_uri()/quack_discover()— prefer the routable100.xIPv4 (the MagicDNS short name stays as a labeled extra; it isn't reachable by the transparent path, only the forwarder).ROUTER_PASSEDprobe doing a directATTACHover the router, gated so the release-binary e2e (which predates this) still passes.Validation
make test24/24.ATTACH 'quack:100.x:9494'with no forwarder returned the server row, andquack_uri()returned the100.xIP.Follow-ups
content_handlerbuffers the whole body (no streaming yet).tailscale_down.Opened as draft pending the compose e2e run + a keep-alive stress check.