Skip to content

Export button: stream full query result to disk as TSV (File System Access API, bypasses result grid) #87

Description

@BorisTyshkevich

Part of #68 (Roadmap to 1.0.0).

Problem

Users sometimes need the full result (e.g. 10M rows) to analyze in Excel/pandas — not to view in the browser. The current paths all buffer in memory: streaming folds every row into tab.result (src/core/stream.js:28), and the existing exportResult (src/ui/app.js:680) serializes already-loaded rows into a Blob. For 10M rows that's gigabytes in RAM and a hung tab.

Auth note (corrected): the obstacle is not "a new browser tab can't authenticate." Opening a blank same-origin tab and letting the opener do the authenticated fetch is already a proven pattern here — openSchemaView (src/ui/explain-graph.js:645) opens app.openWindow('', '_blank') and the opener keeps the token + ch-client so its fetches stay authenticated while rendering into the child tab. The only thing that genuinely can't carry the Authorization: Bearer header is a top-level navigation to a ClickHouse URL (window.open('https://ch/?query=…') / <a href>), since a navigation sends cookies, not that header. So Export uses a fetch from the opener, not a navigation.

The real constraint is streaming a huge response to disk without buffering it in RAM — which needs the File System Access API (showSaveFilePicker + resp.body.pipeTo(writable)), Chromium-only. That's independent of tabs.

Proposed solution

Add an Export button next to Share that runs the current editor query (uncapped) and streams the result to a file as TSV, bypassing the result grid, with progress shown in a child tab.

Settled decisions:

  • Stream to disk via the File System Access API (resp.body.pipeTo(await handle.createWritable())): constant memory regardless of result size.
  • Chromium-only. Feature-detect showSaveFilePicker (Chrome/Edge, secure context). Where unavailable (Firefox/Safari, or non-secure context), the Export button is disabled with a tooltip ("Large export requires Chrome/Edge over HTTPS"). No buffered-Blob fallback — it would defeat the memory goal. (Consistent with the desktop/Chromium-leaning stance; support matrix Document the supported-browser matrix (browsers, ClickHouse versions, IdP requirements) #71.)
  • Progress in a child tab, reusing the openSchemaView pattern: a blank same-origin tab via the existing app.openWindow seam shows download progress (bytes written + elapsed); closing the tab cancels. The opener runs the authenticated streaming fetch and the file write.
  • Format: TabSeparatedWithNames (header row, no types — cleanest for Excel/pandas).
  • Query: current editor SQL, uncapped (explicitly ignores Cap SELECT result rows (default 500) with a 100/500/1000/5000/10000 selector #86's row cap), trailing FORMAT stripped and replaced by TSV.

Gesture ordering (important)

Both showSaveFilePicker and openWindow want the click's transient activation, and openSchemaView opens its window synchronously to survive the gesture. The file picker is essential, the progress tab is a nicety — so on click: (1) call showSaveFilePicker first (consumes the gesture for the essential step), (2) then try app.openWindow for the progress tab; if it's blocked/null, fall back to inline progress in the main page (same graceful-degradation shape as openSchemaViewopenInOverlay).

Scope

  • src/core/ (pure, 100% covered) — prepare the export SQL: strip a trailing FORMAT <x> (reuse detectSqlFormat, src/core/format.js) and force FORMAT TabSeparatedWithNames; build a suggested filename (tab name / timestamp → .tsv).
  • src/net/ch-client.js — an export helper that issues the authed request via authedFetch (token refresh + abort signal) and returns the Response so the caller can pipe resp.body. Non-OK → parsed exception (parseExceptionText).
  • src/ui/app.jsapp.actions.exportDirect(): feature-detect → showSaveFilePicker (direct in the click handler) → app.openWindow('', '_blank') for the progress tab (fallback to inline) → authedFetchresp.body.pipeTo(writable), updating progress as chunks are written. Dedicated AbortController; cancel (tab close, or inline Cancel) aborts the stream, aborts/closes the writable, and issues KILL QUERY. Keep showSaveFilePicker as an injected env seam (like fetch/crypto/openWindow) so the wrapper stays testable.
  • Export buttonexportBtn (class tb-btn) immediately before shareBtn in editorToolbar (src/ui/app.js:921); disabled state + tooltip when unsupported; completion/error via flashToast.

Edge cases

  • User cancels the save-file picker → no-op, no error toast (and don't open the progress tab).
  • Progress tab blocked → inline progress fallback; export still proceeds.
  • Server error mid-stream (after headers) → the partial file is what the browser wrote; surface the error and note the file may be incomplete.
  • enable_http_compression=1 stays on (smaller transfer; the browser decompresses into the written file).

Acceptance

  • Export button sits next to Share; runs the current editor query uncapped and streams TSV (TabSeparatedWithNames) to a user-chosen file via showSaveFilePicker.
  • Memory stays flat for a multi-million-row export (streamed, never fully buffered).
  • Progress shows in a child tab; if the tab is blocked, inline progress is shown instead; both can cancel (aborts the fetch + KILL QUERY).
  • On browsers without showSaveFilePicker (or non-secure context), the button is disabled with an explanatory tooltip.
  • A trailing FORMAT in the user's SQL is overridden to TSV; the result grid / tab.result is untouched by an export.
  • npm test green at the per-file coverage gate (SQL-prep + filename helpers + net helper at 100%; showSaveFilePicker / openWindow seams injected/mocked).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions