Skip to content

Commit 7f3a3c8

Browse files
spicyfalafelclaude
andcommitted
docs: rename Cloud SQL section, simplify initial export, remove perf warnings
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 88d7a6d commit 7f3a3c8

1 file changed

Lines changed: 3 additions & 7 deletions

File tree

docs/tutorials/subscriptions-tutorials/bigquery-aidboxtopicdestination.md

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -355,15 +355,11 @@ To skip the initial export (e.g., the table is already populated or you only nee
355355
3. After all rows are sent, finalizes and commits the stream — data becomes visible atomically
356356
4. Reports progress via the `$status` endpoint (`initialExportProgress_rowsSent`)
357357

358-
The export runs in a single thread. For small to medium datasets (up to ~100K rows) this completes in seconds to minutes. For larger datasets (1M+ rows), consider using the Cloud SQL optimization below.
358+
The export retries up to 3 times on failure.
359359

360-
{% hint style="warning" %}
361-
The pending stream commit is atomic — if the export fails partway through (e.g., on row 999,999 of 1,000,000), **no data is committed** and the export is retried from scratch (up to 3 attempts). This guarantees no partial data in BigQuery, but means large exports may take multiple attempts on transient failures.
362-
{% endhint %}
363-
364-
### Large Datasets: Manual Initial Export via Cloud SQL
360+
### Alternative: Federated Query (Cloud SQL only)
365361

366-
For large datasets (1M+ rows), the default initial export may be slow (single-threaded PG → JVM → gRPC). If your Aidbox runs on [Google Cloud SQL](https://cloud.google.com/sql), you can do the initial export manually using BigQuery's [federated query](https://cloud.google.com/bigquery/docs/cloud-sql-federated-queries) — BigQuery reads directly from Cloud SQL over Google's internal network.
362+
If your Aidbox PostgreSQL runs on [Google Cloud SQL](https://cloud.google.com/sql), you can populate the BigQuery table manually using a [federated query](https://cloud.google.com/bigquery/docs/cloud-sql-federated-queries) instead of the built-in initial export. This can be useful if you want more control over the process or need to re-populate the table without recreating the destination.
367363

368364
1. [Create a BigQuery Connection](https://cloud.google.com/bigquery/docs/create-cloud-sql-connection) to your Cloud SQL instance
369365
2. Run this query in the BigQuery Console:

0 commit comments

Comments
 (0)