Skip to content

Commit 7535e33

Browse files
improvement(mothership): user_table speed parity — limit bounds, async import/delete/update jobs
- query_rows / filter ops clamp limit to the contract maxes; query_rows skips execution metadata. - import_file / create_from_file (large CSV/TSV) and delete_rows_by_filter (>1000 unbounded matches) dispatch background table jobs, claiming the per-table job slot; inline paths claim the slot too. - update_rows_by_filter now escalates the same way: >1000 unbounded matches run as a background table job (new 'update' job type + runTableUpdate worker + tableUpdateTask), so a broad update on a huge table no longer loads every row into the request. Best-effort/non-atomic and skips workflow recompute (documented); unique-column patches stay inline. Pagination is limit/offset. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent cc56408 commit 7535e33

12 files changed

Lines changed: 1579 additions & 95 deletions

File tree

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
import { task } from '@trigger.dev/sdk'
2+
import {
3+
markTableUpdateFailed,
4+
runTableUpdate,
5+
type TableUpdatePayload,
6+
} from '@/lib/table/update-runner'
7+
8+
/**
9+
* `TableUpdatePayload` with the cutoff as an ISO string — task payloads cross a JSON boundary, so
10+
* the Date is rehydrated in `run` rather than trusting payload serialization.
11+
*/
12+
export interface TableUpdateTaskPayload extends Omit<TableUpdatePayload, 'cutoff'> {
13+
cutoff: string
14+
}
15+
16+
/**
17+
* Trigger.dev wrapper around `runTableUpdate`. Errors propagate out of `run` so the retry policy
18+
* fires; the job is marked failed only in `onFailure`, after the final attempt. Retry-safe: the
19+
* worker keysets by id with a `created_at <= cutoff` floor and the JSONB-merge patch is idempotent
20+
* (re-applying the same patch to an already-patched row is a no-op), so a retried attempt re-walks
21+
* and re-applies whatever remains. The `table_jobs` ownership gate stops a retried run that lost
22+
* the job within one page.
23+
*/
24+
export const tableUpdateTask = task({
25+
id: 'table-update',
26+
machine: 'small-1x',
27+
retry: { maxAttempts: 3 },
28+
queue: {
29+
name: 'table-update',
30+
concurrencyLimit: 10,
31+
},
32+
run: async (payload: TableUpdateTaskPayload) => {
33+
await runTableUpdate({ ...payload, cutoff: new Date(payload.cutoff) })
34+
},
35+
onFailure: async ({ payload, error }) => {
36+
await markTableUpdateFailed(payload.tableId, payload.jobId, error)
37+
},
38+
})

apps/sim/lib/copilot/generated/tool-catalog-v1.ts

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3958,7 +3958,8 @@ export const UserTable: ToolCatalogEntry = {
39583958
},
39593959
limit: {
39603960
type: 'number',
3961-
description: 'Maximum rows to return or affect (optional, default 100)',
3961+
description:
3962+
'Maximum rows to return or affect (optional; default 100, max 1000). For delete_rows_by_filter and update_rows_by_filter, omitting it lets matches above 1000 run as a background job.',
39623963
},
39633964
mapping: {
39643965
type: 'object',
@@ -4011,7 +4012,7 @@ export const UserTable: ToolCatalogEntry = {
40114012
},
40124013
offset: {
40134014
type: 'number',
4014-
description: 'Number of rows to skip (optional for query_rows, default 0)',
4015+
description: 'Number of rows to skip for query_rows pagination (optional, default 0).',
40154016
},
40164017
outputColumnNames: {
40174018
type: 'object',

apps/sim/lib/copilot/generated/tool-schemas-v1.ts

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3686,7 +3686,8 @@ export const TOOL_RUNTIME_SCHEMAS: Record<string, ToolRuntimeSchemaEntry> = {
36863686
},
36873687
limit: {
36883688
type: 'number',
3689-
description: 'Maximum rows to return or affect (optional, default 100)',
3689+
description:
3690+
'Maximum rows to return or affect (optional; default 100, max 1000). For delete_rows_by_filter and update_rows_by_filter, omitting it lets matches above 1000 run as a background job.',
36903691
},
36913692
mapping: {
36923693
type: 'object',
@@ -3745,7 +3746,8 @@ export const TOOL_RUNTIME_SCHEMAS: Record<string, ToolRuntimeSchemaEntry> = {
37453746
},
37463747
offset: {
37473748
type: 'number',
3748-
description: 'Number of rows to skip (optional for query_rows, default 0)',
3749+
description:
3750+
'Number of rows to skip for query_rows pagination (optional, default 0).',
37493751
},
37503752
outputColumnNames: {
37513753
type: 'object',

0 commit comments

Comments
 (0)