Skip to content

refactor(datagrid): unify table browse query and drop SUBSTRING column truncation#1373

Merged
datlechin merged 2 commits into
mainfrom
refactor/unify-table-browse-query
May 21, 2026
Merged

refactor(datagrid): unify table browse query and drop SUBSTRING column truncation#1373
datlechin merged 2 commits into
mainfrom
refactor/unify-table-browse-query

Conversation

@datlechin

Copy link
Copy Markdown
Member

Problem

Two things looked wrong when browsing a table:

  1. The editor query and the executed query were different. Opening a table showed SELECT * FROM \t` LIMIT 1000;, but the first page load (and every refresh/pagination/sort/filter) silently replaced it with a different query: SELECT `id`, …, SUBSTRING(`user_agent`, 1, 256) AS `user_agent`, … FROM `t` LIMIT 1000 OFFSET 0. Two separate query builders (QueryTab.buildBaseTableQueryandTableQueryBuilder`) produced different SQL.

  2. The SUBSTRING(col, 1, 256) AS col truncation was an anti-pattern. ColumnExclusionPolicy truncated "very long text" columns server-side and aliased the result back as the real column, so the grid had no idea the value was partial. Grid cell edits had no lazy-load (only the sidebar single-row path did), so editing such a cell and saving wrote the 256-char value back, corrupting the column. The code already refused to do this for BLOBs for exactly this reason, but applied it to text inconsistently.

No mainstream tool (TablePlus, DataGrip, DBeaver) does this. They fetch the full value and truncate at the render layer, or limit transfer while tracking the partial state and refusing to write it back.

Change

  • One query builder. TableQueryBuilder is now the single source of truth. QueryTab.buildBaseTableQuery delegates its SQL branch to it, so the editor query equals the executed query. TableQueryBuilder now emits the mandatory ORDER BY for OFFSET…FETCH dialects when unsorted, and the rebuild path passes the schema so qualification matches.
  • Removed the SUBSTRING truncation and all its now-dead support code: ColumnExclusionPolicy, the columnExclusions plumbing, LazyLoadColumnsService, the sidebar lazy-load path, the write-only column caches, and MultiRowEditState.isTruncated / applyFullValues plus the "truncated" / loading UI in the field editors.

Large text follows the same model already used for BLOBs: fetched in full, truncated only at render, full value available in the cell viewer.

Net ~180 lines removed; 6 files deleted.

Tests

ColumnExclusionPolicyTests, TableQueryBuilderSelectiveTests, and MultiRowEditStateTruncationTests covered the removed behavior and were deleted. TableQueryBuilderMSSQLTests (plugin path) and TableQueryBuilderFilterTests still pass: the MSSQL OFFSET…FETCH + ORDER BY (SELECT NULL) behavior is preserved, and SELECT * is now always emitted.

@datlechin datlechin merged commit 9001ecc into main May 21, 2026
2 checks passed
@datlechin datlechin deleted the refactor/unify-table-browse-query branch May 21, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant