columnar: re-implement parallel/index-only-scan suppression on PG19#8619
Open
ihalatci wants to merge 6 commits into
Open
columnar: re-implement parallel/index-only-scan suppression on PG19#8619ihalatci wants to merge 6 commits into
ihalatci wants to merge 6 commits into
Conversation
DESCRIPTION: Port upstream PG19 ruleutils.c into Citus deparser tree. Mirrors #8010 (PG18) and #7725 (PG17). Replaces the ruleutils_19.c placeholder from the build-foundation PR with a proper port of upstream PG19's src/backend/utils/adt/ruleutils.c, produced by a 3-way merge (git merge-file: ruleutils_18.c as base, upstream PG18 and upstream PG19 as the two sides). All merge conflicts fell in blocks Citus had already stripped from _18.c or in shard-aware deparse variants, and were resolved by keeping the Citus side. The file keeps Citus' existing curation (deparse hooks, shard-aware UPDATE/DELETE branches) while picking up real upstream PG18->PG19 deparse changes. Without this, Citus on a PG19 backend would deparse worker-bound SQL using PG18 semantics -- silent corruption the moment a PG19-only node, clause, or formatting decision appears in the tree. The trailing #endif guard text is corrected to "(PG_VERSION_NUM >= PG_VERSION_19) && (PG_VERSION_NUM < PG_VERSION_20)" (the _17.c/_18.c snapshots carry stale guard comments; not propagated). The generate_function_name fgc_flags fix is absorbed naturally by the merge -- upstream PG19 already passes &fgc_flags at that call site. Excluded from citus-style via .gitattributes (added in the build-foundation PR): the file exceeds the CI uncrustify 10,000-line limit and is upstream-derived. Refs: #8597
- Replace the blanket `#define totaltime query_instr` with a scoped QueryDescTotalTime() accessor (plus a matching pre-PG19 macro) so the rename no longer risks rewriting unrelated identifiers; convert the multi_executor.c call sites accordingly. - Drop the dead currentProc advance at the end of the dclist_foreach loop in AddEdgesForWaitQueue. - Pass a named Size local to ShmemInitStruct in MaintenanceDaemonShmemInit. - Reserve 1024 * PG_CACHE_LINE_SIZE (instead of a magic 128 * 1024) for the PG19 shmem alignment slack, with an explanatory comment. - Point the columnar get_relation_info_hook TODO and the RepackStmt / LWLock tranche comments at their tracking issues (#8614, #8613, #8609).
Enable PG19beta1 across the downstream test matrices now that the build job proves it compiles: - test-citus, test-citus-failure, test-citus-cdc: add pg19_version - test-arbitrary-configs: add pg19_version to the matrix - test-pg-upgrade: add 18->19 and 16->19 upgrade pairs - params: bump pg19_version/upgrade_pg_versions 19devel -> 19beta1 - params: point image_suffix at the PG19beta1 the-process images (-dev-33b3cf5, built from citusdata/the-process#222) test-citus-upgrade is left at PG16/PG17 (no released Citus supports PG19). Closes #8615
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## pg19-columnar-index #8619 +/- ##
====================================================
Coverage 88.95% 88.96%
====================================================
Files 288 288
Lines 64376 64386 +10
Branches 8093 8094 +1
====================================================
+ Hits 57267 57278 +11
- Misses 4776 4777 +1
+ Partials 2333 2331 -2 🚀 New features to boost your workflow:
|
ihalatci
force-pushed
the
pg19-columnar-relinfo
branch
from
d0bf1ee to
004a41d
Compare
ihalatci
force-pushed
the
pg19-columnar-index
branch
from
a31f940 to
83042a9
Compare
Two PG19 runtime crashes that surface in normal Citus operation (not just the regression tests): 1. FuncnameGetCandidates() gained an int *fgc_flags out-parameter that the callee writes to unconditionally; the compat shim passed NULL and crashed on every by-name function lookup (e.g. the maintenance daemon). Pass the address of an int compound literal instead. 2. PG19 added TupleDesc->firstNonCachedOffsetAttr, an offset cache that BlessTupleDesc()/slot_deform_heap_tuple() now assert is populated but no longer populate themselves. Wrap BlessTupleDesc to call TupleDescFinalize() first, add a no-op shim on older majors, and call it explicitly at the three sites that deform hand-built TupleDescs. Also installs ApplicationNameAssignHook via union access on PG19. DESCRIPTION: Fix PG19 runtime crashes in function lookup and TupleDesc handling Closes #8610 Part of #8597
PG19 enables parallel CREATE INDEX by default (max_parallel_maintenance_workers defaults to 2). Columnar's TableAM is always serial: rs_parallel is stored but never read, and the build path flushes pending writes, which is disallowed inside a parallel operation. Three changes: 1. Provide working parallelscan_estimate/initialize/reinitialize that delegate to the table_block_* helpers, so callers that unconditionally size and initialize a ParallelTableScanDesc (e.g. PG19's parallel btree build) do not abort. Columnar ignores the descriptor, so the state is simply unused. 2. In columnar_index_build_range_scan, accept a parallel scan descriptor by discarding it (scan = NULL) and running the serial path. 3. In ColumnarProcessUtility, when the statement is CREATE INDEX on a columnar AM relation, force max_parallel_maintenance_workers=0 via NewGUCNestLevel/AtEOXact_GUC around PrevProcessUtilityHook so the build runs serially without triggering "cannot update tuples during a parallel operation". DESCRIPTION: Serialize CREATE INDEX on columnar tables on PG19 Closes #8611 Part of #8597
PG19 removed get_relation_info_hook, which is where pre-PG19 builds disable
parallel query and index-only scans for columnar relations (columnar scans are
always serial and cannot return tuples from an index). Re-implement that
suppression for PG19 inside ColumnarSetRelPathlistHook:
* forbid future parallel workers (rel_parallel_workers = 0,
consider_parallel = false) and drop any already-generated partial paths;
* clear each index's canreturn flags and strip any IndexOnlyScan paths that
were already created.
The pre-PG19 ColumnarGetRelationInfoHook path is unchanged. Removes the
Phase-1 TODO(PG19, #8614) placeholder in columnar_customscan_init.
DESCRIPTION: Suppress parallel and index-only scans for columnar tables on PG19
Closes #8614
Part of #8597
ihalatci
force-pushed
the
pg19-columnar-relinfo
branch
from
004a41d to
f25e811
Compare
ihalatci
force-pushed
the
pg19-columnar-index
branch
from
83042a9 to
59c5dc3
Compare
This was referenced Jun 13, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Re-implements PG19-specific planner suppression for columnar relations now that upstream removed get_relation_info_hook, ensuring the planner won’t choose unsupported parallel or index-only scan paths for columnar tables.
Changes:
- Adds PG19-only logic in
ColumnarSetRelPathlistHookto disable parallel planning (setrel_parallel_workers = 0,consider_parallel = false, and clearpartial_pathlist). - Clears per-index
canreturnflags and removes already-generatedIndexOnlyScanpaths for columnar relations on PG19. - Replaces the removed PG19 TODO with a comment explaining where the suppression now lives.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ihalatci
force-pushed
the
pg19-columnar-index
branch
from
59c5dc3 to
c70d228
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked PR (PR3b of the PG19 stack). Base =
pg19-columnar-index(#8618) →pg19-runtime-fixes(#8617) →pg19-ci-test-matrices(#8616) →pg19-ruleutils-port(#8602) →pg19-support. Review/merge in stack order.PG19 removed
get_relation_info_hook, which is where pre-PG19 builds disable parallel query and index-only scans for columnar relations (columnar scans are always serial and cannot return tuples from an index). The Phase-1 build left aTODO(PG19, #8614)placeholder; this PR re-implements that suppression insideColumnarSetRelPathlistHook:rel_parallel_workers = 0,consider_parallel = false) and drop any already-generated partial paths;canreturnflags and strip anyIndexOnlyScanpaths that were already created (set_rel_pathlist_hookruns after path generation).The pre-PG19
ColumnarGetRelationInfoHookpath is unchanged.Closes #8614
Part of #8597