Skip to content

fix(iceberg): Tier-1 PostgreSQL-compatibility fixes#689

Draft
EDsCODE wants to merge 1 commit into
mainfrom
fix/iceberg-pg-tier1-compat
Draft

fix(iceberg): Tier-1 PostgreSQL-compatibility fixes#689
EDsCODE wants to merge 1 commit into
mainfrom
fix/iceberg-pg-tier1-compat

Conversation

@EDsCODE
Copy link
Copy Markdown
Contributor

@EDsCODE EDsCODE commented Jun 5, 2026

Summary

Fixes the highest-severity data-integrity gaps found in an exhaustive QA of the Iceberg-backed connection (165 features verified, 148 confirmed gaps, 44 high-severity). All changes are verified live against a local Iceberg backend and covered by unit tests.

Each fix below was independently reproduced before and after.

Fixes

  • DDL hybrid warn/error (Iceberg only). Previously PK/UNIQUE/CHECK/FK, SERIAL, GENERATED, and DEFAULT now() were silently stripped — constraints unenforced, the rest silently NULL. Now: WARNING (still works) for unenforced constraints; ERROR 0A000 for the silently-NULL features (SERIAL/BIGSERIAL, GENERATED ... STORED, DEFAULT <expr>/now()). DEFAULT NULL and NOT NULL preserved. DuckLake keeps its historical silent-strip behavior (sqlmesh/dbt depend on it). Adds a Warnings channel on the transpile Result, surfaced as NoticeResponse in both simple and extended protocols.
  • EXPLAIN (ANALYZE) of a write no longer double-executes. GetQuerySchema returns a synthetic schema for EXPLAIN without running it, and the extended-protocol Describe path no longer probe-executes EXPLAIN. (EXPLAIN ANALYZE INSERT now lands exactly one row.)
  • DROP COLUMN guard. DDL-time WARNING, and the Iceberg "newer schema id" scan failure is mapped to a clear 0A000 message instead of a raw XX000.
  • bytea / bit literals. '\xDEADBEEF'::bytea now decodes to the correct bytes (unhex) instead of dropping the first byte; B'101' maps to '101'::BIT.
  • jsonb ||. Now merges objects (json_merge_patch) instead of silently string-concatenating into invalid JSON; plain string/array || is untouched.
  • Writable-CTE UPDATE ... RETURNING. Rejected when it reads a modified column (would return pre-update values); RETURNING an unmodified key and RETURNING * still work, preserving the existing Airbyte row-identification pattern.

Test plan

  • go test ./transpiler/... ./duckdbservice/ ./server/...
  • Live-verified each fix against a local Iceberg (dbname=iceberg) backend
  • TODO: tests/e2e-mw-dev/harness.sh assertions on both cnpg + ext backends (requires the mw-dev cluster, not the local stack)

Notes / deferred

  • Hybrid warn/error is scoped to Iceberg; DuckLake is intentionally unchanged.
  • Deferred (separate from this batch): INSERT…SELECT RETURNING column-name preservation, jsonb value-equality normalization (the || merge is included here), and a friendlier message for Iceberg's engine-level non-null-literal-DEFAULT rejection.

🤖 Generated with Claude Code

Addresses the highest-severity data-integrity gaps from the Iceberg QA
report (docs/iceberg-pg-syntax-qa.md). All verified live against the
Iceberg backend and covered by unit tests.

- DDL hybrid warn/error (Iceberg only; DuckLake keeps silent-strip for
  sqlmesh/dbt): WARNING for unenforced PK/UNIQUE/CHECK/FK; ERROR (0A000)
  for the silently-NULL features SERIAL/BIGSERIAL, GENERATED ... STORED,
  and DEFAULT <expr>/now(). DEFAULT NULL and NOT NULL preserved. Adds a
  Warnings channel on the transpile Result, surfaced as NoticeResponse in
  both simple and extended protocols.
- EXPLAIN (ANALYZE) of a write no longer double-executes: GetQuerySchema
  returns a synthetic schema for EXPLAIN without running it, and the
  extended-protocol Describe path no longer probe-executes EXPLAIN.
- DROP COLUMN guard: DDL-time WARNING, and the Iceberg "newer schema id"
  scan failure is mapped to a clear 0A000 message instead of a raw XX000.
- bytea hex literals '\xDEADBEEF'::bytea now decode to the correct bytes
  (unhex), and B'101' bit-string literals map to '101'::BIT.
- jsonb || now merges objects (json_merge_patch) instead of silently
  string-concatenating; plain string/array || is left untouched.
- Writable-CTE UPDATE ... RETURNING that reads a modified column is
  rejected (would return pre-update values); RETURNING an unmodified key
  and RETURNING * still work (Airbyte pattern preserved).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@EDsCODE EDsCODE force-pushed the fix/iceberg-pg-tier1-compat branch from ea5bdb6 to 6e0a7a9 Compare June 5, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant