fix(iceberg): Tier-1 PostgreSQL-compatibility fixes#689
Draft
EDsCODE wants to merge 1 commit into
Draft
Conversation
Addresses the highest-severity data-integrity gaps from the Iceberg QA report (docs/iceberg-pg-syntax-qa.md). All verified live against the Iceberg backend and covered by unit tests. - DDL hybrid warn/error (Iceberg only; DuckLake keeps silent-strip for sqlmesh/dbt): WARNING for unenforced PK/UNIQUE/CHECK/FK; ERROR (0A000) for the silently-NULL features SERIAL/BIGSERIAL, GENERATED ... STORED, and DEFAULT <expr>/now(). DEFAULT NULL and NOT NULL preserved. Adds a Warnings channel on the transpile Result, surfaced as NoticeResponse in both simple and extended protocols. - EXPLAIN (ANALYZE) of a write no longer double-executes: GetQuerySchema returns a synthetic schema for EXPLAIN without running it, and the extended-protocol Describe path no longer probe-executes EXPLAIN. - DROP COLUMN guard: DDL-time WARNING, and the Iceberg "newer schema id" scan failure is mapped to a clear 0A000 message instead of a raw XX000. - bytea hex literals '\xDEADBEEF'::bytea now decode to the correct bytes (unhex), and B'101' bit-string literals map to '101'::BIT. - jsonb || now merges objects (json_merge_patch) instead of silently string-concatenating; plain string/array || is left untouched. - Writable-CTE UPDATE ... RETURNING that reads a modified column is rejected (would return pre-update values); RETURNING an unmodified key and RETURNING * still work (Airbyte pattern preserved). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ea5bdb6 to
6e0a7a9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the highest-severity data-integrity gaps found in an exhaustive QA of the Iceberg-backed connection (165 features verified, 148 confirmed gaps, 44 high-severity). All changes are verified live against a local Iceberg backend and covered by unit tests.
Each fix below was independently reproduced before and after.
Fixes
SERIAL,GENERATED, andDEFAULT now()were silently stripped — constraints unenforced, the rest silently NULL. Now: WARNING (still works) for unenforced constraints; ERROR0A000for the silently-NULL features (SERIAL/BIGSERIAL,GENERATED ... STORED,DEFAULT <expr>/now()).DEFAULT NULLandNOT NULLpreserved. DuckLake keeps its historical silent-strip behavior (sqlmesh/dbt depend on it). Adds aWarningschannel on the transpileResult, surfaced asNoticeResponsein both simple and extended protocols.EXPLAIN (ANALYZE)of a write no longer double-executes.GetQuerySchemareturns a synthetic schema forEXPLAINwithout running it, and the extended-protocol Describe path no longer probe-executesEXPLAIN. (EXPLAIN ANALYZE INSERTnow lands exactly one row.)WARNING, and the Iceberg "newer schema id" scan failure is mapped to a clear0A000message instead of a rawXX000.'\xDEADBEEF'::byteanow decodes to the correct bytes (unhex) instead of dropping the first byte;B'101'maps to'101'::BIT.||. Now merges objects (json_merge_patch) instead of silently string-concatenating into invalid JSON; plain string/array||is untouched.UPDATE ... RETURNING. Rejected when it reads a modified column (would return pre-update values);RETURNINGan unmodified key andRETURNING *still work, preserving the existing Airbyte row-identification pattern.Test plan
go test ./transpiler/... ./duckdbservice/ ./server/...dbname=iceberg) backendtests/e2e-mw-dev/harness.shassertions on both cnpg + ext backends (requires the mw-dev cluster, not the local stack)Notes / deferred
RETURNINGcolumn-name preservation, jsonb value-equality normalization (the||merge is included here), and a friendlier message for Iceberg's engine-level non-null-literal-DEFAULTrejection.🤖 Generated with Claude Code