Skip to content

[core][flink] Support view operations in JDBC Catalog#8208

Open
tchivs wants to merge 2 commits into
apache:masterfrom
tchivs:feature/jdbc-catalog-view-support
Open

[core][flink] Support view operations in JDBC Catalog#8208
tchivs wants to merge 2 commits into
apache:masterfrom
tchivs:feature/jdbc-catalog-view-support

Conversation

@tchivs

@tchivs tchivs commented Jun 11, 2026

Copy link
Copy Markdown

Revives #6667 and ports the JDBC catalog view support onto current master.

Hi @JingsongLi, I addressed the questions from the previous review in #6667:

  • Added automatic initialization of the JDBC view metadata table paimon_views.
    It stores (catalog_key, database_name, view_name, view_schema) with a primary key on the catalog/database/view tuple.
  • Stored views as serialized Paimon ViewSchema; the JDBC catalog does not parse or resolve the SQL text during createView.
  • Added tests for views referencing nonexistent tables, including a nonexistent table in another database.
  • Added coverage for cross-database view rename.
  • Added concurrent createView coverage relying on the JDBC primary key constraint.
  • Updated the implementation for the current ViewChange API and listViewDetailsPaged fallback.
  • Kept the SQLite dependency test-scoped for the Flink integration test.

Validation:

  • git diff --check
  • mvn -pl paimon-core spotless:check
  • mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -Dtest=JdbcCatalogTest test
    • JdbcCatalogTest ran 62 tests, 0 failures, 0 errors.

I also attempted:

  • mvn -pl paimon-flink/paimon-flink-common -am -Pfast-build -DfailIfNoTests=false -Dtest=JdbcCatalogViewITCase test

That command was blocked before test execution by dependency resolution timeout from https://jindodata-binary.oss-cn-shanghai.aliyuncs.com/mvn-repo/ for com.aliyun.jindodata:jindo-core:6.9.1 and jindo-sdk:6.9.1 while building unrelated paimon-jindo.

tchivs added 2 commits June 11, 2026 14:21
- Add view methods to CachingCatalog for proper delegation
- Add IT tests for JDBC Catalog view operations
- Add SQLite JDBC dependency for tests
@tchivs

tchivs commented Jun 11, 2026

Copy link
Copy Markdown
Author

@JingsongLi I pushed a follow-up commit to address the old review questions from #6667 and a few local review edge cases. PTAL when you have time.

What changed:

  • The JDBC catalog initializes a paimon_views metadata table automatically. It stores (catalog_key, database_name, view_name, view_schema) with a primary key on the catalog/database/view tuple.
  • View definitions are stored as serialized Paimon ViewSchema; JDBC catalog does not parse or resolve the SQL text during createView. This is covered by tests for nonexistent table references and cross-database references.
  • Cross-database renameView is supported and tested.
  • Concurrent duplicate view creation is covered by a two-thread test and relies on the JDBC metadata primary key. I also centralized uniqueness violation detection for SQLite message text and common SQLState-based drivers.
  • Added hardening for catalog invariants: non-cascade dropDatabase now rejects view-only databases, renameView rejects missing target databases, and table/view same-name collisions are rejected on create and rename paths.
  • Added JDBC catalog view documentation in concepts, Flink SQL DDL, and Spark SQL DDL docs.

Local validation:

  • git diff --check
  • mvn -pl paimon-core spotless:check
  • mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -Dtest=JdbcCatalogTest test -- 67 tests, 0 failures
  • mvn -pl paimon-core -am -DfailIfNoTests=false -Dtest=JdbcCatalogTest test -- 67 tests, 0 failures

I also retried mvn -pl paimon-flink/paimon-flink-common -am -Pfast-build -DfailIfNoTests=false -Dtest=JdbcCatalogViewITCase test, but it was blocked before test execution by the external Jindo dependency repository timeout while resolving com.aliyun.jindodata:jindo-core:6.9.1 / jindo-sdk:6.9.1 for unrelated paimon-jindo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant