[Feature] Enhance the data lake ingestion experience and management.

[hiliuxg]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Support near real-time data ingestion from any data source and manage the data ingestion jobs.

1. Job editing, drag and drop source, transform, router, and sink nodes, edit configuration information, which includes single table to single table synchronization, multiple tables to single table (sharding by database and table) synchronization, multiple tables to multiple tables (entire database) synchronization, schema evolution, and data traffic control.

2. Job operation, job start by specifying start positions (earliest, latest, specified time, specified position, full synchronization, incremental synchronization, full + incremental synchronization), job stop, and job reset.

Solution

Integration with Flink CDC 3.x.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!