Skip to content

[Server] Missing table_bucket_lakeTiering Metrics for Lake-Enabled Tables #2422

@beryllw

Description

@beryllw

Search before asking

  • I searched in the issues and found nothing similar.

Fluss version

0.8.0 (latest release)

Please describe the bug 🐞

When enabling lake functionality after table creation via ALTER TABLE, two issues occur:

  1. The table_bucket_lakeTiering monitoring metrics are not exposed, even though the table is successfully enabled for lake tiering.
  2. Attempting to set or update the table.datalake.freshness option via ALTER TABLE fails with an unsupported operation error.

Steps to Reproduce

CREATE TABLE `fluss-test`.fluss.fluss_order_tiering_test_metrics1 (...);

ALTER TABLE `fluss-test`.fluss.fluss_order_tiering_test_metrics1 
SET ('table.datalake.enabled' = 'true');

-- Observation: The table becomes lake-enabled, but no table_bucket_lakeTiering metrics appear in monitoring.

ALTER TABLE `fluss-test`.fluss.fluss_order_tiering_test_metrics1 
SET (
  'table.datalake.enabled' = 'true',
  'table.datalake.freshness' = '30s'
);

-- Error:Caused by: org.apache.fluss.exception.InvalidAlterTableException: The option 'table.datalake.freshness' is not supported to alter yet.
Execution statement throws exception, rootCauseMessage = FlinkSQLException: org.apache.flink.table.api.TableException: Could not execute AlterTable in path `fluss-test`.`fluss`.`fluss_order_tiering_test_metrics1`. Failed to alter table fluss.fluss_order_tiering_test_metrics1 in fluss-test
	at org.apache.flink.table.catalog.CatalogManager.execute(CatalogManager.java:1952)
	at org.apache.flink.table.catalog.CatalogManager.alterTable(CatalogManager.java:1454)
	at org.apache.flink.table.operations.ddl.AlterTableChangeOperation.execute(AlterTableChangeOperation.java:160)
	at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:1218)
	at org.apache.flink.table.gateway.service.operation.OperationExecutor.callOperation(OperationExecutor.java:797)
	at org.apache.flink.table.gateway.service.operation.OperationExecutor.executeOperation(OperationExecutor.java:537)
	at org.apache.flink.table.gateway.service.operation.OperationExecutor.lambda$executeStatement$1(OperationExecutor.java:277)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
	at org.apache.flink.table.gateway.service.context.SqlGatewaySecurityContext.runSecured(SqlGatewaySecurityContext.java:73)
	at org.apache.flink.table.gateway.service.operation.OperationExecutor.executeStatement(OperationExecutor.java:231)
	at org.apache.flink.table.gateway.service.operation.OperationExecutor.executeStatement(OperationExecutor.java:217)
	at org.apache.flink.table.gateway.service.SqlGatewayServiceImpl.lambda$executeStatement$1(SqlGatewayServiceImpl.java:283)
	at org.apache.flink.table.gateway.service.operation.OperationManager.lambda$submitOperation$1(OperationManager.java:137)
	at org.apache.flink.table.gateway.service.operation.OperationManager$Operation.lambda$run$0(OperationManager.java:288)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:984)
Caused by: org.apache.flink.table.catalog.exceptions.CatalogException: Failed to alter table fluss.fluss_order_tiering_test_metrics1 in fluss-test
	at org.apache.fluss.flink.catalog.FlinkCatalog.alterTable(FlinkCatalog.java:506)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.flink.table.gateway.vvr.module.delegation.CatalogFactoryWrapper$BuiltInFactoryInvocationHandler.invoke0(CatalogFactoryWrapper.java:185)
	at org.apache.flink.table.gateway.vvr.module.delegation.FactoryWrapperBase$ArtifactInvocationHandler.invoke(FactoryWrapperBase.java:106)
	at com.sun.proxy.$Proxy136.alterTable(Unknown Source)
	at org.apache.flink.table.catalog.CatalogManager.lambda$alterTable$30(CatalogManager.java:1457)
	at org.apache.flink.table.catalog.CatalogManager.execute(CatalogManager.java:1944)
	... 22 more
Caused by: org.apache.fluss.exception.InvalidAlterTableException: The option 'table.datalake.freshness' is not supported to alter yet.

Solution

The table.datalake.freshness configuration should be alterable after table creation

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions