Skip to content

[WIP] HIVE-29668: Add -rebuildIndexes utility to reconstruct backend Metastore indexes#6545

Draft
soumyakanti3578 wants to merge 1 commit into
apache:masterfrom
soumyakanti3578:HIVE-29668
Draft

[WIP] HIVE-29668: Add -rebuildIndexes utility to reconstruct backend Metastore indexes#6545
soumyakanti3578 wants to merge 1 commit into
apache:masterfrom
soumyakanti3578:HIVE-29668

Conversation

@soumyakanti3578

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Support -rebuildIndexes through schematool for Postgres, Oracle, MySql/MariaDB, and MSSQL

Why are the changes needed?

Gives users the ability to drop and recreate indexes easily through schematool.

Does this PR introduce any user-facing change?

Yes, adds a new option to schematool

How was this patch tested?

mvn test -pl standalone-metastore/metastore-server -Dtest.groups="" -Dtest="org.apache.hadoop.hive.metastore.tools.schematool.TestSchemaToolTaskRebuildIndexes,org.apache.hadoop.hive.metastore.tools.schematool.TestPostgresIndexRebuilder,org.apache.hadoop.hive.metastore.tools.schematool.TestMySQLIndexRebuilder,org.apache.hadoop.hive.metastore.tools.schematool.TestOracleIndexRebuilder,org.apache.hadoop.hive.metastore.tools.schematool.TestMSSQLIndexRebuilder"

@sonarqubecloud

Copy link
Copy Markdown

@Override
public void rebuildIndex(IndexInfo index) throws HiveMetaException {
PgDdl ddl = ddlMap.get(index.indexName());
executeRebuild(index, ddl.dropDdl(), ddl.createDdl());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, drop+create ddl should be atomic.
For constraint-backed indexes (PKs and UNIQUE constraints), the Postgres path runs two separate DDL statements: DROP CONSTRAINT followed by ADD CONSTRAINT. If the connection has autocommit enabled and the second statement fails, the constraint is permanently dropped with no rollback. It would be safer to ensure these two steps run inside an explicit BEGIN/COMMIT block, or autocommit=false.

.withDescription("Create table for Hive warehouse/compute logs")
.create("createLogsTable");
Option rebuildIndexesOpt = new Option("rebuildIndexes",
"Detect and rebuild corrupt indexes in the metastore backend DB (Postgres only).");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are supporting all DB's, we can remove this "Postgres only" otherwise it can confuse.

}

@Override
public @NotNull String toString() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need @NotNull annotation here ? I think, record will never return null.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants