Skip to content

[fs/azure] Set auth.type=Custom for token delegation#2947

Open
affo wants to merge 1 commit intoapache:mainfrom
affo:la-azure-fix
Open

[fs/azure] Set auth.type=Custom for token delegation#2947
affo wants to merge 1 commit intoapache:mainfrom
affo:la-azure-fix

Conversation

@affo
Copy link
Copy Markdown
Contributor

@affo affo commented Mar 27, 2026

Purpose

Linked issue: close #2946

Token delegation does not seem to work when the client does not configure any entry of the Azure FS configuration:

  1. FlussConnection filters client config:
    FileSystem.initialize(
    Configuration.fromMap(extractPrefix(conf.toMap(), "client.fs.")), null);
    1. extractPrefix filters to client.fs.* keys but does not strip the prefix — so keys remain as client.fs.* in the resulting config.
    2. AzureFileSystemPlugin.getHadoopConfiguration looks for keys with prefix "fs.azure." — but the client config has no such keys (only client.fs.*). The Hadoop config receives no Azure credentials.
    3. setCredentialProvider (AzureFileSystemPlugin.java:64-84): since fs.azure.account.key is absent from the Hadoop config, it falls back to delegation tokens and calls AzureDelegationTokenReceiver.updateHadoopConfig(hadoopConfig).
    4. updateHadoopConfig (AzureDelegationTokenReceiver.java:41-71):
      - Sets fs.azure.account.oauth.provider.type = DynamicTemporaryAzureCredentialsProvider
      - Sets additionInfos entries (just fs.azure.account.oauth2.client.endpoint)
      - DOES NOT set fs.azure.account.auth.type
    5. ABFS driver (AzureBlobFileSystem.initialize → AzureBlobFileSystemStore. → initializeClient): since fs.azure.account.auth.type is not set, defaults to SharedKey auth → calls SimpleKeyProvider.getStorageAccountKey → key not found → "Failure to
      initialize configuration".

Brief change log

Simply hadoopConfig.set("fs.azure.account.auth.type", "Custom"); in case no account key is set (presumably, client side).

Tests

  • Fix existing UT test that was skipping an exception thrown on FS init to actually check the exception is not due to misconfig
  • add check for "fs.azure.account.auth.type", "Custom" to existing UTs
  • add IT test to simulate the token delegation path (the previous IT test only checked the "server path" by setting the complete config)

API and Format

NA

Documentation

NA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Azure FS token delegation does not take effect

1 participant