Skip to content

HDDS-14735. ComponentVersions should not be compared using int values.#9841

Merged
errose28 merged 21 commits intoapache:HDDS-14496-zdufrom
errose28:HDDS-14735-int-version-wrapper
Mar 3, 2026
Merged

HDDS-14735. ComponentVersions should not be compared using int values.#9841
errose28 merged 21 commits intoapache:HDDS-14496-zdufrom
errose28:HDDS-14735-int-version-wrapper

Conversation

@errose28
Copy link
Contributor

@errose28 errose28 commented Feb 27, 2026

What changes were proposed in this pull request?

The int representation of component version should only be used for serialization and should not be checked directly by callers. Direct checking is error prone for various reasons, including FUTURE_VERSION serializing to -1 and mixing up comparison operators like < vs <=. It is also bad for readability. This PR provides utilities to make the serialized value opaque and provides a more robust comparison utility method.

ComponentVersion implementations now contain the following methods:

  • serialize (from HDDS-14723. Create one common interface for LayoutFeature and ComponentVersion #9832): Should only be called when sending a version over the network or to disk
  • deserialize: Should only be called when reading a version from the network or disk
  • isSupportedBy: Should be used to check version compatibility between client and server. This also wraps the deserialization.
    • This will also be wrapped by VersionManager#isAllowed to check against apparent version in
      HDDS-14732
  • For example for the client to check "Does this datanode support read block streaming?" you would write
int dnVersion = dnDetails.getCurrentVersion(); // Get the datanode's version that was sent over the network.
if (HDDSVersion.STREAM_BLOCK_SUPPORT.isSupportedBy(dnVersion)) { ... }

All existing client/server version checks have been migrated to these new methods.

Additionally, CURRENT_VERSION has been renamed to SOFTWARE_VERSION in HDDSVersion and OzoneManagerVersion to reflect the terminology used in the new versioning system. For now ClientVersion continues to use CURRENT_VERSION to refer to the latest version because clients are stateless and will not have apparent versions. TBD if we standardize this terminology so clients also refer to their latest version as SOFTWARE_VERSION in the future.

Some changes were generated by Cursor under my guidance and review.

What is the link to the Apache JIRA

HDDS-14735

How was this patch tested?

  • No functional change is expected. Existing tests pass on my fork (with some unrelated network timeouts)
  • Unit tests for the new isSupportedBy and serialize/deserializemethods added to TestComponentVersionInvariants.

@github-actions github-actions bot added the zdu Pull requests for Zero Downtime Upgrade (ZDU) https://issues.apache.org/jira/browse/HDDS-14496 label Feb 27, 2026
* HDDS-14496-zdu:
  HDDS-14723. Create one common interface for LayoutFeature and ComponentVersion (apache#9832)

Conflicts:
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java
hadoop-hdds/client/src/main/java/org/apache/hadoop/ozone/client/io/BlockInputStreamFactoryImpl.java
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ComponentVersion.java
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HDDSVersion.java
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/upgrade/HDDSLayoutFeature.java
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/ClientVersion.java
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneManagerVersion.java
hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/protocol/TestDatanodeDetails.java
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocolServerSideTranslatorPB.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestBlockDataStreamOutput.java
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/VersionExtractor.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/validation/TestVersionExtractor.java
@errose28 errose28 marked this pull request as ready for review February 27, 2026 22:28
@errose28
Copy link
Contributor Author

errose28 commented Feb 27, 2026

There are some cases that are calling serialize just because they were using ints previously, although it looks like they should be migrated to passing in ComponentVersion and leaving the serialization for a lower level. These include methods in:

  • Pipeline
  • DatanodeDetails
  • DatanodeUsageInfoProto
  • OmKeyInfo
  • StorageContainerLocationProtocolClientSideTranslatorPB
  • Placements into auditMap

Given the size of this change it would probably be better to address these cases in a follow-up. I filed HDDS-14753 for this task, although it is not a blocker to proceeding with implementing the new version managers.

Copy link
Contributor

@sodonnel sodonnel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - I just had one question about why the latest entry is the array length minus 2, rather than 1.

@errose28
Copy link
Contributor Author

errose28 commented Mar 2, 2026

@sodonnel I added a comment explaining the behavior, and filed HDDS-14753 to address this comment. PR should be ready for another review.

Copy link
Contributor

@sodonnel sodonnel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - thanks for adding that comment. CI is green now, so go ahead and commit.

@errose28
Copy link
Contributor Author

errose28 commented Mar 3, 2026

Thanks @sodonnel for the review. Turns out the upgrade acceptance failure was just another network issue:

Error response from daemon: Head "https://ghcr.io/v2/apache/ozone-runner/manifests/20260206-2-jdk21": Get "https://ghcr.io/token?scope=repository%3Aapache%2Fozone-runner%3Apull&service=ghcr.io": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

@errose28 errose28 merged commit 97e1273 into apache:HDDS-14496-zdu Mar 3, 2026
84 of 85 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

zdu Pull requests for Zero Downtime Upgrade (ZDU) https://issues.apache.org/jira/browse/HDDS-14496

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants