HDDS-14701. Consolidate DiskBalancerVolumeChoosingPolicy and ContainerChoosingPolicy#9858
Open
Gargi-jais11 wants to merge 3 commits intoapache:masterfrom
Open
HDDS-14701. Consolidate DiskBalancerVolumeChoosingPolicy and ContainerChoosingPolicy#9858Gargi-jais11 wants to merge 3 commits intoapache:masterfrom
Gargi-jais11 wants to merge 3 commits intoapache:masterfrom
Conversation
Contributor
Author
|
@ChenSammi Please review the patch |
ChenSammi
reviewed
Mar 6, 2026
| public static DiskBalancerVolumeChoosingPolicy getDiskBalancerPolicy(ConfigurationSource conf) { | ||
| Class<?> policyClass = conf.getObject(DiskBalancerConfiguration.class).getVolumeChoosingPolicyClass(); | ||
| return (DiskBalancerVolumeChoosingPolicy) ReflectionUtils.newInstance( | ||
| public static VolumeContainerChoosingPolicy getDiskBalancerContainerPolicy(ConfigurationSource conf) { |
Contributor
There was a problem hiding this comment.
We need a new dedicated Factory for ContainerChoosingPolicy.
ChenSammi
reviewed
Mar 6, 2026
| */ | ||
| Pair<HddsVolume, HddsVolume> chooseVolume(MutableVolumeSet volumeSet, | ||
| double thresholdPercentage, Map<HddsVolume, Long> deltaSizes, long containerSize); | ||
| public final class DiskBalancerVolumeContainerCandidate { |
Contributor
There was a problem hiding this comment.
Let's call it ContainerCandidate, instead of DiskBalancerVolumeContainerCandidate.
ChenSammi
reviewed
Mar 6, 2026
| * Unit tests for the DefaultContainerChoosingPolicy. | ||
| */ | ||
| public class TestDefaultContainerChoosingPolicy { | ||
| public class TestDiskBalancerContainerChoosingLogic { |
Contributor
There was a problem hiding this comment.
Can we merge this one with TestDiskBalancerVolumeChoosingLogic, and
merge TestContainerChoosingPerformance with TestVolumeChoosingPerformance?
ChenSammi
reviewed
Mar 6, 2026
| * **`DefaultVolumeContainerChoosingPolicy`**: This is the default policy that consolidates both volume selection and container | ||
| selection into a single operation. It identifies the most over-utilized volume as the source and the most under-utilized | ||
| volume with sufficient space as the destination, then iterates through containers on the source to pick the first one | ||
| that is in a **CLOSED** state and is not already being moved. Consolidating both steps avoids recalculating ideal |
Contributor
There was a problem hiding this comment.
We don't need to explain why we consolidate the two policies in a final doc, it's a low level implementation detail.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
We have
DiskBalancerVolumeChoosingPolicy#chooseVolumeto choose a pair of {sourceVol, destVol}, and then haveContainerChoosingPolicy#chooseContainerto pick one container from sourceVol to destVol. In each function, overall ideal utilization and specific disk utilization are calculated individually to serve the logic.Consolidate two functions into one say,
DefaultVolumeContainerChoosingPolicywill avoid some recalculation.Another issue is that 5GB is used as the container size when choosing destVolume, while a specific container is picked, it's better for us to adjust destVolume's with container's real size, which could be larger than 5G, or less than 5G.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14701
How was this patch tested?
Updated the existing test files to work according to the new logic.