HDDS-15167. Concurrent Conflict Detection for Conditional MPU Complete#10183
Open
peterxcli wants to merge 2 commits intoapache:masterfrom
Open
Conversation
Signed-off-by: peterxcli <peterxcli@gmail.com>
peterxcli
changed the title
Member
Author
|
cc @YutaLin |
…ction Signed-off-by: peterxcli <peterxcli@gmail.com>
chungen0126
reviewed
May 4, 2026
Contributor
chungen0126
left a comment
chungen0126
left a comment
There was a problem hiding this comment.
Thanks @peterxcli for the patch. Left some comments.
Comment on lines
+1354
to
+1375
| protected void validateAtomicRewriteAtCommit(OmKeyInfo existing, | ||
| OmKeyInfo toCommit, Map<String, String> auditMap) throws OMException { | ||
| validateAtomicRewriteAtCommit(existing, | ||
| toCommit.getExpectedDataGeneration(), auditMap); | ||
| } | ||
|
|
||
| /** | ||
| * Validates an already admitted condition at serialized commit time. | ||
| * | ||
| * This form is for callers that must check the admitted generation before | ||
| * building the final key info to commit. | ||
| */ | ||
| protected void validateAtomicRewriteAtCommit(OmKeyInfo existing, | ||
| Long expectedGen, Map<String, String> auditMap) throws OMException { | ||
| if (expectedGen == null) { | ||
| return; | ||
| } | ||
| validateExpectedDataGeneration(existing, expectedGen, auditMap, | ||
| AtomicRewritePhase.COMMIT); | ||
| } | ||
|
|
||
| private void validateExpectedDataGeneration(OmKeyInfo existing, |
Contributor
There was a problem hiding this comment.
I feel like splitting the validation logic into these three methods might be a bit of over-engineering. Could we simplify it?
Comment on lines
+1323
to
+1328
| KeyArgs resolvedKeyArgs = validateAndRewriteIfMatchAsExpectedGeneration( | ||
| keyArgs, dbKeyInfo); | ||
| if (resolvedKeyArgs.hasExpectedDataGeneration()) { | ||
| addRewriteGenerationToAuditMap( | ||
| resolvedKeyArgs.getExpectedDataGeneration(), auditMap); | ||
| } |
Contributor
There was a problem hiding this comment.
A few quick suggestions to streamline the logic:
- Could we make
validateAndRewriteIfMatchAsExpectedGenerationprivate if it's not used elsewhere? - The
keyArgs.hasExpectedDataGeneration()check insidevalidateAndRewriteIfMatchAsExpectedGenerationis redundant, as the caller already filters it out. - Consider moving
addRewriteGenerationToAuditMapintovalidateAndRewriteIfMatchAsExpectedGeneration. This will make the logic much cleaner.
peterxcli
commented
May 5, 2026
Comment on lines
+419
to
+427
| if (omMetadataManager.getMultipartInfoTable().get(multipartKey) == null) { | ||
| throw new OMException( | ||
| failureMessage(keyArgs.getVolumeName(), keyArgs.getBucketName(), | ||
| keyArgs.getKeyName()), | ||
| OMException.ResultCodes.NO_SUCH_MULTIPART_UPLOAD_ERROR); | ||
| } | ||
|
|
||
| OmKeyInfo existingKeyInfo = getExistingKeyInfo( | ||
| omMetadataManager, keyArgs); |
Member
Author
There was a problem hiding this comment.
I am thinking if calling rocksdb inside preExecute is a good idea
Contributor
There was a problem hiding this comment.
Calling rocksDB inside preExecute is not a big problem. We also check ACLs inside preExecute, it also calls rocksDB.
Member
Author
|
And I think we should apply this concept to conditional put, too.
cc @ivandika3 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
relate: #10164
If two or more independent clients send the exact same Complete Multipart Upload request at the same time, only one client should receive a successful response and the others should get a 409. That way, they know they must re-read the object, re-upload the parts if necessary, and then complete the MPU again using the latest ETag they observed.
ref: docs.aws.amazon.com/AmazonS3/latest/API/API_CompleteMultipartUpload.html
If-Match
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15167
How was this patch tested?
UT/IT
https://github.com/peterxcli/ozone/actions/runs/25326550113