Skip to content

[CELEBORN-2242] PushMergedData RPC supports compatible with low version Server#3573

Closed
cxzl25 wants to merge 2 commits intoapache:mainfrom
cxzl25:CELEBORN-2242
Closed

[CELEBORN-2242] PushMergedData RPC supports compatible with low version Server#3573
cxzl25 wants to merge 2 commits intoapache:mainfrom
cxzl25:CELEBORN-2242

Conversation

@cxzl25
Copy link
Copy Markdown
Contributor

@cxzl25 cxzl25 commented Dec 25, 2025

What changes were proposed in this pull request?

Why are the changes needed?

A client that supports HARD_SPLIT in PushMergedData will report the following error when accessing a lower version of the Server.

java.nio.BufferUnderflowException
	at java.base/java.nio.Buffer.nextGetIndex(Buffer.java:713)
	at java.base/java.nio.DirectByteBuffer.get(DirectByteBuffer.java:334)
	at org.apache.celeborn.client.ShuffleClientImpl$5.onSuccess(ShuffleClientImpl.java:1430)
	at org.apache.celeborn.common.network.client.TransportResponseHandler.handle(TransportResponseHandler.java:373)
	at org.apache.celeborn.common.network.server.TransportChannelHandler.channelRead(TransportChannelHandler.java:158)

Does this PR resolve a correctness bug?

Does this PR introduce any user-facing change?

How was this patch tested?

local test

new RpcResponseCallback() {
@Override
public void onSuccess(ByteBuffer response) {
// Compatible with lower versions of Server
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please add more detail info here? e.g., JIRA tickets, versions, and why it can be treated as success

@SteNicholas SteNicholas requested a review from pan3793 January 5, 2026 10:30
@RexXiong
Copy link
Copy Markdown
Contributor

Celeborn’s release policy is that the server remains compatible with clients from the previous major version for easy user upgrades. If we also commit to ensuring that higher-version clients are compatible with lower-version servers, we would have to ensure every client-side change is compatiable on older servers. This would make development, code review, and releases significantly more difficult. Personally, I suggest that we should not maintain backward compatibility of higher-version clients with lower-version servers. cc @cxzl25 @turboFei @pan3793

@cxzl25
Copy link
Copy Markdown
Contributor Author

cxzl25 commented Jan 12, 2026

I suggest that we should not maintain backward compatibility of higher-version clients with lower-version servers

I agree with you. @RexXiong

@cxzl25 cxzl25 closed this Jan 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants