Ub transport store dev#4
Open
zchuango wants to merge 20 commits into
Open
Conversation
…transport_store_dev
…transport_store_dev
…locator.cpp Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
# Conflicts: # mooncake-store/src/utils.cpp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR enables the UB (Ultra Band) transport for Mooncake Store, along with several bug fixes and improvements for the UB transport layer in the transfer engine.
Key changes:
UB Transport support in Mooncake Store: Added
"ub"protocol handling in the store's client service, real client, and utility functions. This includes:Client::InitTransferEnginewith device name discoveryUbSegmentDeleterfor proper cleanup of UB-allocated segmentsub_allocate_memory/ub_free_memoryin the store's memory allocation pathUB Allocator: Introduced a new
ub_allocatormodule (ub_allocator.h/ub_allocator.cpp) that provides:ub_allocate_memory()— allocates NUMA-local memory vianuma_alloc_localand tracks the allocation rangeub_free_memory()— frees NUMA-allocated memory and removes the tracked rangeub_is_store_memory()— checks if a given address range overlaps with any tracked UB store memory regionUrmaEndpoint bug fixes and improvements:
slices[i] = sliceassignment before the success check so that failed slices are also properly returned to the callerURMA_PORT_ACTIVE_DEFERport state in device openretrieveRemoteSegfails and when a jetty is not importedrjetty.tp_typeandrjetty.flagfields during connection setupBuild & packaging updates:
v25.12.0.B081ub_allocator.cppto the UB sources in CMakeub_allocator.hto the install headers-lurmalinkage in p2p-store build scriptliburma.so*from auditwheel repair in the wheel build scriptModule
mooncake-transfer-engine)mooncake-store)mooncake-p2p-store)mooncake-ep)mooncake-integration)mooncake-wheel)mooncake-pg)mooncake-rl)Type of Change
How Has This Been Tested?
ub_allocate_memory/ub_free_memoryallocation and deallocation pathsub_is_store_memoryrange overlap detection logicChecklist
./scripts/code_format.shbefore submitting.