Skip to content

[Feat] Integrate Mooncake into UCM By UcmMooncakeStoreV1#923

Open
NaganooMei wants to merge 1 commit into
ModelEngine-Group:developfrom
NaganooMei:dev_integrate_mooncake
Open

[Feat] Integrate Mooncake into UCM By UcmMooncakeStoreV1#923
NaganooMei wants to merge 1 commit into
ModelEngine-Group:developfrom
NaganooMei:dev_integrate_mooncake

Conversation

@NaganooMei
Copy link
Copy Markdown
Contributor

Purpose

  • Integrate UcmMooncakeStoreV1 into UCM/vLLM KV transfer flow, including lookup/load/dump/wait/check path support.
  • Wire Mooncake v1 connector registration in store factory and pass runtime config from UCMConnector.
  • Add Mooncake Prefix Cache user guide docs/source/user-guide/prefix-cache/mooncakestore.md.
  • Update Prefix Cache docs index to include mooncakestore.

Modifications

  • ucm/store/mooncakestore/mooncake_connector.py
    • Refactor and integrate Mooncake v1 backend logic.
    • Support async task execution and task lifecycle checks.
    • Add worker-side NPU buffer registration for RDMA transfer.
  • ucm/integration/vllm/ucm_connector.py
    • Pass Mooncake-required runtime fields (device_id, tensor size metadata, optional register buffers, etc.).
  • ucm/store/factory_v1.py
    • Register UcmMooncakeStoreV1 for connector factory creation.
  • docs/source/user-guide/prefix-cache/mooncakestore.md
    • Add user guide for Mooncake Store v1 usage and launch instructions.
  • docs/source/user-guide/prefix-cache/index.md
    • Add mooncakestore to toctree.

Test

Validated on single-machine Atlas A2 with whole-save whole-load enabled and UCM + Mooncake backend.

QwQ-32B (TP=2)

Sequence Length Concurrency Hit Rate Baseline Read Improvement
8k 10 1.0 10819.43 1238.38 88.55%
8k 10 0.8 10819.43 3270.46 69.77%

DSV2-Lite (TP=2)

Sequence Length Concurrency Hit Rate Baseline Read Improvement
8k 10 1.0 2911.66 859.18 70.49%
8k 10 0.8 2911.66 1356.10 53.43%

Comment thread ucm/integration/vllm/ucm_connector.py
Comment thread ucm/integration/vllm/ucm_connector.py
@NaganooMei NaganooMei force-pushed the dev_integrate_mooncake branch 4 times, most recently from 2ebf4ad to 36cb11d Compare April 21, 2026 02:53
@NaganooMei NaganooMei force-pushed the dev_integrate_mooncake branch from 36cb11d to 2cb480c Compare April 21, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants