Skip to content

BBQ Support in SGLang#19

Open
akhauriyash wants to merge 2 commits intoprodfrom
bbq_support
Open

BBQ Support in SGLang#19
akhauriyash wants to merge 2 commits intoprodfrom
bbq_support

Conversation

@akhauriyash
Copy link
Copy Markdown
Collaborator

@akhauriyash akhauriyash commented May 6, 2026

Summary

Adds native SGLang serving support for XllmForCausalLM, used by the 375B BBQ/xLLM MoE checkpoint.

Changes

  • Add sglang.srt.models.xllm.
  • Register EntryClass = XllmForCausalLM.
  • Use SGLang native TopK.forward_native for MoE routing.
  • Preserve xLLM partial-RoPE behavior.
  • Keep the 375B MoE path single-stream while still allowing CUDA graph capture.
  • Increase the unbalanced model-loading timeout for large multi-node loads.
  • Add unit tests for registration, expert-location metadata, partial RoPE, and native TopK routing.

Why

This avoids relying on runtime file overlays and fixes the subtle long-decode failure mode seen with the generic TopK path on 375B MoE serving.

Validation

  • python3 -m py_compile python/sglang/srt/models/xllm.py python/sglang/test/test_xllm_model.py
  • git diff --check

@akhauriyash akhauriyash requested a review from a team as a code owner May 6, 2026 01:13
nightlessbaron
nightlessbaron previously approved these changes May 6, 2026
Copy link
Copy Markdown

@nightlessbaron nightlessbaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@nightlessbaron
Copy link
Copy Markdown

@akhauriyash, can you please add tests to ensure that each of the sub-modules work?

@nightlessbaron nightlessbaron self-requested a review May 6, 2026 03:05
@nightlessbaron nightlessbaron dismissed their stale review May 6, 2026 03:05

need tests

@akhauriyash
Copy link
Copy Markdown
Collaborator Author

@akhauriyash, can you please add tests to ensure that each of the sub-modules work?

Which sub-module? SGLang? I think you mean submodules in RL360 right?

I havent landed my changes in RL360 yet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants