Skip to content

Add GLM-5.1 FP8 GB200 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 FP8 GB200 分离式 dynamo-sglang MTP 配置#2123

Open
RohitNagraj wants to merge 2 commits into
mainfrom
glm5-fp8-gb200-dynamo-sglang-mtp
Open

Add GLM-5.1 FP8 GB200 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 FP8 GB200 分离式 dynamo-sglang MTP 配置#2123
RohitNagraj wants to merge 2 commits into
mainfrom
glm5-fp8-gb200-dynamo-sglang-mtp

Conversation

@RohitNagraj

Copy link
Copy Markdown
Collaborator

Adds glm5-fp8-gb200-dynamo-sglang-mtp, a GB200 FP8 disaggregated
dynamo-sglang MTP config covering ISL/OSL 8k1k and 1k1k across
high-throughput (wide-EP) and low-latency (TP=8) decode topologies.

  • Config points at the zip_override blocks of the vendored recipe at
    benchmarks/multi_node/srt-slurm-recipes/sglang/glm5/gb200-fp8/glm5-mtp.yaml,
    overlaid onto the srt-slurm checkout at launch.
  • Chat template is enabled explicitly in the recipe
    (benchmark.use_chat_template: true) — MTP/EAGLE speculative decoding
    expects chat-formatted inputs.
  • Image: lmsysorg/sglang:v0.5.13.post1-cu130.
  • Adds the corresponding perf-changelog entry.

中文说明

新增 glm5-fp8-gb200-dynamo-sglang-mtp,一个 GB200 FP8 分离式(disaggregated)dynamo-sglang MTP 配置,覆盖 ISL/OSL 8k1k 与 1k1k 的高吞吐(wide-EP)和低延迟(TP=8)解码拓扑。

  • 配置指向内置 recipe benchmarks/multi_node/srt-slurm-recipes/sglang/glm5/gb200-fp8/glm5-mtp.yaml 的 zip_override 块,启动时叠加到 srt-slurm 检出中。
  • 在 recipe 中显式启用 chat template(benchmark.use_chat_template: true)—— MTP/EAGLE 投机解码要求 chat 格式的输入。
  • 镜像:lmsysorg/sglang:v0.5.13.post1-cu130
  • 新增对应的 perf-changelog 条目。

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

1 similar comment
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

@github-actions

Copy link
Copy Markdown
Contributor

1 similar comment
@github-actions

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant