Skip to content

GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8) #2122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cquil11 wants to merge 3 commits into
base: main
Choose a base branch
from
+16 −3
Open

GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8) #2122

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fbb0323
chore: perf-changelog entry for gb200 dsv4 vllm agentic full sweep
cquil11 Jul 8, 2026
dda02fe
fix: pass multinode agentic conc list as JSON in run-sweep
cquil11 Jul 8, 2026
5fc7c84
fix: add prefill allocator headroom to gb200 3p2d agentic recipe
cquil11 Jul 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/run-sweep.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,7 @@ jobs:
model-prefix: ${{ matrix.config.model-prefix }}
framework: ${{ matrix.config.framework }}
precision: ${{ matrix.config.precision }}
conc-list: '[${{ matrix.config.conc }}]'
conc-list: ${{ toJson(matrix.config.conc) }}
spec-decoding: ${{ matrix.config.spec-decoding }}
disagg: ${{ matrix.config.disagg }}
prefill-hardware: ${{ matrix.config.prefill.hardware }}
Expand All @@ -551,7 +551,7 @@ jobs:
decode-ep: ${{ matrix.config.decode.ep }}
decode-dp-attn: ${{ matrix.config.decode.dp-attn }}
decode-additional-settings: ${{ toJson(matrix.config.decode.additional-settings) }}
conc: ${{ matrix.config.conc }}
conc: ${{ matrix.config.conc[0] }}
kv-offloading: ${{ matrix.config.kv-offloading }}
kv-offload-backend: ${{ matrix.config.kv-offload-backend }}
duration: ${{ matrix.config.duration }}
Expand Down
6 changes: 5 additions & 1 deletion ...i_node/srt-slurm-recipes/vllm/deepseek-v4/agentic/disagg-gb200-3p2d-tep8-tp8-agentic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,11 @@ backend:
# FULL capture conflicts with NCCL symmetric-memory registration on the
# TP prefill path; PIECEWISE preserves compilation and safe graph regions.
compilation-config: '{"cudagraph_mode":"PIECEWISE"}'
gpu-memory-utilization: 0.9
# 0.9 leaves <1 GiB free per GPU on the TEP8 prefill workers and both
# sweep attempts died to serving-time transients (sparse-indexer logits
# alloc at c64, DeepGEMM JIT module load at c4). MLA KV is compact, so
# trading KV space for allocator headroom costs prefill nothing.
gpu-memory-utilization: 0.85
no-disable-hybrid-kv-cache-manager: true
tokenizer-mode: deepseek_v4
decode:
Expand Down
9 changes: 9 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4643,3 +4643,12 @@
- "Recipes sourced from NVIDIA/srt-slurm branch sa-submission-q2-2026 (gb300_nvfp4 MTP recipes)"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1799

- config-keys:
- dsv4-fp4-gb200-dynamo-vllm-agentic-3p2d-tep8-tp8
- dsv4-fp4-gb200-dynamo-vllm-agentic-2p1d-dep8-dep8
description:
- "Full agentic-coding sweep of the DSv4 FP4 GB200 dynamo-vllm disagg topologies on the June 21 AgentX corpus"
- "3p2d TEP8 prefill / TP8 decode (40 inference GPUs): c4-c80"
- "2p1d DEP8 prefill / DEP8 decode (24 inference GPUs): c32-c256 including exploratory tail past the c160 normalized-throughput peak"
- "Lower 3p2d prefill gpu-memory-utilization 0.9 -> 0.85: both first-pass attempts OOMed on prefill serving-time transients (sparse-indexer logits at c64, DeepGEMM JIT at c4) with <1 GiB free per GPU"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2122

@claude claude Bot Jul 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The new perf-changelog entry has pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER — a forgotten template token. This isn't just a dead link: utils/validate_perf_changelog.py only accepts either the canonical .../pull/2122 URL or the literal placeholders XXX / .../pull/XXX, so the changelog validator will hard-fail CI until fixed. Replace PLACEHOLDER with 2122.

Extended reasoning...

The bug

The new entry at perf-changelog.yaml:4645 ends with:

pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER

The literal string PLACEHOLDER is a forgotten template token that no other entry in the file uses. The three immediately-preceding entries all use their real PR numbers (pull/2113, pull/2114, pull/2115), so the convention is clear — and per PR metadata this is PR #2122, so the value should be https://github.com/SemiAnalysisAI/InferenceX/pull/2122.

Why this fails CI (not just a dead link)

utils/validate_perf_changelog.py defines the set of accepted placeholder tokens for PR-authored runs:

# utils/validate_perf_changelog.py:24-27
PR_LINK_PLACEHOLDERS = {
    "XXX",
    "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX",
}

and validate_added_pr_link (lines 144-160) requires the link to match either the canonical .../pull/<pr_number> URL or one of those two placeholder tokens:

if link not in PR_LINK_PLACEHOLDERS and link != expected:
    raise ChangelogValidationError(
        f"new PR entry must use {expected!r} or an XXX placeholder; "
        f"found {link!r}"
    )

Step-by-step proof

  1. On this PR, the validator runs with pr_number=2122.
  2. It reads the new entry's pr-link = "https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER".
  3. It computes expected = "https://github.com/SemiAnalysisAI/InferenceX/pull/2122" — does not match.
  4. It checks link in PR_LINK_PLACEHOLDERS — the set only contains "XXX" and ".../pull/XXX", so "PLACEHOLDER" / ".../pull/PLACEHOLDER" is not a member.
  5. Both branches fail → ChangelogValidationError is raised → the changelog-validation CI job fails.

Impact

  • CI blocks the merge on the changelog-validation check until fixed.
  • Even if bypassed, the URL 404s and permanently breaks the changelog's provenance/traceability convention.

Fix

One-token change on line 4645:

-  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PLACEHOLDER
+  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2122

Comment on lines +4646 to +4654

@claude claude Bot Jul 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Nit (pre-existing style rule): per AGENTS.md line 7, PR titles and descriptions must be bilingual — the title needs a / <中文标题> half and the body needs a ## 中文说明 section mirroring the English summary. Recent sibling PRs #2114/#2115 follow this format; please update the PR title and description on GitHub. No code changes needed.

Extended reasoning...

What the rule is. AGENTS.md line 7 states: "PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format: <English title> / <中文标题>. In the PR/issue body, follow the English content with its Chinese translation (e.g. a ## 中文说明 section mirroring the summary). ... This applies to every PR and every issue."

What this PR has. The title is "GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)" — no / <中文标题> half. The description ends after the English bullet list about the two topologies and their concurrency chunks — there is no ## 中文说明 (or equivalent) section mirroring that content.

Sibling PRs confirm the convention. Recently-merged neighbors in perf-changelog.yaml follow the bilingual format, e.g. commit d527fc8 — "Add GLM-5.1 NVFP4 GB300 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 NVFP4 GB300 分离式 dynamo-sglang MTP 配置 (#2114)" — and bed23ea (#2115) has the analogous GB200 Chinese half. #1780 (c57f1e6) does too.

Impact. Purely a policy/metadata issue. No code path is affected, the sweep runs regardless, and the changelog YAML itself has no bilingual requirement. But the rule is documented as mandatory-for-every-PR, so it is worth flagging.

Step-by-step proof.

  1. Open AGENTS.md and read line 7 — the bilingual requirement is stated verbatim with "This applies to every PR and every issue."
  2. Look at this PR's title — "GB200 DSv4 vLLM agentic full sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)". There is no / separator followed by Chinese characters.
  3. Look at this PR's description above — it contains only English prose and no ## 中文说明 heading.
  4. Compare with git log --oneline -5 output, which shows Add GLM-5.1 NVFP4 GB300 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 NVFP4 GB300 分离式 dynamo-sglang MTP 配置 #2114/Add GLM-5.1 NVFP4 GB200 disaggregated dynamo-sglang MTP config / 添加 GLM-5.1 NVFP4 GB200 分离式 dynamo-sglang MTP 配置 #2115/[NV] add glm5-fp4-gb200-dynamo-sglang / 新增 GLM-5 NVFP4 GB200 Dynamo SGLang 分离式基准测试 #1780 all include the / <中文标题> half.

How to fix. In the GitHub PR UI, edit the title to append / <中文标题> (e.g. / GB200 DSv4 vLLM 智能体编码全量 sweep (3p2d TEP8/TP8 + 2p1d DEP8/DEP8)) and append a ## 中文说明 section to the body mirroring the English bullets. No file changes required.