[AMD][AgentX] DeepSeek-V4 MI355X agentic disaggregated benchmarking#2170
Open
ichbinblau wants to merge 16 commits into
Open
[AMD][AgentX] DeepSeek-V4 MI355X agentic disaggregated benchmarking#2170ichbinblau wants to merge 16 commits into
ichbinblau wants to merge 16 commits into
Conversation
matrix.config.conc for multi-node agentic is already a JSON array
(e.g. [16,32,64] from generate_sweep_configs.py). Wrapping it in a
string literal as '[${{ matrix.config.conc }}]' can't correctly
interpolate an array into a string, producing a malformed conc-list
so fromJson(inputs.conc-list) in the reusable template ended up with
an empty/wrong CONC_LIST. Use toJson(matrix.config.conc) instead,
matching the pattern already used for the other two conc-list inputs
in this file.
Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Theresa Shan <theresa.shan@amd.com>
matrix.config.conc for multi-node agentic entries is a JSON array
(chunked concurrencies per allocation), but sweep-multi-node-agentic
passed it directly into benchmark-multinode-tmpl.yml's `conc` input,
which is declared `type: string` ("First concurrency for
agentic-coding scenarios; CONC_LIST carries the full batch"). GitHub
Actions' reusable-workflow input validator rejects a sequence value
for a string-typed input at evaluation time, so the whole job failed
to even load:
evaluate reusable workflow inputs: .github/workflows/run-sweep.yml
(Line: 554, Col: 19): A sequence was not expected
Since the job never materializes when this happens, sweep-multi-node-
agentic silently disappeared from run summaries entirely instead of
showing as failed. Slice to the first element (matching the intended
"first concurrency" semantics and the same fix already applied
elsewhere, e.g. PR #2122) to restore a scalar value.
Co-authored-by: Cursor <cursoragent@cursor.com>
Multi-node agentic sweeps batched up to 4 concurrencies per SLURM allocation, running them sequentially against one shared server session. A slow/hung conc could block the rest of the batch from ever producing results, which is why run #6719 only reported c16 despite a 16/32/48/64 conc-list. Drop the batch size to 1 so each concurrency gets its own task/allocation, matching the granularity already used for single-node agentic sweeps. Cherry-picked from backup/agentx-v1.0-rebase-pre-upstream-rewrite-20260710 (082a59d), adapted for the current test suite. Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Theresa Shan <theresa.shan@amd.com>
patch_decode_tp_queue_agree()'s invocation was already commented out (dead code with no runtime effect), and the reference-only patches/decode_tp_queue_agree.patch it mirrored had no other callers. Drop both plus the README bullet pointing at it. Co-authored-by: Cursor <cursoragent@cursor.com>
…desync patch install_transformers_glm5() was gated on an exact MODEL_NAME == "GLM-5-FP8" match; broaden to any model name containing "GLM" so other GLM variants pick up the same glm_moe_dsa transformers fix. Also disable patch_disagg_prefill_bootstrap_desync's invocation (commented out, matching the already-disabled decode_tp_queue_agree pattern removed earlier). Co-authored-by: Cursor <cursoragent@cursor.com>
…2147) * agentic: add node-0 sibling benchmark-client container for DSv4 sweeps Port the "same-node sibling container" client mode from ROCm/InferenceY: when CLIENT_IMAGE is set (and no CLIENT_NODES), node 0 launches the aiperf trace replay in its own pre-baked container via the host docker socket, instead of rebuilding the aiperf venv inside the server container and running it co-located. This keeps the client's CPU-heavy tokenize/aggregate work off the sglang scheduler + router, which inflates TTFT/E2E and lowers throughput under agentic concurrency. - server_sglang.sh: add IS_AGENTIC_RUN and a CLIENT_IMAGE sibling-container branch that writes client.env and docker-runs the client against the local router (--network host). - job.slurm: define CLIENT_CONT_NAME; when CLIENT_IMAGE is set, mount the host docker socket + CLI into the server container, forward HOST_REPO_DIR/HOST_MODEL_DIR/HOST_BENCH_LOGS/CLIENT_CONT_NAME, pre-pull the client image, and clean up the client container on teardown. - amd-master.yaml: enable the sibling client on dsv4-fp4-mi355x-sglang-disagg-agentic-hicache via CLIENT_IMAGE. The separate-client-NODE mode is intentionally not ported. * agentic: use server image for sibling client so upstream CI can pull it The pre-baked rocm/pytorch-private aiperf client image is not pullable by upstream CI runners. Reuse the (public) server image as CLIENT_IMAGE and build aiperf on the fly from /workspace/utils/aiperf, matching the co-located path. Gate the pre-baked-venv env (AIPERF_USE_PREBUILT / AIPERF_VENV) behind an optional CLIENT_AIPERF_VENV so a real pre-baked client image can still opt in.
Signed-off-by: Theresa Shan <theresa.shan@amd.com>
…it (#2165) The node-0 sibling client pinned AGENTIC_OUTPUT_DIR=/run_logs/slurm_job-* (host /tmp), so the aggregated ${RESULT_FILENAME}_conc<N>.json landed outside GITHUB_WORKSPACE and the workflow result-count guard failed with "expected 1 agentic results, found 0" (run 29095134929, dsv4 c32). The co-located path leaves AGENTIC_OUTPUT_DIR unset, defaulting to INFMAX_CONTAINER_WORKSPACE=/workspace (host repo bind-mount == GITHUB_WORKSPACE). Drop the override so the sibling matches: raw artifacts still go under the trace_replay log dir (/run_logs), only the top-level result JSON moves to /workspace where the guard/upload steps expect it.
…6x inflated (#2168) The sibling client.env omitted IS_MULTINODE / PREFILL_* / DECODE_* / TP, so process_agentic_result._gpu_shape() fell back to the single-node branch (num_gpus=1) in the client container. That left "Throughput per GPU" undivided -> ~16x too high (157484 vs the co-located 9669 tok/s at c32) and recorded wrong tp/ep/num_gpus/is_multinode/disagg in the aggregated JSON. Forward the GPU-shape + aggregation-metadata env (IS_MULTINODE, DISAGG, RUNNER_TYPE, IMAGE, TP/EP_SIZE/DP_ATTENTION/DCP_SIZE/PCP_SIZE, and the PREFILL_*/DECODE_* worker/tp/ep/dp/hardware vars) so the sibling computes the same num_gpus (16) and metadata as the co-located path. Unset vars are skipped by the existing guard.
ichbinblau
requested review from
1am9trash,
billishyahao,
chunfangamd,
seungrokj and
yctseng0211
as code owners
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Resolve conflicts in DSv4 MI355X agentic disaggregated benchmark files and keep perf-changelog.yaml append-only. 中文：同步 main 到 amd/agentx-v1.0-yanfei，并解决 DSv4 MI355X agentic 分离式基准相关文件冲突；保持 perf-changelog.yaml 只在末尾追加。
Update the DSv4 MI355X agentic perf-changelog entry to point at PR 2170 with an accurate description, and move hf_dir into the DeepSeek-V4-Pro YAML block so the SGLang model path extractor can read it. 中文：修复 PR 2170 的 review 反馈：将 DSv4 MI355X agentic 的 perf-changelog 条目改为指向 PR 2170 并更新为准确描述，同时把 hf_dir 移入 DeepSeek-V4-Pro YAML 配置块，确保 SGLang 模型路径提取逻辑可以读取。
Contributor
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29140974686 |
Remove the DeepSeek-V4-Pro hf_dir key from the SGLang models.yaml block; the MI355X recipe resolves through MODEL_PATH/MODEL_DIR and only needs the misleading commented hf_dir fragment removed. 中文：删除 SGLang models.yaml 中未使用的 DeepSeek-V4-Pro hf_dir 字段；MI355X 配方通过 MODEL_PATH/MODEL_DIR 解析模型路径，只需要移除误导性的注释片段。
Contributor
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29141073508 |
Contributor
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29141213005 |
billishyahao
approved these changes
Jul 11, 2026
billishyahao
left a comment
billishyahao
left a comment
Collaborator
There was a problem hiding this comment.
As a PR reviewer and CODEOWNER, I have reviewed this and have:
- Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
- Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse. Approve for day 0
- Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this. https://github.com/SemiAnalysisAI/InferenceX/actions/runs/29141213005
- Verified that this PR passes evals. Please link to GitHub Action workflow that shows this. https://github.com/SemiAnalysisAI/InferenceX/actions/runs/29141213005
- Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
- Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
- If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
- If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
- Verified that the single-node recipes are similar to the official vLLM recipes and/or theSGLang cookbook:
- If they are not, I have verified that a PR has been opened in vLLM recipe repo or SGLang repo and linked it below in the additional detail section:
- If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
Additional detail section:
- insert any additional info here
This is the re-submission of #2127
Signed: @billishyahao
Collaborator
❌❌❌ REJECTED ❌❌❌@billishyahao blocking: no evals ran on the signed-off commit, no authorized ✅ Check 0 (CODEOWNER): PASS — you own |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds end-to-end support for DeepSeek-V4 (DSv4) agentic-coding benchmarks on MI355X in the SGLang disaggregated (prefill/decode) path, plus the multi-node sweep plumbing, result aggregation, and CI fixes needed to make those runs green and ingestible.