Skip to content

Performance Improvements for MI300X with GEMM and FP8 Enhancements #811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
chunfangamd wants to merge 6 commits into
base: main
Choose a base branch
from
+11 −1
Open

Performance Improvements for MI300X with GEMM and FP8 Enhancements #811

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7f78fad
Patch aiter/sgl-kernel versions for MI300X FP8 DSR1 sglang benchmark
chunfangamd Feb 25, 2026
db5eb71
Fix aiter/sgl-kernel API mismatch causing cuda graph capture failure
chunfangamd Feb 25, 2026
1514e4c
Update perf changelog
chunfangamd Feb 26, 2026
9c490e8
Use lmsysorg/sglang-daily:v0.5.9-rocm720-mi30x-20260310 Image
akash-amd Mar 11, 2026
5e4eb8e
Merge branch 'main' into chun_zhentao/dsr1_fp8_mi300x_20260225
akash-amd Mar 11, 2026
6ce2b11
Merge branch 'main' into chun_zhentao/dsr1_fp8_mi300x_20260225
cquil11 Mar 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/configs/amd-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ dsr1-fp4-mi355x-atom-mtp:
- { tp: 8, conc-start: 4, conc-end: 256, spec-decoding: mtp }

dsr1-fp8-mi300x-sglang:
image: lmsysorg/sglang:v0.5.9-rocm700-mi30x
image: lmsysorg/sglang-daily:v0.5.9-rocm720-mi30x-20260310
model: deepseek-ai/DeepSeek-R1-0528
model-prefix: dsr1
runner: mi300x
Expand Down
10 changes: 10 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1041,4 +1041,14 @@
- "Add --exclusive flag to MI355X single-node salloc and multi-node sbatch to prevent node sharing during benchmarks"
- "Only non-TP8 configs listed; TP8 already uses all GPUs on the node"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/934

- config-keys:
- dsr1-fp8-mi300x-sglang
description:
- "Update MI300X FP8 DSR1 SGLang image from v0.5.8 to lmsysorg/sglang-daily:v0.5.9-rocm720-mi30x-20260310"
- "Include configuration files for three GEMM operations: https://github.com/ROCm/aiter/pull/2024"
- "Improve TPOT by using fp8 bmm in MLA and MI300X for DSR1/V3: https://github.com/sgl-project/sglang/pull/18624"
- "Broaden the optimized paths to all HIP platforms and add tuned FP8 GEMM configs: https://github.com/sgl-project/sglang/pull/18242"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/811