Skip to content

[Clang][AMDGPU] Search TheRock-based device libraries #170590

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
a4lg wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[Clang][AMDGPU] Search TheRock-based device libraries #170590

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bb52fec
[Clang][AMDGPU] Fix a typo
a4lg Dec 4, 2025
b426723
[Clang][AMDGPU] Search TheRock-based device libraries
a4lg Dec 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions clang/lib/Driver/ToolChains/AMDGPU.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -436,15 +436,21 @@ void RocmInstallationDetector::detectDeviceLibrary() {
if (HasDeviceLibrary)
return;

// Find device libraries in a legacy ROCm directory structure
// ${ROCM_ROOT}/amdgcn/bitcode/*
// Find device libraries in a ROCm directory structure
auto &ROCmDirs = getInstallationPathCandidates();
for (const auto &Candidate : ROCmDirs) {
// Legacy: ${ROCM_PATH}/amdgcn/bitcode/*
Copy link

@b-sumner b-sumner Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The true path is $ROCM/lib/llvm/lib/clang/NN/amdgcn/bitcode . I'm not clear on the reason for moving it.

Copy link

@b-sumner b-sumner Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"clang -print-resource-dir" will get you up to the amdgcn directory.

Copy link
Contributor Author

@a4lg a4lg Dec 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@b-summer Wait, in my environment (ROCm 7.1.0), I only find device libraries of the ROCm SDK release at /opt/rocm/amdgcn/bitcode. Am I missing something?

Copy link
Contributor Author

@a4lg a4lg Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, seems distribution-dependency.
On rocm/vllm-dev:rocm7.1.1_navi_ubuntu24.04_py3.12_pytorch_2.8_vllm_0.10.2rc1 container with Ubuntu 24.04,
/opt/rocm/amdgcn is a symbolic link to /opt/rocm/lib/llvm/lib/clang/20/lib/amdgcn.

Would you enlighten me what do I really need to change?

Copy link
Contributor

@arsenm arsenm Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You seem to be manually reconstructing the path to the resource directory, which you shouldn't need to do? Is the problem you are scraping the resource directory of a different clang?

We really, really should not have to do this. These should be treated as an integral part of the compiler, taken solely from the resource directory of the current build, and nowhere else. We're going to be stuck handling these cross build uses for a while though

Copy link
Contributor

@arsenm arsenm Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we still have the symlink, I'd probably prefer to just leave it as-is and not directly look into the resource directory

LibDevicePath = Candidate.Path;
llvm::sys::path::append(LibDevicePath, "amdgcn", "bitcode");
HasDeviceLibrary = CheckDeviceLib(LibDevicePath, Candidate.StrictChecking);
if (HasDeviceLibrary)
return;
// TheRock: ${ROCM_PATH}/lib/llvm/amdgcn/bitcode/*
LibDevicePath = Candidate.Path;
llvm::sys::path::append(LibDevicePath, "lib", "llvm", "amdgcn", "bitcode");
HasDeviceLibrary = CheckDeviceLib(LibDevicePath, Candidate.StrictChecking);
if (HasDeviceLibrary)
return;
}
}

Expand Down
1 change: 1 addition & 0 deletions clang/test/Driver/Inputs/rocm-therock/include
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../rocm/include
1 change: 1 addition & 0 deletions clang/test/Driver/Inputs/rocm-therock/lib/llvm/amdgcn
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../../../rocm/amdgcn
1 change: 1 addition & 0 deletions clang/test/Driver/Inputs/rocm-therock/share/hip/version
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../../../rocm/bin/.hipVersion
10 changes: 9 additions & 1 deletion clang/test/Driver/hip-device-libs.hip
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,FLUSHD,ROCMDIR


// Test subtarget with flushing off by ddefault.
// Test subtarget with flushing off by default.
// RUN: %clang -### --target=x86_64-linux-gnu \
// RUN: --cuda-gpu-arch=gfx900 \
// RUN: --rocm-path=%S/Inputs/rocm \
Expand Down Expand Up @@ -85,6 +85,13 @@
// RUN: %S/Inputs/hip_multiple_inputs/b.hip \
// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,FLUSHD,ROCMDIR

// Test TheRock toolchain layout
// RUN: %clang -### --target=x86_64-linux-gnu \
// RUN: --offload-arch=gfx803 -nogpuinc \
// RUN: --rocm-path=%S/Inputs/rocm-therock \
// RUN: %S/Inputs/hip_multiple_inputs/b.hip \
// RUN: 2>&1 | FileCheck %s --check-prefixes=ALL,FLUSHD,ROCMDIR-THEROCK

// Test finding device lib in resource dir
// RUN: %clang -### --target=x86_64-linux-gnu \
// RUN: --offload-arch=gfx803 -nogpuinc \
Expand Down Expand Up @@ -210,6 +217,7 @@

// RESDIR-SAME: "-mlink-builtin-bitcode" "[[DEVICELIB_DIR:[^"]+(/|\\\\)rocm_resource_dir(/|\\\\)lib(64)?(/|\\\\)amdgcn(/|\\\\).*]]ocml.bc"
// ROCMDIR-SAME: "-mlink-builtin-bitcode" "[[DEVICELIB_DIR:[^"]+(/|\\\\)rocm(/|\\\\)amdgcn(/|\\\\).*]]ocml.bc"
// ROCMDIR-THEROCK-SAME: "-mlink-builtin-bitcode" "[[DEVICELIB_DIR:[^"]+(/|\\\\)rocm-therock(/|\\\\)lib(/|\\\\)llvm(/|\\\\)amdgcn(/|\\\\).*]]ocml.bc"

// ALL-SAME: "-mlink-builtin-bitcode" "[[DEVICELIB_DIR]]ockl.bc"

Expand Down
Loading