Skip to content

[inductor] allow reorder_for_locality on training graphs (opt-in) #3235

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
reger-men wants to merge 1 commit into
base: develop
Choose a base branch
from
+124 −1

[inductor] allow reorder_for_locality on training graphs (opt-in)

3edffd2
Select commit
Loading
Failed to load commit list.
Open

[inductor] allow reorder_for_locality on training graphs (opt-in) #3235

[inductor] allow reorder_for_locality on training graphs (opt-in)
3edffd2
Select commit
Loading
Failed to load commit list.
ROCm Repo Management API / Jenkins failed May 21, 2026 in 1h 58m 30s

Tests/Test Inductor/Run pytorch_inductor_1: error in 'error' step

Tests / Test PyTorch / Test PyTorch / Run pytorch_test_1 / Shell Script

Error in sh step, with arguments ./test_pytorch_test.sh.

script returned exit code 1
Build log 
[2026-05-21T11:22:56.451Z] + ./test_pytorch_test.sh
[2026-05-21T11:22:56.451Z] + [[ -z '' ]]
[2026-05-21T11:22:56.451Z] + DEFAULT_EXECUTION_OPTIONS='-e CUSTOM_TEST_ARTIFACT_BUILD_DIR=build/custom_test_artifacts         -e CUSTOM_TEST_ARTIFACTS_FILE=test_artifacts.zip         --copy-whls'
[2026-05-21T11:22:56.451Z] + source prepare_docker_env.sh -e 'EXTENSION_BUILD_GFXARCH=gfx90a;gfx908;gfx942' -e CI=1 -e TEST_CONFIG=default -e SHARD_NUMBER=1 -e CUSTOM_TEST_ARTIFACT_BUILD_DIR=build/custom_test_artifacts -e CUSTOM_TEST_ARTIFACTS_FILE=test_artifacts.zip --copy-whls
[2026-05-21T11:22:56.451Z] ++ set -ex
[2026-05-21T11:22:56.451Z] ++ set -o pipefail
[2026-05-21T11:22:56.451Z] ++ copy_whls=0
[2026-05-21T11:22:56.451Z] ++ envvars=()
[2026-05-21T11:22:56.451Z] ++ [[ 13 -gt 0 ]]
[2026-05-21T11:22:56.451Z] ++ key=-e
[2026-05-21T11:22:56.451Z] ++ case $key in
[2026-05-21T11:22:56.451Z] ++ envvars+=("-e $2")
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ [[ 11 -gt 0 ]]
[2026-05-21T11:22:56.451Z] ++ key=-e
[2026-05-21T11:22:56.451Z] ++ case $key in
[2026-05-21T11:22:56.451Z] ++ envvars+=("-e $2")
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ [[ 9 -gt 0 ]]
[2026-05-21T11:22:56.451Z] ++ key=-e
[2026-05-21T11:22:56.451Z] ++ case $key in
[2026-05-21T11:22:56.451Z] ++ envvars+=("-e $2")
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ [[ 7 -gt 0 ]]
[2026-05-21T11:22:56.451Z] ++ key=-e
[2026-05-21T11:22:56.451Z] ++ case $key in
[2026-05-21T11:22:56.451Z] ++ envvars+=("-e $2")
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ [[ 5 -gt 0 ]]
[2026-05-21T11:22:56.451Z] ++ key=-e
[2026-05-21T11:22:56.451Z] ++ case $key in
[2026-05-21T11:22:56.451Z] ++ envvars+=("-e $2")
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ [[ 3 -gt 0 ]]
[2026-05-21T11:22:56.451Z] ++ key=-e
[2026-05-21T11:22:56.451Z] ++ case $key in
[2026-05-21T11:22:56.451Z] ++ envvars+=("-e $2")
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ [[ 1 -gt 0 ]]
[2026-05-21T11:22:56.451Z] ++ key=--copy-whls
[2026-05-21T11:22:56.451Z] ++ case $key in
[2026-05-21T11:22:56.451Z] ++ copy_whls=1
[2026-05-21T11:22:56.451Z] ++ shift
[2026-05-21T11:22:56.451Z] ++ [[ 0 -gt 0 ]]
[2026-05-21T11:22:56.451Z] ++ envvars+=("-e PREBUILT=${PREBUILT:-false}")
[2026-05-21T11:22:56.451Z] ++ envvars+=("-e TEST_CORE=${TEST_CORE:-false}")
[2026-05-21T11:22:56.451Z] ++ [[ false == \t\r\u\e ]]
[2026-05-21T11:22:56.451Z] ++ [[ true == \t\r\u\e ]]
[2026-05-21T11:22:56.451Z] ++ envvars+=("-e TEST_CORE=${TEST_CORE}")
[2026-05-21T11:22:56.451Z] ++ docker ps -a
[2026-05-21T11:22:56.451Z] CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
[2026-05-21T11:22:56.451Z] ++ docker rm -f pytorch-ci-container
[2026-05-21T11:22:56.451Z] ++ docker pull rocm/pytorch-ci-private:pytorch-linux-noble-rocm7.2.3-py3.12-c91466b3554b6f3ca5f4eac76875bdb67eb66a91-gfx908_gfx90a_gfx942
[2026-05-21T11:22:56.451Z] Error response from daemon: Head "https://registry-1.docker.io/v2/rocm/pytorch-ci-private/manifests/pytorch-linux-noble-rocm7.2.3-py3.12-c91466b3554b6f3ca5f4eac76875bdb67eb66a91-gfx908_gfx90a_gfx942": unauthorized: personal access token is expired

Tests / Test PyTorch / Test PyTorch / Run pytorch_test_1 / Error signal

Error in error step, with arguments pytorch_test_1 failed.

pytorch_test_1 failed

Tests / Test PyTorch / Test PyTorch / Run pytorch_test_1 / Archive JUnit-formatted test results

Error in junit step.

No test report files were found. Configuration error?
Build log 
[2026-05-21T11:22:56.877Z] Recording test results
[2026-05-21T11:22:56.926Z] No test report files were found. Configuration error?

Tests / Test PyTorch / Test PyTorch / Run pytorch_test_1 / Error signal

Error in error step, with arguments Failed to publish test reports xml files.

Failed to publish test reports xml files

Tests / Test PyTorch / Test PyTorch / Run pytorch_test_2 / Shell Script

Error in sh step, with arguments ./test_pytorch_test.sh.

script returned exit code 1
Build log 
[2026-05-21T11:27:56.278Z] + ./test_pytorch_test.sh
[2026-05-21T11:27:56.278Z] + [[ -z '' ]]
[2026-05-21T11:27:56.278Z] + DEFAULT_EXECUTION_OPTIONS='-e CUSTOM_TEST_ARTIFACT_BUILD_DIR=build/custom_test_artifacts         -e CUSTOM_TEST_ARTIFACTS_FILE=test_artifacts.zip         --copy-whls'
[2026-05-21T11:27:56.278Z] + source prepare_docker_env.sh -e 'EXTENSION_BUILD_GFXARCH=gfx90a;gfx908;gfx942' -e CI=1 -e TEST_CONFIG=default -e SHARD_NUMBER=2 -e CUSTOM_TEST_ARTIFACT_BUILD_DIR=build/custom_test_artifacts -e CUSTOM_TEST_ARTIFACTS_FILE=test_artifacts.zip --copy-whls
[2026-05-21T11:27:56.278Z] ++ set -ex
[2026-05-21T11:27:56.278Z] ++ set -o pipefail
[2026-05-21T11:27:56.278Z] ++ copy_whls=0
[2026-05-21T11:27:56.278Z] ++ envvars=()
[2026-05-21T11:27:56.278Z] ++ [[ 13 -gt 0 ]]
[2026-05-21T11:27:56.278Z] ++ key=-e
[2026-05-21T11:27:56.278Z] ++ case $key in
[2026-05-21T11:27:56.278Z] ++ envvars+=("-e $2")
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ [[ 11 -gt 0 ]]
[2026-05-21T11:27:56.278Z] ++ key=-e
[2026-05-21T11:27:56.278Z] ++ case $key in
[2026-05-21T11:27:56.278Z] ++ envvars+=("-e $2")
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ [[ 9 -gt 0 ]]
[2026-05-21T11:27:56.278Z] ++ key=-e
[2026-05-21T11:27:56.278Z] ++ case $key in
[2026-05-21T11:27:56.278Z] ++ envvars+=("-e $2")
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ [[ 7 -gt 0 ]]
[2026-05-21T11:27:56.278Z] ++ key=-e
[2026-05-21T11:27:56.278Z] ++ case $key in
[2026-05-21T11:27:56.278Z] ++ envvars+=("-e $2")
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ [[ 5 -gt 0 ]]
[2026-05-21T11:27:56.278Z] ++ key=-e
[2026-05-21T11:27:56.278Z] ++ case $key in
[2026-05-21T11:27:56.278Z] ++ envvars+=("-e $2")
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ [[ 3 -gt 0 ]]
[2026-05-21T11:27:56.278Z] ++ key=-e
[2026-05-21T11:27:56.278Z] ++ case $key in
[2026-05-21T11:27:56.278Z] ++ envvars+=("-e $2")
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ [[ 1 -gt 0 ]]
[2026-05-21T11:27:56.278Z] ++ key=--copy-whls
[2026-05-21T11:27:56.278Z] ++ case $key in
[2026-05-21T11:27:56.278Z] ++ copy_whls=1
[2026-05-21T11:27:56.278Z] ++ shift
[2026-05-21T11:27:56.278Z] ++ [[ 0 -gt 0 ]]
[2026-05-21T11:27:56.278Z] ++ envvars+=("-e PREBUILT=${PREBUILT:-false}")
[2026-05-21T11:27:56.278Z] ++ envvars+=("-e TEST_CORE=${TEST_CORE:-false}")
[2026-05-21T11:27:56.278Z] ++ [[ false == \t\r\u\e ]]
[2026-05-21T11:27:56.278Z] ++ [[ true == \t\r\u\e ]]
[2026-05-21T11:27:56.278Z] ++ envvars+=("-e TEST_CORE=${TEST_CORE}")
[2026-05-21T11:27:56.278Z] ++ docker ps -a
[2026-05-21T11:27:56.278Z] CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
[2026-05-21T11:27:56.278Z] ++ docker rm -f pytorch-ci-container
[2026-05-21T11:27:56.278Z] ++ docker pull rocm/pytorch-ci-private:pytorch-linux-noble-rocm7.2.3-py3.12-c91466b3554b6f3ca5f4eac76875bdb67eb66a91-gfx908_gfx90a_gfx942
[2026-05-21T11:27:56.278Z] Error response from daemon: Head "https://registry-1.docker.io/v2/rocm/pytorch-ci-private/manifests/pytorch-linux-noble-rocm7.2.3-py3.12-c91466b3554b6f3ca5f4eac76875bdb67eb66a91-gfx908_gfx90a_gfx942": unauthorized: personal access token is expired

Tests / Test PyTorch / Test PyTorch / Run pytorch_test_2 / Error signal

Error in error step, with arguments pytorch_test_2 failed.

pytorch_test_2 failed

Tests / Test PyTorch / Test PyTorch / Run pytorch_test_2 / Archive JUnit-formatted test results

Error in junit step.

No test report files were found. Configuration error?
Build log 
[2026-05-21T11:27:56.615Z] Recording test results
[2026-05-21T11:27:56.663Z] No test report files were found. Configuration error?

Tests / Test PyTorch / Test PyTorch / Run pytorch_test_2 / Error signal

Error in error step, with arguments Failed to publish test reports xml files.

Failed to publish test reports xml files

Tests / Test Distributed / Test Distributed / Run pytorch_distributed_1 / Shell Script

Error in sh step, with arguments ./test_pytorch_test_distributed.sh.

script returned exit code 1
Build log 
[2026-05-21T11:28:32.204Z] + ./test_pytorch_test_distributed.sh
[2026-05-21T11:28:32.204Z] + [[ -z '' ]]
[2026-05-21T11:28:32.204Z] + DEFAULT_EXECUTION_OPTIONS='-e CUSTOM_TEST_ARTIFACT_BUILD_DIR=build/custom_test_artifacts         -e CUSTOM_TEST_ARTIFACTS_FILE=test_artifacts.zip         --copy-whls'
[2026-05-21T11:28:32.204Z] + source prepare_docker_env.sh -e 'EXTENSION_BUILD_GFXARCH=gfx90a;gfx908;gfx942' -e CI=1 -e TEST_CONFIG=distributed -e SHARD_NUMBER=1 -e CUSTOM_TEST_ARTIFACT_BUILD_DIR=build/custom_test_artifacts -e CUSTOM_TEST_ARTIFACTS_FILE=test_artifacts.zip --copy-whls
[2026-05-21T11:28:32.204Z] ++ set -ex
[2026-05-21T11:28:32.204Z] ++ set -o pipefail
[2026-05-21T11:28:32.204Z] ++ copy_whls=0
[2026-05-21T11:28:32.204Z] ++ envvars=()
[2026-05-21T11:28:32.204Z] ++ [[ 13 -gt 0 ]]
[2026-05-21T11:28:32.204Z] ++ key=-e
[2026-05-21T11:28:32.204Z] ++ case $key in
[2026-05-21T11:28:32.204Z] ++ envvars+=("-e $2")
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ [[ 11 -gt 0 ]]
[2026-05-21T11:28:32.204Z] ++ key=-e
[2026-05-21T11:28:32.204Z] ++ case $key in
[2026-05-21T11:28:32.204Z] ++ envvars+=("-e $2")
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ [[ 9 -gt 0 ]]
[2026-05-21T11:28:32.204Z] ++ key=-e
[2026-05-21T11:28:32.204Z] ++ case $key in
[2026-05-21T11:28:32.204Z] ++ envvars+=("-e $2")
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ [[ 7 -gt 0 ]]
[2026-05-21T11:28:32.204Z] ++ key=-e
[2026-05-21T11:28:32.204Z] ++ case $key in
[2026-05-21T11:28:32.204Z] ++ envvars+=("-e $2")
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ [[ 5 -gt 0 ]]
[2026-05-21T11:28:32.204Z] ++ key=-e
[2026-05-21T11:28:32.204Z] ++ case $key in
[2026-05-21T11:28:32.204Z] ++ envvars+=("-e $2")
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ [[ 3 -gt 0 ]]
[2026-05-21T11:28:32.204Z] ++ key=-e
[2026-05-21T11:28:32.204Z] ++ case $key in
[2026-05-21T11:28:32.204Z] ++ envvars+=("-e $2")
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ [[ 1 -gt 0 ]]
[2026-05-21T11:28:32.204Z] ++ key=--copy-whls
[2026-05-21T11:28:32.204Z] ++ case $key in
[2026-05-21T11:28:32.204Z] ++ copy_whls=1
[2026-05-21T11:28:32.204Z] ++ shift
[2026-05-21T11:28:32.204Z] ++ [[ 0 -gt 0 ]]
[2026-05-21T11:28:32.204Z] ++ envvars+=("-e PREBUILT=${PREBUILT:-false}")
[2026-05-21T11:28:32.204Z] ++ envvars+=("-e TEST_CORE=${TEST_CORE:-false}")
[2026-05-21T11:28:32.204Z] ++ [[ false == \t\r\u\e ]]
[2026-05-21T11:28:32.204Z] ++ [[ true == \t\r\u\e ]]
[2026-05-21T11:28:32.204Z] ++ envvars+=("-e TEST_CORE=${TEST_CORE}")
[2026-05-21T11:28:32.204Z] ++ docker ps -a
[2026-05-21T11:28:32.204Z] CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
[2026-05-21T11:28:32.204Z] ++ docker rm -f pytorch-ci-container
[2026-05-21T11:28:32.204Z] ++ docker pull rocm/pytorch-ci-private:pytorch-linux-noble-rocm7.2.3-py3.12-c91466b3554b6f3ca5f4eac76875bdb67eb66a91-gfx908_gfx90a_gfx942
[2026-05-21T11:28:32.204Z] Error response from daemon: Head "https://registry-1.docker.io/v2/rocm/pytorch-ci-private/manifests/pytorch-linux-noble-rocm7.2.3-py3.12-c91466b3554b6f3ca5f4eac76875bdb67eb66a91-gfx908_gfx90a_gfx942": unauthorized: personal access token is expired

Tests / Test Distributed / Test Distributed / Run pytorch_distributed_1 / Error signal

Error in error step, with arguments pytorch_distributed_1 failed.

pytorch_distributed_1 failed

Tests / Test Distributed / Test Distributed / Run pytorch_distributed_1 / Archive JUnit-formatted test results

Error in junit step.

No test report files were found. Configuration error?
Build log 
[2026-05-21T11:28:32.622Z] Recording test results
[2026-05-21T11:28:32.693Z] No test report files were found. Configuration error?

Tests / Test Distributed / Test Distributed / Run pytorch_distributed_1 / Error signal

Error in error step, with arguments Failed to publish test reports xml files.

Failed to publish test reports xml files

Tests / Test Distributed / Test Distributed / Run pytorch_distributed_2 / Shell Script

Error in sh step, with arguments ./test_pytorch_test_distributed.sh.

script returned exit code 1
Build log 
[2026-05-21T11:34:16.347Z] + ./test_pytorch_test_distributed.sh
[2026-05-21T11:34:16.347Z] + [[ -z '' ]]
[2026-05-21T11:34:16.347Z] + DEFAULT_EXECUTION_OPTIONS='-e CUSTOM_TEST_ARTIFACT_BUILD_DIR=build/custom_test_artifacts         -e CUSTOM_TEST_ARTIFACTS_FILE=test_artifacts.zip         --copy-whls'
[2026-05-21T11:34:16.348Z] + source prepare_docker_env.sh -e 'EXTENSION_BUILD_GFXARCH=gfx90a;gfx908;gfx942' -e CI=1 -e TEST_CONFIG=distributed -e SHARD_NUMBER=2 -e CUSTOM_TEST_ARTIFACT_BUILD_DIR=build/custom_test_artifacts -e CUSTOM_TEST_ARTIFACTS_FILE=test_artifacts.zip --copy-whls
[2026-05-21T11:34:16.348Z] ++ set -ex
[2026-05-21T11:34:16.348Z] ++ set -o pipefail
[2026-05-21T11:34:16.348Z] ++ copy_whls=0
[2026-05-21T11:34:16.348Z] ++ envvars=()
[2026-05-21T11:34:16.348Z] ++ [[ 13 -gt 0 ]]
[2026-05-21T11:34:16.348Z] ++ key=-e
[2026-05-21T11:34:16.348Z] ++ case $key in
[2026-05-21T11:34:16.348Z] ++ envvars+=("-e $2")
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ [[ 11 -gt 0 ]]
[2026-05-21T11:34:16.348Z] ++ key=-e
[2026-05-21T11:34:16.348Z] ++ case $key in
[2026-05-21T11:34:16.348Z] ++ envvars+=("-e $2")
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ [[ 9 -gt 0 ]]
[2026-05-21T11:34:16.348Z] ++ key=-e
[2026-05-21T11:34:16.348Z] ++ case $key in
[2026-05-21T11:34:16.348Z] ++ envvars+=("-e $2")
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ [[ 7 -gt 0 ]]
[2026-05-21T11:34:16.348Z] ++ key=-e
[2026-05-21T11:34:16.348Z] ++ case $key in
[2026-05-21T11:34:16.348Z] ++ envvars+=("-e $2")
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ [[ 5 -gt 0 ]]
[2026-05-21T11:34:16.348Z] ++ key=-e
[2026-05-21T11:34:16.348Z] ++ case $key in
[2026-05-21T11:34:16.348Z] ++ envvars+=("-e $2")
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ [[ 3 -gt 0 ]]
[2026-05-21T11:34:16.348Z] ++ key=-e
[2026-05-21T11:34:16.348Z] ++ case $key in
[2026-05-21T11:34:16.348Z] ++ envvars+=("-e $2")
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ [[ 1 -gt 0 ]]
[2026-05-21T11:34:16.348Z] ++ key=--copy-whls
[2026-05-21T11:34:16.348Z] ++ case $key in
[2026-05-21T11:34:16.348Z] ++ copy_whls=1
[2026-05-21T11:34:16.348Z] ++ shift
[2026-05-21T11:34:16.348Z] ++ [[ 0 -gt 0 ]]
[2026-05-21T11:34:16.348Z] ++ envvars+=("-e PREBUILT=${PREBUILT:-false}")
[2026-05-21T11:34:16.348Z] ++ envvars+=("-e TEST_CORE=${TEST_CORE:-false}")
[2026-05-21T11:34:16.348Z] ++ [[ false == \t\r\u\e ]]
[2026-05-21T11:34:16.348Z] ++ [[ true == \t\r\u\e ]]
[2026-05-21T11:34:16.348Z] ++ envvars+=("-e TEST_CORE=${TEST_CORE}")
[2026-05-21T11:34:16.348Z] ++ docker ps -a
[2026-05-21T11:34:16.348Z] CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
[2026-05-21T11:34:16.348Z] ++ docker rm -f pytorch-ci-container
[2026-05-21T11:34:16.348Z] ++ docker pull rocm/pytorch-ci-private:pytorch-linux-noble-rocm7.2.3-py3.12-c91466b3554b6f3ca5f4eac76875bdb67eb66a91-gfx908_gfx90a_gfx942
[2026-05-21T11:34:16.348Z] Error response from daemon: Head "https://registry-1.docker.io/v2/rocm/pytorch-ci-private/manifests/pytorch-linux-noble-rocm7.2.3-py3.12-c91466b3554b6f3ca5f4eac76875bdb67eb66a91-gfx908_gfx90a_gfx942": unauthorized: personal access token is expired

Tests / Test Distributed / Test Distributed / Run pytorch_distributed_2 / Error signal

Error in error step, with arguments pytorch_distributed_2 failed.

pytorch_distributed_2 failed

Tests / Test Distributed / Test Distributed / Run pytorch_distributed_2 / Archive JUnit-formatted test results

Error in junit step.

No test report files were found. Configuration error?
Build log 
[2026-05-21T11:34:16.896Z] Recording test results
[2026-05-21T11:34:16.966Z] No test report files were found. Configuration error?

Tests / Test Distributed / Test Distributed / Run pytorch_distributed_2 / Error signal

Error in error step, with arguments Failed to publish test reports xml files.

Failed to publish test reports xml files

Tests / Test Inductor / Test Inductor / Run pytorch_inductor_1 / Shell Script

Error in sh step, with arguments ./test_pytorch_inductor.sh.

script returned exit code 1
Build log 
[2026-05-21T11:25:05.933Z] + ./test_pytorch_inductor.sh
[2026-05-21T11:25:05.933Z] + [[ -z '' ]]
[2026-05-21T11:25:05.933Z] + DEFAULT_EXECUTION_OPTIONS='-e CUSTOM_TEST_ARTIFACT_BUILD_DIR=build/custom_test_artifacts         -e CUSTOM_TEST_ARTIFACTS_FILE=test_artifacts.zip         --copy-whls'
[2026-05-21T11:25:05.933Z] + source prepare_docker_env.sh -e 'EXTENSION_BUILD_GFXARCH=gfx90a;gfx908;gfx942' -e CI=1 -e TEST_CONFIG=inductor -e CUSTOM_TEST_ARTIFACT_BUILD_DIR=build/custom_test_artifacts -e CUSTOM_TEST_ARTIFACTS_FILE=test_artifacts.zip --copy-whls
[2026-05-21T11:25:05.933Z] ++ set -ex
[2026-05-21T11:25:05.933Z] ++ set -o pipefail
[2026-05-21T11:25:05.933Z] ++ copy_whls=0
[2026-05-21T11:25:05.933Z] ++ envvars=()
[2026-05-21T11:25:05.933Z] ++ [[ 11 -gt 0 ]]
[2026-05-21T11:25:05.933Z] ++ key=-e
[2026-05-21T11:25:05.933Z] ++ case $key in
[2026-05-21T11:25:05.933Z] ++ envvars+=("-e $2")
[2026-05-21T11:25:05.933Z] ++ shift
[2026-05-21T11:25:05.933Z] ++ shift
[2026-05-21T11:25:05.933Z] ++ [[ 9 -gt 0 ]]
[2026-05-21T11:25:05.933Z] ++ key=-e
[2026-05-21T11:25:05.933Z] ++ case $key in
[2026-05-21T11:25:05.933Z] ++ envvars+=("-e $2")
[2026-05-21T11:25:05.933Z] ++ shift
[2026-05-21T11:25:05.933Z] ++ shift
[2026-05-21T11:25:05.933Z] ++ [[ 7 -gt 0 ]]
[2026-05-21T11:25:05.933Z] ++ key=-e
[2026-05-21T11:25:05.933Z] ++ case $key in
[2026-05-21T11:25:05.933Z] ++ envvars+=("-e $2")
[2026-05-21T11:25:05.933Z] ++ shift
[2026-05-21T11:25:05.933Z] ++ shift
[2026-05-21T11:25:05.933Z] ++ [[ 5 -gt 0 ]]
[2026-05-21T11:25:05.933Z] ++ key=-e
[2026-05-21T11:25:05.933Z] ++ case $key in
[2026-05-21T11:25:05.933Z] ++ envvars+=("-e $2")
[2026-05-21T11:25:05.933Z] ++ shift
[2026-05-21T11:25:05.933Z] ++ shift
[2026-05-21T11:25:05.933Z] ++ [[ 3 -gt 0 ]]
[2026-05-21T11:25:05.933Z] ++ key=-e
[2026-05-21T11:25:05.933Z] ++ case $key in
[2026-05-21T11:25:05.933Z] ++ envvars+=("-e $2")
[2026-05-21T11:25:05.933Z] ++ shift
[2026-05-21T11:25:05.933Z] ++ shift
[2026-05-21T11:25:05.933Z] ++ [[ 1 -gt 0 ]]
[2026-05-21T11:25:05.933Z] ++ key=--copy-whls
[2026-05-21T11:25:05.933Z] ++ case $key in
[2026-05-21T11:25:05.933Z] ++ copy_whls=1
[2026-05-21T11:25:05.933Z] ++ shift
[2026-05-21T11:25:05.933Z] ++ [[ 0 -gt 0 ]]
[2026-05-21T11:25:05.933Z] ++ envvars+=("-e PREBUILT=${PREBUILT:-false}")
[2026-05-21T11:25:05.933Z] ++ envvars+=("-e TEST_CORE=${TEST_CORE:-false}")
[2026-05-21T11:25:05.933Z] ++ [[ false == \t\r\u\e ]]
[2026-05-21T11:25:05.933Z] ++ [[ true == \t\r\u\e ]]
[2026-05-21T11:25:05.933Z] ++ envvars+=("-e TEST_CORE=${TEST_CORE}")
[2026-05-21T11:25:05.933Z] ++ docker ps -a
[2026-05-21T11:25:05.933Z] CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
[2026-05-21T11:25:05.933Z] ++ docker rm -f pytorch-ci-container
[2026-05-21T11:25:05.933Z] ++ docker pull rocm/pytorch-ci-private:pytorch-linux-noble-rocm7.2.3-py3.12-c91466b3554b6f3ca5f4eac76875bdb67eb66a91-gfx908_gfx90a_gfx942
[2026-05-21T11:25:05.933Z] Error response from daemon: Head "https://registry-1.docker.io/v2/rocm/pytorch-ci-private/manifests/pytorch-linux-noble-rocm7.2.3-py3.12-c91466b3554b6f3ca5f4eac76875bdb67eb66a91-gfx908_gfx90a_gfx942": unauthorized: personal access token is expired

Tests / Test Inductor / Test Inductor / Run pytorch_inductor_1 / Error signal

Error in error step, with arguments pytorch_inductor_1 failed.

pytorch_inductor_1 failed

Tests / Test Inductor / Test Inductor / Run pytorch_inductor_1 / Archive JUnit-formatted test results

Error in junit step.

No test report files were found. Configuration error?
Build log 
[2026-05-21T11:25:06.333Z] Recording test results
[2026-05-21T11:25:06.386Z] No test report files were found. Configuration error?

Tests / Test Inductor / Test Inductor / Run pytorch_inductor_1 / Error signal

Error in error step, with arguments Failed to publish test reports xml files.

Failed to publish test reports xml files

Details

  • Kill older PR Builds (3.3 sec)
  • Initialize (1 hr 43 min)
    • Download CI scripts (30 sec)
    • Checkout Pytorch (2 min 58 sec)
    • Check base Docker image existence (7.6 sec)
    • Pull Docker Image (15 min)
    • Build PyTorch (1 hr 23 min)
  • Tests (14 min)
    • Test PyTorch (7 ms)
      • Test PyTorch (8 min 11 sec)
        • Run pytorch_test_1 (2 min 57 sec)
          Error: script returned exit code 1 - logs
          Error: pytorch_test_1 failed - logs
          Error: No test report files were found. Configuration error? - logs
          Error: Failed to publish test reports xml files - logs
        • Run pytorch_test_2 (4 min 59 sec)
          Error: script returned exit code 1 - logs
          Error: pytorch_test_2 failed - logs
          Error: No test report files were found. Configuration error? - logs
          Error: Failed to publish test reports xml files - logs
    • Test Distributed (6 ms)
      • Test Distributed (14 min)
        • Run pytorch_distributed_1 (8 min 33 sec)
          Error: script returned exit code 1 - logs
          Error: pytorch_distributed_1 failed - logs
          Error: No test report files were found. Configuration error? - logs
          Error: Failed to publish test reports xml files - logs
        • Run pytorch_distributed_2 (5 min 44 sec)
          Error: script returned exit code 1 - logs
          Error: pytorch_distributed_2 failed - logs
          Error: No test report files were found. Configuration error? - logs
          Error: Failed to publish test reports xml files - logs
    • Test Inductor (6 ms)
      • Test Inductor (5 min 21 sec)
        • Run pytorch_inductor_1 (5 min 6 sec)
          Error: script returned exit code 1 - logs
          Error: pytorch_inductor_1 failed - logs
          Error: No test report files were found. Configuration error? - logs
          Error: Failed to publish test reports xml files - logs
    • Test PyTorch Slow (8 ms)
      • Test PyTorch Slow (13 sec)
    • Microbenchmark (27 sec)
      • Microbenchmark (13 sec)
  • Post Build (2 sec)
  • Declarative: Post Actions (4.7 sec)
View more details on ROCm Repo Management API