Add nightly regression tests for CUDA 12.8 and 12.9 on H100/A100

[jainapurva]

We need to test TorchAO more extensively with the various cuda versions (12.8, 12.9). A nightly test run for those will help us catch regressions faster. This is a test PR to see what failures occur on these versions

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3448

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Replace all macOS instances with nextjs due CVE-2025-55182

❌ 2 New Failures

As of commit 16c30ce with merge base aa21b80 ():

NEW FAILURES - The following jobs have failed:

• Run CUDA Nightly Regression Tests (12.8, 12.9) / test (CUDA 12.8 H100, linux.aws.h100, --pre torch --index-url https://download.pytorch.org/whl/ni... / linux-job (gh)
  test/test_utils.py::TestTorchAOBaseTensor::test_default_impls_with_optional_data
• Run CUDA Nightly Regression Tests (12.8, 12.9) / test (CUDA 12.9 H100, linux.aws.h100, --pre torch --index-url https://download.pytorch.org/whl/ni... / linux-job (gh)
  test/test_utils.py::TestTorchAOBaseTensor::test_default_impls_with_optional_data

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[jerryzh168]

is this too many tests to run?

cc @huydhn @atalman

[jainapurva]

is this too many tests to run?

cc @huydhn @atalman

If these are too many tests, we can make this a weekly test, or as a test run for release process, rather than running it nightly