[ROCm] Upgrade ROCm CI to ROCm6.4 #151368
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/151368
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 2 Unrelated FailuresAs of commit 429a48c with merge base 7cf8049 ( NEW FAILURE - The following job has failed:
FLAKY - The following job failed but was likely due to flakiness present on trunk:
UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@pytorchbot rebase -b viable/strict |
|
@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here |
|
Successfully rebased |
975c819 to
ec493f9
Compare
|
@pytorchbot rebase |
|
@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here |
|
Successfully rebased |
74f5734 to
da03b48
Compare
|
@pytorchbot rebase main |
|
❌ 🤖 pytorchbot command failed: Try |
|
@pytorchbot rebase -b main |
|
@pytorchbot started a rebase job onto refs/remotes/origin/main. Check the current status here |
|
Successfully rebased |
da03b48 to
688d6e8
Compare
|
Had to disallow hipblaslt TT fp32 due to https://rocm.docs.amd.com/en/latest/about/release-notes.html#incorrect-computation-results-in-hipblaslt-for-specific-transpose-configuration. |
|
@pytorchbot rebase |
|
@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here |
|
Successfully rebased |
91ade79 to
c5607c2
Compare
|
Rebase failed due to Command Raised by https://github.com/pytorch/pytorch/actions/runs/14863706606 |
|
@pytorchbot rebase -b main |
|
@pytorchbot started a rebase job onto refs/remotes/origin/main. Check the current status here |
|
Rebase failed due to Command Raised by https://github.com/pytorch/pytorch/actions/runs/14864031889 |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command |
|
@jithunnair-amd inductor-cpp failures look relevant, please don't |
|
I reverted the only change that could possible have affected CUDA jobs. All other changes are inside #ifdef USE_ROCM or unit test skips. After reverting this PR the first time the same cuda inductor cpp test failed again. The test is marked as unstable now. |
|
@malfet inductor_cpp_wrapper is currently failing on main without this PR merged. It can't be related. |
|
@pytorchbot merge -i |
Merge startedYour change will be merged while ignoring the following 1 checks: inductor / unit-test / cuda12.6-py3.10-gcc9-sm86 / test (inductor_cpp_wrapper, 1, 2, linux.g5.4xlarge.nvidia.gpu) Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 mandatory check(s) failed. The first few are: Dig deeper by viewing the failures on hud |
|
@jeffdaily The link-url lint check failure can be ignored since that's not related to changes made in this PR, and the same check fails on other commits that touch However, the failure for https://github.com/pytorch/pytorch/actions/runs/14890367417/job/41830892272?pr=151368 is only observed in this PR so far. |
|
@pytorchbot merge -i |
Merge startedYour change will be merged while ignoring the following 3 checks: Lint / Link checks / lint-urls / linux-job, periodic / linux-jammy-rocm-py3.10 / test (distributed, 2, 3, linux.rocm.gpu.4, module:rocm, oncall:distributed), inductor / unit-test / cuda12.6-py3.10-gcc9-sm86 / test (inductor_cpp_wrapper, 1, 2, linux.g5.4xlarge.nvidia.gpu) Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command |
|
Can't merge closed PR #151368 |
cc @H-Huang @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @jeffdaily @sunway513 @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang @naromero77amd @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov