[CUDA][CUDNN] Dispatch to cuDNN for non-batch-splittable 64-bit NCHW convolutions #153101
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/153101
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (2 Unrelated Failures)As of commit 0a9c6da with merge base 89ebd29 ( FLAKY - The following jobs failed but were likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
| c = nn.Conv2d(2, 2, kernel_size=3, stride=1, padding=1, groups=2) | ||
| yref = c(x) | ||
| y = c.to(device=device)(x.to(device=device)) | ||
| self.assertEqual(yref, y) |
There was a problem hiding this comment.
Huh, this is really numerically stable enough on for assertEqual instead of assertClose?
There was a problem hiding this comment.
assertEqual does use tolerances under the hood
|
@pytorchmergebot rebase |
|
@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here |
|
Successfully rebased |
b339a07 to
fa66195
Compare
|
@pytorchmergebot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 jobs have failed, first few of them are: trunk / linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2) Details for Dev Infra teamRaised by workflow job |
aten/src/ATen/native/Convolution.cpp
Outdated
| @@ -467,8 +467,19 @@ struct ConvParams { | |||
| // always use cudnn_depthwise for channels_last format | |||
| return true; | |||
| } | |||
| static long cudnn_version = detail::getCUDAHooks().versionCuDNN(); | |||
| // native kernel doesn't support 64-bit non-splittable case | |||
| if (needs_64bit_indexing_no_split(input, weight) && detail::getCUDAHooks().compiledWithCuDNN() && cudnn_enabled) { | |||
There was a problem hiding this comment.
Start with cudnn_enabled for efficient short circuiting here.
aten/src/ATen/native/Convolution.cpp
Outdated
| @@ -467,8 +467,19 @@ struct ConvParams { | |||
| // always use cudnn_depthwise for channels_last format | |||
| return true; | |||
| } | |||
| static long cudnn_version = detail::getCUDAHooks().versionCuDNN(); | |||
There was a problem hiding this comment.
This static variable is now run even if CUDNN is not enabled or compiled with, is this the source of the ROCM issues? Might not be able to be static as a result (unless you can put it in a constexpr or something).
|
@pytorchmergebot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
@pytorchbot revert -m "Seems to have introduced breakages on main, tentative revert: https://github.com/pytorch/pytorch/actions/runs/15024667248/job/42224521705" -c nosignal |
|
@pytorchbot successfully started a revert job. Check the current status here. |
|
@pytorchmergebot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 mandatory check(s) failed. The first few are: Dig deeper by viewing the failures on hud |
|
@pytorchmergebot rebase |
|
@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here |
|
Successfully rebased |
18ba66e to
0a9c6da
Compare
|
@pytorchmergebot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
For #152816
cc @ptrblck @msaroufim @jerryzh168 @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10