backwards before using CUDA #1198Description
When I use TorchSharp and I perform a gradient calculation before loading in the CUDA binaries, then TorchSharp no longer recognizes that CUDA is available.
For example:
var lin = torch.nn.Linear(10, 1, false);
lin.forward(torch.rand(10)).backward();
Console.WriteLine(torch.cuda.is_available()); // FalseOn the other hand, if I call a CUDA function before the backward() call, it will work:
Console.WriteLine(torch.cuda.is_available()); // True
var lin = torch.nn.Linear(10, 1, false);
lin.forward(torch.rand(10)).backward();
Console.WriteLine(torch.cuda.is_available()); // TrueI've traced this issue back to LibTorch, over here. The exact quote is:
// NB: The once_flag here implies that if you try to call any CUDA
// functionality before libATen_cuda.so is loaded, CUDA is permanently
// disabled for that copy of ATen.@NiklasGustafsson How do you think this should be handled?
In PyTorch I'm pretty sure that they load in the CUDA binaries with the first import torch call. This would be easy to add to TorchSharp, but is that the desired behavior?