-
Notifications
You must be signed in to change notification settings - Fork 24.2k
8000
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/145000
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 2f1782e with merge base 85bfaf8 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR makes the necessary changes in order to upgrade PyTorch DLPack support to version 1.0. In summary, we add support for the following: - Support both `DLManagedTensor` and `DLManagedTensorVersioned` when producing and consuming DLPack capsules - New parameter for `__dlpack__` method: `max_version` - Version checks: - Fallback to old implementation if no `max_version` or if version lower than 1.0 - Check that the to-be-consumed capsule is of version up to 1.X In order to accommodate these new specifications, this PR adds the following main changes: - `torch._C._to_dlpack_versioned` Python API (Module.cpp): new Python API for creating a versioned DLPack capsule (called by `__dlpack__` method) - `DLPackTraits<T>` class (DLConvertor.h): select the correct capsule name depending on which DLPack tensor class is being used - `toDLPackImpl<T>` function (DLConvertor.cpp): populates the common fields of both classes - `fillVersion<T>` function (DLConvertor.cpp): populates the version field for `DLManagedTensorVersioned` (no-op for `DLManagedTensor`) - `fromDLPackImpl<T>` function (tensor_new.cpp): common function for creating an `at::Tensor` for both classes, leaving the possible version check for its caller ghstack-source-id: 3ca1169 Pull Request resolved: #145000
This PR makes the necessary changes in order to upgrade PyTorch DLPack support to version 1.0. In summary, we add support for the following: - Support both `DLManagedTensor` and `DLManagedTensorVersioned` when producing and consuming DLPack capsules - New parameter for `__dlpack__` method: `max_version` - Version checks: - Fallback to old implementation if no `max_version` or if version lower than 1.0 - Check that the to-be-consumed capsule is of version up to 1.X In order to accommodate these new specifications, this PR adds the following main changes: - `torch._C._to_dlpack_versioned` Python API (Module.cpp): new Python API for creating a versioned DLPack capsule (called by `__dlpack__` method) - `DLPackTraits<T>` class (DLConvertor.h): select the correct capsule name depending on which DLPack tensor class is being used - `toDLPackImpl<T>` function (DLConvertor.cpp): populates the common fields of both classes - `fromDLPackImpl<T>` function (DLConvertor.cpp): constructs a tensor from a DLPAck capsule - `fillVersion<T>` function (DLConvertor.cpp): populates the version field for `DLManagedTensorVersioned` (no-op for `DLManagedTensor`) - `tensor_fromDLPackImpl<T>` function (tensor_new.cpp): outer function for constructing a tensor out of a DLPack capsule that also marks the capsule as used ghstack-source-id: e2d39d9 Pull Request resolved: #145000
This PR makes the necessary changes in order to upgrade PyTorch DLPack support to version 1.0. In summary, we add support for the following: - Support both `DLManagedTensor` and `DLManagedTensorVersioned` when producing and consuming DLPack capsules - New parameter for `__dlpack__` method: `max_version` - Version checks: - Fallback to old implementation if no `max_version` or if version lower than 1.0 - Check that the to-be-consumed capsule is of version up to 1.X In order to accommodate these new specifications, this PR adds the following main changes: - `torch._C._to_dlpack_versioned` Python API (Module.cpp): new Python API for creating a versioned DLPack capsule (called by `__dlpack__` method) - `DLPackTraits<T>` class (DLConvertor.h): select the correct capsule name depending on which DLPack tensor class is being used - `toDLPackImpl<T>` function (DLConvertor.cpp): populates the common fields of both classes - `fromDLPackImpl<T>` function (DLConvertor.cpp): constructs a tensor from a DLPAck capsule - `fillVersion<T>` function (DLConvertor.cpp): populates the version field for `DLManagedTensorVersioned` (no-op for `DLManagedTensor`) - `tensor_fromDLPackImpl<T>` function (tensor_new.cpp): outer function for constructing a tensor out of a DLPack capsule that also marks the capsule as used ghstack-source-id: e58ba67 Pull Request resolved: #145000
This PR makes the necessary changes in order to upgrade PyTorch DLPack support to version 1.0. In summary, we add support for the following: - Support both `DLManagedTensor` and `DLManagedTensorVersioned` when producing and consuming DLPack capsules - New parameter for `__dlpack__` method: `max_version` - Version checks: - Fallback to old implementation if no `max_version` or if version lower than 1.0 - Check that the to-be-consumed capsule is of version up to 1.X In order to accommodate these new specifications, this PR adds the following main changes: - `torch._C._to_dlpack_versioned` Python API (Module.cpp): new Python API for creating a versioned DLPack capsule (called by `__dlpack__` method) - `DLPackTraits<T>` class (DLConvertor.h): select the correct capsule name depending on which DLPack tensor class is being used - `toDLPackImpl<T>` function (DLConvertor.cpp): populates the common fields of both classes - `fromDLPackImpl<T>` function (DLConvertor.cpp): constructs a tensor from a DLPAck capsule - `fillVersion<T>` function (DLConvertor.cpp): populates the version field for `DLManagedTensorVersioned` (no-op for `DLManagedTensor`) - `tensor_fromDLPackImpl<T>` function (tensor_new.cpp): outer function for constructing a tensor out of a DLPack capsule that also marks the capsule as used ghstack-source-id: 1f9150c Pull Request resolved: #145000
This PR makes the necessary changes in order to upgrade PyTorch DLPack support to version 1.0. In summary, we add support for the following: - Support both `DLManagedTensor` and `DLManagedTensorVersioned` when producing and consuming DLPack capsules - New parameter for `__dlpack__` method: `max_version` - Version checks: - Fallback to old implementation if no `max_version` or if version lower than 1.0 - Check that the to-be-consumed capsule is of version up to 1.X In order to accommodate these new specifications, this PR adds the following main changes: - `torch._C._to_dlpack_versioned` Python API (Module.cpp): new Python API for creating a versioned DLPack capsule (called by `__dlpack__` method) - `DLPackTraits<T>` class (DLConvertor.h): select the correct capsule name depending on which DLPack tensor class is being used - `toDLPackImpl<T>` function (DLConvertor.cpp): populates the common fields of both classes - `fromDLPackImpl<T>` function (DLConvertor.cpp): constructs a tensor from a DLPAck capsule - `fillVersion<T>` function (DLConvertor.cpp): populates the version field for `DLManagedTensorVersioned` (no-op for `DLManagedTensor`) - `tensor_fromDLPackImpl<T>` function (tensor_new.cpp): outer function for constructing a tensor out of a DLPack capsule that also marks the capsule as used ghstack-source-id: 063107b Pull Request resolved: #145000
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Change sounds ok but what is the expectations in terms of BC when interracting with libraries that didn't upgrade to latest dlpack yet? Is that ok for all Tensors to be of the new version?
TORCH_API Tensor fromDLPack(DLManagedTensor* src); | ||
TORCH_API Tensor | ||
fromDLPack(DLManagedTensor* src, std::function<void(void*)> deleter); | ||
TORCH_API DLManagedTensorVersioned* toDLPack(const Tensor& src); |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Is this API used by C++ libraries? This would be a BC-breaking change for these users right?
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I think it's being used by PyTorch/XLA. Yes, it's definitely BC-breaking. I was thinking that, since DLPack 1.0 should be the new default, the old version should have the name with a suffix. However, now that you brought this up, not being BC-breaking sounds more important.
In summary, I will change the names so that we are not BC-breaking.
TORCH_API DLManagedTensor* toDLPackUnversioned(const Tensor& src); | ||
TORCH_API Tensor fromDLPack( | ||
DLManagedTensorVersioned* src, | ||
std::optional<std::function<void(void*)>> deleter = std::nullopt); |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
std::function is optional by default and doesn't require std::optional right?
@@ -2270,7 +2270,7 @@ def compiled_with_cxx11_abi() -> builtins.bool: | |||
matrix_rank, | |||
solve, | |||
) | |||
from torch.utils.dlpack import from_dlpack, to_dlpack | |||
from torch.utils.dlpack import from_dlpack, to_dlpack, to_dlpack_unversioned |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
While I understand this matches current code, I don't think we want to have this as a new top level API? Being part of torch.utils.dlpack is enough?
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
That comment sounds right - I don't think it should be needed. For the introduction strategy to a versioned protocol, see the explanation and prototype code (if max_version is None: ....
) at https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.__dlpack__.html
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
If it's useful/needed to have a Python function to use here so that can be called from within Tensor.__dlpack__
, then making it a private function by prepending an underscore should be fine I think.
@@ -1599,6 +1616,7 @@ static std::initializer_list<PyMethodDef> TorchMethods = { | |||
METH_NOARGS, | |||
nullptr}, | |||
{"_to_dlpack", THPModule_toDLPack, METH_O, nullptr}, | |||
{"_to_dlpack_unversioned", THPModule_toDLPackUnversioned, METH_O, nullptr}, |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Why do we still need this one?
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
This does look necessary to me. For DLPack to change the ABI once, there's a dance that needs doing to be not-super-disruptive: continue returning the old (0.8) version, unless the consumer indicates it can handle the new (1.X) version by passing in max_version=(1, 0)
(or (1, x)
in the future).
try: | ||
# Try running __dlpack__ while specifying `max_version` argument. | ||
dlpack = ext_tensor.__dlpack__(**kwargs) | ||
except TypeError: |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Is it guaranteed that they will fail with TypeError?
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
That should be the case - it's the standard error in Python for using a non-existing keyword.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Thanks for tackling this @ysiraichi! This looks like a good start. A few high-level comments:
__dlpack__
gained 3 new keywords, so far this adds only max_version
. I suspect it's safer to add all at once, because other libraries are probably going to assume that if max_version
is present, the 1.0 support is complete.Line 239 in e9f6e27
# TODO: add interchange tests once NumPy 1.22 (dlpack support) is required |
DLPACK_FLAG_BITMASK_READ_ONLY
field, which in the future can feed into PyTorch's copy-on-write (COW) feature.try: | ||
# Try running __dlpack__ while specifying `max_version` argument. | ||
dlpack = ext_tensor.__dlpack__(**kwargs) | ||
except TypeError: |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
That should be the case - it's the standard error in Python for using a non-existing keyword.
@@ -2270,7 +2270,7 @@ def compiled_with_cxx11_abi() -> builtins.bool: | |||
matrix_rank, | |||
solve, | |||
) | |||
from torch.utils.dlpack import from_dlpack, to_dlpack | |||
from torch.utils.dlpack import from_dlpack, to_dlpack, to_dlpack_unversioned |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
That comment sounds right - I don't think it should be needed. For the introduction strategy to a versioned protocol, see the explanation and prototype code (if max_version is None: ....
) at https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.__dlpack__.html
|
||
max_version (tuple[int, int] or None): An optional Python tuple with | ||
2 integers, representing the maximum version the caller supports. If | ||
None is passed, then PyTorch will fallback to DLPack 0.X, where versions |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I'd rephrase as "If None (default), PyTorch will use DLPack 0.8".
@@ -2270,7 +2270,7 @@ def compiled_with_cxx11_abi() -> builtins.bool: | |||
matrix_rank, | |||
solve, | |||
) | |||
from torch.utils.dlpack import from_dlpack, to_dlpack | |||
from torch.utils.dlpack import from_dlpack, to_dlpack, to_dlpack_unversioned |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
If it's useful/needed to have a Python function to use here so that can be called from within Tensor.__dlpack__
, then making it a private function by prepending an underscore should be fine I think.
@@ -1661,7 +1661,7 @@ def __torch_function__(cls, func, types, args=(), kwargs=None): | |||
|
|||
__torch_dispatch__ = _C._disabled_torch_dispatch_impl | |||
|
|||
def __dlpack__(self, stream=None): | |||
def __dlpack__(self, stream=None, max_version=None): |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Note that there are also new dl_device
and copy
keywords (API docs).
@@ -1599,6 +1616,7 @@ static std::initializer_list<PyMethodDef> TorchMethods = { | |||
METH_NOARGS, | |||
nullptr}, | |||
{"_to_dlpack", THPModule_toDLPack, METH_O, nullptr}, | |||
{"_to_dlpack_unversioned", THPModule_toDLPackUnversioned, METH_O, nullptr}, |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
This does look necessary to me. For DLPack to change the ABI once, there's a dance that needs doing to be not-super-disruptive: continue returning the old (0.8) version, unless the consumer indicates it can handle the new (1.X) version by passing in max_version=(1, 0)
(or (1, x)
in the future).
@albanD @rgommers Since you have already reviewed this PR, I was thinking about adding the new
Let me know if that works for you. |
@ysiraichi a stack with two separate PRs, with the |
Separate PR on the stack sounds ok. |
This PR makes the necessary changes in order to upgrade PyTorch DLPack support to version 1.0. In summary, we add support for the following: - Support both `DLManagedTensor` and `DLManagedTensorVersioned` when producing and consuming DLPack capsules - New parameter for `__dlpack__` method: `max_version` - Version checks: - Fallback to old implementation if no `max_version` or if version lower than 1.0 - Check that the to-be-consumed capsule is of version up to 1.X In order to accommodate these new specifications, this PR adds the following main changes: - `torch._C._to_dlpack_versioned` Python API (Module.cpp): new Python API for creating a versioned DLPack capsule (called by `__dlpack__` method) - `DLPackTraits<T>` class (DLConvertor.h): select the correct capsule name depending on which DLPack tensor class is being used - `toDLPackImpl<T>` function (DLConvertor.cpp): populates the common fields of both classes - `fromDLPackImpl<T>` function (DLConvertor.cpp): constructs a tensor from a DLPAck capsule - `fillVersion<T>` function (DLConvertor.cpp): populates the version field for `DLManagedTensorVersioned` (no-op for `DLManagedTensor`) - `tensor_fromDLPackImpl<T>` function (tensor_new.cpp): outer function for constructing a tensor out of a DLPack capsule that also marks the capsule as used ghstack-source-id: 2b774ce Pull Request resolved: #145000
Gentle ping @ysiraichi, any chance we can get this work wrapped up in the near future? Note that we just tagged DLPack v1.1: https://github.com/dmlc/dlpack/releases/tag/v1.1, which added a few more dtype enums. |
Yeah. I'm still working on this. Have 2 more PRs to be opened. |
Here is a summary of the latest updates after the last reviews:
@albanD @rgommers |
|
||
template <class T> | ||
at::Tensor tensor_fromDLPackImpl(PyObject* data, T* tensor) { | ||
// HACK: Ensure that we hold the GIL here just in case the |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
n00b q: what's the context here?
@@ -1690,9 +1690,14 @@ def __dlpack__(self, stream=None): | |||
both streams. If None or -1 is passed then no synchronization is performed. | |||
If 1 (on CUDA) or 0 (on ROCM) then the default stream is used for | |||
synchronization. | |||
2851 |
|
||
max_version (tuple[int, int] or None): An optional Python tuple with |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
whats the tradeoff with being conservative on the default value?
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I guess we follow the default from https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.__dlpack__.html ?
class DLDeviceType(enum.IntEnum): | ||
# Enums as in DLPack specification (aten/src/ATen/dlpack.h) | ||
kDLCPU = 1, | ||
kDLGPU = 2, | ||
kDLCPUPinned = 3, |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
why change the name?
Will defer to @albanD for merge but this seems like a fairly safe PR to land considering BC guarantees are there and despite an upgrade it doesn't change the way dlpack works in pytorch all that much and if anyhing adds more layers of tests in every new PR |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Sounds good!
Mostly small comments.
The main thing would be to add a BC-breaking note on exactly which user facing APIs are changed.
* | ||
* \note This is the current standard DLPack exchange data structure. | ||
*/ | ||
struct DLManagedTensorVersioned { |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I assume this struct is defined by the dlpack standard?
I'm curious why we define all of these by hand vs using a standard include from dlpack? I guess it doesn't exist?
def test(device, **kwargs): | ||
inp = make_tensor((5,), dtype=torch.float32, device=device) | ||
out = torch.from_dlpack(inp.__dlpack__(**kwargs)) | ||
self.assertEqual(inp, out) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Can we query the version field on the dlpack Tensor to ensure we get the object we want?
@@ -1673,7 +1673,7 @@ def __torch_function__(cls, func, types, args=(), kwargs=None): | |||
|
|||
__torch_dispatch__ = _C._disabled_torch_dispatch_impl | |||
|
|||
def __dlpack__(self, stream=None): | |||
def __dlpack__(self, stream=None, max_version=None): |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
These are kwargs only as well.
This will help with adding more kwargs later without any issue
@@ -1690,9 +1690,14 @@ def __dlpack__(self, stream=None): | |||
both streams. If None or -1 is passed then no synchronization is performed. | |||
If 1 (on CUDA) or 0 (on ROCM) then the default stream is used for | |||
synchronization. | |||
|
|||
max_version (tuple[int, int] or None): An optional Python tuple with |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I guess we follow the default from https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.__dlpack__.html ?
""" | ||
if has_torch_function_unary(self): | ||
return handle_torch_function(Tensor.__dlpack__, (self,), self, stream) | ||
args = (self, stream, max_version) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Move to kwargs
""" | ||
if has_torch_function_unary(self): | ||
return handle_torch_function(Tensor.__dlpack__, (self,), self, stream) | ||
args = (self, stream, max_version) | ||
return handle_torch_function(Tensor.__dlpack__, (self,), *args) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
bug below not from this PR: we should check both is_conj() and is_neg() as they will fail the same way!
class DLDeviceType(enum.IntEnum): | ||
# Enums as in DLPack specification (aten/src/ATen/dlpack.h) | ||
kDLCPU = 1, | ||
kDLGPU = 2, | ||
kDLCPUPinned = 3, | ||
kDLCUDA = 2, |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
These are BC-breaking right?
try: | ||
# Try running __dlpack__ while specifying `max_version` argument. | ||
dlpack = ext_tensor.__dlpack__(**kwargs) | ||
except TypeError: |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
According to the standard, it says BufferError should be raised if you don't get a good max_version. Should we handle that here?
Stack from ghstack (oldest at bottom):
BufferError
for DLPack buffer-related errors. #150691This PR makes the necessary changes in order to upgrade PyTorch DLPack
support to version 1.0. In summary, we add support for the following:
DLManagedTensor
andDLManagedTensorVersioned
whenproducing and consuming DLPack capsules
__dlpack__
method:max_version
max_version
or if versionlower than 1.0
In order to accommodate these new specifications, this PR adds the
following main changes:
torch._C._to_dlpack_versioned
Python API (Module.cpp): new PythonAPI for creating a versioned DLPack capsule (called by
__dlpack__
method)
DLPackTraits<T>
class (DLConvertor.h): select the correcttraits (e.g. capsule name, conversion functions) depending on which
DLPack tensor class is being used
toDLPackImpl<T>
function (DLConvertor.cpp): populates thecommon fields of both classes
fromDLPackImpl<T>
function (DLConvertor.cpp): constructs a tensorfrom a DLPAck capsule
fillVersion<T>
function (DLConvertor.cpp): populates the versionfield for
DLManagedTensorVersioned
(no-op forDLManagedTensor
)tensor_fromDLPackImpl<T>
function (tensor_new.cpp): outer functionfor constructing a tensor out of a DLPack capsule that also marks the
capsule as used