-
Notifications
You must be signed in to change notification settings - Fork 25.8k
Update torch::stable::Tensor() default constructor #159507
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
5635bca
6808f0f
daef220
20ee0fb
7eeb740
5b28d27
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
Allows things like
```cpp
Tensor cu_seqlens_q;
if (...) {
cu_seqlens_q = ...
}
...
```
Also adds `torch::stable::Tensor.defined()`
[ghstack-poisoned]- Loading branch information
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -29,11 +29,13 @@ class Tensor { | |||||||||||||||||
| std::shared_ptr<AtenTensorOpaque> ath_; | ||||||||||||||||||
|
|
||||||||||||||||||
| public: | ||||||||||||||||||
| // Construct a stable::Tensor with an uninitialized AtenTensorHandle (ATH) | ||||||||||||||||||
| // Steals ownership from the ATH | ||||||||||||||||||
| Tensor() { | ||||||||||||||||||
mikaylagawarecki marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||
| AtenTensorHandle ret; | ||||||||||||||||||
| AOTI_TORCH_ERROR_CODE_CHECK(aoti_torch_new_uninitialized_tensor(&ret)); | ||||||||||||||||||
| TORCH_ERROR_CODE_CHECK(aoti_torch_new_uninitialized_tensor(&ret)); | ||||||||||||||||||
| ath_ = std::shared_ptr<AtenTensorOpaque>(ret, [](AtenTensorHandle ath) { | ||||||||||||||||||
| AOTI_TORCH_ERROR_CODE_CHECK(aoti_torch_delete_tensor_object(ath)); | ||||||||||||||||||
| TORCH_ERROR_CODE_CHECK(aoti_torch_delete_tensor_object(ath)); | ||||||||||||||||||
| }); | ||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
@@ -123,7 +125,7 @@ class Tensor { | |||||||||||||||||
|
|
||||||||||||||||||
| bool defined() const { | ||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the same semantic as the one in TensorBase.h right? And to make sure I'm understanding correctly: an undefined tensor is an uninitialized tensor which I know has no memory but is it basically a nullptr?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes to the first pytorch/torch/csrc/inductor/aoti_torch/shim_common.cpp Lines 759 to 764 in 5f5f508
pytorch/aten/src/ATen/templates/TensorBody.h Line 103 in 5f5f508
pytorch/aten/src/ATen/core/TensorBase.h Line 95 in 5f5f508
My understanding is I think there is a special case where |
||||||||||||||||||
| bool defined; | ||||||||||||||||||
| AOTI_TORCH_ERROR_CODE_CHECK(aoti_torch_is_defined(ath_.get(), &defined)); | ||||||||||||||||||
| TORCH_ERROR_CODE_CHECK(aoti_torch_is_defined(ath_.get(), &defined)); | ||||||||||||||||||
| return defined; | ||||||||||||||||||
| } | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
woah a way to test CPU memory! how confident are you that this test is testing the right thing? This looks okay to me but I am not an expert
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried creating a list of tensors on CPU and the reserved set size increased, so I felt somewhat confident this was doing the right thing
cc @albanD do you know :P
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will most likely be flaky as we don't control what else might grab cpu memory.
I would recommend running this test on cuda and check for cuda memory which is very reliable for this kind of things.
Don't forget to add @serialTest() decorator to avoid cross polution of course ;)
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test is testing that the following code doesn't memory leak
I don't think we can make this use any cuda memory unfortunately, should we just remove the test? @janeyx99