ENH: Add a force argument to `numpy()` #78564

HaoZeke · 2022-05-31T19:56:55Z

Reopened to help with merge issues. See #59790 for full context.

Fixes #20778. Helps #71688.

Finalizes @martinPasen's force argument for Tensor.numpy(). It is set to False by default. If it's set to True then we:

detatch the Tensor, if requires_grad == True
move to cpu, if not on cpu already
Uses .resolve_conj() if .is_conj() == True
Uses .resolve_neg() if .is_neg() == True

cc @albanD

… force = True). Also it throws error if tensor.is_conj() = True and force = False and uses tensor.resolve_conj() if tensor.is_coj() = True and force = True

…ore specific with assertRaisesRegex

facebook-github-bot · 2022-05-31T19:57:00Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/78564
📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓Need help or want to give feedback on the CI? Visit our office hours

❌ 2 New Failures

As of commit 41c0d18 (more details on the Dr. CI page):

Expand to see more

2/2 failures introduced in this PR

🕵️ 2 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages

trunk / linux-bionic-rocm5.1-py3.7 / test (default, 2, 2, linux.rocm.gpu) (1/2)

Step: "Set up job" (full log | diagnosis details | 🔁 rerun)

2022-05-31T20:46:56.2318912Z ##[error]The reque...igured HttpClient.Timeout of 100 seconds elapsing.

2022-05-31T20:41:15.0120205Z ##[endgroup]
2022-05-31T20:41:15.0123962Z Secret source: Actions
2022-05-31T20:41:15.0124542Z Prepare workflow directory
2022-05-31T20:41:15.1333898Z Prepare all required actions
2022-05-31T20:41:15.1598102Z Getting action download info
2022-05-31T20:41:15.6492322Z Download action repository 'pytorch/pytorch@master' (SHA:96c134854d4dbc418cdc0ec82959476ddac8068e)
2022-05-31T20:42:55.6886828Z ##[warning]Failed to download action 'https://api.github.com/repos/pytorch/pytorch/tarball/96c134854d4dbc418cdc0ec82959476ddac8068e'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
2022-05-31T20:42:55.6897854Z ##[warning]Back off 18.681 seconds before retry.
2022-05-31T20:44:54.3818242Z ##[warning]Failed to download action 'https://api.github.com/repos/pytorch/pytorch/tarball/96c134854d4dbc418cdc0ec82959476ddac8068e'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
2022-05-31T20:44:54.3822540Z ##[warning]Back off 21.833 seconds before retry.
2022-05-31T20:46:56.2318912Z ##[error]The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

trunk / linux-bionic-rocm5.1-py3.7 / test (default, 1, 2, linux.rocm.gpu) (2/2)

Step: "Set up job" (full log | diagnosis details | 🔁 rerun)

2022-05-31T20:47:03.8554861Z ##[error]The reque...igured HttpClient.Timeout of 100 seconds elapsing.

2022-05-31T20:41:15.0442630Z ##[endgroup]
2022-05-31T20:41:15.0445960Z Secret source: Actions
2022-05-31T20:41:15.0446445Z Prepare workflow directory
2022-05-31T20:41:15.1476430Z Prepare all required actions
2022-05-31T20:41:15.1782574Z Getting action download info
2022-05-31T20:41:15.4770493Z Download action repository 'pytorch/pytorch@master' (SHA:96c134854d4dbc418cdc0ec82959476ddac8068e)
2022-05-31T20:42:55.5305279Z ##[warning]Failed to download action 'https://api.github.com/repos/pytorch/pytorch/tarball/96c134854d4dbc418cdc0ec82959476ddac8068e'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
2022-05-31T20:42:55.5330094Z ##[warning]Back off 23.07 seconds before retry.
2022-05-31T20:44:58.6178024Z ##[warning]Failed to download action 'https://api.github.com/repos/pytorch/pytorch/tarball/96c134854d4dbc418cdc0ec
8000
82959476ddac8068e'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
2022-05-31T20:44:58.6181716Z ##[warning]Back off 25.214 seconds before retry.
2022-05-31T20:47:03.8554861Z ##[error]The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

albanD

Looks good!
Thanks for the new PR.

albanD · 2022-06-01T13:54:29Z

@pytorchbot merge this please

HaoZeke · 2022-06-06T13:54:57Z

@janeyx99 could you look into the PyTorchBot issue?

albanD · 2022-06-06T14:12:53Z

Ho I didn't realize the bot didn't merge that sorry!

albanD · 2022-06-06T14:12:57Z

@pytorchbot merge

pytorchmergebot · 2022-06-06T14:14:11Z

@pytorchbot successfully started a merge job. Check the current status here

github-actions · 2022-06-06T14:15:05Z

Hey @HaoZeke.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

janeyx99 · 2022-06-06T15:37:52Z

@janeyx99 could you look into the PyTorchBot issue?

Ah! We've migrated off the natural language commands and moved to a more CLI format--please check our wiki for the valid commands now: https://github.com/pytorch/pytorch/wiki/Bot-commands

Summary: **Reopened** to help with merge issues. See #59790 for full context. Fixes #20778. Helps #71688. Finalizes martinPasen's force argument for `Tensor.numpy()`. It is set to False by default. If it's set to True then we: 1. detatch the Tensor, if requires_grad == True 2. move to cpu, if not on cpu already 3. Uses .resolve_conj() if .is_conj() == True 4. Uses .resolve_neg() if .is_neg() == True cc albanD Pull Request resolved: #78564 Approved by: https://github.com/albanD Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/3f58dd18dc6fc18ed82fb1632cea48373c0a7798 Reviewed By: seemethere Differential Revision: D36935606 Pulled By: seemethere fbshipit-source-id: dc2dd7f569feb8da29add55db3d1625241ff8d77

vadimkantorov · 2023-07-18T16:24:12Z

@albanD Can we allow automatic detach / force = True when doing automatic numpy conversion?

E.g. when visualizing Torch tensors with matplotlib: plt.plot(x, y.requires_grad_()) will fail because y is not detached, but it's quite useful for visualization purposes.

albanD · 2023-07-18T18:15:09Z

Do you have specific examples in mind?
The plotting code should be able to use force() explicitly there so that the end user doesn't have to specify it?

vadimkantorov · 2023-07-18T20:07:44Z

6D40

My treatment is: the better and better Pytorch --> NumPy interop becomes, the more value is in doing detach (or even gpu->cpu) automatically (if want to protect, can add a readonly bit in NumPy tensors).

At least for PyTorch->matplotlib quick vis the explicit .numpy() calls do make the code less readable.

Device transfer is a bit more understandable, but why can't attached-to-graph tensors be passed for zero-copy NumPy conversion?

martinPasen and others added 18 commits May 29, 2022 17:09

addition of basic test for Tensor.numpy(force = True)

e2bf93f

implemented argument force in Tensor.numpy()

da00113

adding test for sparse tensors

e438e96

modified .numpy() to always throw error, if tensor is sparse (even if…

390f2d2

… force = True). Also it throws error if tensor.is_conj() = True and force = False and uses tensor.resolve_conj() if tensor.is_coj() = True and force = True

updated tests about sparse tensors and extend tests to cover conj()

4937857

documentation

737a6ac

makeing force kwarg only argument

d9ee027

cleaning of code

f30497f

minor documentation fixes and deleting redundant ternary operator

1ada0c9

nit docs update

e4f17ee

documentation typo

a7eed4f

unified error messages in tensor.numpy(), fixing test, making tests m…

5905324

…ore specific with assertRaisesRegex

documentation refactor

c5b51ae

fixed failing test(reason was changed error message)

a472c3c

undelet of deletion

56a2688

DOC: Minor cleanup to include negative resolution

682fbdf

BUG: Fix idiotic typo

887562a

TST: Rework docstrings to pass regex assertions

41c0d18

HaoZeke requested review from albanD and soulitzer as code owners May 31, 2022 19:56

facebook-github-bot added the cla signed label May 31, 2022

HaoZeke mentioned this pull request May 31, 2022

Addition of a force argument to Tensor.numpy() #59790

Closed

pytorchbot added the open source label May 31, 2022

albanD approved these changes May 31, 2022

View reviewed changes

albanD added the ciflow/trunk Trigger trunk jobs on your pull request label May 31, 2022

vadimkantorov mentioned this pull request Jun 1, 2022

tolist called on torch scalar does not return a list => proposal to support new arg force = True to provide a work around #52262

Open

pytorchmergebot added the Merged label Jun 6, 2022

pytorchmergebot closed this in 3f58dd1 Jun 6, 2022

rgommers mentioned this pull request Jun 20, 2022

Allow __array__ to automatically detach and move to CPU #36560

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ENH: Add a force argument to `numpy()` #78564

ENH: Add a force argument to `numpy()` #78564

Uh oh!

Uh oh!

🕵️ 2 new failures recognized by patterns

trunk / linux-bionic-rocm5.1-py3.7 / test (default, 2, 2, linux.rocm.gpu) (1/2)

trunk / linux-bionic-rocm5.1-py3.7 / test (default, 1, 2, linux.rocm.gpu) (2/2)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ENH: Add a force argument to numpy() #78564

ENH: Add a force argument to numpy() #78564

Uh oh!

Conversation

Uh oh!

Uh oh!

Uh oh!

🔗 Helpful links

❌ 2 New Failures

🕵️ 2 new failures recognized by patterns

trunk / linux-bionic-rocm5.1-py3.7 / test (default, 2, 2, linux.rocm.gpu) (1/2)

trunk / linux-bionic-rocm5.1-py3.7 / test (default, 1, 2, linux.rocm.gpu) (2/2)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ENH: Add a force argument to `numpy()` #78564

ENH: Add a force argument to `numpy()` #78564