8000 [pytree] hardcode values for `none_is_leaf` and `namespace` in C++ pytree by XuehaiPan · Pull Request #114858 · pytorch/pytorch · GitHub
[go: up one dir, main page]
Skip to content

[pytree] hardcode values for none_is_leaf and namespace in C++ pytree #114858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 8 commits into from
Closed

[pytree] hardcode values for none_is_leaf and namespace in C++ pytree #114858

wants to merge 8 commits into from

Conversation

XuehaiPan
Copy link
Collaborator
@XuehaiPan XuehaiPan commented Nov 30, 2023 •
edited
Loading

@XuehaiPan XuehaiPan mentioned this pull request Nov 30, 2023
Closed
@pytorch-bot PyTorch Bot
Copy link
pytorch-bot bot commented Nov 30, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/114858

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 5e0601d with merge base 25b8352 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Nov 30, 2023
Closed
Closed
Closed
Open
@XuehaiPan
Update on "[pytree] hardcode values for none_is_leaf and `namespace… < 8000 /code>
c81c2eb 
…` in C++ pytree"

[ghstack-poisoned]
@zou3519
Copy link
Contributor
zou3519 commented Nov 30, 2023

@XuehaiPan, what's the plan for after this stack is done? It would be cool to move to having C++ pytree turned on by default for PyTorch. In order to get there, we'll need to figure out:

  • how to make optree a required dependency. We can either add optree as a submodule of PyTorch or copy-paste it into PyTorch. submodule is likely the easiest.
  • how to get torch.compile to support optree. This is why we think we may need to keep around the Python pytree implementation, but maybe there is a better design that allows us to remove Python pytree.

XuehaiPan added a commit that referenced this pull request Nov 30, 2023
@XuehaiPan
[pytree] hardcode values for none_is_leaf and namespace in C++ py…
fed4436 
…tree

ghstack-source-id: 2f73472
Pull Request resolved: #114858
XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Nov 30, 2023
@XuehaiPan
[pytree] hardcode values for none_is_leaf and namespace in C++ py…
076ee7f 
…tree

ghstack-source-id: 1817ee8
Pull Request resolved: pytorch#114858
@XuehaiPan
Copy link
Collaborator Author

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Nov 30, 2023
@XuehaiPan
Copy link
Collaborator Author

what's the plan for after this stack is done? It would be cool to move to having C++ pytree turned on by default for PyTorch. In order to get there, we'll need to figure out:

  • how to make optree a required dependency. We can either add optree as a submodule of PyTorch or copy-paste it into PyTorch. submodule is likely the easiest.

@zou3519 I think the submodule approach would be fine for me.

  • how to get torch.compile to support optree. This is why we think we may need to keep around the Python pytree implementation, but maybe there is a better design that allows us to remove Python pytree.

To the best of my knowledge, the Python pytree is required by dynamo. We still have many polyfill implementations for the CPython builtin functions.

pytorch/torch/_dynamo/polyfill.py

Lines 1 to 28 in 7b3429d

"""
Python polyfills for common builtins.
"""
import math
def all(iterator):
for elem in iterator:
if not elem:
return False
return True
def index(iterator, item, start=0, end=-1):
for i, elem in enumerate(list(iterator) 8000 )[start:end]:
if item == elem:
return i
# This will not run in dynamo
raise ValueError(f"{item} is not in {type(iterator)}")
def repeat(item, count):
for i in range(count):
yield item
def radians(x):
return math.pi / 180.0 * x

Here is a minimal list of required Python implementations for the pytree API:

https://github.com/metaopt/optree/blob/main/optree/_C.pyi

def flatten(tree: PyTree[T]) -> tuple[list[T], PyTreeSpec]: ...
def is_namedtuple(obj: object | type) -> bool: ...
def is_namedtuple_class(cls: type) -> bool: ...
def is_structseq(obj: object | type) -> bool: ...
def is_structseq_class(cls: type) -> bool: ...

class PyTreeSpec:
    num_nodes: int
    num_leaves: int
    num_children: int
    type: builtins.type | None
    def unflatten(self, leaves: Iterable[T]) -> PyTree[T]: ...
    def flatten_up_to(self, full_tree: PyTree[T]) -> list[PyTree[T]]: ...
    def paths(self) -> list[tuple[Any, ...]]: ...
    def entries(self) -> list[Any]: ...
    def children(self) -> list[PyTreeSpec]: ...
    def is_leaf(self, strict: bool = True) -> bool: ...
    def __eq__(self, other: object) -> bool: ...
    def __ne__(self, other: object) -> bool: ...
    def __hash__(self) -> int: ...

def register_node(
    cls: type,
    flatten_func: FlattenFunc,
    unflatten_func: UnflattenFunc,
) -> None: ...

@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: This PR needs a release notes: label
If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Details for Dev Infra team Raised by workflow job

@XuehaiPan XuehaiPan added the topic: not user facing topic category label Nov 30, 2023
@XuehaiPan
Copy link
8000
Collaborator Author

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

Dig deeper by viewing the failures on hud

Details for Dev Infra team Raised by workflow job

Failing merge rule: Core Maintainers

@XuehaiPan
Update on "[pytree] hardcode values for none_is_leaf and `namespace…
< 8000 code> feede94 
…` in C++ pytree"

[ghstack-poisoned]
@XuehaiPan
Copy link
Collaborator Author

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: PR #113257 has not been reviewed yet

Details for Dev Infra team Raised by workflow job

Failing merge rule: Core Maintainers

@XuehaiPan
Copy link
Collaborator Author

@pytorchbot merge

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Dec 1, 2023
@XuehaiPan
[pytree] hardcode values for none_is_leaf and namespace in C++ py…
16e991c 
…tree

ghstack-source-id: e4e3c3d
Pull Request resolved: pytorch#114858
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@facebook-github-bot facebook-github-bot deleted the gh/XuehaiPan/15/head branch December 4, 2023 15:29
dmenig pushed a commit to dmenig/pytorch that referenced this pull request Dec 21, 2023
@XuehaiPan @dmenig
[pytree] hardcode values for none_is_leaf and namespace in C++ py…
9f79392 
…tree (pytorch#114858)

Pull Request resolved: pytorch#114858
Approved by: https://github.com/zou3519
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zou3519 zou3519 zou3519 approved these changes

Assignees
No one assigned
Labels
ciflow/trunk Trigger trunk jobs on your pull request Merged open source topic: not user facing topic category
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@XuehaiPan @zou3519 @pytorchmergebot @pytorchbot
0