[torchgen] Refactor torchgen.utils.FileManager to accept pathlib.Path
#150726
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/150726
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ❌ 1 New FailureAs of commit 0a437b7 with merge base e2ce17c ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
ghstack-source-id: 6c802eb Pull Request resolved: pytorch#150726
torchgen.utils.FileManager to accept pathlib.Pathtorchgen.utils.FileManager to accept pathlib.Path
There was a problem hiding this comment.
Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.
Comments suppressed due to low confidence (1)
torchgen/utils.py:180
- [nitpick] The error message does not properly interpolate the file variable; consider using an f-string (e.g. f"duplicate file write {file}") to enhance debugging clarity.
assert file not in self.files, "duplicate file write {file}"
…ath` ghstack-source-id: 6c802eb Pull Request resolved: pytorch#150726
…ath` ghstack-source-id: 6c802eb Pull Request resolved: pytorch#150726
|
Rebased |
…ath` ghstack-source-id: a576b3b Pull Request resolved: pytorch#150726
|
Rebased |
|
Starting merge as part of PR stack under #150732 |
…ath` ghstack-source-id: a576b3b Pull Request resolved: pytorch#150726
…ath` ghstack-source-id: 1969649 Pull Request resolved: pytorch#150726
…ath` ghstack-source-id: d3f2fb7 Pull Request resolved: pytorch#150726
…ath` ghstack-source-id: e910556 Pull Request resolved: pytorch#150726
…ath` ghstack-source-id: 4f12c6e Pull Request resolved: pytorch#150726
…ath` ghstack-source-id: 4f12c6e Pull Request resolved: pytorch#150726
…ath` ghstack-source-id: b1198ff Pull Request resolved: pytorch#150726
| self.filenames = set() | ||
| def __init__( | ||
| self, | ||
| install_dir: str | Path, |
There was a problem hiding this comment.
This syntax is python 3.10+ and we still need to support 3.9. Please change these (here and below) to Union[str, Path]
There was a problem hiding this comment.
There is a future import from __future__ import annotations at the top of the file.
| @@ -97,6 +97,15 @@ def context(msg_fn: Callable[[], str]) -> Iterator[None]: | |||
| raise | |||
|
|
|||
|
|
|||
| if TYPE_CHECKING: | |||
There was a problem hiding this comment.
I'm not sure if the BC checker is "TYPE_CHECKING" or not... but if this passes the BC checker then I'm okay with it.
There was a problem hiding this comment.
typing_extensions.assert_never provides the exact same function.
…ath` ghstack-source-id: 2ebb98e Pull Request resolved: pytorch#150726
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 mandatory check(s) failed. The first few are: Dig deeper by viewing the failures on hud |
…ath` ghstack-source-id: 2ebb98e Pull Request resolved: pytorch#150726
|
@pytorchbot merge -i |
Merge startedYour change will be merged while ignoring the following 1 checks: pull / linux-focal-py3.13-clang10 / test (dynamo_wrapped, 2, 3, linux.2xlarge) Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
…s (PEP 585) and Union Type (PEP 604) (#150727) #129001 (comment) is the motivation for the whole stack of PRs. In `torch/__init__.py`, `torch._C.Type` shadows `from typing import Type`, and there is no type stub for `torch._C.Type` in `torch/_C/__init__.pyi`. So we need to use `from typing import Type as _Type`. After enabling [Generic TypeAlias (PEP 585)](https://peps.python.org/pep-0585) in the `.pyi` type stub files, we can use `type` instead of `typing.Type` or `from typing import Type as _Type`. ------ - [Generic TypeAlias (PEP 585)](https://peps.python.org/pep-0585): e.g. `typing.List[T] -> list[T]`, `typing.Dict[KT, VT] -> dict[KT, VT]`, `typing.Type[T] -> type[T]`. - [Union Type (PEP 604)](https://peps.python.org/pep-0604): e.g. `Union[X, Y] -> X | Y`, `Optional[X] -> X | None`, `Optional[Union[X, Y]] -> X | Y | None`. Note that in `.pyi` stub files, we do not need `from __future__ import annotations`. So this PR does not violate issue #117449: - #117449 ------ Pull Request resolved: #150727 Approved by: https://github.com/aorenste ghstack dependencies: #150726
…nd Union Type (PEP 604) (#150728) #129001 (comment) is the motivation for the whole stack of PRs. In `torch/__init__.py`, `torch._C.Type` shadows `from typing import Type`, and there is no type stub for `torch._C.Type` in `torch/_C/__init__.pyi`. So we need to use `from typing import Type as _Type`. After enabling [Generic TypeAlias (PEP 585)](https://peps.python.org/pep-0585) in the `.pyi` type stub files, we can use `type` instead of `typing.Type` or `from typing import Type as _Type`. ------ - [Generic TypeAlias (PEP 585)](https://peps.python.org/pep-0585): e.g. `typing.List[T] -> list[T]`, `typing.Dict[KT, VT] -> dict[KT, VT]`, `typing.Type[T] -> type[T]`. - [Union Type (PEP 604)](https://peps.python.org/pep-0604): e.g. `Union[X, Y] -> X | Y`, `Optional[X] -> X | None`, `Optional[Union[X, Y]] -> X | Y | None`. Note that in `.pyi` stub files, we do not need `from __future__ import annotations`. So this PR does not violate issue #117449: - #117449 ------ Pull Request resolved: #150728 Approved by: https://github.com/cyyever, https://github.com/aorenste ghstack dependencies: #150726, #150727
This PR allows
FileManagerto acceptpathlib.Pathas arguments while keeping the originalstrpath support.This allows us to simplify the code such as:
os.path.join(..., ...)withPath.__floordiv__(..., ...).pytorch/torchgen/utils.py
Line 155 in 95a5958
pytorch/torchgen/utils.py
Line 176 in 95a5958
os.path.basename(...)withPath(...).name.pytorch/torchgen/utils.py
Line 161 in 95a5958
Manual file extension split with
Path(...).with_stem(new_stem)pytorch/torchgen/utils.py
Lines 241 to 256 in 95a5958
Stack from ghstack (oldest at bottom):
lintrunneron generated.pyistub files #150732.pyistub files #150731gen_pyiare properly formatted #150730torch/utils/data/datapipes/gen_pyi.pywithtorchgen#150626__all__totorch/nn/functional.pyiandtorch/return_types.pyi#150729.pyistub template to use Generic TypeAlias (PEP 585) and Union Type (PEP 604) #150728gen_pyi.pyto use Generic TypeAlias (PEP 585) and Union Type (PEP 604) #150727torchgen.utils.FileManagerto acceptpathlib.Path#150726