8000 [feature request] allow torch.compile calls with compiler options, but without modifying global dynamo options · Issue #124505 · pytorch/pytorch · GitHub
[go: up one dir, main page]
Skip to content

[feature request] allow torch.compile calls with compiler options, but without modifying globa 8000 l dynamo options #124505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
vadimkantorov opened this issue Apr 19, 2024 · 5 comments
Open

[feature request] allow torch.compile calls with compiler options, but without modifying global dynamo options #124505

vadimkantorov opened this issue Apr 19, 2024 · 5 comments
Labels
feature A request for a proper, new feature. module: dynamo oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@vadimkantorov
Copy link
Contributor
vadimkantorov commented Apr 19, 2024 •
edited by pytorch-bot bot
Loading

🚀 The feature, motivation and pitch

Originally discussed in: #106614

As I understood, any torch.compile call with passed options will modify global compiler options (and thus require torch._dynamo.reset() or sth like this). I suggest to introduce a "local mode" which applies passed options but only to a given function. Or maybe also introduce a "compile options context" - this would be a "semi-local" mode

Alternatives

No response

Additional context

No response

cc @ezyang @msaroufim @bdhirsh @anijain2305 @chauhang @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng
@masnesral masnesral added module: dynamo feature A request for a proper, new feature. triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Apr 22, 2024
@ezyang
Copy link
Contributor
ezyang commented Apr 23, 2024

#111299 is this but it needs to be rewritten to be more obviously correct

@vadimkantorov
Copy link
Contributor Author

One thing that confused (and I think is confusing truly) is that the global option change was applied at the call site of the compiled function :) This is especially confusing if torch compile is used as a decorator, so the compiled functions and new options are specified in a file far, far away

@vadimkantorov vadimkantorov mentioned this issue Aug 26, 2024
Open
@vadimkantorov
Copy link
Contributor Author
vadimkantorov commented Aug 26, 2024 •
edited
Loading

This would be very useful for having torch.compile calls in the PyTorch nn library (e.g. to have RMSNorm defined as torch.nn.RMSNorm = torch.compile(torch.nn.RMSNorm) in the core nn library for having fast RMSNorm code, even if the user is using eager/not compiling for the full model) - to guard them against accidental option-modification by the user in the client code

Maybe even a different decorator could be introduced like torch.compile_scoped

@vadimkantorov vadimkantorov mentioned this issue Sep 11, 2024
Open
@vadimkantorov vadimkantorov mentioned this issue Dec 9, 2024
Open
@cora-codes
Copy link

What ended up happening with #111299? This would be a super useful feature

@ezyang
Copy link
Contributor
ezyang commented Dec 18, 2024

@cora-codes it caused a SEV due to either a bug in its implementation or triggering a latent bug, so we need to reimplement it from scratch I think.

@vadimkantorov vadimkantorov mentioned this issue Jan 25, 2025
Open
@vadimkantorov vadimkantorov mentioned this issue Apr 22, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature A request for a proper, new feature. module: dynamo oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@ezyang @vadimkantorov @bdhirsh @masnesral @cora-codes
0