Document dynamo #146736
Closed
Document dynamo #146736
+1,337
−86
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@Raymo111 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
38cce9e to
2232ecd
Compare
|
@Raymo111 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
2232ecd to
bef2193
Compare
4dc32b7 to
8e725f4
Compare
|
Ignore the fbsource diff imports, that was for testing purposes and D69337511 is now abandoned. |
jansel
requested changes
Feb 9, 2025
torch/_dynamo/variables/constant.py
Outdated
Comment on lines
7
to
8
| values during compilation, ensuring proper handling of Python literals and | ||
| maintaining type safety through the compilation process. |
There was a problem hiding this comment.
Whoops that was a copy-paste mistake when I moved the docstring up, fixed.
torch/_dynamo/variables/lazy.py
Outdated
Comment on lines
3
to
4
| It optimizes memory usage and performance by deferring the creation of VariableTracker | ||
| objects until they are actually needed. |
There was a problem hiding this comment.
... creation of VariableTracker objects (and thereby guard installation) until ...
| The key classes are: | ||
| - UserDefinedVariable: Base class for representing custom Python objects | ||
| - UserDefinedClassVariable: Handles Python class objects/types | ||
| - UserDefinedObjectVariable: Main class for instance objects, with support for method calls, |
There was a problem hiding this comment.
s/Main/Fallback
The idea is that Dynamo would specialize to other VariableTracker subclasses like FrozenDataClassVariable first, and if no subclass qualifies, we fallback to UserDefinedObjectVariable:
pytorch/torch/_dynamo/variables/builder.py
Lines 1293 to 1294 in b826135
There was a problem hiding this comment.
Updated and note added!
torch/_dynamo/side_effects.py
Outdated
| - Tracks mutations to Python objects, lists, and dictionaries | ||
| - Manages attribute modifications and deletions | ||
| - Handles tensor hooks and backward pass state | ||
| - Supports cell variable mutations and global variable changes |
torch/_dynamo/side_effects.py
Outdated
| and other side effects that occur during program execution. | ||
|
|
||
| Key responsibilities: | ||
| - Tracks mutations to Python objects, lists, and dictionaries |
There was a problem hiding this comment.
I'd change this one, because mutations to list/dict/etc. are actually tracked along the VariableTracker instances (e.g., ConstDictVariable).
There was a problem hiding this comment.
Tweaked the comment and moved it down to be the class docstring since there's only one class in the file.
8e725f4 to
3c5caf7
Compare
jansel
approved these changes
Feb 10, 2025
anijain2305
reviewed
Feb 12, 2025
torch/_dynamo/__init__.py
Outdated
Comment on lines
6
to
11
| This module provides the main interface for TorchDynamo's functionality, including: | ||
| - Various compilation backends and their registration | ||
| - Optimization and export capabilities | ||
| - Debugging and profiling utilities | ||
| - Graph manipulation and control flow tools | ||
| - Decorators for fine-grained control over compilation behavior |
There was a problem hiding this comment.
Can we remove these lines. Its too generic, and seems off.
anijain2305
reviewed
Feb 12, 2025
Comment on lines
+7
to
+9
| This module lazily imports and registers the TorchInductor compiler to avoid loading it | ||
| into memory when it is not being used. This helps reduce memory overhead when using | ||
| other backends. |
anijain2305
reviewed
Feb 12, 2025
torch/_dynamo/config.py
Outdated
Comment on lines
13
to
15
| Many settings can be overridden via environment variables. The environment variable | ||
| name typically matches the setting name in uppercase with a "TORCH_DYNAMO_" or | ||
| "TORCH_COMPILE_" prefix. |
anijain2305
reviewed
Feb 12, 2025
torch/_dynamo/decorators.py
Outdated
Comment on lines
7
to
25
| Key functionality includes: | ||
| - Function marking decorators (disable, skip, allow_in_graph, disallow_in_graph) | ||
| - Tensor dimension handling (mark_dynamic, mark_static, mark_unbacked) | ||
| - Graph compilation control (run, graph_break) | ||
| - Module attribute handling (mark_static for nn.Module) | ||
|
|
||
| The decorators and utilities in this module are thread-safe and can be used to: | ||
| - Control which functions are included/excluded from graph compilation | ||
| - Specify tensor dimension properties for optimization | ||
| - Configure compilation behavior for specific functions | ||
| - Mark tensors and modules as static/dynamic for better performance | ||
|
|
||
| Important decorators: | ||
| - disable: Prevents TorchDynamo from operating on a function | ||
| - allow_in_graph: Permits direct function inclusion in compiled graphs | ||
| - mark_dynamic: Specifies dynamic tensor dimensions | ||
| - mark_static: Marks tensor dimensions or nn.Modules as static | ||
|
|
||
| Note: Most marking functions should be called before compilation, not during tracing. |
There was a problem hiding this comment.
Remove this. Mostly because this will be outdated soon. And the docs should live in the decorators itself.
anijain2305
reviewed
Feb 12, 2025
torch/_dynamo/variables/lazy.py
Outdated
Comment on lines
2
to
15
| This module implements lazy evaluation mechanisms for Dynamo's variable tracking system. | ||
| It optimizes memory usage and performance by deferring the creation of VariableTracker | ||
| objects (and thereby guard installation) until they are actually needed. | ||
|
|
||
| The lazy evaluation strategy is particularly beneficial when dealing with large | ||
| containers or complex object graphs, where eagerly creating VariableTrackers for | ||
| all elements would be expensive and potentially wasteful. Key components include: | ||
|
|
||
| - LazyCache: Stores the original value and source until realization is needed | ||
| - LazyVariableTracker: Provides a proxy interface that triggers realization on access | ||
| - Realization system: Handles the actual creation of VariableTrackers on demand | ||
|
|
||
| This approach allows Dynamo to efficiently handle large data structures and complex | ||
| object relationships while maintaining good performance characteristics. |
There was a problem hiding this comment.
This is somewhat wrong. Lets delete it for now, I can put the right doc later on.
anijain2305
reviewed
Feb 12, 2025
torch/_dynamo/variables/lists.py
Outdated
Comment on lines
18
to
21
| Special handling is provided for torch.Size to work with symbolic shapes, | ||
| and for named tuples to preserve their field names and type information. | ||
| The module also includes restricted list subclass support for user-defined | ||
| list subclasses that don't override core list methods. |
anijain2305
approved these changes
Feb 12, 2025
3c5caf7 to
19c2609
Compare
19c2609 to
9b5a22c
Compare
zou3519
approved these changes
Feb 12, 2025
|
@pytorchbot merge -f "Errors are unrelated to my docstring changes" |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
facebook-github-bot
pushed a commit
to pytorch/benchmark
that referenced
this pull request
Feb 13, 2025
Summary: Many files in dynamo are currently lacking file/module-level documentation, which makes it hard to know what they do at a glance and without digging into the code. This fixes that. Note: documentation was AI-generated and could be incorrect, please review carefully. X-link: pytorch/pytorch#146736 Approved by: https://github.com/jansel, https://github.com/StrongerXi, https://github.com/anijain2305, https://github.com/zou3519 Reviewed By: huydhn Differential Revision: D69565828 fbshipit-source-id: 093b4575224b914a263f3e8e8994a63015a5b66d
There was a problem hiding this comment.
I didn't have a chance to review this before it got merged - I put in my comments for modules that I'm more familiar with.
Can we also add a disclaimer for each of the generated docstrings stating that they are AI-generated and partially audited? This will help to highlight that the generated docstrings may not necessarily be correct or precise.
You can make these changes in a followup PR.
|
|
||
| Key Debugging Backends: | ||
| - eager: Simple pass-through backend that runs models in eager mode | ||
| - eager_noexcept: Similar to eager but with additional exception handling |
There was a problem hiding this comment.
Suggested change
| - eager_noexcept: Similar to eager but with additional exception handling | |
| - eager_noexcept: Similar to `eager` backend but ensures that the given graph doesn't raise any errors. |
| - eager_debug: Adds schema validation checks for custom operators | ||
| - aot_eager: Uses AOT Autograd with nop compiler for debugging | ||
| - aot_eager_decomp_partition: Uses TorchInductor decompositions for debugging | ||
| - torchscript: Compiles using TorchScript for debugging JIT-related issues |
There was a problem hiding this comment.
This is not a relevant backend.
|
|
||
| Testing and Development Tools: | ||
| - Backends for inducing specific errors (compile/runtime/accuracy) | ||
| - ExplainOutput class for detailed graph compilation analysis |
There was a problem hiding this comment.
This is not really used anymore
| 1241 |
|
|
| These backends are primarily used for: | ||
| 1. Debugging graph breaks and compilation failures | ||
| 2. Testing error handling and recovery mechanisms |
| These backends are primarily used for: | ||
| 1. Debugging graph breaks and compilation failures | ||
| 2. Testing error handling and recovery mechanisms | ||
| 3. Analyzing performance bottlenecks |
| @@ -1,5 +1,27 @@ | |||
| # This module contains functions that *will be allowed* by dynamo | |||
|
|
|||
| """ | |||
| This module contains utility functions that are explicitly allowed to be called during | |||
There was a problem hiding this comment.
Suggested change
| This module contains utility functions that are explicitly allowed to be called during | |
| This module contains utility functions that are explicitly allowed to be traced/inlined during |
Comment on lines
+4
to
+15
| This module provides functionality for resuming Python execution at specific points in code, | ||
| primarily used by PyTorch Dynamo for control flow handling and optimization. It implements | ||
| bytecode transformation and execution state management to enable: | ||
|
|
||
| - Resuming execution at arbitrary points in Python bytecode | ||
| - Managing context managers and their state across execution boundaries | ||
| - Transforming and generating new code objects with preserved execution state | ||
| - Supporting Python 3.11+ exception handling and block management | ||
| - Restoring torch function mode stacks and other execution context | ||
|
|
||
| The module is critical for PyTorch Dynamo's ability to optimize code while preserving | ||
| Python semantics and execution state. |
There was a problem hiding this comment.
Suggested change
| This module provides functionality for resuming Python execution at specific points in code, | |
| primarily used by PyTorch Dynamo for control flow handling and optimization. It implements | |
| bytecode transformation and execution state management to enable: | |
| - Resuming execution at arbitrary points in Python bytecode | |
| - Managing context managers and their state across execution boundaries | |
| - Transforming and generating new code objects with preserved execution state | |
| - Supporting Python 3.11+ exception handling and block management | |
| - Restoring torch function mode stacks and other execution context | |
| The module is critical for PyTorch Dynamo's ability to optimize code while preserving | |
| Python semantics and execution state. | |
| This module is responsible for constructing the bytecode for graph break continuation functions. | |
| Graph break continuation functions allow Dynamo to handle unsupported bytecode instructions by | |
| breaking the operation graph, compiling the traced partial graph, and resuming execution after | |
| the unsupported instruction. | |
| The continuation function: | |
| - Restores contexts at the time of the graph break (e.g. supported context managers and torch function mode stacks) | |
| - Jumps to the instruction after the unsupported bytecode |
Comment on lines
+4
to
+24
| Core module responsible for converting Python bytecode into TorchDynamo's symbolic execution format. | ||
|
|
||
| This module implements the bytecode-level tracing system that allows TorchDynamo to analyze | ||
| and transform Python code. It converts Python bytecode instructions into a symbolic format | ||
| that tracks the flow of tensors and other values through the program. | ||
|
|
||
| Key components: | ||
| - InstructionTranslatorBase: Base class for converting bytecode to symbolic execution | ||
| - InstructionTranslator: Main translator for function bytecode | ||
| - InliningInstructionTranslator: Handles inlining of called functions | ||
| - SpeculationLog: Manages state for speculative execution and rollback | ||
|
|
||
| The symbolic conversion process handles: | ||
| - Control flow (loops, conditionals, etc.) | ||
| - Function inlining and call stack management | ||
| - Tracking of program values and side effects | ||
| - Graph breaks and resumption points | ||
| - Exception handling and stack frame management | ||
|
|
||
| This is a core part of TorchDynamo's tracing system that enables ahead-of-time | ||
| optimization of PyTorch programs. |
There was a problem hiding this comment.
Core module responsible for symbolically tracing Python bytecode.
This module implements the bytecode-level tracing system that allows TorchDynamo to analyze
and transform Python code.
Key components:
- InstructionTranslatorBase: Base class for converting bytecode to symbolic execution
- InstructionTranslator: Translator for the top-level function bytecode
- InliningInstructionTranslator: Translator for any nested function bytecode
- SpeculationLog: Logs bytecode locations that Dynamo can graph break on. Used to restore state
if a bytecode instruction is determined to graph break. Dynamo will restart symbolic analysis and
trace until the final speculation log entry.
The symbolic conversion process handles things like:
- Control flow (loops, conditionals, etc.)
- Nested function calls
- Closures
- Side effects
- Basic exception handling
If tracing fails for some reason (including unsupported bytecode operations), we attempt to graph break.
This is a core part of TorchDynamo's tracing system that enables ahead-of-time
optimization of PyTorch programs.
|
|
||
| """Common utilities for testing Dynamo's minifier functionality. | ||
|
|
||
| This module provides the base infrastructure for running minification tests in Dynamo. |
There was a problem hiding this comment.
Suggested change
| This module provides the base infrastructure for running minification tests in Dynamo. | |
| This module provides the base infrastructure for running minifier tests in Dynamo. |
| - Running tests in isolated environments | ||
| - Managing temporary directories and configurations | ||
| - Executing minifier launcher scripts | ||
| - Running and validating reproduction scripts |
There was a problem hiding this comment.
Suggested change
| - Running and validating reproduction scripts | |
| - Running and validating minifier reproduction scripts |
Tytskiy
pushed a commit
to Tytskiy/pytorch
that referenced
this pull request
Feb 18, 2025
Many files in dynamo are currently lacking file/module-level documentation, which makes it hard to know what they do at a glance and without digging into the code. This fixes that. Note: documentation was AI-generated and could be incorrect, please review carefully. Pull Request resolved: pytorch#146736 Approved by: https://github.com/jansel, https://github.com/StrongerXi, https://github.com/anijain2305, https://github.com/zou3519
Raymo111
added a commit
that referenced
this pull request
Feb 20, 2025
Many files in dynamo are currently lacking file/module-level documentation, which makes it hard to know what they do at a glance and without digging into the code. This fixes that. Note: documentation was AI-generated and could be incorrect, please review carefully. Pull Request resolved: #146736 Approved by: https://github.com/jansel, https://github.com/StrongerXi, https://github.com/anijain2305, https://github.com/zou3519
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Many files in dynamo are currently lacking file/module-level documentation, which makes it hard to know what they do at a glance and without digging into the code. This fixes that.
Note: documentation was AI-generated and could be incorrect, please review carefully.
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @StrongerXi @xmfan @svekars @brycebortree @sekyondaMeta @AlannaBurke