8000 Update on "[DTensor] Rewrite doc of TupleStrategy" · pytorch/pytorch@a7839d6 · GitHub
[go: up one dir, main page]

Skip to content
8000

Commit a7839d6

Browse files
committed
Update on "[DTensor] Rewrite doc of TupleStrategy"
cc H-Huang awgu wanchaol fegin fduwjj wz337 d4l3k [ghstack-poisoned]
1 parent f1ed390 commit a7839d6

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

torch/distributed/tensor/_op_schema.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -176,14 +176,14 @@ class TupleStrategy(StrategyType):
176176
TupleStrategy is a special case for operators that are fundamentally compound or batched such that some subset
177177
of the inputs and outputs are completely unrelated to some other subset.
178178
179-
Generally, foreach_* ops are the most common use-case for TupleStrategy, becuase they accept lists of inputs,
179+
Generally, foreach_* ops are the most common use-case for TupleStrategy, because they accept lists of inputs,
180180
but operate independently on each input or tuple of zipped inputs.
181181
182182
For example, [out_a, out_b] = torch.foreach_add([a, b], scalar): input a's sharding only affects out_a's sharding,
183183
independent of b and out_b.
184184
185-
An example of an operator that should NOT use TupleStrategy is torch.split. It produces a tuple of
186-
tensors as its output, but the sharding decision of one output is bound together with the decision
185+
An example of an operator that should NOT use TupleStrategy is torch.split. It produces a List[Tensor]
186+
as its output, but the sharding decision of one output is bound together with the decision
187187
of each other output and the common input.
188188
"""
189189

0 commit comments

Comments
 (0)
0