Torch Distributed Elastic

Created On: May 04, 2021 | Last Updated On: Jun 04, 2024

Makes distributed PyTorch fault-tolerant and elastic.

Get Started

Usage

• Quickstart
• Train script
• Examples

Documentation

API

• torchrun (Elastic Launch)
• Elastic Agent
• Multiprocessing
• Error Propagation
• Rendezvous
• Expiration Timers
• Metrics
• Events
• Subprocess Handling
• Control Plane

Advanced

• Customization

Plugins

• TorchElastic Kubernetes