8000 [c10d] Simplify ProcessGroupNCCL into single-device style by kwen2501 · Pull Request #118674 · pytorch/pytorch · GitHub
[go: up one dir, main page]
Skip to content

[c10d] Simplify ProcessGroupNCCL into single-device style #118674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 16 commits into from
Closed

[c10d] Simplify ProcessGroupNCCL into single-device style #118674

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
e8bbc7b
Simplify ProcessGroupNCCL into single-device style
kwen2501 Jan 30, 2024
8338feb
Refactor overriden functions in tests
kwen2501 Jan 30, 2024
a923440
Fix segv
kwen2501 Feb 1, 2024
18c1f5e
Fix test_gather_checks
kwen2501 Feb 1, 2024
87b14fb
Correctly print device
kwen2501 Feb 2, 2024
204253c
Refactor ProcessGroupNCCLTest from multi-device to multi-thread style
kwen2501 Feb 2, 2024
f13b03b
Refactor ProcessGroupNCCLErrorsTest
kwen2501 Feb 2, 2024
efe2ace
Fix NcclErrorHandlingTest
kwen2501 Feb 2, 2024
a4fb6fd
Add logging in barrier wait
kwen2501 Feb 2, 2024
dff9b73
Fix AG and RS recording
kwen2501 Feb 2, 2024
87816df
Make flight recorder happy
kwen2501 Feb 2, 2024
bb435ee
Bump seq number once only for batch p2p
kwen2501 Feb 2, 2024
bd4a7f0
Lint
kwen2501 Feb 2, 2024
32704d4
Improve error message
kwen2501 Feb 2, 2024
c144c1b
Fix unused-variable warning
kwen2501 Feb 2, 2024
db92422
Re-enable flatten path of AG and RS
kwen2501 Feb 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Correctly print device
  • Loading branch information
@kwen2501
kwen2501 committed Feb 2, 2024
commit 87b14fb28df54bcd193f53a1fa1b22fa7d283352
2 changes: 1 addition & 1 deletion torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1917,7 +1917,7 @@ std::shared_ptr<NCCLComm> ProcessGroupNCCL::getNCCLComm(
#endif

LOG(INFO) << logPrefix() << "ProcessGroupNCCL created ncclComm_ "
<< ncclComm->ncclComm_ << " on CUDA device: " << device.index();
<< ncclComm->ncclComm_ << " on CUDA device: " << deviceIndex;

// At this point NCCL should have been initialized, hence we can accurately
// get the env value even if NCCL sets it by reading from nccl.conf file
Expand Down
0