8000 Record the XPU and XCCL build settings in the compiled binary by pkourdis · Pull Request #147161 · pytorch/pytorch · GitHub
[go: up one dir, main page]
Skip to content

Record the XPU and XCCL build settings in the compiled binary #147161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Record the XPU and XCCL build settings in the compiled binary #147161

wants to merge 6 commits into from

Conversation

pkourdis
Copy link
@pkourdis pkourdis commented Feb 14, 2025 •
edited
Loading

Fixes #ISSUE_NUMBER

Currently the XPU and XCCL build settings are not recorded in the compiled binary and are not shown using the torch.__config__.show() which is a quick way to check if the binary has been built with such support.

Below is the output adding them (see end of last line):

Python 3.12.8 | packaged by conda-forge | (main, Dec  5 2024, 14:24:40) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> print(torch.__config__.show())
PyTorch built with:
  - GCC 13.3
  - C++ Version: 201703
  - Intel(R) oneAPI Math Kernel Library Version 2025.1-Product Build 20250203 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v3.5.3 (Git Hash 66f0cb9eb66affd2da3bf5f8d897376f04aae6af)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - CPU capability usage: AVX512
XPU backend  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=RelWithDebInfo, COMMIT_SHA=43eb39d7c832b5560f7bfa8d29cc7919ac21c0ca, CXX_COMPILER=/home/pkourdis/compilers/gcc-13.3.0/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=OFF -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-dangling-reference -Wno-error=dangling-reference -Wno-error=redundant-move -DUSE_XPU -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, TORCH_VERSION=2.7.0, USE_CUDA=0, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=1, USE_MPI=0, USE_NCCL=OFF, USE_NNPACK=0, USE_OPENMP=ON, USE_ROCM=0, USE_ROCM_KERNEL_ASSERT=OFF, USE_XCCL=1, USE_XPU=1,

@pytorch-bot pytorch-bot bot added the caffe2 label Feb 14, 2025
@linux-foundation-easycla Linux Foundation: EasyCLA
Copy link
linux-foundation-easycla bot commented Feb 14, 2025 •
edited
Loading

CLA Signed

The committers listed above are authorized under a signed CLA.

@pytorch-bot PyTorch Bot
Copy link
pytorch-bot bot commented Feb 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/147161

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ No Failures

As of commit 9072ce0 with merge base 084c4aa (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pkourdis
Copy link
Author

@pytorchbot label "topic: not user facing"

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Feb 14, 2025
@pkourdis
Copy link
Author

@gujinghui @zhangxiaoli73 @EikanWang please help review.

@guangyey guangyey requested a review from EikanWang February 18, 2025 06:24
@guangyey guangyey added release notes: xpu release notes category ciflow/xpu Run XPU CI tasks labels Feb 18, 2025
@pytorch-bot PyTorch Bot
Copy link
pytorch-bot bot commented Feb 18, 2025

To add the ciflow label ciflow/xpu please first approve the workflows that are awaiting approval (scroll to the bottom of this page).

This helps ensure we don't trigger CI on this PR until it is actually authorized to do so. Please ping one of the reviewers if you do not have access to approve and run workflows.

@pytorch-bot pytorch-bot bot removed the ciflow/xpu Run XPU CI tasks label Feb 18, 2025
@guangyey
Copy link
Collaborator
guangyey commented Feb 18, 2025 •
edited
Loading

@pkourdis Could you help to add a UT in test_xpu.py to check if USE_XPU and USE_XCCL are in torch.__config__.show as expected.

guangyey
guangyey requested changes Feb 18, 2025
View reviewed changes
@guangyey guangyey mentioned this pull request Feb 18, 2025
Merged
@guangyey guangyey self-requested a review February 18, 2025 08:30
@guangyey guangyey added the ciflow/xpu Run XPU CI tasks label Feb 18, 2025
@pkourdis
Copy link
Author
pkourdis commented Feb 19, 2025 •
edited
Loading

@pkourdis Could you help to add a UT in test_xpu.py to check if USE_XPU and USE_XCCL are in torch.__config__.show as expected.

Added.

@pkourdis pkourdis force-pushed the panos/record-xpu-build-settings branch from 6749ff3 to 2f26639 Compare February 20, 2025 00:40
@pkourdis pkourdis requested a review from gujinghui as a code owner February 20, 2025 00:40
@pkourdis
Copy link
Author
pkourdis commented Feb 20, 2025 •
edited
Loading

Failing unit test expected to pass when intel/torch-xpu-ops#1377 is merged and PyTorch gets this update.

@pkourdis pkourdis requested a review from guangyey February 20, 2025 05:03
@pkourdis pkourdis force-pushed the panos/record-xpu-build-settings branch from 2f26639 to f7ce842 Compare February 20, 2025 06:18
@pkourdis pkourdis marked this pull request as draft February 20, 2025 06:35
@guangyey guangyey marked this pull request as ready for review March 7, 2025 06:32
@guangyey
Copy link
Collaborator
guangyey commented Mar 7, 2025

@pytorchbot rebase

@guangyey guangyey added the ciflow/trunk Trigger trunk jobs on your pull request label Mar 7, 2025
@pytorch-bot PyTorch Bot
Copy link
pytorch-bot bot commented Mar 7, 2025

To add the ciflow label ciflow/trunk please first approve the workflows that are awaiting approval (scroll to the bottom of this page).

This helps ensure we don't trigger CI on this PR until it is actually authorized to do so. Please ping one of the reviewers if you do not have access to approve and run workflows.

@pytorch-bot pytorch-bot bot removed the ciflow/trunk Trigger trunk jobs on your pull request label Mar 7, 2025
@guangyey guangyey added the ciflow/trunk Trigger trunk jobs on your pull request label Mar 7, 2025
@pytorchmergebot
Copy link
Collaborator

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

@pytorchmergebot
Copy link
Collaborator

Successfully rebased panos/record-xpu-build-settings onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout panos/record-xpu-build-settings && git pull --rebase)

@pytorchmergebot pytorchmergebot force-pushed the panos/record-xpu-build-settings branch from 372d5b0 to 4cf8daa Compare March 7, 2025 06:34
@pytorch-bot pytorch-bot bot removed ciflow/trunk Trigger trunk jobs on your pull request ciflow/xpu Run XPU CI tasks labels Mar 7, 2025
@guangyey guangyey added the ciflow/trunk Trigger trunk jobs on your pull request label Mar 7, 2025
@pkourdis
Copy link
Author
pkourdis commented Mar 8, 2025

@pytorchbot rebase

@pytorch-bot PyTo 8000 rch Bot
Copy link
pytorch-bot bot commented Mar 8, 2025

You don't have permissions to rebase this PR since you are a first time contributor. If you think this is a mistake, please contact PyTorch Dev Infra.

@pkourdis pkourdis force-pushed the panos/record-xpu-build-settings branch from 4cf8daa to ecea689 Compare March 8, 2025 05:57
@pytorch-bot pytorch-bot bot removed the ciflow/trunk Trigger trunk jobs on your pull request label Mar 8, 2025
@github-actions GitHub Actions
Copy link
Contributor
github-actions bot commented May 7, 2025

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

@github-actions github-actions bot added the Stale label May 7, 2025
@pkourdis pkourdis force-pushed the panos/record-xpu-build-settings branch from ecea689 to 71ffd3c Compare May 16, 2025 02:27
@pkourdis
Copy link
Author

@guangyey @Chao1Han prerequisite PR #147593 was merged. Please help review this one.

@EikanWang
Copy link
Collaborator

@pytorchbot rebase

@pytorchmergebot
Copy link
Collaborator

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

@pytorchmergebot
Copy link
Collaborator

Successfully rebased panos/record-xpu-build-settings onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout panos/record-xpu-build-settings && git pull --rebase)

@pytorchmergebot pytorchmergebot force-pushed the panos/record-xpu-build-settings branch from 71ffd3c to 9072ce0 Compare May 17, 2025 04:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@guangyey guangyey guangyey approved these changes

@albanD albanD albanD approved these changes

@EikanWang EikanWang EikanWang approved these changes

@malfet malfet Awaiting requested review from malfet

@atalman atalman Awaiting requested review from atalman

@gujinghui gujinghui Awaiting requested review from gujinghui gujinghui is a code owner

Assignees
No one assigned
Labels
caffe2 open source release notes: xpu release notes category Stale topic: not user facing topic category
Projects
PyTorch Intel
Status: In Progress
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@pkourdis @guangyey @gujinghui @pytorchmergebot @EikanWang @albanD @pytorchbot
0