8000 Conflicting OMP implementations linked when building with MKLDNN · Issue #126211 · pytorch/pytorch · GitHub
[go: up one dir, main page]
Skip to content

Conflicting OMP implementations linked when building with MKLDNN #126211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
mwlon op 8000 ened this issue May 14, 2024 · 2 comments · May be fixed by #138834
Open

Conflicting OMP implementations linked when building with MKLDNN #126211

mwlon opened this issue May 14, 2024 · 2 comments · May be fixed by #138834
Assignees
@jingxu10
Labels
module: build Build system issues module: mkldnn Related to Intel IDEEP or oneDNN (a.k.a. mkldnn) integration module: openmp Related to OpenMP (omp) support in PyTorch triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@mwlon
Copy link
Contributor
mwlon commented May 14, 2024 •
edited
Loading

🐛 Describe the bug

Custom builds with MKLDNN can link to both libomp and libgomp, which could cause invalid or slow behavior if some symbols are missing from one.

This happens because the MKLDNN extension always builds with libgomp, whereas pytorch by default prefers libomp.

Assuming libomp is present on the system, this can be replicated via a custom build with

export CMAKE_PREFIX_PATH="/path/to/mkl/"
cmake -DUSE_MKLDNN=1 -DUSE_STATIC_MKL=1 -DBLAS=MKL

Producing things like this:

$ readelf -d libtorch_cpu.so 

Dynamic section at offset 0x12425918 contains 40 entries:
  Tag        Type                         Name/Value
...
 0x0000000000000001 (NEEDED)             Shared library: [libomp.so]
 0x0000000000000001 (NEEDED)             Shared library: [libgomp.so.1]
...

I suspect the fix here would be to always use libgomp in FindOpenMP.cmake when USE_MKLDNN is true, but perhaps there's a way to make MKLDNN use whatever OMP we've discovered?

Versions

I think this has been the case for a while.

cc @malfet @seemethere @gujinghui @PenghuiCheng @XiaobingSuper @jianyuh @jgong5 @mingfeima @sanchitintel @ashokei @jingxu10 @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen @snadampal
@malfet malfet added module: mkldnn Related to Intel IDEEP or oneDNN (a.k.a. mkldnn) integration module: openmp Related to OpenMP (omp) support in PyTorch module: build Build system issues triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels May 14, 2024
@mwlon mwlon linked a pull request Oct 24, 2024 that will close this issue
Open
@mwlon
8000 Copy link
Contributor Author
mwlon commented Jan 7, 2025

@malfet @seemethere @gujinghui @PenghuiCheng @XiaobingSuper @jianyuh @jgong5 @mingfeima @sanchitintel @ashokei @jingxu10 @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen @snadampal

I've pinged a few times in the PR, which is approved but unmerged. Does someone want to merge #138834 ?

@mwlon
Copy link
Contributor Author
mwlon commented Feb 7, 2025

@malfet @seemethere @gujinghui @PenghuiCheng @XiaobingSuper @jianyuh @jgong5 @mingfeima @sanchitintel @ashokei @jingxu10 @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen @snadampal

One more ping. I realize there might be a deferred responsibility problem here, but could someone please look at the PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jingxu10 jingxu10

Labels
module: build Build system issues module: mkldnn Related to Intel IDEEP or oneDNN (a.k.a. mkldnn) integration module: openmp Related to OpenMP (omp) support in PyTorch triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
PyTorch Intel
Status: No status
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Prioritize building with libgomp over libomp mwlon/pytorch
3 participants
@malfet @mwlon @jingxu10
0