8000 Cross-compiled macosx_arm64: do the extensions that link with npymath.a (npyrandom.a) actually work? · Issue #173 · MacPython/scipy-wheels · GitHub
[go: up one dir, main page]
Skip to content
This repository was archived by the owner on Feb 15, 2023. It is now read-only.

Cross-compiled macosx_arm64: do the extensions that link with npymath.a (npyrandom.a) actually work? #173

Closed
andyfaff opened this issue Aug 15, 2022 · 2 comments
Closed

Cross-compiled macosx_arm64: do the extensions that link with npymath.a (npyrandom.a) actually work? #173

andyfaff opened this issue Aug 15, 2022 · 2 comments

Comments

@andyfaff
Copy link

@matthew-brett, I have a question for a cross-compile scipy build for macosx_arm64. When I carry out a cross-build I get several warnings when an extension tries to link to npymath.a. This is because the installed version of numpy on the build computer is for x86_64.

  ld: warning: ignoring file /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-build-env-dlv4s5j4/overlay/lib/python3.10/site-packages/numpy/core/include/../lib/libnpymath.a, building for macOS-arm64 but attempting to link with file built for macOS-x86_64

The build warning is here.

Given that the MacPython/scipy-wheels repo is trying to do the same thing I thought I'd have a look and see if it'd solved this issue. However, I think it's got the same issue:

/opt/gfortran-darwin-arm64/bin/arm64-apple-darwin20.0.0-gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle -arch arm64 -L/opt/arm64-builds/lib -Wl,-rpath,/opt/arm64-builds/lib -L/opt/gfortran-darwin-arm64/lib/gcc/arm64-apple-darwin20.0.0/10.2.1 -Wl,-rpath,/opt/gfortran-darwin-arm64/lib/gcc/arm64-apple-darwin20.0.0/10.2.1 build/temp.macosx-11.0-arm64-3.8/scipy/special/_ufuncs.o build/temp.macosx-11.0-arm64-3.8/scipy/special/sf_error.o build/temp.macosx-11.0-arm64-3.8/scipy/special/amos_wrappers.o build/temp.macosx-11.0-arm64-3.8/scipy/special/cdf_wrappers.o build/temp.macosx-11.0-arm64-3.8/scipy/special/specfun_wrappers.o build/temp.macosx-11.0-arm64-3.8/scipy/special/_cosine.o -L/opt/arm64-builds/lib -L/Users/runner/work/1/s/venv/lib/python3.8/site-packages/numpy/core/lib -L/opt/gfortran-darwin-arm64/bin/../lib/gcc/arm64-apple-darwin20.0.0/10.2.1 -L/opt/gfortran-darwin-arm64/bin/../lib/gcc/arm64-apple-darwin20.0.0/10.2.1 -L/opt/gfortran-darwin-arm64/bin/../lib/gcc/arm64-apple-darwin20.0.0/10.2.1 -Lbuild/temp.macosx-11.0-arm64-3.8 -Wl,-rpath,/opt/arm64-builds/lib -lsc_amos -lsc_cephes -lsc_mach -lsc_cdf -lsc_specfun -lopenblas -lopenblas -lnpymath -lgfortran -o build/lib.macosx-11.0-arm64-3.8/scipy/special/_ufuncs.cpython-38-darwin.so
ld: warning: ignoring file /Users/runner/work/1/s/venv/lib/python3.8/site-packages/numpy/core/lib/libnpymath.a, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
C compiler: clang++ -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -g -arch arm64 -arch arm64 -arch arm64 -I/opt/arm64-builds/include -arch arm64

The build log for that output is here. I think the installed version of numpy for the arm64 part of the multibuild is macosx_x86_64

My question is, do these cross-compiled arm64 extensions that link to npymath (or npyrandom) actually work? We can't actually test them on the build machine because its the wrong architecture.
@rgommers
Copy link
Contributor

Detecting the correct libnpymath.a is also discussed at scipy/scipy#16783. I am actively working on this. My goal is to get rid of libnpymath.a completely, that is what all my large cleanup PRs for scipy.special last week are working towards.

I'll have a look at why the current macOS arm64 builds do work (because they do, people use wheels on PyPI just fine). My guess would be that either the right library is picked up further down in the build log, or Rosetta happens to make it work somehow.

@rgommers
Copy link
Contributor

What is happening here is that linking in libnpymath is broken, however distutils happily ignores that and continues. When importing an extension, the correct libnpymath symbols are already loaded when numpy is imported, so things happen to work. I verified this by creating a clean conda env, installing the arm64 scipy wheel from PyPI, and then cd'ing into site-packages/scipy/special and doing:

% python -c "import _ufuncs"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: dlopen(/Users/rgommers/mambaforge/envs/scipy-macos-arm64-wheel/lib/python3.10/site-packages/scipy/special/_ufuncs.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '_npy_asinh'

% python -c "import scipy.special._ufuncs"
# No error

@rgommers rgommers mentioned this issue Aug 16, 2022
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rgommers @andyfaff
0