10000 rebase-gh-5237 - BUG: Workaround segfault in Apple Accelerate framework SGEMV by charris · Pull Request #5831 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

rebase-gh-5237 - BUG: Workaround segfault in Apple Accelerate framework SGEMV #5831

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 3, 2015
Merged

rebase-gh-5237 - BUG: Workaround segfault in Apple Accelerate framework SGEMV #5831

merged 1 commit into from
May 3, 2015

Conversation

charris
Copy link
Member
@charris charris commented May 3, 2015

Rebased #5237.

SGEMV in Accelerate framework will segfault on MacOS X version 10.9
(aka Mavericks) if arrays are not aligned to 32 byte boundaries and the
CPU supports AVX instructions. This can produce segfaults in np.dot.
This patch overshadows the symbols cblas_sgemv, sgemv_ and sgemv
exported by Accelerate to produce the correct behavior. The MacOS X
version and CPU specs are checked on module import. If Mavericks and AVX
are detected the call to SGEMV is emulated with a call to SGEMM if the
arrays are not 32 byte aligned. If the exported symbols cannot be
overshadowed on module import, a fatal error is produced and the process
aborts. All the fixes are in a self-contained C file and do not alter
the multiarray C code. The patch is not applied unless NumPy is
configured to link with Apple's Accelerate framework.

@sturlamolden @charris
BUG: Workaround segfault in Apple Accelerate framework SGEMV
cf30776 
SGEMV in Accelerate framework will segfault on MacOS X version 10.9
(aka Mavericks) if arrays are not aligned to 32 byte boundaries and the
CPU supports AVX instructions. This can produce segfaults in np.dot.
This patch overshadows the symbols cblas_sgemv, sgemv_ and sgemv
exported by Accelerate to produce the correct behavior. The MacOS X
version and CPU specs are checked on module import. If Mavericks and AVX
are detected the call to SGEMV is emulated with a call to SGEMM if the
arrays are not 32 byte aligned. If the exported symbols cannot be
overshadowed on module import, a fatal error is produced and the process
aborts. All the fixes are in a self-contained C file and do not alter
the multiarray C code. The patch is not applied unless NumPy is
configured to link with Apple's Accelerate framework.
@charris charris added this to the 1.10.0 release milestone May 3, 2015
@charris charris mentioned this pull request May 3, 2015
Closed
@charris
Copy link
Member Author
charris commented May 3, 2015

@matthew-brett Could you test this to see if it works on OS X 10.9?

@matthew-brett
Copy link
Contributor

Compiles, links to Accelerate, tests pass. Test from #4007

import numpy
mean = numpy.random.rand(200).astype(numpy.float32)
syn0norm = numpy.random.rand(10000, 200).astype(numpy.float32)
numpy.dot(syn0norm, mean)

runs OK.

@sturlamolden
Copy link
Contributor 
$ git remote add charris-numpy https://github.com/charris/numpy.git
$ git fetch charris-numpy
$ git checkout -b rebase-gh-5237 charris-numpy/rebase-gh-5237
$ conda create -n rebase-gh-5237 python=3.4
$ source activate rebase-gh-5237
$ conda install cython
$ conda install nose
$ export CC=clang
$ python setup.py build_ext
$ python setup.py install
$ python
>>> import numpy
>>> numpy.__version__
'1.10.0.dev0+cf30776'
>>> numpy.test()

Ran 5861 tests in 27.745s

OK (KNOWNFAIL=4, SKIP=11)
<nose.result.TextTestResult run=5861 errors=0 failures=0>

Looks ok to me (on OSX 10.9 with Python 3.4). Accelerate is used by default when no BLAS/LAPACK is specified.

@charris
Copy link
Member Author
charris commented May 3, 2015

Thanks Matthew and Sturla. I'll put this in now. One more step towards 1.10.0.

charris added a commit that referenced this pull request May 3, 2015
@charris
Merge pull request #5831 from charris/rebase-gh-5237
63e8bab 
rebase-gh-5237 - BUG: Workaround segfault in Apple Accelerate framework SGEMV
@charris charris merged commit 63e8bab into numpy:master May 3, 2015
@charris charris deleted the rebase-gh-5237 branch May 3, 2015 20:27
@charris
Copy link
Member Author
charris commented May 3, 2015

@rgommers Note that this isn't part of the bento build. Not sure how to add that.

@sturlamolden
Copy link
Contributor

SciPy has the same issue with the Bento build as well. Currently the fix is not used when building with Bento. Bento has a static list of files to build, but here we have to conditionally add a file to the build depending on the OS and the BLAS library. I am sorry I don't know how to do this with Bento either.

@rgommers
Copy link
Member
rgommers commented May 3, 2015

No worries, I'll have a look. Note that the bscript files contain executable code.

@sturlamolden
Copy link
Contributor

Hm, the fix should perhaps/probably also be added to the numpy.linalg module, which means a small change to this file:

https://github.com/numpy/numpy/blob/master/numpy/linalg/setup.py

@rgommers
Copy link
Member
rgommers commented May 3, 2015

scipy.linalg already contains it also for Bento: https://github.com/scipy/scipy/blob/master/scipy/linalg/bscript#L19. But it's missing here and in scipy.sparse. Will look at fixing that up tomorrow.

rgommers pushed a commit to rgommers/numpy that referenced this pull request May 4, 2015
BLD: use fix for Accelerate SGEMV issue in Bento build. See numpygh-5831
7a02544 
.
@rgommers rgommers mentioned this pull request May 4, 2015
Merged
@rgommers
Copy link
Member
rgommers commented May 4, 2015

Bento build updated in gh-5836.

Didn't touch numpy/linalg/setup.py. It wasn't immediately clear what to do there - needs a test anyway.

8000

charris added a commit that referenced this pull request May 4, 2015
@charris
Merge pull request #5836 from rgommers/bento-sgemv-fix
93bcedf 
BLD: use fix for Accelerate SGEMV issue in Bento build.  See gh-5831.
@charris
Copy link
Member Author
charris commented May 4, 2015

Hmm, I didn't even think of the linalg module. It doesn't use cblas, does that make a difference?

@sturlamolden
Copy link
Contributor

I'm not sure. Obviously numpy.linalg can call Fortran sgemv internally in LAPACK and segfault. But if we overshadow sgemv, which sgemv will then be called from within the framework? It might not be a fixable issue, so we should perhaps just leave it and hope that OSX 10.9 is abandoned before it becomes a problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
00 - Bug component: build
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@charris @matthew-brett @sturlamolden @rgommers
0