8000 NumPy "dot" hangs when used with multiprocessing (potentially Apple Accelerate related?) · Issue #5752 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content
NumPy "dot" hangs when used with multiprocessing (potentially Apple Accelerate related?) #5752
Closed
@josePhoenix

Description

@josePhoenix

I'm having a devil of a time making a minimal test case, but this seems to be my issue: http://stackoverflow.com/questions/23963997/python-child-process-crashes-on-numpy-dot-if-pyside-is-imported

In my case, I have code that farms out a bunch of calculations, including matrix products to a multiprocessing.Pool with pool.map. The computation hangs partway through, and some hacky print-based debugging shows it hanging on a call to np.dot down in the guts of the program.

Replacing pool.map with the built-in (serial) map makes everything work.

I used not to have this issue, then something changed (lunar eclipse?) and now my computation hangs consistently whenever multiprocessing is used. A minimal test case continues to elude me. (It's not enough to simply generate 10 random NxN arrays and dot them in a multiprocessing-based way.)

>>> numpy.show_config()
lapack_opt_info:
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    extra_compile_args = ['-msse3', '-DAPPLE_ACCELERATE_SGEMV_PATCH']
    define_macros = [('NO_ATLAS_INFO', 3)]
openblas_lapack_info:
  NOT AVAILABLE
atlas_3_10_blas_threads_info:
  NOT AVAILABLE
atlas_threads_info:
  NOT AVAILABLE
atlas_3_10_threads_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
atlas_3_10_blas_info:
  NOT AVAILABLE
atlas_blas_threads_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
blas_opt_info:
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    extra_compile_args = ['-msse3', '-DAPPLE_ACCELERATE_SGEMV_PATCH', '-I/System/Library/Frameworks/vecLib.framework/Headers']
    define_macros = [('NO_ATLAS_INFO', 3)]
atlas_info:
  NOT AVAILABLE
atlas_3_10_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
mkl_info:
  NOT AVAILABLE

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0