Description
People are probably already aware of this issue, but adding my $0.02 as to what I think may be going on.
In a nutshell: NumPy fails to build on OS 11 and python 3.9.1. The error message is attached below, but NumPy is installed and works. (you can import numpy as np from a python interpreter) However, if you try to import numpy in a notebook, you get an error message saying that numpy is using a buggy "accelerate" backend.
The error about the backend occurs when numpy is being built from source incorrectly instead of using the provided wheel. That known issue was supposedly fixed in a pip upgrade (20.3). However, I get the same error below, even when I have updated pip to 20.3.
However, when i create a new pipenv environment, it defaults to pip 20.2. So my guess is that even if I subsequently update pip to 20.3, that pipenv install numpy is using the old version of pip.
Can anybody confirm and/or offer a workaround?
(blog) shane@Olympus ~/github/blog dev ● pipenv install numpy
Installing numpy...
Adding numpy to Pipfile's [packages]...
✔ Installation Succeeded
Pipfile.lock (16c839) out of date, updating to (49bf85)...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✘ Locking Failed!
ERROR:pip.subprocessor:Command errored out with exit status 1:
command: /Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/bin/python /usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/tmpk03mt0_f
cwd: /private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy
Complete output (209 lines):
Running from numpy source directory.
setup.py:480: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
run_build = parse_setuppy_commands()
Cythonizing sources
Processing numpy/random/_bounded_integers.pxd.in
Processing numpy/random/_philox.pyx
Processing numpy/random/_bounded_integers.pyx.in
Processing numpy/random/_sfc64.pyx
Processing numpy/random/_mt19937.pyx
Processing numpy/random/bit_generator.pyx
Processing numpy/random/mtrand.pyx
Processing numpy/random/_generator.pyx
Processing numpy/random/_pcg64.pyx
Processing numpy/random/_common.pyx
blas_opt_info:
blas_mkl_info:
customize UnixCCompiler
libraries mkl_rt not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
blis_info:
libraries blis not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
openblas_info:
libraries openblas not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_3_10_blas_threads_info:
Setting PTATLAS=ATLAS
libraries tatlas not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_3_10_blas_info:
libraries satlas not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_blas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_blas_info:
libraries f77blas,cblas,atlas not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
accelerate_info:
libraries accelerate not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
Library accelerate was not found. Ignoring
libraries veclib not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
Library veclib was not found. Ignoring
FOUND:
extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
FOUND:
extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
non-existing path in 'numpy/distutils': 'site.cfg'
lapack_opt_info:
lapack_mkl_info:
libraries mkl_rt not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
openblas_lapack_info:
libraries openblas not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
openblas_clapack_info:
libraries openblas,lapack not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
flame_info:
libraries flame not found in ['/Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
atlas_3_10_threads_info:
Setting PTATLAS=ATLAS
libraries lapack_atlas not found in /Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib
libraries tatlas,tatlas not found in /Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib
libraries lapack_atlas not found in /usr/local/lib
libraries tatlas,tatlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib
libraries tatlas,tatlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
NOT AVAILABLE
atlas_3_10_info:
libraries lapack_atlas not found in /Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib
libraries satlas,satlas not found in /Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib
libraries lapack_atlas not found in /usr/local/lib
libraries satlas,satlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib
libraries satlas,satlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_3_10_info'>
NOT AVAILABLE
atlas_threads_info:
Setting PTATLAS=ATLAS
libraries lapack_atlas not found in /Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib
libraries ptf77blas,ptcblas,atlas not found in /Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib
libraries lapack_atlas not found in /usr/local/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_threads_info'>
NOT AVAILABLE
atlas_info:
libraries lapack_atlas not found in /Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib
libraries f77blas,cblas,atlas not found in /Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/lib
libraries lapack_atlas not found in /usr/local/lib
libraries f77blas,cblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/lib
libraries f77blas,cblas,atlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_info'>
NOT AVAILABLE
FOUND:
extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
/usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py:274: UserWarning: Unknown distribution option: 'define_macros'
warnings.warn(msg)
running dist_info
running build_src
build_src
building py_modules sources
creating build
creating build/src.macosx-11-x86_64-3.9
creating build/src.macosx-11-x86_64-3.9/numpy
creating build/src.macosx-11-x86_64-3.9/numpy/distutils
building library "npymath" sources
Traceback (most recent call last):
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_vendor/pep517/_in_process.py", line 257, in <module>
main()
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_vendor/pep517/_in_process.py", line 240, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_vendor/pep517/_in_process.py", line 110, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-build-env-5hhkc_mx/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 157, in prepare_metadata_for_build_wheel
self.run_setup()
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-build-env-5hhkc_mx/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 248, in run_setup
super(_BuildMetaLegacyBackend,
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-build-env-5hhkc_mx/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 142, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 508, in <module>
setup_package()
File "setup.py", line 500, in setup_package
setup(**metadata)
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/core.py", line 169, in setup
return old_setup(**new_attr)
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-build-env-5hhkc_mx/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 165, in setup
return distutils.core.setup(**attrs)
File "/usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-build-env-5hhkc_mx/overlay/lib/python3.9/site-packages/setuptools/command/dist_info.py", line 31, in run
egg_info.run()
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/command/egg_info.py", line 24, in run
self.run_command("build_src")
File "/usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/command/build_src.py", line 144, in run
self.build_sources()
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/command/build_src.py", line 155, in build_sources
self.build_library_sources(*libname_info)
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/command/build_src.py", line 288, in build_library_sources
sources = self.generate_sources(sources, (lib_name, build_info))
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/command/build_src.py", line 378, in generate_sources
source = func(extension, build_dir)
File "numpy/core/setup.py", line 658, in get_mathlib_info
st = config_cmd.try_link('int main(void) { return 0;}')
File "/usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/config.py", line 241, in try_link
self._check_compiler()
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/command/config.py", line 80, in _check_compiler
self.fcompiler = new_fcompiler(compiler=self.fcompiler,
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/fcompiler/__init__.py", line 880, in new_fcompiler
compiler = get_default_fcompiler(plat, requiref90=requiref90,
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/fcompiler/__init__.py", line 851, in get_default_fcompiler
compiler_type = _find_existing_fcompiler(matching_compiler_types,
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/fcompiler/__init__.py", line 802, in _find_existing_fcompiler
c.customize(dist)
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/fcompiler/__init__.py", line 526, in customize
linker_so_flags = self.flag_vars.linker_so
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/fcompiler/environment.py", line 37, in __getattr__
return self._get_var(name, conf_desc)
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/fcompiler/environment.py", line 53, in _get_var
var = self._hook_handler(name, hook)
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/fcompiler/__init__.py", line 705, in _environment_hook
return hook()
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/fcompiler/gnu.py", line 346, in get_flags_linker_so
flags = GnuFCompiler.get_flags_linker_so(self)
File "/private/var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/pip-resolver-id_2dgms/numpy/numpy/distutils/fcompiler/gnu.py", line 136, in get_flags_linker_so
os.environ['MACOSX_DEPLOYMENT_TARGET'] = target
File "/usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 684, in __setitem__
value = self.encodevalue(value)
File "/usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 756, in encode
raise TypeError("str expected, not %s" % type(value).__name__)
TypeError: str expected, not int
----------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/resolver.py", line 764, in <module>
main()
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/resolver.py", line 758, in main
_main(parsed.pre, parsed.clear, parsed.verbose, parsed.system, parsed.write,
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/resolver.py", line 741, in _main
resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages, dev)
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/resolver.py", line 702, in resolve_packages
results, resolver = resolve(
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/resolver.py", line 684, in resolve
return resolve_deps(
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/utils.py", line 1395, in resolve_deps
results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/utils.py", line 1108, in actually_resolve_deps
resolver.resolve()
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/utils.py", line 823, in resolve
results = self.resolver.resolve(max_rounds=environments.PIPENV_MAX_ROUNDS)
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/piptools/resolver.py", line 180, in resolve
has_changed, best_matches = self._resolve_one_round()
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/piptools/resolver.py", line 268, in _resolve_one_round
their_constraints.extend(self._iter_dependencies(best_match))
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/piptools/resolver.py", line 383, in _iter_dependencies
dependencies = self.repository.get_dependencies(ireq)
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 229, in get_dependencies
legacy_results = self.get_legacy_dependencies(ireq)
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 350, in get_legacy_dependencies
results, ireq = self.resolve_reqs(
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 306, in resolve_reqs
results = resolver._resolve_one(reqset, ireq)
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/legacy_resolve.py", line 339, in _resolve_one
abstract_dist =
670F
self._get_abstract_dist_for(req_to_install)
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/legacy_resolve.py", line 287, in _get_abstract_dist_for
abstract_dist = self.preparer.prepare_linked_requirement(req)
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/operations/prepare.py", line 508, in prepare_linked_requirement
abstract_dist = _get_prepared_distribution(
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/operations/prepare.py", line 95, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(finder, build_isolation)
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/distributions/sdist.py", line 40, in prepare_distribution_metadata
self.req.prepare_metadata()
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/req/req_install.py", line 564, in prepare_metadata
self.metadata_directory = self._generate_metadata()
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/req/req_install.py", line 549, in _generate_metadata
return generate_metadata(
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/operations/build/metadata.py", line 36, in generate_metadata
distinfo_dir = backend.prepare_metadata_for_build_wheel(
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_vendor/pep517/wrappers.py", line 167, in prepare_metadata_for_build_wheel
return self._call_hook('prepare_metadata_for_build_wheel', {
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_vendor/pep517/wrappers.py", line 245, in _call_hook
self._subprocess_runner(
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/utils/subprocess.py", line 271, in runner
call_subprocess(
File "/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_internal/utils/subprocess.py", line 242, in call_subprocess
raise InstallationError(exc_msg)
pipenv.patched.notpip._internal.exceptions.InstallationError: Command errored out with exit status 1: /Users/shane/.local/share/virtualenvs/blog-kKpxKzEP/bin/python /usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/patched/notpip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/yz/l6qh44196z70lty4rs6dm3nc0000gn/T/tmpk03mt0_f Check the logs for full command output.