ImportError: Numerical Python (NumPy) is not installed. #5904

napsternxg · 2015-11-23T01:34:58Z

I am trying to upgrade my sklearn from version 0.16.1 to the latest using pip however the sudo pip install sklearn --upgrade command fails with the following message:

$ sudo pip install sklearn --upgrade
Requirement already up-to-date: sklearn in /usr/local/lib/python2.7/dist-packages
Collecting scikit-learn (from sklearn)
  Using cached scikit-learn-0.17.tar.gz
Installing collected packages: scikit-learn
  Found existing installation: scikit-learn 0.16.1
    Uninstalling scikit-learn-0.16.1:
      Successfully uninstalled scikit-learn-0.16.1
  Running setup.py install for scikit-learn
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-IEvJKv/scikit-learn/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-Qo0d9N-record/install-record.txt --single-version-externally-managed --compile:
    Partial import of sklearn during the build process.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-IEvJKv/scikit-learn/setup.py", line 247, in <module>
        setup_package()
      File "/tmp/pip-build-IEvJKv/scikit-learn/setup.py", line 227, in setup_package
        .format(numpy_req_str, instructions))
    ImportError: Numerical Python (NumPy) is not installed.
    scikit-learn requires NumPy >= 1.6.1.
    Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html


    ----------------------------------------
  Rolling back uninstall of scikit-learn
Command "/usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-IEvJKv/scikit-learn/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-Qo0d9N-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-IEvJKv/scikit-learn

I already have numpy 1.10.1 installed as found in

$ python -c "import numpy; print numpy.__version__"
1.10.1

My pip version is also up to date

$ sudo pip --version
pip 7.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7)

I manually installed numpy from source using OpenBLAS 0.2.16.dev using the pip command from the source directory.

# Install OpenBLAS at /usr/local/
cd numpy-1.10.1
sudo pip install . --upgrade

The text was updated successfully, but these errors were encountered:

raghavrv · 2015-11-23T02:53:25Z

Try sudo pip install numpy

srinivasreddy · 2015-11-23T04:33:42Z

@napsternxg Please post in Pip issues. https://github.com/pypa/pip/issues

GaelVaroquaux · 2015-11-23T06:28:00Z

$ sudo pip install sklearn --upgrade
Requirement already up-to-date: sklearn in /usr/local/lib/python2.7/dist-packages

$ python -c "import numpy; print numpy.version"
1.10.1

Appartently the setup.py of scikit-learn fails to import numpy. I do not
know why. Are you sure that, in the above commands, 'pip' and 'python'
denote the same Python install?

napsternxg · 2015-11-23T06:45:49Z

@GaelVaroquaux How to determine if the pip and python are from same python install ?

Here is all I could find:

$ python --version
Python 2.7.3
$ which python
/usr/bin/python
$ pip --version
pip 7.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7)

Also, I can find numpy in the following location: ls /usr/local/lib/python2.7/dist-packages/ , but can't find it in /usr/lib/python2.7/dist-packages/

Also, numpy, scipy and sklearn can all be imported from inside of python run from command line.

>>> import numpy as np
>>> import scipy
>>> import sklearn
>>> np.__version__
'1.10.1'
>>> scipy.__version__
'0.16.1'
>>> sklearn.__version__
'0.16.1'
>>> import site
>>> site.getsitepackages()
['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']

GaelVaroquaux · 2015-11-23T07:38:02Z

@GaelVaroquaux How to determine if the pip and python are from same python
install ?

$ which python
/usr/bin/python

If you look at your earlier traceback, you'll see that pip is most likely
using also /usr/bin/python.

This is very, very strange. I tried installing scikit-learn 0.17 with pip
to see if I could reproduce, but I cannot.

raghavrv · 2015-11-23T07:53:35Z

Since you are running pip with elevated privileges (sudo pip) for installing scikit-learn, but checking for numpy without sudo, I think numpy may not have been installed for root? (just a wild guess) If that is the case sudo pip install numpy may fix this...

GaelVaroquaux · 2015-11-23T07:55:02Z

Since you are running pip with elevated privileges (sudo pip) for installing scikit-learn, but check for numpy without sudo, I think numpy may not have been installed for root? (just a wild guess)

Good idea. Could you check the imports when running as root.

napsternxg · 2015-11-23T07:56:04Z

Already did that.

$ sudo pip install numpy
[sudo] password for user: 
Requirement already satisfied (use --upgrade to upgrade): numpy in /usr/local/lib/python2.7/dist-packages

raghavrv · 2015-11-23T08:03:28Z

Ok could you try this? (sorry I am just curious ;))

$ echo "With sudo"
$ for p in "numpy" "scipy" "sklearn"; do {sudo python -c "import $p; print('$p', $p.__version__, $p.__file__)"}; done
$ sudo which python; sudo which pip
$ echo "W/o sudo"
$ for p in "numpy" "scipy" "sklearn"; do {python -c "import $p; print('$p', $p.__version__, $p.__file__)"}; done
$ which python; which pip

Russell-Jones · 2015-11-24T08:15:46Z

@napsternxg what're in ~/.local/lib/python2.7 and ~root/.local/lib/python2.7 ?

napsternxg · 2015-11-24T20:45:49Z

@rvraghav93 I don't have permissions to run python as sudo so can't present some of the results in your script. But this is ourput from what I can run with my current permissions.

$ for p in "numpy" "scipy" "sklearn"; do { python -c "import $p; print('$p', $p.__version__, $p.__file__)"; } done
('numpy', '1.10.1', '/usr/local/lib/python2.7/dist-packages/numpy/__init__.pyc')
('scipy', '0.16.1', '/usr/local/lib/python2.7/dist-packages/scipy/__init__.pyc')
('sklearn', '0.16.1', '/usr/local/lib/python2.7/dist-packages/sklearn/__init__.pyc')

$ sudo pip --version
pip 7.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7)

$ pip --version
pip 7.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7)

$ which python
/usr/bin/python

Some extra information which I can provide about the sudo and non sudo versions of pip are the following:

Using sudo pip:

Site packages folder:  ['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']
User packages folder:  /root/.local/lib/python2.7/site-packages
User base folder:  /root/.local

Using pip without sudo:

Site packages folder:  ['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']
User packages folder:  /homed/username/.local/lib/python2.7/site-packages
User base folder:  /homed/username/.local

raghavrv · 2015-11-24T22:13:16Z

I don't have permissions to run python as sudo

Could that be the reason? (since (sudo )pip does (sudo )python setup.py?

Also any specific reason why you don't do pip install --upgrade scikit-learn? If you get an error while doing so can you post it here?

napsternxg · 2015-11-24T23:32:24Z

@rvraghav93

$ pip install --upgrade scikit-learn
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Collecting scikit-learn
  Downloading scikit-learn-0.17.tar.gz (7.8MB)
    100% |â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆ| 7.8MB 47kB/s 
Installing collected packages: scikit-learn
  Found existing installation: scikit-learn 0.16.1
    Uninstalling scikit-learn-0.16.1:
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 640, in install
    requirement.uninstall(auto_confirm=True)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 716, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/usr/local/lib/python2.7/dist-packages/pip/utils/__init__.py", line 315, in renames
    shutil.move(old, new)
  File "/usr/lib/python2.7/shutil.py", line 299, in move
    rmtree(src)
  File "/usr/lib/python2.7/shutil.py", line 252, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/usr/lib/python2.7/shutil.py", line 250, in rmtree
    os.remove(fullname)
OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/scikit_learn-0.16.1-py2.7.egg-info/SOURCES.txt'

And here is for sudo pip install

$ sudo pip install --upgrade scikit-learn
[sudo] password for username: 
Sorry, try again.
[sudo] password for username: 
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Collecting scikit-learn
  Using cached scikit-learn-0.17.tar.gz
Installing collected packages: scikit-learn
  Found existing installation: scikit-learn 0.16.1
    Uninstalling scikit-learn-0.16.1:
      Successfully uninstalled scikit-learn-0.16.1
  Running setup.py install for scikit-learn
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-kiDgr5/scikit-learn/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-vY272s-record/install-record.txt --single-version-externally-managed --compile:
    Partial import of sklearn during the build process.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-kiDgr5/scikit-learn/setup.py", line 247, in <module>
        setup_package()
      File "/tmp/pip-build-kiDgr5/scikit-learn/setup.py", line 227, in setup_package
        .format(numpy_req_str, instructions))
    ImportError: Numerical Python (NumPy) is not installed.
    scikit-learn requires NumPy >= 1.6.1.
    Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html


    ----------------------------------------
  Rolling back uninstall of scikit-learn
Command "/usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-kiDgr5/scikit-learn/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-vY272s-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-kiDgr5/scikit-learn

raghavrv · 2015-11-25T00:02:35Z

Ah! Try this?

$ sudo chown -R username:username /usr/local/lib/python2.7/
$ pip install --upgrade scikit-learn

I think this should work (assuming you are able to run chown with elevated privileges...)

napsternxg · 2015-11-26T01:39:07Z

@rvraghav93 sorry don't have privileges for running chown. Also can you edit your last comment to remove my username from it. Thanks.

napsternxg · 2015-11-26T01:40:29Z

BTW I also ran the following command which ensures that Numpy is indeed installed for root python as shown below:

$ sudo pip install --upgrade numpy
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Requirement already up-to-date: numpy in /usr/local/lib/python2.7/dist-packages

napsternxg · 2015-11-26T01:56:57Z

Some more information:

I wrote a small package of my own with similar numpy check as the one used in Sklearn. My setup.py file looks like the following:

DOCLINES = __doc__.split("\n")

import os
import sys
import site

# Always prefer setuptools over distutils
from setuptools import setup, find_packages
from pkg_resources import parse_version

def check_numpy_status():
  try:
    import numpy as np
    print "Imported numpy version %s" % np.__version__
    print "Version >= %s: %s" % ('1.6.1', parse_version(np.__version__) >= parse_version('1.6.1'))
  except ImportError:
    print "Couldn't import Numpy."



print("Shubh Pip Tester.")
print("Site packages folder: ", site.getsitepackages())
print("User packages folder: ", site.getusersitepackages())
print("User base folder: ", site.getuserbase())
check_numpy_status()
setup(name="shubh_pip_tester", py_modules=["shubh_test"])
if __name__ == '__main__':
  print("Hello World")

When I install the following using sudo pip install . -v from the folder of the package, I see the same message saying Couldn't import Numpy.. Here is the full install message:

$ sudo pip install . -v
Processing /content/username/SOFTWARE/pip_testing
  Running setup.py (path:/tmp/pip-IUB9V9-build/setup.py) egg_info for package from file:///content/username/SOFTWARE/pip_testing
    Running command python setup.py egg_info

    Shubh Pip Tester.
    ('Site packages folder: ', ['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages'])
    ('User packages folder: ', '/root/.local/lib/python2.7/site-packages')
    ('User base folder: ', '/root/.local')
    Couldn't import Numpy.
    running egg_info
    creating pip-egg-info/shubh_pip_tester.egg-info
    writing pip-egg-info/shubh_pip_tester.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/shubh_pip_tester.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/shubh_pip_tester.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/shubh_pip_tester.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'pip-egg-info/shubh_pip_tester.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/shubh_pip_tester.egg-info/SOURCES.txt'
    Hello World
  Source in /tmp/pip-IUB9V9-build has version 0.0.0, which satisfies requirement shubh-pip-tester==0.0.0 from file:///content/username/SOFTWARE/pip_testing
Installing collected packages: shubh-pip-tester
  Running setup.py install for shubh-pip-tester
    Running command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-IUB9V9-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-_SCoFG-record/install-record.txt --single-version-externally-managed --compile
    Shubh Pip Tester.
    ('Site packages folder: ', ['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages'])
    ('User packages folder: ', '/root/.local/lib/python2.7/site-packages')
    ('User base folder: ', '/root/.local')
    Couldn't import Numpy.
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    copying shubh_test.py -> build/lib.linux-x86_64-2.7
    running install_lib
    copying build/lib.linux-x86_64-2.7/shubh_test.py -> /usr/local/lib/python2.7/dist-packages
    byte-compiling /usr/local/lib/python2.7/dist-packages/shubh_test.py to shubh_test.pyc
    running install_egg_info
    running egg_info
    creating shubh_pip_tester.egg-info
    writing shubh_pip_tester.egg-info/PKG-INFO
    writing top-level names to shubh_pip_tester.egg-info/top_level.txt
    writing dependency_links to shubh_pip_tester.egg-info/dependency_links.txt
    writing manifest file 'shubh_pip_tester.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'shubh_pip_tester.egg-info/SOURCES.txt'
    writing manifest file 'shubh_pip_tester.egg-info/SOURCES.txt'
    Copying shubh_pip_tester.egg-info to /usr/local/lib/python2.7/dist-packages/shubh_pip_tester-0.0.0-py2.7.egg-info
    running install_scripts
    writing list of installed files to '/tmp/pip-_SCoFG-record/install-record.txt'
    Hello World
  Removing source in /tmp/pip-IUB9V9-build
Successfully installed shubh-pip-tester-0.0.0
Cleaning up...

I do believe it is some issue on my side and not with sklearn. I will investigate more and will post here if I find any issues. I will close the issue for the time being.

raghavrv · 2015-11-26T02:49:37Z

👍 good luck :)

napsternxg · 2015-11-26T05:32:45Z

Fixed this issue by:

Uninstalling numpy
Adding openblas.conf in /etc/ld.so.conf.d/
Installing numpy
Install other packages like scipy and scikit-learn.

GaelVaroquaux · 2015-11-26T06:37:12Z

I do believe it is some issue on my side and not with sklearn. I will investigate more and will post here if I find any issues. I will close the issue for the time being.

Please do report your conclusions here, however, so that we learn a bit the kind of problems our users have.

GaelVaroquaux · 2015-11-26T06:39:57Z

1. Uninstalling numpy 2. Adding openblas.conf in /etc/ld.so.conf.d/ 3. Installing numpy 4. Install other packages like scipy and scikit-learn.

I really don't understand a thing. Maybe it's because the installs of numpy that you had were missing a .egg-info? It used to be that installs of numpy via Linux distributions didn't play well with Python packaging tools. But I thought that this was mostly over.

JnBrymn-EB · 2016-06-07T20:10:12Z

I was having similar trouble today and noticed a potential problem with sklearn's setup.py. According to python setuptool docs you need to include the requires keyword argument and list numpy. Searching for "requires" in the sklearn setup.py I don't see this happening. Could this resolve our issues?

lesteve · 2016-06-08T06:47:29Z

According to python setuptool docs you need to include the requires keyword argument and list numpy. Searching for "requires" in the sklearn setup.py I don't see this happening.

There are arguments both ways. Look at #4164 (comment) and related issues/PRs for example about why we decided to do it this way.

napsternxg mentioned this issue Nov 23, 2015

ImportError: Numerical Python (NumPy) is not installed while installing sklearn pypa/pip#3260

Closed

GaelVaroquaux mentioned this issue Nov 23, 2015

Original traceback in setup.py should be displayed #5905

Closed

napsternxg mentioned this issue Nov 24, 2015

Segmentation fault with TSNE fit_transform #5903

Closed

napsternxg closed this as completed Nov 26, 2015

nomisto mentioned this issue Jan 14, 2020

Changed import structure so that openbiolink is root repository, resolved all relative imports into absolute imports OpenBioLink/OpenBioLink#15

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ImportError: Numerical Python (NumPy) is not installed. #5904

ImportError: Numerical Python (NumPy) is not installed. #5904

ImportError: Numerical Python (NumPy) is not installed. #5904

ImportError: Numerical Python (NumPy) is not installed. #5904

Comments