can't setup.py install without numpy #4164
Comments
|
What is your current install setup? Scikit-learn requires numpy and if you install everything together, it should work fine. |
Agreed.
How are you installing things? If you are compiling from source, you do
I would be surprised that scipy behaves different. |
|
Scipy does behave differently: |
Hum, interesting. There are pros and cons for doing it this way for numpy. The pro is that My hunch would be to leave it the way it is, but I see the point of the |
|
Demo: output: |
|
Maybe we could make this demo work, but chances are that you would have a Really, you should be building from source only if you know well what you If you are not an expert, please use prebuild packages. At least this is |
|
You are right, installing this way really is a bad idea in most cases... |
|
This is a serious issue, since virtualenv and pip are really becoming the standard to set up a test or development environment. How can I run tox-based contiuous integration tests for projects that depend on sklearn then? I agree that using prebuilt packages provided by a decent Linux distribution is the way for production/real life applications, but CI and automatic tests are required tools too. scikit-learn should not be installed using "pip install" or setuptools in general, but scikit-learn must be installable that way. David |
|
Hum, continuous integration is a good point. |
|
@douardda can you try if the scipy hack works for you? But I guess it will be more complicated than that... |
|
Hum, our setup.py seems pretty identical to the scipy one, I'm not sure what makes theirs work. |
|
Humm, on my jessie laptop, in a fresh virtualenv, "pip install scipy" also fails with a "ImportError: No module named numpy.distutils.core". |
Pass build_requires if numpy is not found Fixes scikit-learn#4164 Signed-off-by: Saket Choudhary <saketkc@gmail.com>
|
Interesting, for me scipy works but scikit-learn fails. |
|
@amueller Any ideas on this? I read through #4332 and don't see a solution being proposed that actually installs scikit-learn correctly like scipy does when both are in a requirements file (please correct me if I am mistaken). We're using pre-built wheel packages to avoid compile/configuration times taking forever, and I'm running into this same issue. I've got another project using scipy that works flawlessly when numpy is in the requirements file, but I get this error specifically for scikit-learn. I'll try adding scipy to the requirements.txt to see if that helps, but from the errors I'm seeing it's bombing out before even trying to install. edit Actually it looks like @saketkc's work over in #4371 might address this. What's the timeline looking like on getting that merged in? I can work around it by installing numpy, THEN going back through the requirements.txt in my config management tool but it's a pain and it increases build times. |
It is not possible to build scipy from source without installing blas, lapack and gfortran. At this point users will have to read the scipy doc to build from source and install an optimized BLAS / LAPACK. |
|
+1 for using the |
|
Actually I changed my mind as explained in my last comment in #4371. Let's close this for now. |
|
If I understand it, It's still not possible to have a project that installs numpy and sklearn in the same step, and you all don't plan to fix it. Am I right? |
|
From what @ogrisel said #4371 (comment) it is non-trivial to make this work with ubuntu stable pip. |
|
didn't @GaelVaroquaux mention a flag to force installing dependencies? |
|
Various processes at our company involve 'pip install -r requirements.txt'. On Fri, May 8, 2015, 4:29 PM Andreas Mueller notifications@github.com
|
|
Various processes at our company involve 'pip install -r requirements.txt'.
For production or testing? If it's for production, I would be worried
about inefficiencies in the binaries produced.
This works fine for any group of packages not containing sklearn.
numerics / data analytics are a different beast than say web services. It
is very hard to have a one-size-fits-all solution. The ecosystem is
slowly evolving toward one, but it takes time.
|
|
I agree that if that is how you produce production binaries, then you are most certainly doing it wrong. |
|
Any progress on this issue? It'd be really great if there was a way to use scikit-learn inside a virtualenv by including it (and whatever versions of numpy/scipy/etc. one wants) in a requirements.txt file. Virtualenvs are a very standard way to deploy applications these days, so this is a pretty common use case. Since there was a solution under discussion as of a few months ago, maybe it makes sense to reopen the issue at least? |
|
I never got a chance to work on this. I still think it should be possible/straightforward to set up a |
|
Seconding 6D40 @timabbott. |
|
PR up at #6990. It's a very straightforward change. |
|
Temporary fix by overriding the import pip
from setuptools import setup
from setuptools.command.install import install
from pip.req import parse_requirements
install_reqs = parse_requirements('./requirements.txt', session=False)
reqs = [str(ir.req) for ir in install_reqs]
class OverrideInstall(install):
"""
Emulate sequential install of pip install -r requirements.txt
To fix numpy bug in scipy, scikit in py2
"""
def run(self):
for req in reqs:
pip.main(["install", req])
# the setup
setup(
...
cmdclass={'install': OverrideInstall}
....
)Then run |
Re: feedback #2840 (comment) I think we can live with double-installs, now that I have an open PR and the attention of the scikit-image maintainers (#2841 (comment)) we can probably expect that we can do away with this workaround sooner than later. This re-adds numpy in alphabetical order without the misleading note about "installing first" because ordering requirements.txt got deprecated at some point (e.g. scikit-learn/scikit-learn#4164 (comment)) and that's why we need an entirely separate `pip install numpy` in the first place.
It turns out there was a good reason for installing numpy explicitly, the line that we removed in #2751, because scikit-learn needs it to build from source but doesn't declare it: scikit-image/scikit-image#4919 PyPA doesn't give a clear or reliable way to declare build dependencies: some projects are using pip everywhere, some are using legacy setuptools, some distutils. It's confusing for everyone right now. They're working on it. I think we can live with double-installs, now that I have an open PR and the attention of the scikit-image maintainers (#2841 (comment)) we can probably expect that we can do away with this workaround sooner than later. This also moves numpy to alphabetical order and removes the misleading note about "installing first" because ordering requirements.txt got deprecated at some point (e.g. scikit-learn/scikit-learn#4164 (comment))
It turns out there was a good reason for installing numpy explicitly, the line that we removed in #2751, because scikit-learn needs it to build from source but doesn't declare it: scikit-image/scikit-image#4919 PyPA doesn't give a clear or reliable way to declare build dependencies: some projects are using pip everywhere, some are using legacy setuptools, some distutils. It's confusing for everyone right now. They're working on it. I think we can live with double-installs, now that I have an open PR and the attention of the scikit-image maintainers (#2841 (comment)) we can probably expect that we can do away with this workaround sooner than later. This also moves numpy to alphabetical order and removes the misleading note about "installing first" because ordering requirements.txt got deprecated at some point (e.g. scikit-learn/scikit-learn#4164 (comment))
It turns out there was a good reason for installing numpy explicitly, the line that we removed in spinalcordtoolbox#2751, because scikit-learn needs it to build from source but doesn't declare it: scikit-image/scikit-image#4919 PyPA doesn't give a clear or reliable way to declare build dependencies: some projects are using pip everywhere, some are using legacy setuptools, some distutils. It's confusing for everyone right now. They're working on it. I think we can live with double-installs, now that I have an open PR and the attention of the scikit-image maintainers (spinalcordtoolbox#2841 (comment)) we can probably expect that we can do away with this workaround sooner than later. This also moves numpy to alphabetical order and removes the misleading note about "installing first" because ordering requirements.txt got deprecated at some point (e.g. scikit-learn/scikit-learn#4164 (comment))
At minimum, this could use a better error message. Is it not possible to have a project that depends on sklearn and installs all its dependencies to a virtualenv in a single pass? Must I do one pass to install numpy and everything else, and a second pass just to install sklearn?
The text was updated successfully, but these errors were encountered: