8000 BUG: fix segfault at import of the scikit by GaelVaroquaux · Pull Request #1268 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

BUG: fix segfault at import of the scikit #1268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

BUG: fix segfault at import of the scikit #1268

wants to merge 1 commit into from

Conversation

GaelVaroquaux
Copy link
Member

Hot fix to solve the segfault on the buildbot.

In general to use the C API of numpy (cimport numpy in Cython), you
should always insert 'np.import_array()' after the import. If not, you
can have race conditions during the imports.

@GaelVaroquaux
BUG: fix segfault at import of the scikit
6430d22 
In general to use the C API of numpy (cimport numpy in Cython), you
should always insert 'np.import_array()' after the import. If not, you
can have race conditions during the imports.
@GaelVaroquaux
Copy link
Member Author

OK, it seems that this does not fix the problem. The segfault is not 100% reproducible, but I still get it every once in a while.

@larsmans
Copy link
Member

Isn't there a way to do this centrally, say from sklearn/__init__.py?

@GaelVaroquaux
Copy link
Member Author

Isn't there a way to do this centrally, say from sklearn/init.py?

I don't know, but anyhow it seems that it's not the problem.

G

@larsmans
Copy link
Member

Interesting: I can import sklearn and do some stuff, but IPython segfaults on exit:

Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48) 
Type "copyright", "credits" or "license" for more information.

IPython 0.10 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

  Welcome to pylab, a matplotlib-based Python environment.
  For more information, type 'help(pylab)'.

In [1]: import sklearn

In [2]: sklearn
Out[2]: <module 'sklearn' from 'sklearn/__init__.pyc'>

In [3]: import sklearn.metrics

In [4]: sklearn.metrics.euclidean_distances
Out[4]: <function euclidean_distances at 0x309a050>

In [5]: sklearn.metrics.euclidean_distances([1,2,3])
Out[5]: array([[ 0.]])

In [6]: sklearn.metrics.euclidean_distances([1,2,3], [4,5,6])
Out[6]: array([[ 5.19615242]])

In [7]: 
Do you really want to exit ([y]/n)? 
Segmentation fault

This doesn't happen every time, though.

@GaelVaroquaux
Copy link
Member Author

On Tue, Oct 23, 2012 at 08:48:16AM -0700, Lars Buitinck wrote:

Interesting: I can import sklearn and do some stuff, but IPython segfaults on
exit:

That's probably more debuggable than my current way of triggering the
bug: at import.

I'll keep digging...

@mrjbq7
Copy link
Contributor
mrjbq7 commented Oct 23, 2012

Does it trigger with an explicit gc?

@mrjbq7
Copy link
Contributor
mrjbq7 commented Oct 23, 2012

(Was thinking maybe something doesn't __del__ properly)...

@GaelVaroquaux
Copy link
Member Author

Does it trigger with an explicit gc?

Good idea. However the answer is: "not reliably".

@GaelVaroquaux
Copy link
Member Author

Does it trigger with an explicit gc?

Hey Jon,

Looking at the gdb backtrace, which I have a hard time to trigger, it
does seem to happen during garbage collection (not surprising). I put the
information online at:
#1006 (comment)

@erg
Copy link
Contributor
erg commented Oct 24, 2012

Can't debug because of this, reported by @amueller on the mailing list too:

erg@ommegang ~/python/scikit2/sklearn $ [master*] nosetests
E
======================================================================
ERROR: Failure: ImportError (/home/erg/python/scikit2/sklearn/metrics/_euclidean_fast.so: undefined symbol: cblas_dsyrk)
----------------------------------------------------------------------
...

@GaelVaroquaux
Copy link
Member Author

On Tue, Oct 23, 2012 at 05:51:52PM -0700, erg wrote:

erg@ommegang ~/python/scikit2/sklearn $ [master*] nosetests

E

ERROR: Failure: ImportError (/home/erg/python/scikit2/sklearn/metrics/_euclidean_fast.so: undefined symbol: cblas_dsyrk)

...

Thanks for trying. It's always good to catch such a linking error. We'll
be reverting this change. It has too many adverse consequences.

@mblondel
Copy link
Member

Did anyone try to add the missing blas routine to see if it fixes the problem?

@amueller
Copy link
Member

I'm pretty sure adding it would resolve this issue (but not the segfault).
Don't think anyone has tried.

@amueller
Copy link
Member

What happend to @vene's PR btw?

@GaelVaroquaux
Copy link
Member Author

What happend to @vene's PR btw?

Merged and then reverted... So it's lost, althought the code is still
somewhere :(.

G

@amueller
Copy link
Member

On 11/27/2012 09:37 AM, Gael Varoquaux wrote:

What happend to @vene's PR btw?

Merged and then reverted... So it's lost, althought the code is still
somewhere :(.
@vene care to reopen?

@larsmans
Copy link
Member

The revert is a single commit so

git revert 0dd841f6ee1a747324124485e902d88fdba1eaf0

will get the code back.

@amueller
Copy link
Member

@larsmans I guess the issue is more that someone needs to pick it up, merge the blas dependencies and find the segfault ;)

@larsmans
Copy link
Member

Is this still current?

@larsmans larsmans closed this Jul 6, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@GaelVaroquaux @larsmans @mrjbq7 @erg @mblondel @amueller
0