Segmentation fault when using memory allocation tracker #4834
Comments
|
can you provide a backtrace with gdb? or provide the code snippet that crashes. |
|
I need a bit more time to reproduce/isolate it. The example shipped with track_allocations.py works for me as well. |
|
Ok I have more details but couldn't isolate the error completely yet. Contrary to what I said earlier it doesn't happen the first time an array is allocated. My program actually runs a while, calculating things, plotting things (with Matplotlib/Basemap) and at one point when it wants to draw a map with Basemap it seg faults: |
|
You might try valgrind. It tends to throw up false positives when used with
|
|
thanks for the traceback, it shows that the hook is called from the possibly you can also get away with just disabling the garbage collector during the python hook ( @njsmith, by quoting the traceback you referenced all PRs 1-118, it seems github is not smart enough to ignore email quotes :) |
|
Doh, sorry :-) I don't understand the problem. In general it is legal to call arbitrary
|
|
|
|
Ok, at least the cause of the error is clear now. But I'm afraid that this is a bit too heavy for me, I just wrote some simple Cython wrappers, nothing really complicated with numpy's API or so. |
|
can you try if putting gc.disable() and gc.enable() around the code in the hook function helps? |
|
Your suggestion seems to work. I'll let it run overnight and see if it crashes. Still, I have the feeling that garbage collection could theoretically still happen between the moment of invoking hook and executing the |
|
I would also expect that calling gc.enable() has the potential to trigger We might want to ask the cython folks if they have any understanding or On Thu, Jul 3, 2014 at 4:41 PM, Maik Riechert notifications@github.com
Nathaniel J. Smith |
|
yes adding the disable in the python code will involve a python function call which can trigger garbage collection so it will only decrease the probability of a crash. But as long as it works most of the time it should be a fine workaround, the hooks are only for development/debugging anyway. |
|
Fair enough I guess. Another reason perhaps to look forward to tracemalloc, which IIRC On Thu, Jul 3, 2014 at 5:23 PM, Julian Taylor notifications@github.com
Nathaniel J. Smith |
|
I didn't have any other crashes, so I think this workaround is fine for now and could be integrated. |
|
k, thanks for the feedback |
When a cython object contains numpy arrays the pure python allocation hook can trigger during cythons __dealloc__ method and trigger a second deletion of the object currently being deleted. To minimize the probabily that that happens disable the garbage collector during the hook. As this involves python calls it is still possible that a double delete occurs but chances are lowered, a proper solution would be C only hook like python 3.4 tracemalloc module. Closes numpygh-4834
I use Python 2.7.6, numpy 1.8.1 and cython 0.20.2 under Red Hat 6.5. I tried out the code in numpy/tools/allocation_tracking, that is, I installed the extension (alloc_hook) and then copied track_allocations.py into my own code and used it according to the example in the main block. But the moment I allocate a numpy array I get a segmentation fault. In case it is related, here is the extension compile output:
@thouis @charris
The text was updated successfully, but these errors were encountered: