ENH: Add memory leak test #10302
ENH: Add memory leak test #10302
Conversation
Related: numpy#10157
|
IMO find out how to upgrade the debug Python on Travis-CI to 3.5 and then run pytest-leaks. |
|
Can anyone help me an tell me why the chroot job in https://travis-ci.org/numpy/numpy/jobs/323433566 fails? It seems as if valgrind is not found by What do you suggest to skip the test when it is run by appveyor? I could add a skip decorator, which checks if valgrind is installed and skips the test if not. |
|
As I understand If we are going to do valgrind though, this is now how we should do it. Instead, we should create another job that runs valgrind on every single test. This is not difficult to do with |
This sounds like a good idea. I need some assistance with that, since this requires that pytest runs each test transparently in a valgrind subprocess. |
|
The basic procedure is:
|
|
Writing a script to do this should not be difficult. |
This strikes me as in scope for a pytest plugin, which could run as part of the main |
@nicoddemus, are you aware of a pytest plugin which runs each test in a subprocess, e.g. (gdb, valgrind, ..) ? |
|
Hmm pytest-xdist? |
|
For valgrind you probably want python 3.6 because it has
PYTHONMALLOC=malloc.
1.1.2018 23.08 "Bruno Oliveira" <notifications@github.com> kirjoitti:
… Hmm pytest-xdist?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#10302 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AACI5hiVoOb8KLvLibND0652QHhDjSQ-ks5tGUlhgaJpZM4RPtrQ>
.
|
|
How slowly will this run? See scipy/scipy#7800 where it slows down tests significantly |
|
xref #12718, which was not merged |
|
I am leaning toward closing this as "too complicated for current CI". Any opposition? |
|
I am ok with closing, not sure I feel the I ran the numpy test suit in valgrind with leak checking after every test (which is certainly faster then actually starting a new process for every test (although it appeared not 100% reliable – I wonder if a debug python might help with it, although not sure why it should). That took more then the whole night, much longer than just running it all in valgrind. Of course a leak check after every module or so would be enough. But overall it still seems like it might be a lot of long test runs compared to the gain. One thing I could imagine trying, would be to make my pytest-valgrind thingy more reasonble, e.g. to only check for leaks every module or so. But considering how slow valgrind runs... Even if we want to see if it can work, I think it should run only occasionally or on request, and I am not sure if there even a nice way to set that up. pytest-leaks may be better (certainly faster, if also slow, although it doesn't get the additional benefit of finding other valgrind errors). |
|
Closing. |
|
@seberg if the job could be made easier with a script, maybe add something to |
Related: #10157
In #10286 @charris suggested to add a new environment variable for (de)activating the valgrind test. I don't know yet how we want to integrate this environment variable into the CI config files.