Memory leaks on matplotlib 3.4.2 (and 3.4.0) #20490
Comments
|
If you put at the top of your script do these issues go away? I suspect that adding When you create a figure using a GUI backend, we are also creating GUI objects behind the scenes. These objects can out-live the attn @richardsheridan |
|
Agree, linking relevant PRs #9956, #17789, #19959. Also curious if other backends leak like this, for instance Qt or wx. In this comment I said that |
|
Tinkering on master to identify the leak, here is an interesting MRE: import psutil, objgraph
import matplotlib
matplotlib.use('tkagg') # leaks
# matplotlib.use('qt5agg') # leaks (pyside2)
# matplotlib.use('agg') # no leaks
import matplotlib.pyplot as plt
proc = psutil.Process()
for i in range(50):
print(i, proc.memory_info().rss)
objgraph.show_growth()
plt.plot([1,2,5])
plt.savefig(f"fig{i}.png")
# plt.pause(.1) # doesn't patch the leak, but causes QT to crash
plt.clf() # or cla() to simulate use case of plotting fresh figures
plt.close() # comment to patch the leak
# plt.pause(.1) # only calls time.sleep(.1) hereSpecifically regarding the tk backend, |
|
I have also encountered this issue. I have an example that is even more minimal. No plotting or saving is necessary to leak memory. In a fresh virtual environment with python 3.9.9 and only matplotlib the following code leaks: import matplotlib
from matplotlib import pyplot as plt
matplotlib.use('tkagg')
for i in range(100_000):
fig = plt.figure()
plt.close(fig)any of the following workarounds make no difference: fig.clf()
plt.close('all')
del fig
gc.collect()however using I have tested matplotlib versions 3.5.1, 3.5.0, 3.4.0, 3.4.0rc1 and 3.3.4. |
|
The core of the problem is that if you are using a GUI backend we are creating c++ side objects for the GUI framework. Those have their own life cycle rules and if you never run the GUI main loop in some cases they can not clean them selves up.
I suggest |
|
I see, so technically the memory is not leaked (references still exist) but there is no way to directly free it. If I need to do a lot of plotting it seems like the only option is to detect if tkagg is active and switch to something else instead since I can manually switch back in the rare case when I need I'm trying to track down some memory leaks which are likely a result of some exotic uses of the library and this was the only leak I could pin down definitively as not being my fault. |
|
The example by @mbway can be improved with import matplotlib
from matplotlib import pyplot as plt
import psutil
p = psutil.Process()
matplotlib.use('tkagg')
for i in range(100_000):
fig = plt.figure()
plt.close(fig)
print(p.memory_full_info().uss // 1_000_000)This figure-based leak bisects to e201189, which is no great surprise. I am working on a patch for that. What is surprising is that the leak persists (although at a slower rate of growth) on I bisected again with that reversion applied and got back 741ee02, which is part of #19167 (?!) so @QuLogic might have to dig in to find something leaky in there... at a glance I can't pick anything suspicious out. |
I cannot reproduce this; if I revert that commit, then the memory leak amount from #22002 remains the same. |
|
Something may have happened on It does suggest to me that we should put in at least the memory leak check (and possibly others) to prevent more regressions. |
|
With #22002 as it is now, the above test is down to about 1-2MB, but once you insert a |
tkinter variables get cleaned up with normal `destroy` and `gc` semantics but tkinter's implementation of trace is effectively global and keeps the callback object alive until the trace is removed. Additionally extend and clean up the tests. Closes #20490 Co-authored-by: Elliott Sales de Andrade <quantum.analyst@gmail.com>
…ory growth regressions FIX: TkAgg memory leaks and test for memory growth regressions (matplotlib#22002) tkinter variables get cleaned up with normal `destroy` and `gc` semantics but tkinter's implementation of trace is effectively global and keeps the callback object alive until the trace is removed. Additionally extend and clean up the tests. Closes matplotlib#20490 Co-authored-by: Elliott Sales de Andrade <quantum.analyst@gmail.com> (cherry picked from commit 1a016f0)
…ory growth regressions FIX: TkAgg memory leaks and test for memory growth regressions (matplotlib#22002) tkinter variables get cleaned up with normal `destroy` and `gc` semantics but tkinter's implementation of trace is effectively global and keeps the callback object alive until the trace is removed. Additionally extend and clean up the tests. Closes matplotlib#20490 Co-authored-by: Elliott Sales de Andrade <quantum.analyst@gmail.com> (cherry picked from commit 1a016f0)
Bug report
Bug summary
Multiple different memory related errors when running as a part of script creating and saving lots of figures.
Code for reproduction
Sorry, could not create a minimal example that reproduces this. Just wanted to leave a note.
Actual outcome
There are multiple different errors that happen. Running the same script will produce different results randomly.
Example 1
Just prints this. No Exceptions, nothing. Script stops running.
Example 2
Example 3
Expected outcome
No errors.
Matplotlib version
import matplotlib; print(matplotlib.__version__)): 3.4.2pri 8000 nt(matplotlib.get_backend())): TkAggWorkaround
As a side note, tried on version 3.3.4 and everything works. Does not work on 3.4.0. I'm calling
after every
plt.savefig()just to be sure.The text was updated successfully, but these errors were encountered: