[Bug]: plt.figure(), plt.close() leaks memory #23701
Comments
|
The following is with matplotlib.use('GTK3Agg')
|
|
Have you tried other backends? Is this GTK and/or AGG specific? Interesting would be "qtagg", "Tkagg", "agg" and "gtk4cairo". |
|
The issue is that you are never letting the GUI event loop run so there are a growing number of GUI windows just hanging out in the background waiting for the event loop to run so they can finish their tear down cycle (because most UI frameworks have a way to notify that things are being destroyed etc). If you need to run tight loops like this either use a non-interactive backend (like |
|
I've tried the suggestion. Outcome: All backends have the problem, but
When adding
Code: import matplotlib
import matplotlib.pyplot as plt
matplotlib.use("GTK4Agg")
# [
# "GTK3Agg",
# "GTK3Cairo",
# "GTK4Agg",
# "GTK4Cairo",
# "MacOSX",
# "nbAgg",
# "QtAgg",
# "QtCairo",
# "Qt5Agg",
# "Qt5Cairo",
# "TkAgg",
# "TkCairo",
# "WebAgg",
# "WX",
# "WXAgg",
# "WXCairo",
# "agg",
# "cairo",
# "pdf",
# "pgf",
# "ps",
# "svg",
# "template",
# ]
for k in range(4000): # higher numbers for agg, svg
plt.figure()
plt.close()
if k % 1000 == 0:
plt.pause(1.0) |
|
Can you please try adding Can you also try with https://github.com/matplotlib/matplotlib/blob/main/tools/memleak.py on your system? import gc
import matplotlib.pyplot as plt
import matplotlib as mpl
mpl.use('agg')
while True:
plt.figure()
plt.close()
gc.collect()This stays under 90MB for me (where as with out the |
import matplotlib.figure as mfigure
while True:
mfigure.Figure()Also seems to work so likely something in the |
|
|
Indeed: import gc
import matplotlib.pyplot as plt
import matplotlib as mpl
import matplotlib.figure as mfigure
mpl.use('agg')
for k in range(100000):
mfigure.Figure()
plt.close()gives
-- no explicit gc. |
|
That long delay at the end is probably GC finally running on the way out
I understand the problem, TL;DR either use manual
You can reproduce this issue with: import gc
import matplotlib.figure as mfigure
print("about to start")
for j in range(100_000):
fig = mfigure.Figure()
gc.collect(1)
print(gc.get_count())A way to "fix" this is from itertools import count
import gc
import matplotlib.pyplot as plt
import matplotlib as mpl
mpl.use('agg')
class MyObject:
...
gc.collect()
print("about to start")
for j in range(100):
print(gc.get_count())
plt.figure()
print(gc.get_count())
plt.close()
print(gc.get_count())
c = [MyObject() for j in range(700)]which will force enough generation 0 objects to be created that it will cause a full gc eventually. |
Matplotlib has a large number of circular references (between figure and manager, between axes and figure, axes and artist, figure and canvas, and ...) so when the user drops their last reference to a `Figure` (and clears it from pyplot's state), the objects will not immediately deleted. To account for this we have long (goes back to e34a333 the "reorganize code" commit in 2004 which is the end of history for much of the code) had a `gc.collect()` in the close logic in order to promptly clean up after our selves. However, unconditionally calling `gc.collect` and be a major performance issue (see matplotlib#3044 and matplotlib#3045) because if there are a large number of long-lived user objects Python will spend a lot of time checking objects that are not going away are never going away. Instead of doing a full collection we switched to clearing out the lowest two generations. However this both not doing what we want (as most of our objects will actually survive) and due to clearing out the first generation opened us up to having unbounded memory usage. In cases with a very tight loop between creating the figure and destroying it (e.g. `plt.figure(); plt.close()`) the first generation will never grow large enough for Python to consider running the collection on the higher generations. This will lead to un-bounded memory usage as the long-lived objects are never re-considered to look for reference cycles and hence are never deleted because their reference counts will never go to zero. closes matplotlib#23701
Matplotlib has a large number of circular references (between figure and manager, between axes and figure, axes and artist, figure and canvas, and ...) so when the user drops their last reference to a `Figure` (and clears it from pyplot's state), the objects will not immediately deleted. To account for this we have long (goes back to e34a333 the "reorganize code" commit in 2004 which is the end of history for much of the code) had a `gc.collect()` in the close logic in order to promptly clean up after our selves. However, unconditionally calling `gc.collect` and be a major performance issue (see matplotlib#3044 and matplotlib#3045) because if there are a large number of long-lived user objects Python will spend a lot of time checking objects that are not going away are never going away. Instead of doing a full collection we switched to clearing out the lowest two generations. However this both not doing what we want (as most of our objects will actually survive) and due to clearing out the first generation opened us up to having unbounded memory usage. In cases with a very tight loop between creating the figure and destroying it (e.g. `plt.figure(); plt.close()`) the first generation will never grow large enough for Python to consider running the collection on the higher generations. This will lead to un-bounded memory usage as the long-lived objects are never re-considered to look for reference cycles and hence are never deleted because their reference counts will never go to zero. closes matplotlib#23701
Uh oh!
There was an error while loading. Please reload this page.
Bug summary
MWE:
Either run this code and monitor the mem consumption on top, or run
(with memory-profiler) to get
Might be related to #22448.
Operating system
Ubuntu 22.10
Matplotlib Version
3.5.3
Matplotlib Backend
GTK4Agg
Python version
3.10.6
Jupyter version
No response
Installation
pip
The text was updated successfully, but these errors were encountered: