Failure to allocate when free memory is very large - severe fragmentation? #2057
Comments
|
What hardware are you running on? You can use |
|
@dpgeorge I wasn't aware of that function. Will provide when I'm with the device again, some time in the next 12ish hours, maybe sooner. Using STM32F439 (64k more ram allocated to heap than f405) I took the ~100KB of code (tabs not spaces!) and built it in as frozen mpy. I've now got 100k total free ram and the issue doesnt occur frequently enough to be of concern (for now...) |
I assume you mean using mpy-cross, the config option MICROPY_MODULE_FROZEN_MPY, and mpy-tool? Yes, doing this is the best way to free up RAM. |
|
@dpgeorge I'm compiling with the following arguments, that seems to take care of the hard work for me. make -j12 BOARD=STM32F439 FROZEN_MPY_DIR=scripts |
|
What do the letters in the dump refer to? |
|
The letters correspond to objects in the heap. The dots are free memory The letters correspond to different types of objects in the heap. See: Each object consists of one head block and 0-n tail blocks ('t') On Tue, May 10, 2016 at 6:36 PM, Ryan Shaw notifications@github.com wrote:
Dave Hylands |
|
@dhylands thanks for the clarification. Why does the gc mem free function report 47k free but this report 20k? |
|
I'm not sure. There are 1285 dots = 1285 x 16 = 20560 bytes. gc.mem_free reports the exact same thing that is used to report the free memory in this line: i.e. both places call gc_info and report info.free, so I'd have to assume that 27K of memory was allocated for something between the 2 calls. I do see 4 x 4k blocks, but without knowing what the heap looked like when 47k was free, its hard to saw. |
|
The router task that I've written generates a lot of immediately collectable objects (read packet from buffer, process metadata, write to uart, repeat many times). The "free memory" printed by mem_info(1) seems to be before a GC - that's probably why it's ~27k less than when I print mem_free (I collect between every "task") Why doesn't it automatically garbage collect and then attempt the allocation? Or is it doing this and for some reason is unable to free 582 bytes of contiguous memory? I'll rewrite this code at a later date to pass a memoryview to uart.write and see how that goes. |
|
My understanding is that if the allocation fails, then it will perform a gc, and only if that fails to free up enough memory will it be considered a memory allocation failure. |
It is. As Dave says it does a collection then tries to allocate again, then it'll fail if it can't get enough contiguous memory.
Probably there's just not 582 bytes (37 blocks) of contiguos heap. I would run a gc.collect() and then a micropython.mem_info(1) to make sure that it's really this that's the issue. Is the above output of mem_info(1) with frozen bytecode? It sounds like you are doing a lot of allocations and churning through the heap very quickly. And it looks like you operate with around 71%-87% of heap usage which is pretty tight for a program that has a large churn. I would try rewriting some parts to not allocate so much memory, eg by preallocating buffers and using memoryview when possible. |
|
Try applying the following patch to master (using git am < file >): It might help to collect more unused heap. |
|
@dpgeorge the above output is straight after crashing, prior freezing bytecode and changes that freed around 60k of ram. It now runs with 109k free. Since these changes our device hasn't been crashing (yet? - we are auto rebooting hourly) I also intend to rewrite parts further to allocate even less by using memoryviews, but I'm yet to investigate how well supported they are (eg can I pass it without concerns to a uart.write()) |
|
Will investigate with the patch tomorrow when I'm with the device again. I appreciate your help, keep up the solid work. |
memoryview should be at the same level of support as bytes and bytearray (and array) objects. You should be able to pass a memoryview to anything that needs a buffer (readable or writable). |
|
Excellent. We will be able to reduce memory consumption much further. Is there currently any way to "freeze" boot.py and main.py? Will report back with more logs and test results within a day. |
No. boot.py/main.py are files, not modules (and only modules can be frozen). But in main.py you can just do |
|
@dpgeorge Tested the patch, after 5 to 10 minutes a device running with the patch has 4-5k more free memory than devices running without the patch. An increase in free memory was only noted after routing a large number of packets, not after idling, which is probably what you'd expect. |
|
@ryannathans thanks for testing, good that it helped a bit. |
|
@dpgeorge has that patch made it in to master? Eyeballed the commit list but didn't notice it |
No. It would impact performance, so needs to be considered properly. |
This fixes a problem with USB MIDI messages 0xc and 0xd and Closes: micropython#2057
|
Also try using |
Hi all
We're experiencing the following issue at least once per hour of runtime.
We're failing to allocate 581 bytes of memory when we have over 45k of free memory, what's going on? I can only assume this is some kind of severe fragmentation?
I've done a bit of investigation and uncovered the following discussions. My understanding is that memory fragmentation is a known issue and isn't currently being worked on. Progress towards a solution occurs slowly and in small increments.
#1161
#1168
#1604
http://forum.micropython.org/viewtopic.php?f=3&t=455
https://github.com/micropython/micropython/wiki/Memory-Manager
Do you guys have any suggestions for improving robustness or techniques we can use to avoid this? Is there a way to measure memory fragmentation and restart the device safely when fragmentation approaches a threshold? Are there certain objects that significantly contribute to fragmentation that I should avoid using when writing code?
I like @dpgeorge 's ideas on the forum. Either having a memory-manager that defrags the memory or is able to allocate in fragmented memory using filesystem-like blocks.
The text was updated successfully, but these errors were encountered: