py/gc: When heap autosplit is enabled, limit new heap size. #13035

projectgus · 2023-11-21T01:14:47Z

Previously the "auto split" function (added in #12141) would try to "greedily" double the heap size each time, and only tries to grow by a smaller increment if it would be not possible to allocate the memory.

Greedily growing the heap like this reduces the chance of Python fragmentation, but increases the chance of prematurely running out of system heap (even when there's quite a lot of Python heap free.)

This adds a new function micropython.heap_sys_reserve() and a port-specific macro, MP_PLAT_DEFAULT_HEAP_SYS_RESERVE. This is the amount of total system heap to try and keep free, and a function gc_get_total_free() to get the total free system heap. These are added on esp32, as this is the only port that enables "auto split" by default.

On original ESP32 without PSRAM, I've been running this test program that first allocates around 60,000 bytes of Python data (in small chunks), then initializes ESP32 Wi-Fi STA mode, and then creates as many concurrent TLS connections as it can.

MicroPython commit d325ee4 (last commit before auto-split), the Python total heap size is 108032 (constant) and one TLS socket can be created before ESP-IDF system heap runs out.
Current master branch, the Python total heap size grows from 64000 to 128000 bytes. Zero TLS sockets can be created after the heap size grows.
Previous version of this PR ("don't greedily take more than 75% of free system heap"), the pattern is actually the same as current master: heap size grows to 128000 bytes, zero TLS sockets can be created after this.
Current version of this PR, with default 80KB heap_sys_reserve for the system heap, the Python total heap size grows from 64000 to 111168 bytes. One TLS socket can be created.
Calling micropython.heap_sys_reserve(100000) at the beginning allows two TLS sockets to be created. The Python total heap size grows from 64000 to 75712 bytes. Setting the value higher doesn't improve this further (limited by actual RAM requirements.)

So , this seems promising but the side effects may be a little unpredictable, still:

The Python heap will still grow into the "reserved" space if it has to, but that memory may be more fragmented than before due to the new "conservative" growing strategy that's only adding the minimum necessary each time.
If Wi-Fi is enabled or large system memory allocations like TLS happen before the Python heap grows (including over a soft reset) then it will unnecessarily grow the Python heap conservatively, as the heaviest memory use has already happened but it doesn't "know" this. However it seems like most often the heap growth happens the other way around.

I've experimented with this version a little bit and it seems to perform well, but I'm not 100% sure. Anyone who is experiencing issues running out of system heap and wants to test it, please report back if it helps you.

This work was funded through GitHub Sponsors.

projectgus · 2023-11-21T01:15:26Z

@tannewt CircuitPython may find this interesting.

github-actions · 2023-11-21T01:22:17Z

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS

codecov · 2023-11-21T01:31:09Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (f3889db) 98.36% compared to head (de5fae0) 98.36%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #13035   +/-   ##
=======================================
  Coverage   98.36%   98.36%           
=======================================
  Files         159      159           
  Lines       20978    20978           
=======================================
  Hits        20636    20636           
  Misses        342      342

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

bobveringa · 2023-11-21T10:43:42Z

py/gc.c

+    // into allocation failures elsewhere in the system.
+
+    size_t total_free = gc_get_total_free();
+    size_t grow_limit = total_free * 75 / 100;


Could this grow limit be implemented to be configurable? People who want to use all the memory possible aren't limited in those instances, and advanced users can configure this to whatever value they like.

Thanks @bobveringa, it's an idea worth considering for sure.

To be clear on a small difference, this isn't a hard limit - the Python heap will still grow to use all of the available RAM if the alternative is a MemoryError.

This limit is to limit "greedily" adding more memory than is needed to the Python heap, it eases back to avoid prematurely taking all of the system heap for Python.

projectgus · 2023-11-21T23:40:34Z

I've made some additional changes after discussion with @dpgeorge and @jimmo . See PR main description for an updated account.

The original commit which applied a different "don't greedily take more than 75% of the free system heap" approach is still in this branch too, for now.

bobveringa · 2023-11-23T14:57:51Z

I just tried out these changes. The default setting doesn't really seem to have an effect. The total still jumps from 64k to 128k. So just to be sure, updated the definition on my (custom) board to #define MP_PLAT_TRY_RESERVE_SYSTEM_HEAP (120 * 1024)
This seemed to work considerably better with the total being capped at 96448. This is a little too aggressive I think, but it works alright for now.

I don't have the time to figure out the precise workings of this system yet. But if you have things that would like me to test, then I'm happy to help.

projectgus · 2023-11-28T00:28:23Z

Thanks @bobveringa . That experience suggests to me either that either this will need to be runtime tunable, or that it's the wrong approach...

bobveringa · 2023-11-28T14:06:12Z

For me, after adjusting the setting to try and preserve some more memory, it worked well. My use-case is a bit different from most, I suspect, as I also have a bunch of user C modules that need large blocks of continuous memory (3x 5000 bytes).

It is working well enough that if this isn't in the next release (or another solution) I will probably use this solution anyway as a patch just so I can upgrade my MicroPython version.

Previously the "auto split" function would try to "greedily" double the heap size each time, and only tries to grow by a smaller increment if it would be not possible to allocate the memory. Now it will stop adding large chunks of heap once the free system heap is lower than a limit. The limit is exposed as micropython.heap_sys_reserve() so it can be tweaked for particular firmware use cases. With this change and no tuning of the limit, the Python heap growth on original ESP32 is closer to the v1.20 Python heap size after the first time it expands. Signed-off-by: Angus Gratton <angus@redyak.com.au>

projectgus · 2023-12-05T00:01:27Z

Thanks @bobveringa for the update. I've made some changes, this now has a runtime tuneable value "micropython.heap_sys_reserve()`. See the main PR text for some details.

dpgeorge · 2023-12-18T02:49:11Z

After discussing with @jimmo, we think this change is too complex/fine-tuned. And the semantics of the new micropython.heap_sys_reserve() function are hard to define: is it a hard limit or soft limit? What is actually reserved? Adding this function now means we need to commit to it, and I don't think it's general enough in its scope to tune the heap-growing algorithm.

As an alternative I've prepared #13219. That's a simple change, and users compiling their own board can change the new MICROPY_GC_INITIAL_HEAP_SIZE if needed.

dpgeorge · 2023-12-19T07:37:41Z

Closing in favour of #13219.

dpgeorge · 2023-12-19T07:38:21Z

@bobveringa if it's still not working for you, you can tune MICROPY_GC_INITIAL_HEAP_SIZE in your board config file.

projectgus added port-esp32 py-core Relates to py/ directory in source labels Nov 21, 2023

bobveringa reviewed Nov 21, 2023

View reviewed changes

dpgeorge added this to the release-1.22.0 milestone Nov 29, 2023

projectgus force-pushed the bugfix/gc_heap_autosplit_conservative branch from 560734e to de5fae0 Compare December 4, 2023 23:54

projectgus marked this pull request as ready for review December 5, 2023 00:06

dpgeorge mentioned this pull request Dec 18, 2023

esp32: Add MICROPY_GC_INITIAL_HEAP_SIZE option and tune it. #13219

Merged

dpgeorge closed this Dec 19, 2023

dhalbert mentioned this pull request Dec 19, 2023

Look at recent split heap changes in MicroPython adafruit/circuitpython#8729

Closed

projectgus deleted the bugfix/gc_heap_autosplit_conservative branch November 1, 2024 05:12

projectgus mentioned this pull request Mar 10, 2025

ESP32 without psram. Socket and Memory Split or ... . #14421

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

py/gc: When heap autosplit is enabled, limit new heap size. #13035

py/gc: When heap autosplit is enabled, limit new heap size. #13035

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

py/gc: When heap autosplit is enabled, limit new heap size. #13035

py/gc: When heap autosplit is enabled, limit new heap size. #13035

Uh oh!

Conversation

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!