Memory allocation failed when using machine.lightsleep on Arduino Portenta (STM32H747) #16832

snapchaturvedi · 2025-02-28T03:22:57Z

snapchaturvedi
Feb 28, 2025

Setup:
I am using Arduino Portenta H7 (STM32H747) with Vision Shield with a couple of external modules like RTC and PIR sensor; running on OpenMV firmware 4.5.9 based on Micropython

Issue:
I am running an image classification model on 10 images clicked by Portenta when motion is detected. I want to use machine.lightsleep() to save power when the device is simply waiting for motion. In other words, I have defined an external interrupt based on the PIR sensor. The problem is when the device sleeps for a while before waking up, I get Invoke failed error in my logfile (since machine.lightsleep() disconnects the IDE, I am throwing all errors and inline flags to an external .csv file using a simple function called send_to_log(msg) here).

# Model (this is outside the main loop)
def invoke_model():
    return ml.Model("/model/trained.tflite", load_to_fb=uos.stat('/model/trained.tflite')[6] > (gc.mem_free() - (64*1024)))

net = invoke_model()

# Inference (part of the main loop)
def inference(images):
    send_to_log("Starting inference")                           # Send string to .csv log file
    prob = [None]*len(images)                                   # Empty array to store predictions
    for i in range(len(images)):                                # Predict using loaded model
        predict = list(zip(labels, net.predict([images[i]])[0].flatten().tolist()))
        prob[i] = predict[1][1]
    return([prob])

As a quick fix to this, I thought of invoking the model at each iteration and deleting after use for the garbage collector to clear:

# Model (this is outside the main loop)
def invoke_model():
    return ml.Model("/model/trained.tflite", load_to_fb=uos.stat('/model/trained.tflite')[6] > (gc.mem_free() - (64*1024)))

# Inference (part of the main loop)
def inference(images):
    send_to_log("Starting inference")                           # Send string to .csv log file
    net = invoke_model()                                        # Call model
    prob = [None]*len(images)                                   # Empty array to store predictions
    for i in range(len(images)):                                # Predict using loaded model
        predict = list(zip(labels, net.predict([images[i]])[0].flatten().tolist()))
        prob[i] = predict[1][1]
    del net                                                     # Delete model
    return([prob])

This works fine for a few iterations until I get memory allocation failed, allocating 2561328 bytes. The memory size is of course different each time, but always close to this number. I have also been printing gc.mem_free() in the beginning of each iteration to confirm it's big enough w.r.t. memory required (eg., 2740656).

Ideally, invoking the model just once during boot should be sufficient as invoking again and again could potentially lead to complications in memory management with every iteration (like it probably is). Upon discussing on OpenMV's forum in this thread, I have some clarity that this could be a memory corruption issue due to machine.lightsleep().

Could someone please help me understand what I could do to

ensure the model when called only once does not get lost/deleted, or
manage enough memory to be able to invoke the model in each iteration

PC

jonnor · 2025-03-03T20:59:50Z

jonnor
Mar 3, 2025
Collaborator

You are looping over images. How many images do you have in there?

2 561 328 bytes is 2.5 megabytes. That is a very large number, and I would say a potential. problem in itself, as the Porenta H7 only has 8MB SDRAM in total. What is allocating such large chunks of memory?

I do not see any reason to believe that lightsleep is to blame. But you have not even shown the code related to lightsleep, so it is hard to verify.

You mention that you have an interrupt handler for the PIR sensor. That might cause trouble if done incorrectly, for example if it triggers several times quickly. Can you show the code for that?

0 replies

snapchaturvedi · 2025-03-03T23:19:31Z

snapchaturvedi
Mar 3, 2025
Author

Hi @jonnor, thanks for your response!

How many images do you have in there? Burst of 10 images are clicked and fed for inference
What is allocating such large chunks of memory? I'll look into this and get back to you ASAP
I do not see any reason to believe that lightsleep is to blame. You might be right here but I have been working on this very program for weeks without machine.lightsleep() and did not encounter this issue even once.
I was advised to work on a minimal script to replicate the issue and have been working on it since posting this question. Seems to me that this issue occurs when I use LoRa in my application 'cause I've been testing it without LoRa and it's not encountering the issue. I don't have enough evidence yet to say that the issue absolutely does not occur when LoRa is not in use though (working on it still).
Here's the minimal working example where I'm invoking the model just once and waking up from lightsleep every 60 seconds. Please find the used model and label files here:

import machine, pyb, sensor, ml, uos, gc
from lora import *
from LORA_APP_KEY import * # Make a .py file with variable APP_KEY

# Paths
save_path = "save"  # To save pictures
log_path = "log"    # To save log in a .csv file since IDE disconnects when lightsleeping

# Create folders if does not exist
paths = [save_path, log_path]
for path in paths:
    if path not in uos.listdir():
        uos.mkdir(path)

# Create log file if does not exist
if "log.csv" not in uos.listdir(log_path):
    with open(f"/{log_path}/log.csv", "a") as file:
        file.write("timestamp, event\n")

# Define RTC
rtc = pyb.RTC()

# Format datetime for convenient usage
def dt_format():
    return "_".join(str(i) for i in rtc.datetime()[0:3]+rtc.datetime()[4:7])

# Camera setup
sensor.reset()
sensor.set_auto_exposure(1)
sensor.set_auto_rotation(1)

sensor.set_pixformat(sensor.GRAYSCALE)
sensor.set_framesize(sensor.QVGA)
sensor.skip_frames(time = 2000)

# Pin define for external interrupt (PIR sensor in my case)
pir = pyb.Pin("PA8
10000
", pyb.Pin.IN, pull=pyb.Pin.PULL_DOWN)        # PIR sensor object

# Load labels for model
labels = [line.rstrip('\n') for line in open("/model/labels.txt")]

# Function to invoke model
def invoke_model():
    save_to_log("Invoking model")
    return ml.Model("/model/trained.tflite", load_to_fb=uos.stat('/model/trained.tflite')[6] > (gc.mem_free() - (64*1024)))

# Register logs
def save_to_log(event):
    with open(f"/{log_path}/log.csv", "a") as file:
        file.write(f"{dt_format()}, {event}\n")

# Take photos for inference
def click(n):
    img = [None]*n                                              # Empty array of size n to store images

    pyb.LED(2).on()

    save_to_log("Taking pictures")
    for i in range(n):                                          # Click images, save in array and path
        img[i] = sensor.snapshot().rotation_corr(z_rotation=180)
        img[i].save(f"{save_path}/{dt_format()}_{i}.jpeg")
        pyb.delay(500)

    pyb.LED(2).off()
    return(img)

# Inference
def inference(images):
    save_to_log("Starting inference")
#    net = invoke_model()
    prob = [None]*len(images)                                   # Empty array to store predictions
    for i in range(len(images)):                                # Predict using loaded model
        save_to_log(f"Inference {i}")
        predict = list(zip(labels, net.predict([images[i]])[0].flatten().tolist()))
        prob[i] = predict[1][1]
#    del net
    return([prob, len([i for i in prob if i>0.5])])             # Return no. images that predict cat

# Define external interrupt and callback instance for pin
def callback(line):
    pass

#ext = pyb.ExtInt(pir, pyb.ExtInt.IRQ_RISING, pyb.Pin.PULL_DOWN, callback)

# LoRa
# APP_KEY imported from LORA_APP_KEY library
N_ATTEMPTS = 1
status = "not connected"
BAND = BAND_AS923
JOIN_EUI = "0000000000000000"

# Estb. LoRa connection
for attempt in range(N_ATTEMPTS):
    print(f"Attempt {attempt} to connect to TTN")
    try:
        pyb.LED(3).on()
        lora = Lora(band=BAND, poll_ms=60000, debug=False)
        lora.join_OTAA(JOIN_EUI, APP_KEY, timeout=10000)
        status = "connected"
        print("\nLoRa Connected")
        pyb.LED(3).off()
        break
    except Exception as e:
        print(e)
        attempt+=1
        pyb.delay(1000)
        pyb.LED(3).off()
        pass

# Invoke model
net = invoke_model()

# main
while True:
    try:
        gc.collect()
        save_to_log(gc.mem_free())

        pyb.LED(1).on()
        pyb.delay(3000)
        pyb.LED(1).off()

        machine.lightsleep(1*60*1000)

        pics = click(10)
        preds = inference(pics)
        save_to_log(preds[1])

        save_to_log("Sending message via LoRa")

        if status == "connected":
            try:
                if lora.send_data(message, True):
                    save_to_log("Message confirmed.")
                else:
                    save_to_log("Message wasn't confirmed")

            except Exception as e:
                print(e)

    except Exception as e:
        pyb.LED(1).on()
        pyb.LED(2).on()
        pyb.LED(3).on()
        save_to_log(e)

Important thing to note is that it takes time for the issue to come up. Sometimes it's just the 2nd or 3rd iteration of the loop but most of the times I've come to it after a few iterations. Please find the attached log file of my latest run from last evening (37 successful iterations before issue): log.csv

Re Interrupt Handler: Can you show the code for that? Here you go: I have commented it in the above script but basically it's ext = pyb.ExtInt(pir, pyb.ExtInt.IRQ_RISING, pyb.Pin.PULL_DOWN, callback). I don't have experience working with external interruptions so happy to take suggestions.

PC

EDIT: MORE CONTEXT TO POINT 4

I started running this last evening and left it to run overnight. I used a multimeter to monitor power consumption (specifically current draw) and figure where the process stops.
The point at which the process was stuck was drawing 9 mA current, which is how much the Portenta draws on average when it's on lightsleep (verified on previous tests); meaning, the system was not waking up from lightsleep.

PC

3 replies

jonnor Mar 5, 2025
Collaborator

Your interrupt handler does nothing in this code.

You are also keeping all the 10 the images in memory while doing all the inference, and also until the next loop.
I would at the very least do a "delete pics" after you have finished using them. And probably do the gc.collect() there as well, to be explicit that it is the "cleanup" step for the iteration.

snapchaturvedi Mar 5, 2025
Author

Your interrupt handler does nothing in this code. It is because for testing my focus was on reiterating the process systematically so the lightsleep is interrupted by a timer here. The issue persists, nonetheless.
You are also keeping all the 10 the images in memory while doing all the inference...: That's a good idea. I'll test deleting the pictures and calling gc immediately after that to clear memory. Thanks a ton!

Will keep you posted.

PC

snapchaturvedi Mar 7, 2025
Author

Update: the issue persists even when I cleanup images and inference related objects after inference at each iter

PC

F5CC

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MicroPython

Memory allocation failed when using machine.lightsleep on Arduino Portenta (STM32H747) #16832

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

MicroPython

Memory allocation failed when using machine.lightsleep on Arduino Portenta (STM32H747) #16832

Uh oh!

Uh oh!

snapchaturvedi Feb 28, 2025

Replies: 2 comments · 3 replies

Uh oh!

jonnor Mar 3, 2025 Collaborator

Uh oh!

Uh oh!

snapchaturvedi Mar 3, 2025 Author

Uh oh!

jonnor Mar 5, 2025 Collaborator

Uh oh!

Uh oh!

snapchaturvedi Mar 5, 2025 Author

Uh oh!

snapchaturvedi Mar 7, 2025 Author

snapchaturvedi
Feb 28, 2025

Replies: 2 comments 3 replies

jonnor
Mar 3, 2025
Collaborator

snapchaturvedi
Mar 3, 2025
Author

jonnor Mar 5, 2025
Collaborator

snapchaturvedi Mar 5, 2025
Author

snapchaturvedi Mar 7, 2025
Author