try/except leads to infinite loop with growing memory usage #1

pfalcon · 2013-12-29T12:31:32Z

Running a following script (./py exc.py) leads to an infinite loop with a growing memory usage (i.e. uPy keeps allocating something in an infinite loop):

a = {}
try:
    a[1]
except KeyError:
    print("caught")

Running on x64.

The text was updated successfully, but these errors were encountered:

dpgeorge · 2013-12-29T14:54:49Z

There is currently no memory manager (no garbage collection) for the unix version, so that's why it eats memory. But it's certainly a bug that it runs an infinite loop.

dpgeorge · 2013-12-29T17:23:03Z

Added proper nested exception handling, and basic exception type matching. Above code should now work.

pfalcon · 2013-12-29T18:26:47Z

Thanks for quick response, hope it doesn't steal cycles from hardware hacking ;-). (Though this particular issue is important enough, so I really appreciate looking into it).

Well, exception-throwing path now works (including on x86/32bit, will submit patch once cleaned up), but exception-free path leads to:

a = {1:2}
try:
    a[1]
except KeyError:
    print("caught")

./py exc.py 
py: ../py/vm.c:308: mp_execute_byte_code_2: Assertion `0' failed.
Aborted

Corresponding opcode doesn't appear to be implemented.

dpgeorge · 2013-12-29T18:45:09Z

Yep, it's not fully implemented. Micro Python doesn't use reference counting (like CPython) so the action of push/pop blocks in the interpreter is a bit simpler for Micro Python. Thus, the byte code POP_BLOCK is probably only needed when exiting an exception handling block.

dpgeorge · 2013-12-29T18:54:19Z

Okay, it should now work. There will still be issues with "finally" and "with" blocks.

pfalcon · 2013-12-29T20:40:05Z

Yes, confirmed working. Let's close and track finally/with somewhere else then. (Note that I didn't try to find which Python features don't work, just read that "It works only for 64-bit machines due to a small piece of x86-64 assembler for
the exception handling.", so after building for x86, I proceeded to test exactly exception handling ;-) ).

dpgeorge · 2013-12-29T23:38:02Z

By "exception handling" I actually meant the underlying C exception handling that I'm using (nlr_jump). You don't see it, but it's working when you get a Python error, eg at the command line just type "foo" and it'll give you a NameError. That's using nlr_jump, written in assembler.

In new hardware API, these classes implement master modes of interfaces, and "mode" parameter is not accepted. Trying to implement new HW API in terms of older pyb module leaves variuos corner cases: In new HW API, I2C(1) means "I2C #1 in master mode" (? depends on interpretation), while in old API, it means "I2C #1, with no settings changes". For I2C class, it's easy to make mode optional, because that's last positional param, but for SPI, there's "baudrate" after it (which is inconsistent with I2C, which requires "baudrate" to be kwonly-arg).

@pfalcon

Storing a chain of pbuf was an original design of @pfalcon's lwIP socket module. The problem with storing just one, like modlwip does is that "peer closed connection" notification is completely asynchronous and out of band. So, there may be following sequence of actions: 1. pbuf #1 arrives, and stored in a socket. 2. pbuf #2 arrives, and rejected, which causes lwIP to put it into a queue to re-deliver later. 3. "Peer closed connection" is signaled, and socket is set at such status. 4. pbuf #1 is processed. 5. There's no stored pbufs in teh socket, and socket status is "peer closed connection", so EOF is returned to a client. 6. pbuf #2 gets redelivered. Apparently, there's no easy workaround for this, except to queue all incoming pbufs in a socket. This may lead to increased memory pressure, as number of pending packets would be regulated only by TCP/IP flow control, whereas with previous setup lwIP had a global overlook of number packets waiting for redelivery and could regulate them centrally.

@pfalcon

Storing a chain of pbuf was an original design of @pfalcon's lwIP socket module. The problem with storing just one, like modlwip does is that "peer closed connection" notification is completely asynchronous and out of band. So, there may be following sequence of actions: 1. pbuf #1 arrives, and stored in a socket. 2. pbuf #2 arrives, and rejected, which causes lwIP to put it into a queue to re-deliver later. 3. "Peer closed connection" is signaled, and socket is set at such status. 4. pbuf #1 is processed. 5. There's no stored pbufs in teh socket, and socket status is "peer closed connection", so EOF is returned to a client. 6. pbuf #2 gets redelivered. Apparently, there's no easy workaround for this, except to queue all incoming pbufs in a socket. This may lead to increased memory pressure, as number of pending packets would be regulated only by TCP/IP flow control, whereas with previous setup lwIP had a global overlook of number packets waiting for redelivery and could regulate them centrally.

*: Heading to 2015-07-11

Closes micropython#1. Also adds TX and RX led support on the Arduino Zero.

added support for hardware temperature sensor

Patch 1

stm32 explicitily enable/disable vbus sense, fix warning

merging from upstream

update to adafruit master

asan considers that memcmp(p, q, N) is permitted to access N bytes at each of p and q, even for values of p and q that have a difference earlier. Accessing additional values is frequently done in practice, reading 4 or more bytes from each input at a time for efficiency, so when completing "non_exist<TAB>" in the repl, this causes a diagnostic: ==16938==ERROR: AddressSanitizer: global-buffer-overflow on address 0x555555cd8dc8 at pc 0x7ffff726457b bp 0x7fffffffda20 sp 0x7fff READ of size 9 at 0x555555cd8dc8 thread T0 #0 0x7ffff726457a (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a) #1 0x555555b0e82a in mp_repl_autocomplete ../../py/repl.c:301 #2 0x555555c89585 in readline_process_char ../../lib/mp-readline/re #3 0x555555c8ac6e in readline ../../lib/mp-readline/readline.c:513 #4 0x555555b8dcbd in do_repl /home/jepler/src/micropython/ports/uni #5 0x555555b90859 in main_ /home/jepler/src/micropython/ports/unix/ #6 0x555555b90a3a in main /home/jepler/src/micropython/ports/unix/m #7 0x7ffff619a09a in __libc_start_main ../csu/libc-start.c:308 #8 0x55555595fd69 in _start (/home/jepler/src/micropython/ports/uni 0x555555cd8dc8 is located 0 bytes to the right of global variable 'import_str' defined in '../../py/repl.c:285:23' (0x555555cd8dc0) of size 8 'import_str' is ascii string 'import ' Signed-off-by: Jeff Epler <jepler@gmail.com>

asan considers that memcmp(p, q, N) is permitted to access N bytes at each of p and q, even for values of p and q that have a difference earlier. Accessing additional values is frequently done in practice, reading 4 or more bytes from each input at a time for efficiency, so when completing "non_exist<TAB>" in the repl, this causes a diagnostic: ==16938==ERROR: AddressSanitizer: global-buffer-overflow on address 0x555555cd8dc8 at pc 0x7ffff726457b bp 0x7fffffffda20 sp 0x7fff READ of size 9 at 0x555555cd8dc8 thread T0 #0 0x7ffff726457a (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xb857a) micropython#1 0x555555b0e82a in mp_repl_autocomplete ../../py/repl.c:301 micropython#2 0x555555c89585 in readline_process_char ../../lib/mp-readline/re micropython#3 0x555555c8ac6e i 10000 n readline ../../lib/mp-readline/readline.c:513 micropython#4 0x555555b8dcbd in do_repl /home/jepler/src/micropython/ports/uni micropython#5 0x555555b90859 in main_ /home/jepler/src/micropython/ports/unix/ micropython#6 0x555555b90a3a in main /home/jepler/src/micropython/ports/unix/m micropython#7 0x7ffff619a09a in __libc_start_main ../csu/libc-start.c:308 micropython#8 0x55555595fd69 in _start (/home/jepler/src/micropython/ports/uni 0x555555cd8dc8 is located 0 bytes to the right of global variable 'import_str' defined in '../../py/repl.c:285:23' (0x555555cd8dc0) of size 8 'import_str' is ascii string 'import ' Signed-off-by: Jeff Epler <jepler@gmail.com>

RTCounter starts up the 2nd RTC unit (micropython#1) and demonstrates retrieving the currently elapsed seconds. Signed-off-by: RetiredWizard <github@retiredwizard.com>

Update zh_Latn_pinyin.po

Add board def for MagiClick S3

Add senseBox MCU-S2

RTCounter starts up the 2nd RTC unit (micropython#1) and demonstrates retrieving the currently elapsed seconds. Signed-off-by: RetiredWizard <github@retiredwizard.com>

Added additional pin names to pins.c for vidi_x board

Fix formatting.

Merged main to fork

A crash like the following occurs in the unix port: ``` Program received signal SIGSEGV, Segmentation fault. 0x00005555555a2d7a in mp_obj_module_set_globals (self_in=0x55555562c860 <ulab_user_cmodule>, globals=0x55555562c840 <mp_module_ulab_globals>) at ../../py/objmodule.c:145 145 self->globals = globals; (gdb) up micropython#1 0x00005555555b2781 in mp_builtin___import__ (n_args=5, args=0x7fffffffdbb0) at ../../py/builtinimport.c:496 496 mp_obj_module_set_globals(outer_module_obj, (gdb) micropython#2 0x00005555555940c9 in mp_import_name (name=824, fromlist=0x555555621f10 <mp_const_none_obj>, level=0x1) at ../../py/runtime.c:1392 1392 return mp_builtin___import__(5, args); ``` I don't understand how it doesn't happen on the embedded ports, because the module object should reside in ROM and the assignment of self->globals should trigger a Hard Fault. By checking VERIFY_PTR, we know that the pointed-to data is on the heap so we can do things like mutate it.

Um s2 boards

It was incorrect to NULL out the pointer to our heap allocated buffer in `reset`, because subsequent to framebuffer_reset, but while the heap was still active, we could call `get_bufinfo` again, leading to a fresh allocation on the heap that is about to be destroyed. Typical stack trace: ``` micropython#1 0x0006c368 in sharpdisplay_framebuffer_get_bufinfo micropython#2 0x0006ad6e in _refresh_display micropython#3 0x0006b168 in framebufferio_framebufferdisplay_background micropython#4 0x00069d22 in displayio_background micropython#5 0x00045496 in supervisor_background_tasks micropython#6 0x000446e8 in background_callback_run_all micropython#7 0x00045546 in supervisor_run_background_tasks_if_tick micropython#8 0x0005b042 in common_hal_neopixel_write micropython#9 0x00044c4c in clear_temp_status micropython#10 0x000497de in spi_flash_flush_keep_cache micropython#11 0x00049a66 in supervisor_external_flash_flush micropython#12 0x00044b22 in supervisor_flash_flush micropython#13 0x0004490e in filesystem_flush micropython#14 0x00043e18 in cleanup_after_vm micropython#15 0x0004414c in run_repl micropython#16 0x000441ce in main ``` When this happened -- which was inconsistent -- the display would keep some heap allocation across reset which is exactly what we need to avoid. NULLing the pointer in reconstruct follows what RGBMatrix does, and that code is a bit more battle-tested anyway. If I had a motivation for structuring the SharpMemory code differently, I can no longer recall it. Testing performed: Ran my complicated calculator program over multiple iterations without observing signs of heap corruption. Closes: micropython#3473

merge from adafruit

dpgeorge closed this as completed Dec 29, 2013

pfalcon reopened this Dec 29, 2013

pfalcon closed this as completed Dec 29, 2013

Chananyah mentioned this issue Jan 2, 2014

The unix version crashes #38

Closed

SteveGinter mentioned this issue Jun 4, 2014

Compilation error: gccollect.c:129:5: error: unknown type name 'regs_t' #659

Closed

nuraci mentioned this issue Apr 1, 2015

STM32 Nucleo F4x1 boards support #989

Closed

dhylands mentioned this issue Oct 30, 2015

stmhal: uart chars are being dropped, even when only at 115200 baud #1555

Open

robert-hh mentioned this issue Dec 16, 2015

Control-C isn't handled in all cases #1722

Closed

prusnak mentioned this issue Mar 31, 2016

UNIX port: add option to start repl after a script is executed. #1951

Closed

EcmaXp added a commit to OpenPythons/micropython-opencom-legacy that referenced this issue Aug 28, 2016

Merge pull request micropython#1 from micropython/master

c847e06

*: Heading to 2015-07-11

benalb mentioned this issue Sep 5, 2016

undefined reference to `mp_hal_stdout_tx_strn_cooked' #2390

Closed

turbinenreiter pushed a commit to turbinenreiter/micropython that referenced this issue Oct 16, 2016

atmel-samd: Support raw repl and soft reset to support ampy.

7fd84e9

Closes micropython#1. Also adds TX and RX led support on the Arduino Zero.

fisehara mentioned this issue Oct 24, 2016

esp8266: Crashes on concurrent WebREPL access #2537

Closed

Adam5Wu mentioned this issue Jan 12, 2017

ESP8266: Extremely bad clock accuracy #2724

Open

elfman2 mentioned this issue Jul 1, 2017

Enable btree duplicate key functionality #3187

Closed

aykevl referenced this issue in aykevl/micropython Oct 6, 2017

Merge pull request #1 from ajbaabf/nrf5_no_sdk

c1ffac9

added support for hardware temperature sensor

nickzoic pushed a commit to nickzoic/micropython that referenced this issue Aug 21, 2018

Merge pull request micropython#1 from arofarn/patch-1

1ac0cc1

Patch 1

mhe-fr mentioned this issue Sep 22, 2018

extmod/moducryptolib: implement RSA PKCS1 signature from mbed_tls #4169

Closed

nickzoic pushed a commit to nickzoic/micropython that referenced this issue Apr 18, 2020

Merge pull request micropython#1 from hathach/hierophect-update-tusb

409b5ff

stm32 explicitily enable/disable vbus sense, fix warning

nickzoic pushed a commit to nickzoic/micropython that referenced this issue Apr 18, 2020

Merge pull request micropython#1 from adafruit/master

db8365c

merging from upstream

nickzoic pushed a commit to nickzoic/micropython that referenced this issue Apr 18, 2020

Merge pull request micropython#1 from adafruit/master

7bdc8b3

update to adafruit master

stinos mentioned this issue Sep 22, 2020

Using LWIP on Windows #6463

Open

andreBook pushed a commit to andreBook/micropython that referenced this issue Mar 14, 2021

Add RSA PKCS micropython#1 v 1.5

30a998b

8bitgeek mentioned this issue Mar 20, 2021

ports/stm32/sdram.c STM32 16Mb 512K x 16bit x 2bank SDRAM Patch #7057

Closed

schamu mentioned this issue Sep 10, 2022

Thread and irq don't go #9271

Closed

This was referenced Sep 12, 2022

esp32: build error on windows with sed expression #9281

Open

ESP32 build error #9282

Closed

NiekIlmer referenced this issue in NiekIlmer/micropython Mar 8, 2023

Merge pull request #1 from NiekIlmer/dynamic_gatt

81a19a8

RetiredWizard pushed a commit to RetiredWizard/micropython that referenced this issue May 6, 2023

Merge pull request micropython#1 from ChenxiXiao/ChenxiXiao-ZH_PINYIN

47a901d

Update zh_Latn_pinyin.po

klora-sun mentioned this issue May 26, 2023

When compiling and porting stm32H750 to micro python, an error is reported #11629

Closed

natecarlson added a commit to natecarlson/rm520-micropython that referenced this issue Jul 6, 2023

Attempt micropython#1 to build a release workflow

ce1472d

RetiredWizard pushed a commit to RetiredWizard/micropython that referenced this issue Oct 24, 2023

Merge pull request micropython#1 from MakerM0/magiclick_s3_n4r2

0bf4f19

Add board def for MagiClick S3

RetiredWizard pushed a commit to RetiredWizard/micropython that referenced this issue Mar 8, 2024

Merge pull request micropython#1 from mariopesch/add-sensebox-mcus2

74ad733

Add senseBox MCU-S2

RetiredWizard pushed a commit to RetiredWizard/micropython that referenced this issue Jul 26, 2024

Merge pull request micropython#1 from Erasertm/board-support-for-vidi-x

d3bd06f

Added additional pin names to pins.c for vidi_x board

agatti mentioned this issue Sep 10, 2024

tools/verifygitlog.py: Apply stricter rules on git subject line. #15823

Merged

yoann-darche mentioned this issue Feb 19, 2025

esp32/PWM: Reduce inconsitencies between ports. #10854

Merged

RetiredWizard pushed a commit to RetiredWizard/micropython that referenced this issue Feb 25, 2025

Merge pull request micropython#1 from deisterhold/lilygo-tdongle-s3

e3c8907

Fix formatting.

dpgeorge mentioned this issue Mar 1, 2025

samd: Allow using default devices for UART, I2C and SPI. #16735

Merged

smurfix pushed a commit to M-o-a-T/micropython that referenced this issue Mar 26, 2025

Merge pull request micropython#1 from adafruit/main

c010cc2

Merged main to fork

smurfix pushed a commit to M-o-a-T/micropython that referenced this issue Mar 26, 2025

Merge pull request micropython#1 from tannewt/UM_S2_Boards

1a09996

Um s2 boards

smurfix pushed a commit to M-o-a-T/micropython that referenced this issue Mar 26, 2025

Merge pull request micropython#1 from adafruit/main

cf21c4d

merge from adafruit

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

try/except leads to infinite loop with growing memory usage #1

try/except leads to infinite loop with growing memory usage #1

try/except leads to infinite loop with growing memory usage #1

try/except leads to infinite loop with growing memory usage #1

Comments