Compiling Rust, GoLang, Zig, WebAssembly to MicroPython modules #15702
Replies: 9 comments 13 replies
-
|
Very cool work! I believe that building on, and expanding the usage of, .mpy native modules has a lot of potential for MicroPython. It enables creating modules with near-native performance (the bindings boundary does have a slight overhead), while preserving the "just mip install it" developer experience. Which is what people expect from Python on desktop, but is quite unique on embedded/microcontrollers. Having WASM also as an option here, allows code in more languages to be used. And over time we will likely see more and more libraries built in a WASM-first manner, which can then be utilized on MicroPython. Often this comes from "web land" (with very different constraints) but there are some areas where I think code reuse might be realistic - for example compression algorithms, hashing, file formats, digital signal processing, algorithms, numeric computing, machine learning, et.c. |
Beta Was this translation helpful? Give feedback.
-
|
It seems that in your approach, WASM is ahead-of-time compiled? That is excellent, because one avoids the FLASH/RAM footprint of an interpreter/runtime, and the difficulty of making two different runtimes interoperate. Also: Congratulations on Independence Day of Ukraine! А завзяття, праця щира свого ще докаже, Ще ся волі в Україні піснь гучна розляже. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the warm wishes and for quoting such a powerful line! As for the approach - you guessed it right. The wasm module is compiled AOT to the native target architecture. It will still impose some overhead, but it can be negligible in many cases. Deploying such modules via |
Beta Was this translation helpful? Give feedback.
-
|
Indeed, it does. I am happy to know many from Ukraine, and hope to go there soon. As far as I know, mip does currently not understand native modules and the target architecture / .mpy binary version. This is something I would like to see addressed (I use dynamic modules extensively in https://github.com/emlearn/emlearn-micropython/). But that is a bit wider scope - so ideally we would have a separate discussion thread for that. Btw, there is currently a blocker issue with non-trivial .mpy native modules (those that have BSS data) - #6769 - hopefully it can be resolved soon. |
Beta Was this translation helpful? Give feedback.
-
|
Sorry for derailing the discussion but, @jonnor do you reckon that it's the same issue I see in #15603? |
Beta Was this translation helpful? Give feedback.
-
|
I have also updated https://github.com/wasm3/embedded-wasm-apps Since 2021, many dependencies had breaking changes.. some significant efforts were required to get everything working again. |
Beta Was this translation helpful? Give feedback.
-
|
small update... 😁 |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, that is a known issue/limitation at this moment. Discussion, including some fixes/workaround at #5629 |
Beta Was this translation helpful? Give feedback.
-
|
Yes, thanks. I already got my first impressions from this process 😁 Looks like Also, as already discussed in other threads, it requires some automation. I was struggling to apply the same process to Also, I figured out that XIP (executing from flash directly) of native modules is still not happening even if modules are frozen. |
Beta Was this translation helpful? Give feedback.
-
|
On the positive note, here are the current results on ESP32: These are real WebAssembly apps built with different languages and running on ESP32-S3 as MicroPython import zig
zig.run() |
Beta Was this translation helpful? Give feedback.
-
|
Very Nice progress. |
Beta Was this translation helpful? Give feedback.
-
|
The progress stands on the shoulders of These are just normal MicroPython modules, so they can interact as usual. I don't think "deep" async is possible, but "shallow" async should be possible. |
Beta Was this translation helpful? Give feedback.
-
|
I was finally able to run something more complicated... MicroPython v1.24.0-preview.193.gba49efe6b on 2024-08-20; Generic ESP32S3 module with ESP32S3
Type "help()" for more information.
>>> import machine
>>> machine.freq(240000000)
>>> import coremark
>>> coremark.setup()
Running CoreMark 1.0...
Result: 135.796The declared CoreMark for ESP32-S3 is
|
Beta Was this translation helpful? Give feedback.
-
|
With some small optimizations, I'm getting |
Beta Was this translation helpful? Give feedback.
-
|
UPD: |
Beta Was this translation helpful? Give feedback.
-
|
You can now access and modify the wasm module memory: >>> import cpp
>>> cpp.setup()
🤩 C++ is running!
>>> cpp._memory[4096:4096+32]
bytearray(b' Blink\x00\xf0\x9f\xa4\xa9 C++ is running!\x00\n\x00\x00\x00')
>>> new_data = b"Hello C++ world"
>>> cpp._memory[4108:4108+len(new_data)] = new_data
>>> cpp.setup()
🤩 Hello C++ world |
Beta Was this translation helpful? Give feedback.
-
|
🚀🚀🚀 |
Beta Was this translation helpful? Give feedback.
-
|
While I think this is a very cool project, I'm wondering what the limitations of the combined toolchains are. More specifically: Python is an object oriented language and has nice predefined types like: list, tuple, dictionary, ... Also the source languages you mention like Rust, Go, Zig are object oriented, aren't they? I don't know webassembly. But I know that C is not an object oriented language. Can you confirm this? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, I think this is what you're looking for: https://component-model.bytecodealliance.org/design/why-component-model.html |
Beta Was this translation helpful? Give feedback.
-
|
Support of RISC-V ( |
Beta Was this translation helpful? Give feedback.
-
|
Congratulations ! :-) |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Back in 2021, I experimented with the concept of
embedded-wasm-apps.This allowed to run any of the WebAssembly supported languages on microcontrollers:
Recently, I discovered that MicroPython has the ability to compile plain C code into a dynamically loadable
.mpyfile. This revelation immediately sparked an idea:So, here is how to convert a
WASMfile into aMPYmodule and run it dynamically on different MCUs:https://github.com/vshymanskyy/wasm2mpy
Compile
Upload to the board
Run
Potential benefits of this approach are already described in embedded-wasm-apps.
Also, it will play nicely with
VfsMapFurther reading:
Beta Was this translation helpful? Give feedback.
All reactions