unix port: Get x86_64 binary under 400000 bytes. #2908
Comments
|
To avoid the "MicroPython is big bloat" impression, might it not make sense to implement a custom build service like they have for nodemcu? It seems to me that this would allow more freedom to add nice-to-have features while still potentially providing very compact builds. |
This disables the help in the unix port by default but keeps the configuration option in place to allow users to enable it if desired.
|
It seems that 400000 bytes is a rather arbitrary metric to go by. With ports like microbit and 512k-esp8266 modules there are hard size constraints, and the features that are enabled are because of the type of hardware and intended users. Eg microbit has a speech synthesiser and a small non-fatfs filesystem, but not complex numbers; and esp8266 has socket and ussl modules because its main hardware feature is wifi. But the unix port has no real size constraint and the features it wants are not well defined (eg does it need btree?). Anyway, the simplest thing to do to reduce unix binary size in disable assertions. That gets it down to 378375 bytes for me. |
|
One use for the Unix port is as a fast means of testing MicroPython code before transferring to the target platform. For example you might want to test with large datasets which would be impractical on the target. On this basis I'd say include as much as possible; definitely complex numbers and btree as features typical of types of code you might want to test. |
It has, as explained in https://github.com/micropython/micropython/wiki/ContributorGuidelines
Also pretty well defined - we want it to provide most of the features uPy provides (while not growing further ;-) ).
Yep, as was discussed. We want to provide solution for full-stack development, which requires DB support, and we want that to be fully portable from the MCUs up, hence btree module.
It should be pretty obvious - it's now ~408K, and as a balancing measure against patches which propose to bloat it further, here's alternative proposal. |
Well, maybe we should do that soon, after making another pass on asserts to make sure we converted relevant ones to exceptions. But my original motivation is again, to ring a bell to all the new folks who exhibit following pattern: "What a little cute thing! Let's do something to it!" And that "something" turns out to be "grow it bigger, bigger, BIGGER!" Sorry guys, don't steal the fun from the future generations - they want to discover this cute little thing too. In that regard, I think that codesize-grow check for the minimal port was a helpful thing, but it doesn't fail enough patches, so I'd be looking forward to enable it for unix port too. |
Yes, but it's difficult to measure code size when dynamically linked libraries are in play. Eg how big is CPython3 really? Why don't we disable axtls and use a dynamic library instead?
How about enabling framebuf module then? Otherwise one has to run the coverage build to test scripts that use framebuf. |
So, we're looking in the opposite direction - not rely on any dynamic library and have a fully static build. libffi is the last dependency which is scheduled to go: #2191 . The reason for that is well known: there's no guarantee that there's a needed dynamic library on the target system, and no guarantee that it can be built easily, or built at all. (Try your router, try dozen of other embedded Linux systems). And "MicroPython runs everywhere".
Well, that's not how "full stack" idea works. It should provide a solution for data storage and UI. We already make a compromise on data storage part by not providing a full SQL database, and there's one default way to make UIs - with HTML over network. Besides, does framebuf support linuxfb, OpenGL, OpenGLES, OpenVG? No? That's what I thought ;-).
Yes, we came to such solution, and it works pretty well. Can start to think about offloading the production build from assertions, etc. indeed. |
So then people should realise this when complaining that uPy is "bloated".
A framebuf is exactly what you need for a UI on a device that does not have a network! So it's needed in the default build.
Of course it does. It also supports ASCII. You just copy the data out to the hardware. |
But hardware which runs unix ports has a network in like 95% of cases, while headless in probably 50% of cases.
Well, this ticket discusses how to decrease the size, not how to increase it ;-). |
|
@pfalcon I'd like to close this issue as I don't see it ever being resolved to any level of satisfaction. Currently unix can be built without debugging to get it (a lot) under 400,000. The 64 and 32 bit version have different sizes (32 bit is 360k with debugging and 340k without debugging) and so does the nanboxing version. In the future adding support for things like VFS posix and mbedtls (yes we need it, or hope axtls adds features very soon) are going to change the size such that 400,000 doesn't have a real meaning. |
|
Why this ticket is worse than 300 other open? It waits its turn, as many other. There're action items here: disable assertions, establish Travis code size check. 400000 is just a figure to keep the title short, the real meaning of this ticket is "keep uPy from growing, find ways to shrink it instead". That's not really attainable, and it'll keep growing, but tickets like this should remind that it should be well-considered exceptions, not extensive/explosive growth. |
It's distracting becuse, as I said, there's no clear goal (especially if 400000 is just a figure to keep the title short!). And 400000 is anyway already met by 32-bit port with debug enabled.
It shouldn't be a ticket because it can never be closed. |
|
Tickets should be a reminders of work to do, and there's work to do. The title clearly states it's for x86_64 port. Of course, if you really want to close it, feel free to it, I don't see how that helps with the issues we discuss. |
Isn't that sort of covered by the guidelines, the code size check in the build process, and (less visibly, but only at first sight) because it has been mentioned often in the issue comments? |
I'd like individual tickets to have definite goals that can be realistically reached, rather than a vauge description like "make MicroPython better", which of course everyone is trying to do.
Yes, sorry, I missed that. So, assertions for unix build were disabled in 1e6fd9f, and 64-bit executable is now 377000. |
|
Err, but did you go thru assertions one by one to check which need to be converted e.g. to exceptions? Need to do this was the reason why this idea waited in queue. |
Yep, and a lot people still manage to miss it, so this was made to ease the reference to it. And the code size check exists only for minimal build, this ticket proposes to add it for unix port too (the obvious drawback is that 95% of PRs will fail). |
I checked most of them, and all I found was d0db93c. Likely there are a few assertions remaining that should be converted. |
|
In the end binary size seems to be meaningless on unix, since the latest gcc v7.1.1 compiles the unix port to 43300 bytes, up from 377000 (64-bit build). |
|
Of course, there's a meaning, it's just there's bloat everywhere, rampaging. And we should not succumb to it. |
|
More info on the "latest" gcc version: it's Arch Linux's gcc-multilib 7.1.1-4, built on 1/07/2017. It seems to use position independent code by default, for both 32-bit and 64-bit x86 archs. Code size is increased by about 15%. The previous Arch Linux version (7.1.1-3) doesn't have this issue. |
|
I'd like to reopen this issue, to remind everyone (including myself) that the code size growth, and without reflecting on that and controlling that (the purpose of this ticket), the growth will uncontrollable. So, some thought and action should be put into ways to compensate the growth. (E.g. I opened #3319 and dumping stuff there). |
|
I'll close this issue again because I don't see how it's useful to keep it open. |
Uh oh!
There was an error while loading. Please reload this page.
Time to do some clean up and optimizations to avoid "MicroPython is big bloat" impression ;-).
The text was updated successfully, but these errors were encountered: