8000 Tool to compile entire subdirectory to .mpy files · Issue #3040 · micropython/micropython · GitHub
[go: up one dir, main page]

Skip to content

Tool to compile entire subdirectory to .mpy files #3040

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pfalcon opened this issue Apr 24, 2017 · 9 comments
Closed

Tool to compile entire subdirectory to .mpy files #3040

pfalcon opened this issue Apr 24, 2017 · 9 comments

Comments

@pfalcon
Copy link
Contributor
pfalcon commented Apr 24, 2017

I think we should think of developers' experience using .mpy files and add a tool to convert entire directory/project to .mpy form. For starters, it can be written in Python. It would accept an input directory, and optionally, output directory to .mpy files (in the expected hierarchical form).

@dpgeorge
Copy link
Member

I'm not quite sure I follow the use/need of this tool: isn't it just a 1-line shell command? Eg something like for file in $(find *.py); do mpy-cross $file; done

@pfalcon
Copy link
Contributor Author
pfalcon commented Apr 30, 2017

Apparently not a one-line shell command, because it needs to pass arguments down to mpy-cross, be able to compile .mpy in place or in a dedicated (non existing) dir, then likely few more features pop up. But it definitely going to be pretty simple (for starters) Python script, and that point that we should standardize on it.

@dpgeorge
Copy link
Member

Apparently not a one-line shell command, because it needs to pass arguments down to mpy-cross, be able to compile .mpy in place or in a dedicated (non existing) dir, then likely few more features pop up.

This is all work for a Makefile, or whatever build system the user is using. Also, a user would want to install the files on the board which requires copying, something that we don't yet provide, so that would be the first thing to do (eg in pyboard.py).

@pfalcon
Copy link
Contributor Author
pfalcon commented May 1, 2017

This is all work for a Makefile

Right, and Makefile just calls that tool, instead of containing bunch of glitchy shell magic.

Also, a user would want to install the files on the board which requires copying

In my case, I just want to easily and cleanly get compiled bytecode to run with unix port, see
#2622 (comment) . It shows that just trying .mpy didn't get me far, so I had to use frozen .mpy's, which is kind handled by shell/make magic, which not very unsurprisingly let me down, as comment spam at #2322 (comment) shows. Just imagine what an average user experiences trying it.

We really can't go for advanced tools like #3034, and leave large basic gaps in developer convenience inbetween.

I have a prototype of the tool, will post it for review tomorrow.

@dpgeorge
Copy link
Member
dpgeorge commented May 1, 2017

Right, and Makefile just calls that tool, instead of containing bunch of glitchy shell magic.

Shell is much more widely supported than a Python script to do the same job. And such a script, if it existed, would want to check timestamps to only recompile those .py files that needed it, exactly what a Makefile is for.

so I had to use frozen .mpy's, which is kind handled by shell/make magic, which not very unsurprisingly let me down, as comment spam at #2322 (comment) shows. Just imagine what an average user experiences trying it.

Frozen .mpy is used only in the coverage build of unix, but you can also use it in the normal build if you define FROZEN_MPY_DIR. Did that work? If not it's bug that needs fixing. As for other ports that directly support frozen .mpy file (ie esp8266, stmhal) they work fine: just put your .py files in the modules/ directory.

@pfalcon
Copy link
Contributor Author
pfalcon commented May 1, 2017

Shell is much more widely supported than a Python script to do the same job.

No. There's no standard shell - MacOSX, Wondows have different ones.

And such a script, if it existed, would want to check timestamps to only recompile those .py files that needed it

No such requirement for the initial version of the tool, mpy-cross is fast, so compiling couple of dozens of source files (on user request) is not a problem.

exactly what a Makefile is for.

Unfortunately, Makefile can't have a target which depends on "all files matching a pattern in a subdir, recursively, including any additions or deletions", which leads to workarounds of different level of smartness, with it being a pure wonder when they work, and when they don't (which happens regularly), it looks like comments above.

@pfalcon
Copy link
Contributor Author
pfalcon commented May 1, 2017

you can also use it in the normal build if you define FROZEN_MPY_DIR. Did that work?

Yes, eventually it did. Apparently, frozen_mpy.c wasn't properly generated at the try # 1.

@pfalcon pfalcon assigned pfalcon and unassigned pfalcon May 1, 2017
@pfalcon
Copy link
Contributor Author
pfalcon commented May 14, 2017

Implemented in PR above.

@pfalcon pfalcon closed this as completed May 14, 2017
@amotl
Copy link
Contributor
amotl commented Jul 28, 2019

Hi there,

we are just getting started with mpy-cross and so it took some time to actually find mpy_cross_all.py added through #3057.

Implemented in PR above.

As the reference to the actual PR somehow got lost, we just decided to re-add this reference for others also coming here.

Last but not least, we would like to send kudos regarding the amazing system you are building with MicroPython over there. Keep up the spirit!

With kind regards,
Andreas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
0