tools/mpy-cross-all.py: Helper tool to run mpy-cross on the entire project. #3057
Conversation
|
Proposed tool for #3040 |
|
Things to decide:
|
|
@dpgeorge : I'd like to have some progress with this (have a code which depends on it: pfalcon/notes-pico@e316d59). If there're no ideas how to improve this immediately, let's have it as is, and improve as ideas arise. |
| if f.endswith(".py"): | ||
| fpath = path + "/" + f | ||
| #print(fpath) | ||
| out_fpath = args.out + "/" + fpath[path_prefix_len:-3] + ".mpy" |
There was a problem hiding this comment.
I don't think fpath[path_prefix_len:-3] is a robust way of stripping off the the prefix of fpath. Eg if I do os.walk('.') or os.walk('./') then I get the same output (same value for path) but len(args.dir) will differ by 1, and with './' the code will strip an extra character.
There was a problem hiding this comment.
I can't reproduce the issue, if you pass a path with extra trailing slash, you would get double-slashes somewhere in the constructed paths. But let me strip it.
| out_dir = os.path.dirname(out_fpath) | ||
| if not os.path.isdir(out_dir): | ||
| os.makedirs(out_dir) | ||
| cmd = "mpy-cross -v -v %s %s -o %s" % (TARGET_OPTS.get(args.target, ""), fpath, out_fpath) |
There was a problem hiding this comment.
I think it's a good idea to use the -s option to mpy-cross to strip the prefix off fpath. Eg if I pass in /some/long/path as the input argument to this script then I don't really want all that path taking up room in the .mpy file and then in qstrs when it gets loaded. When you freeze a whole directory you only really care about the path/filename relative to the root of that dir. So I suggest adding the option in the form -s fpath[path_prefix_len:]. See py/mkrules.mk line 117 to see how the Makefile does a similar thing.
Note that this has nothing to do with storing qstrs multiple times. It's just the name that is stored in the .mpy file for when exceptions are raised, to identify the file.
There was a problem hiding this comment.
Yep, I noticed too long paths, didn't give enough thought of how much to strip. Given that it repeats multiple times, even having an option to strip a barename might make sense. Will do as you suggest for now.
There was a problem hiding this comment.
Added, but that likely will need tweaking later.
Will do. We probably should just standardize on underscores as Python's golden (well, silver, as it's less beautiful than hyphens) standard, the questions what to do with existing tools which use hyphens? |
We seriously should start to dogfeed, so this targets MicroPython, and by extension, Python3 ;-). |
I agree, things like tools/mpy-tool.py should be renamed to use underscore, or maybe even remove it altogether, ie mpytool.py. Probably not wise to rename mpy-cross, but that's a C program so can stay that way.
Sounds good. |
Well, that's a guaranteed way to resolve underscore vs hyphen controversy, but I'm not sure it scales to all cases ;-).
Well, and that's the problem - the base tool is called mpy-cross, but suddenly wrapper is called mpy_cross_all. So, let's go over it again: what's the real usecase for importing mpy_cross_all into another script? |
It's just the general principle of software engineering, to make reusable components. But think about it this way: go back in time to the point when deciding what to call mpy-cross, would it have been a sensible argument to say "let's call it mpy_cross just in case one day we want to write a wrapper script mpy_cross_XXX.py". I don't think such an argument is sensible, and at the same time calling it mpy-cross should not preculde making reusable components called mpy_cross_XXX.py |
No, such arguments aren't necessary sensible, but short and consistent arguments may as well be, like: 1. Given Python peculiarities, let's standardize on using underscores for everything. 2. Let's rename everything to be consistent. Ok, let's make a weaker form now: 1. Given Python peculiarities, let's standardize on using underscores for everything Python. 2. Let's rename everything Python to be consistent. Note this is very similar to the situation in #3025 - I'm trying to follow entailments which stem from the simple, consistent requirements, while you try to bring in peculiarities of current context, etc. Well, I perfectly know that simpl(istic) rules often aren't enough for real world, and you perfectly know that not following consistent principles leads to a mess. It's hard to tell which approach is better for each particular case. But it's fair to say that humans err on the side on keeping mess, and indeed, there's a lot of mess ensues ;-). |
|
Merged. Further evolution of this tool is certainly expected. |
|
Btw, just noticed that CPython has a module in stdlib for a similar purpose: https://docs.python.org/3/library/compileall.html |
…n-main Translations update from Weblate
No description provided.