8000 tools/mpy-cross-all.py: Helper tool to run mpy-cross on the entire project. by pfalcon · Pull Request #3057 · micropython/micropython · GitHub
[go: up one dir, main page]
Skip to content

tools/mpy-cross-all.py: Helper tool to run mpy-cross on the entire project. #3057

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

tools/mpy-cross-all.py: Helper tool to run mpy-cross on the entire project. #3057

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions tools/mpy-cross-all.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/usr/bin/env python3
import argparse
import os
import os.path

argparser = argparse.ArgumentParser(description="Compile all .py files to .mpy recursively")
argparser.add_argument("-o", "--out", help="output directory (default: input dir)")
argparser.add_argument("--target", help="select MicroPython target config")
argparser.add_argument("-mcache-lookup-bc", action="store_true", help="cache map lookups in the bytecode")
argparser.add_argument("dir", help="input directory")
args = argparser.parse_args()

TARGET_OPTS = {
"unix": "-mcache-lookup-bc",
"baremetal": "",
}

if not args.out:
args.out = args.dir

path_prefix_len = len(args.dir) + 1

for path, subdirs, files in os.walk(args.dir):
for f in files:
if f.endswith(".py"):
fpath = path + "/" + f
#print(fpath)
out_fpath = args.out + "/" + fpath[path_prefix_len:-3] + ".mpy"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think fpath[path_prefix_len:-3] is a robust way of stripping off the the prefix of fpath. Eg if I do os.walk('.') or os.walk('./') then I get the same output (same value for path) but len(args.dir) will differ by 1, and with './' the code will strip an extra character.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't reproduce the issue, if you pass a path with extra trailing slash, you would get double-slashes somewhere in the constructed paths. But let me strip it.

out_dir = os.path.dirname(out_fpath)
if not os.path.isdir(out_dir):
os.makedirs(out_dir)
cmd = "mpy-cross -v -v %s %s -o %s" % (TARGET_OPTS.get(args.target, ""), fpath, out_fpath)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a good idea to use the -s option to mpy-cross to strip the prefix off fpath. Eg if I pass in /some/long/path as the input argument to this script then I don't really want all that path taking up room in the .mpy file and then in qstrs when it gets loaded. When you freeze a whole directory you only really care about the path/filename relative to the root of that dir. So I suggest adding the option in the form -s fpath[path_prefix_len:]. See py/mkrules.mk line 117 to see how the Makefile does a similar thing.

Note that this has nothing to do with storing qstrs multiple times. It's just the name that is stored in the .mpy file for when exceptions are raised, to identify the file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I noticed too long paths, didn't give enough thought of how much to strip. Given that it repeats multiple times, even having an option to strip a barename might make sense. Will do as you suggest for now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added, but that likely will need tweaking later.

#print(cmd)
res = os.system(cmd)
assert res == 0
0