8000 tools/mpremote: Add `mpremote mip install` to install packages. · micropython/micropython@81f118d · GitHub
[go: up one dir, main page]
Skip to content

Commit 81f118d

Browse files
jimmojimmo
committed
tools/mpremote: Add mpremote mip install to install packages.
This supports the same package sources as the new `mip` tool. - micropython-lib (by name) - http(s) & github packages with json description - directly downloading a .py/.mpy file The version is specified with an optional `@version` on the end of the package name. The target dir, index, and mpy/no-mpy can be set through command line args. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
1 parent d8e9b73 commit 81f118d

File tree

6 files changed

+249
-25
lines changed

6 files changed

+249
-25
lines changed

‎docs/reference/mpremote.rst

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,14 @@ The full list of supported commands are:
146146
variable ``$EDITOR``). If the editor exits successfully, the updated file will
147147
be copied back to the device.
148148

149+
- install packages from :term:`micropython-lib` (or GitHub) using the ``mip`` tool:
150+
151+
.. code-block:: bash
152+
153+
$ mpremote mip install <packages...>
154+
155+
See :ref:`packages` for more information.
156+
149157
- mount the local directory on the remote device:
150158

151159
.. code-block:: bash
@@ -269,3 +277,9 @@ Examples
269277
mpremote cp -r dir/ :
270278
271279
mpremote cp a.py b.py : + repl
280+
281+
mpremote mip install aioble
282+
283+
mpremote mip install github:org/repo@branch
284+
285+
mpremote mip install --target /flash/third-party functools

‎docs/reference/packages.rst

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -78,17 +78,17 @@ The :term:`mpremote` tool also includes the same functionality as ``mip`` and
7878
can be used from a host PC to install packages to a locally connected device
7979
(e.g. via USB or UART)::
8080

81-
$ mpremote install pkgname
82-
$ mpremote install pkgname@x.y
83-
$ mpremote install http://example.com/x/y/foo.py
84-
$ mpremote install github:org/repo
85-
$ mpremote install github:org/repo@branch-or-tag
81+
$ mpremote mip install pkgname
82+
$ mpremote mip install pkgname@x.y
83+
$ mpremote mip install http://example.com/x/y/foo.py
84+
$ mpremote mip install github:org/repo
85+
$ mpremote mip install github:org/repo@branch-or-tag
8686

8787
The ``--target=path``, ``--no-mpy``, and ``--index`` arguments can be set::
8888

89-
$ mpremote install --target=/flash/third-party pkgname
90-
$ mpremote install --no-mpy pkgname
91-
$ mpremote install --index https://host/pi pkgname
89+
$ mpremote mip install --target=/flash/third-party pkgname
90+
$ mpremote mip install --no-mpy pkgname
91+
$ mpremote mip install --index https://host/pi pkgname
9292

9393
Installing packages manually
9494
----------------------------

‎tools/mpremote/README.md

Lines changed: 24 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -11,23 +11,28 @@ This will automatically connect to the device and provide an interactive REPL.
1111

1212
The full list of supported commands are:
1313

14-
mpremote connect <device> -- connect to given device
15-
device may be: list, auto, id:x, port:x
16-
or any valid device name/path
17-
mpremote disconnect -- disconnect current device
18-
mpremote mount <local-dir> -- mount local directory on device
19-
mpremote eval <string> -- evaluate and print the string
20-
mpremote exec <string> -- execute the string
21-
mpremote run <file> -- run the given local script
22-
mpremote fs <command> <args...> -- execute filesystem commands on the device
23-
command may be: cat, ls, cp, rm, mkdir, rmdir
24-
use ":" as a prefix to specify a file on the device
25-
mpremote repl -- enter REPL
26-
options:
27-
--capture <file>
28-
--inject-code <string>
29-
--inject-file <file>
30-
mpremote help -- print list of commands and exit
14+
mpremote connect <device> -- connect to given device
15+
device may be: list, auto, id:x, port:x
16+
or any valid device name/path
17+
mpremote disconnect -- disconnect cur F438 rent device
18+
mpremote mount <local-dir> -- mount local directory on device
19+
mpremote eval <string> -- evaluate and print the string
20+
mpremote exec <string> -- execute the string
21+
mpremote run <file> -- run the given local script
22+
mpremote fs <command> <args...> -- execute filesystem commands on the device
23+
command may be: cat, ls, cp, rm, mkdir, rmdir
24+
use ":" as a prefix to specify a file on the device
25+
mpremote repl -- enter REPL
26+
options:
27+
--capture <file>
28+
--inject-code <string>
29+
--inject-file <file>
30+
mpremote mip install <package...> -- Install packages (from micropython-lib or third-party sources)
31+
options:
32+
--target <path>
33+
--index <url>
34+
--no-mpy
35+
mpremote help -- print list of commands and exit
3136

3237
Multiple commands can be specified and they will be run sequentially. Connection
3338
and disconnection will be done automatically at the start and end of the execution
@@ -73,3 +78,5 @@ Examples:
7378
mpremote cp :main.py .
7479
mpremote cp main.py :
7580
mpremote cp -r dir/ :
81+
mpremote mip install aioble
82+
mpremote mip install github:org/repo@branch

‎tools/mpremote/mpremote/main.py

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@
3636
do_resume,
3737
do_soft_reset,
3838
)
39+
from .mip import do_mip
3940
from .repl import do_repl
4041

4142
_PROG = "mpremote"
@@ -162,6 +163,29 @@ def argparse_filesystem():
162163
return cmd_parser
163164

164165

166+
def argparse_mip():
167+
cmd_parser = argparse.ArgumentParser(
168+
description="install packages from micropython-lib or third-party sources"
169+
)
170+
_bool_flag(cmd_parser, "mpy", "m", True, "download as compiled .mpy files (default)")
171+
cmd_parser.add_argument(
172+
"--target", type=str, required=False, help="destination direction on the device"
173+
)
174+
cmd_parser.add_argument(
175+
"--index",
176+
type=str,
177+
required=False,
178+
help="package index to use (defaults to micropython-lib)",
179+
)
180+
cmd_parser.add_argument("command", nargs=1, help="mip command (e.g. install)")
181+
cmd_parser.add_argument(
182+
"packages",
183+
nargs="+",
184+
help="list package specifications, e.g. name, name@version, github:org/repo, github:org/repo@branch",
185+
)
186+
return cmd_parser
187+
188+
165189
def argparse_none(description):
166190
return lambda: argparse.ArgumentParser(description=description)
167191

@@ -216,6 +240,10 @@ def argparse_none(description):
216240
do_filesystem,
217241
argparse_filesystem,
218242
),
243+
"mip": (
244+
do_mip,
245+
argparse_mip,
246+
),
219247
"help": (
220248
do_help,
221249
argparse_none("print help and exit"),

‎tools/mpremote/mpremote/mip.py

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
# Micropython package installer
2+
# Ported from micropython-lib/micropython/mip/mip.py.
3+
# MIT license; Copyright (c) 2022 Jim Mussared
4+
5+
import urllib.error
6+
import urllib.request
7+
import json
8+
import tempfile
9+
import os
10+
11+
from .commands import CommandError
12+
13+
14+
_PACKAGE_INDEX = "https://micropython.org/pi/v2"
15+
16+
17+
# This implements os.makedirs(os.dirname(path))
18+
def _ensure_path_exists(pyb, path):
19+
import os
20+
21+
split = path.split("/")
22+
23+
# Handle paths starting with "/".
24+
if not split[0]:
25+
split.pop(0)
26+
split[0] = "/" + split[0]
27+
28+
prefix = ""
29+
for i in range(len(split) - 1):
30+
prefix += split[i]
31+
if not pyb.fs_exists(prefix):
32+
pyb.fs_mkdir(prefix)
33+
prefix += "/"
34+
35+
36+
def _rewrite_url(url, branch=None):
37+
if not branch:
38+
branch = "HEAD"
39+
if url.startswith("github:"):
40+
url = url[7:].split("/")
41+
url = (
42+
"https://raw.githubusercontent.com/"
43+
+ url[0]
44+
+ "/"
45+
+ url[1]
46+
+ "/"
47+
+ branch
48+
+ "/"
49+
+ "/".join(url[2:])
50+
)
51+
return url
52+
53+
54+
def _download_file(pyb, url, dest):
55+
try:
56+
with urllib.request.urlopen(url) as src:
57+
fd, path = tempfile.mkstemp()
58+
try:
59+
with os.fdopen(fd, "wb") as f:
60+
f.write(src.read())
61+
print("Copying:", dest)
62+
_ensure_path_exists(pyb, dest)
63+
pyb.fs_put(path, dest)
64+
finally:
65+
os.unlink(path)
66+
except urllib.error.HTTPError as e:
67+
if e.status == 404:
68+
raise CommandError(f"File not found: {url}")
69+
else:
70+
raise CommandError(f"Error {e.status} requesting {url}")
71+
except urllib.error.URLError as e:
72+
raise CommandError(f"{e.reason} requesting {url}")
73+
74+
75+
def _install_json(pyb, package_json_url, index, target, version, mpy):
76+
try:
77+
with urllib.request.urlopen(_rewrite_url(package_json_url, version)) as response:
78+
package_json = json.load(response)
79+
except urllib.error.HTTPError as e:
80+
if e.status == 404:
81+
raise CommandError(f"Package not found: {package_json_url}")
82+
else:
83+
raise CommandError(f"Error {e.status} requesting {package_json_url}")
84+
except urllib.error.URLError as e:
85+
raise CommandError(f"{e.reason} requesting {package_json_url}")
86+
for target_path, short_hash in package_json.get("hashes", ()):
87+
fs_target_path = target + "/" + target_path
88+
file_url = f"{index}/file/{short_hash[:2]}/{short_hash}"
89+
_download_file(pyb, file_url, fs_target_path)
90+
for target_path, url in package_json.get("urls", ()):
91+
fs_target_path = target + "/" + target_path
92+
_download_file(pyb, _rewrite_url(url, version), fs_target_path)
93+
for dep, dep_version in package_json.get("deps", ()):
94+
_install_package(pyb, dep, index, target, dep_version, mpy)
95+
96+
97+
def _install_package(pyb, package, index, target, version, mpy):
98+
if (
99+
package.startswith("http://")
100+
or package.startswith("https://")
101+
or package.startswith("github:")
102+
):
103+
if package.endswith(".py") or package.endswith(".mpy"):
104+
print(f"Downloading {package} to {target}")
105+
_download_file(
106+
pyb, _rewrite_url(package, version), target + "/" + package.rsplit("/")[-1]
107+
)
108+
return
109+
else:
110+
if not package.endswith(".json"):
111+
if not package.endswith("/"):
112+
package += "/"
113+
package += "package.json"
114+
print(f"Installing {package} to {target}")
115+
else:
116+
if not version:
117+
version = "latest"
118+
print(f"Installing {package} ({version}) from {index} to {target}")
119+
120+
mpy_version = "py"
121+
if mpy:
122+
pyb.exec("import sys")
123+
mpy_version = (
124+
int(pyb.eval("getattr(sys.implementation, '_mxpy', 0) & 0xFF").decode()) or "py"
125+
)
126+
127+
package = f"{index}/package/{mpy_version}/{package}/{version}.json"
128+
129+
_install_json(pyb, package, index, target, version, mpy)
130+
131+
132+
def do_mip(state, args):
133+
state.did_action()
134+
135+
if args.command[0] == "install":
136+
state.ensure_raw_repl()
137+
138+
for package in args.packages:
139+
version = None
140+
if "@" in package:
141+
package, version = package.split("@")
142+
143+
print("Install", package)
144+
145+
if args.index is None:
146+
args.index = _PACKAGE_INDEX
147+
148+
if args.target is None:
149+
state.pyb.exec("import sys")
150+
lib_paths = state.pyb.eval("'\\n'.join(p for p in sys.path if p.endswith('/lib'))").decode().split('\n')
151+
if lib_paths and lib_paths[0]:
152+
args.target = lib_paths[0]
153+
else:
154+
raise CommandError(
155+
"Unable to find lib dir in sys.path, use --target to override"
156+
)
157+
158+
if args.mpy is None:
159+
args.mpy = True
160+
161+
try:
162+
_install_package(
163+
state.pyb, package, args.index.rstrip("/"), args.target, version, args.mpy
164+
)
165+
except CommandError:
166+
print("Package may be partially installed")
167+
raise
168+
print("Done")

‎tools/pyboard.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -476,6 +476,13 @@ def get_time(self):
476476
t = str(self.eval("pyb.RTC().datetime()"), encoding="utf8")[1:-1].split(", ")
477477
return int(t[4]) * 3600 + int(t[5]) * 60 + int(t[6])
478478

479+
def fs_exists(self, src):
480+
try:
481+
self.exec_("import uos\nuos.stat(%s)" % (("'%s'" % src) if src else ""))
482+
return True
483+
except PyboardError:
484+
return False
485+
479486
def fs_ls(self, src):
480487
cmd = (
481488
"import uos\nfor f in uos.ilistdir(%s):\n"

0 commit comments

Comments
 (0)
0