tools/mpr: Add new command-line utility to interact with remote device. #6375

dpgeorge · 2020-08-26T07:44:54Z

This PR builds on and supersedes #3034, adding a new tool called mpr (name subject to change).

The new tool provides a set of utilities to interact with and automate a MicroPython device over a serial connection It's considered a more user-friendly version of pyboard.py (and uses that module to create the connection to the device). So far the commands supported are:

mpr                         -- auto-detect, connect and enter REPL

mpr <device-shortcut>       -- connect to given device
mpr connect <device>        -- connect to given device
mpr mount <local-dir>       -- mount local directory on device

mpr eval <string>           -- evaluate and print the string
mpr exec <string>           -- execute the string
mpr fs <command> <args...>  -- execute filesystem commands on the device
mpr repl                    -- enter REPL
mpr run <script>            -- run the given local script

Multiple commands can be specified and they will be run sequentially. The serial device will be automatically detected if not specified. If no actionis specified then the REPL will be entered.

Example usage:

mpr
mpr a1
mpr connect /dev/ttyUSB0 repl
mpr ls
mpr a1 ls
mpr exec "import micropython; micropython.mem_info()"
mpr eval 1/2 eval 3/4
mpr mount .
mpr mount . exec "import local_script"
mpr ls
mpr cat boot.py

The mount command is quite powerful and is the main thing inherited from #3034. For example the bootloader on an stm32 board can be updated by using (at the PC prompt):

$ cd ports/stm32/mboot
$ make BOARD=PYBD_SF2
$ mpr mount . run fwupdate.py exec 'update_mboot("build-PYBD_SF2/firmware.dfu")'
<
8000
/pre>

That will:

automatically detect the connected device (eg /dev/ttyACM0)
mount the PC's local filesystem on the device (and chdir to it)
run the local fwupdate.py script which defines some functions
execute the update_mboot(...) function (defined in fwupdate.py) to update the bootloader
when update_mboot(...) is executing it reads in the firmware file from the local PC's filesystem

TODO:

decide on name of the tool, currently mpr means "MicroPython remote"
publish to pip, including something that works on Windows

UnexpectedMaker · 2020-08-26T23:22:08Z

This is sooo awesome! Thanks heaps for spending time getting this working.
I personally don't mind mpr for the name.

cwalther · 2020-09-04T09:56:16Z

Cool! Looks like a worthy successor to mprepl.py. Everything I tried works.

I’d appreciate a --help output to make the commands a bit more discoverable.

The preconfigured device shortcuts/autodetection are a bit Linux-centric, useful for me would be an additional /dev/cu.SLAB_USBtoUART for Silicon-Labs-based devices on macOS.

Now I just need to fix that WebREPL protocol…

rolandvs · 2020-09-05T21:15:39Z

This really rocks! This helps testing hardware even quicker!

robert-hh · 2020-09-18T06:31:00Z

Looks very good. Just a little note: You are using assignment expressions at three places. That does not work with python3 < 3.8, which is on some systems (like my debian 10) still the default version. Considering about what to change, I changed the script.

dpgeorge · 2020-09-18T06:34:40Z

You are using assignment expressions at three places. That does not work with python3 < 3.8,

Ok, noted. I'll remove use of them.

UnexpectedMaker · 2020-09-23T07:52:45Z

How's progress on this going @dpgeorge ? Is it really just waiting on choosing a name? or are there more outstanding issues to be worked on before this can get merged?

amirgon · 2020-09-23T08:12:39Z

Could be nice to be able to mount remote FS on ESP32 over socket.
If I understand correctly, this tool is currently limited to serial.

cwalther · 2020-09-23T11:44:57Z

I agree, which is why I once worked on adding WebREPL support. The roadblock there was that the WebREPL protocol is flawed, it is incapable of transferring arbitrary binary data, which cannot be fixed compatibly in the current design, and I haven’t spent the effort to come up with a redesign proposal.

amirgon · 2020-09-23T12:39:12Z

agree, which is why I once worked on adding WebREPL support.

I was thinking of a service that runs on the ESP32 side (started by boot.py), waits for Wifi and listens on some port for incoming connections. Once connected, it can automatically mount remote FS.

cwalther · 2020-09-23T15:32:50Z

Since this tool runs over the REPL, that would essentially be a Telnet service. Which seems supported already, so it might just work?

dpgeorge · 2020-09-24T02:35:00Z

How's progress on this going @dpgeorge ? Is it really just waiting on choosing a name?

Yes, it's really just picking a name. Maybe mpremote, then one can easily alias it to something shorter (eg mp) if desired.

dpgeorge · 2020-09-24T02:36:18Z

Could be nice to be able to mount remote FS on ESP32 over socket.
If I understand correctly, this tool is currently limited to serial.

The idea is that this tool will work with any serial IO, eg USB, UART, I2C, CAN, telnet, etc. It just needs a way of getting bytes in and out. At the moment USB/UART/serial should work, along with telnet.

robert-hh · 2020-10-01T15:01:34Z

I mad a small change to the Windows readchar function to a) support Home/End/PgUP&PgDown b) not stop with an unknown function key.

    def readchar(self):
        if msvcrt.kbhit():
            ch = msvcrt.getch()
            while ch in b"\x00\xe0":  # arrow or function key prefix?
                if not msvcrt.kbhit():
                    return None
                ch = msvcrt.getch()  # second call returns the actual key code
                try:
                    ch = b"\x1b[" + {b"H": b"A", b"P": b"B", b"M": b"C", b"K": b"D",
                                     b"G": b"H", b"O": b"F", b"I": b"5~", b"Q": b"6~"}[ch]
                except KeyError:
                    return None
            return ch

scy · 2021-02-23T11:24:46Z

Please be careful with the syntax for having multiple commands in one invocation. Right now, there’s no delimiter between them. In my opinion, this looks strange (e.g. in your example mpr mount . exec "import local_script"), but more importantly it will cause issues with commands that accept a variable number of arguments (e.g. something like a “copy” command, in case that will be implemented at some point, or the fs command).

Of course using ; as the delimiter would come with its own set of issues, especially that it usually has to be escaped in the shell, but I can’t think of a better alternative. It would probably have to be its own shell argument then (to simplify parsing in mpr), like

mpr mount . \; run fwupdate.py \; exec 'update_mboot("build-PYBD_SF2/firmware.dfu")'

tve · 2021-02-25T16:24:47Z

Have you considered using pyinvoke? mpr could use it as a library and thereby allow users to write pyinvoke tasks files in their projects that use the mpr functionality to do things, plus combine with other tasks as well.

dpgeorge · 2021-03-11T05:35:33Z

I mad a small change to the Windows readchar function to a) support Home/End/PgUP&PgDown b) not stop with an unknown function key.

Thanks, that changed is pushed here.

Please be careful with the syntax for having multiple commands in one invocation. Right now, there’s no delimiter between them. In my opinion, this looks strange

This tool is intended to be optimised for ergonomics (fast typing) and I think not having a delimiter when not needed helps that aim. Once you get used to it it seems quite natural, IMO.

but more importantly it will cause issues with commands that accept a variable number of arguments (e.g. something like a “copy” command

Copy is already supported. It consumes all its arguments.

Of course using ; as the delimiter would come with its own set of issues, especially that it usually has to be escaped in the shell, but I can’t think of a better alternative

Yes ; would be a natural choice but won't work.

I think it would be good to support optional delimiters that can be used when needed to disambiguate chains of commands. Maybe something like --?

dpgeorge · 2021-03-11T05:36:06Z

Have you considered using pyinvoke?

I've never seen this library. Might be a good way to customise things, will look into it.

robert-hh · 2021-03-15T07:24:38Z

Just before I forget it again: I played a little bit more with the Windows terminal window and extended the translation list for keys to all combinations of cursor + modifier that issued a key. No functions keys included:

    def readchar(self):
        if msvcrt.kbhit():
            ch = msvcrt.getch()
            while ch in b"\x00\xe0":  # arrow or function key prefix?
                if not msvcrt.kbhit():
                    return None
                ch = msvcrt.getch()  # second call returns the actual key code
                try:
                    ch = b"\x1b[" + {b"H": b"A",  # UP
                                     b"P": b"B",  # DOWN
                                     b"M": b"C",  # RIGHT
                                     b"K": b"D",  # LEFT
                                     b"G": b"H",  # POS1
                                     b"O": b"F",  # END
                                     b"Q": b"6~",  # PGDN
                                     b"I": b"5~",  # PGUP
                                     b"s": b"1;5D",  # CTRL-LEFT,
                                     b"t": b"1;5C",  # CTRL-RIGHT,
                                     b"\x8d": b"1;5A",  #  CTRL-UP,
                                     b"\x91": b"1;5B",  # CTRL-DOWN,
                                     b"w" : b"1;5H",  # CTRL-POS1
                                     b"u" : b"1;5F", # CTRL-END
                                     b"\x98": b"1;3A",  #  ALT-UP,
                                     b"\xa0": b"1;3B",  # ALT-DOWN,
                                     b"\x9d": b"1;3C",  #  ALT-RIGHT,
                                     b"\x9b": b"1;3D",  # ALT-LEFT,
                                     b"\x97": b"1;3H",  #  ALT-POS1,
                                     b"\x9f": b"1;3F",  # ALT-END,
                                     b"S" : b"3~",  # DEL,
                                     b"\x93": b"3;5~",  # CTRL-DEL
                                     b"R" : b"2~",  # INS
                                     b"\x92": b"2;5~",  # CTRL-INS
                                     b"\x94" : b"Z",  # Ctrl-Tab = BACKTAB,
                                    }[ch]
                except KeyError:
                    return None
            return ch

I also added port shortcuts for a few 'COMx' devices as 'cx', like c3 for COM3, etc.

glenn20 · 2021-04-29T05:14:18Z

For what it's worth, a tiny fix (for what seems to be unused code anyway :-):
mpr:100 if self.use_second_port:

should be:
mpr:100 if use_second_port:

Loving mpr. Thanks so much for this tool @dpgeorge.

glenn20 · 2021-05-04T16:22:25Z

I am loving mpr - such a clever solution to the micropython dev workflow.

I do find that on the ESP32, filesystem operations fail in an ugly way if I type a single key while they are in progress (eg. while reading a file from the /remote filesystem). I know, I could just be more careful - but I do have a proposed fix.

Cause:

RemoteCommand.begin() discards any incoming chars before sending ctrl-X - BUT only on the pyboard!

Fix (mostly):

(Sorry - I can't post a PR against this PR ;-).

In RemoteCommand.init() after:

        except:
            # TODO sys.stdio doesn't support polling
            import sys
            self.fout = sys.stdout.buffer
            self.fin = sys.stdin.buffer

insert (sys.stdio does support polling on the ESP32 - I don't know which boards don't have this capability):

            import select
            self.poller = select.poll()
            self.poller.register(self.fin, select.POLLIN)

In RemoteCommand.begin(): after

        if hasattr(self.fin, 'any'):
            while self.fin.any():
                self.fin.readinto(buf4, 1)

insert:

        elif hasattr(self, 'poller'):
            while self.poller.poll(0):
                self.fin.readinto(buf4, 1)

This make remote filesystem operations robust against most casual keystrokes, but not entirely.

More Complete Fix:

After RemoteCommand.begin() sends ctrl-X to the serial port there may be some characters still being sent from the computer that will be swallowed on the next data read.

A suggested further fix:

In RemoteCommand.begin(): after sending ctrl-X and type char, wait for an ack from mpr (discarding chars till the expected ack).
In SerialIntercept._check_input(): Send an ack char after receiving ctrl-X and type char.
- I used ctrl-X as the ack char - and now the filesystem ops are more robust against my impatient typing.

The downside is a little extra handshaking latency. That could be alleviated with a more complex fix by looking for the ack just before the first read in each operation. With this fix, the clearing of chars before sending ctrl-X can be deleted.

dpgeorge · 2021-05-06T05:41:55Z

I played a little bit more with the Windows terminal window and extended the translation list for keys to all combinations of cursor + modifier that issued a key.

Now added, thanks!

I also added port shortcuts for a few 'COMx' devices as 'cx', like c3 for COM3, etc.

Also added.

For what it's worth, a tiny fix (for what seems to be unused code anyway :-):
mpr:100 if self.use_second_port:

should be:
mpr:100 if use_second_port:

Fixed.

This make remote filesystem operations robust against most casual keystrokes, but not entirely.

Thanks for that @glenn20 , I've applied the fix using the poller for all platforms (they should now all support stdin polling). Can fix properly using "ctrl-X" at a later date.

dpgeorge · 2021-05-06T07:14:10Z

This tool is now available for installation via pip:

$ pip install mpremote

Then it's installed as the executable mpremote, which I personally symlink as mp.

peterhinch · 2021-05-06T17:47:22Z

This works really well. I had one confusing issue which I guess is my own fault. The following line produced an odd error message:

from color_setup import ssd

The cause proved to be that the directory on my PC contained a file color_setup.py but also a directory color_setup. The filesystem on the target device is identical and it has caused no problem to MicroPython but for some reason it confuses this tool.

I guess this is a real outlier. Renaming the directory has fixed the problem for me.

peterhinch · 2021-05-07T11:44:45Z

So far I haven't managed to replicate the error message but I have uncovered this issue #7229 which may be the underlying cause.

dpgeorge · 2021-05-07T11:59:09Z

The filesystem on the target device is identical and it has caused no problem to MicroPython but for some reason it confuses this tool.

@peterhinch this is what confused me: it sounds like when the board is running from its own filesystem then the import works as expected, but when it runs via mpremote mount then it fails (and the filesystem layout is the same in both cases). Is that true? If true then there's a problem with mpremote mount.

peterhinch · 2021-05-07T12:18:30Z

The application runs on the device under its own filesystem and has been tested on Pyboard, ESP32, ESP8266 and RP2. When run under mpremote it produces the above error message.

However I have failed to produce a test case which produces a similar error message.

Further, issue #7229 has me foxed as to how the application ever worked.

In my attempts to produce a simple test case the error messages are a) consistent with #7229 and b) identical under mpremote and Unix. I'm puzzled that I can't produce a test case as very little code has run at the point where the exception occurs.

peterhinch · 2021-05-09T07:53:26Z

I've looked at this again and figured out what was going on.

There is nothing wrong with mpremote.
The error message is a consequence of Module import fails if a directory exists with same name #7229 (comment) this MP behaviour.
Given Module import fails if a directory exists with same name #7229 the behaviour of my code under mpremote is expected.
The reason the code works on various targets is that the installation procedure does not copy the color_setup directory to the target, whereas mpremote sees the whole directory structure. The problem does not arise on a hardware target.

mpremote is awesome, btw :)

dpgeorge · 2021-05-09T14:24:46Z

I've looked at this again and figured out what was going on.
* There is nothing wrong with `mpremote`.

Ok, great! Thanks for testing/confirming that. Then it's just #7229 to deal with.

glenn20 · 2021-05-11T10:02:52Z

Just a heads-up that the existing mpremote tool allows access to files and directories outside the mounted directory, eg os.listdir('/remote/..'). It will also follow symlinks outside the mounted dir as well.

For my own personal use I can't see this being a big security issue, but I can envision scenarios where it could be - especially for unwary users accessing boards with code from other sources. The following patch causes an OSError(errno.EPERM) to be raised when trying to access files or dirs outside the mounted directory (including symlinks).

And while I am here, I have also included below the very simple patch to make the FS handling completely robust against keystrokes. On the ESP32 without this patch, pressing a few key strokes during a read of a 30k file will cause it to fail most times. With the patch - never - (so far ;-).

And finally, I am just noting some surprising behaviour with os.chdir():

os.chdir('/remote/non-existent-directory') succeeds, and
os.chdir('/remote'); os.chdir('..'); os.getcwd() will return '/remote/..'
I have no fixes for that.

--- mpremote/pyboardextended-orig.py    2021-05-11 16:50:06.009401663 +1000
+++ mpremote/pyboardextended.py 2021-05-11 19:56:03.937357402 +1000
@@ -1,6 +1,8 @@
 import os, re, serial, struct, time
 from .console import VT_ENABLED
 
+from errno import EPERM
+
 try:
     from .pyboard import Pyboard, PyboardError, stdout_write_bytes, filesystem_command
 except ImportError:
@@ -262,7 +267,10 @@
         c = self.cmd
         c.begin(CMD_ILISTDIR_START)
         c.wr_str(self.path + path)
+        res = c.rd_s8()
         c.end()
+        if res < 0:
+            raise OSError(-res)
         def next():
             while True:
                 c.begin(CMD_ILISTDIR_NEXT)
@@ -350,10 +358,17 @@
     def log_cmd(self, msg):
         print(f"[{msg}]", end="\r\n")
 
+    def path_check(self, path):
+        parent = os.path.realpath(self.root)
+        child = os.path.realpath(path)
+        if parent != os.path.commonpath([parent, child]):
+            raise OSError(EPERM)  # File is outside mounted dir
+
     def do_stat(self):
         path = self.root + self.rd_str()
         # self.log_cmd(f"stat {path}")
         try:
+            self.path_check(path)
             stat = os.stat(path)
         except OSError as er:
             self.wr_s8(-abs(er.args[0]))
@@ -368,8 +383,14 @@
 
     def do_ilistdir_start(self):
         path = self.root + self.rd_str()
-        self.data_ilistdir[0] = path
-        self.data_ilistdir[1] = os.listdir(path)
+        try:
+            self.path_check(path)
+            self.wr_s8(0)
+        except OSError as er:
+            self.wr_s8(-abs(er.args[0]))
+        else:
+            self.data_ilistdir[0] = path
+            self.data_ilistdir[1] = os.listdir(path)
 
     def do_ilistdir_next(self):
         if self.data_ilistdir[1]:
@@ -389,6 +410,7 @@
         mode = self.rd_str()
         # self.log_cmd(f"open {path} {mode}")
         try:
+            self.path_check(path)
             f = open(path, mode)
         except OSError as er:
             self.wr_s8(-abs(er.args[0]))
@@ -437,6 +459,7 @@
         path = self.root + self.rd_str()
         # self.log_cmd(f"remove {path}")
         try:
+            self.path_check(path)
             os.remove(path)
             ret = 0
         except OSError as er:
@@ -448,6 +471,8 @@
         new = self.root + self.rd_str()
         # self.log_cmd(f"rename {old} {new}")
         try:
+            self.path_check(old)
+            self.path_check(new)
             os.rename(old, new)
             ret = 0
         except OSError as er:

To fix keystrokes crashing FS commands:

--- mpremote-orig/pyboardextended.py    2021-05-11 16:50:06.009401663 +1000
+++ mpremote/pyboardextended.py 2021-05-11 16:47:06.743581447 +1000
@@ -70,11 +70,14 @@
     def begin(self, type):
         micropython.kbd_intr(-1)
         buf4 = self.buf4
-        while self.poller.poll(0):
-            self.fin.readinto(buf4, 1)
         buf4[0] = 0x18
         buf4[1] = type
         self.fout.write(buf4, 2)
+        for i in range(40):  # Don't get stuck here forever
+            self.poller.poll(1000)
+            self.fin.readinto(buf4, 1)
+            if buf4[0] == 0x18:
+                break
 
     def end(self):
         micropython.kbd_intr(3)
@@ -481,6 +484,7 @@
             if c == b"\x18":
                 # a special command
                 c = self.orig_serial.read(1)[0]
+                self.orig_serial.write(b"\x18")  # Acknowledge with ctrl-X
                 PyboardCommand.cmd_table[c](self.cmd)
             elif not VT_ENABLED and c == b"\x1b":
                 # ESC code, ignore these on windows

peterhinch · 2021-05-11T10:56:33Z

A small point re user friendliness. If the user pulls the plug on the device while the session is in progress a device disconnected message is output followed by a traceback. Arguably the traceback carries no useful information. It could be discarded with mpremote issuing the message and returning to the shell.

dpgeorge · 2021-05-11T14:03:31Z

Thanks @glenn20 , both those improvements are now in.

If the user pulls the plug on the device while the session is in progress a device disconnected message is output followed by a traceback.

@peterhinch I do not see a traceback. If I unplug a device I just see "device disconnected".

peterhinch · 2021-05-11T16:17:35Z

Here is what I get:

[adminpete@capybara]: ~
$ mpremote mount .
Local directory . is mounted at /remote
Connected to MicroPython at /dev/ttyACM0
Use Ctrl-] to exit this shell
>
MicroPython v1.14 on 2021-03-28; Raspberry Pi Pico with RP2040
Type "help()" for more information.
>>> device disconnected
                       Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/serial/serialposix.py", line 537, in write
    n = os.write(self.fd, d)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/adminpete/.local/bin/mpremote", line 8, in <module>
    sys.exit(main())
  File "/home/adminpete/.local/lib/python3.8/site-packages/mpremote/main.py", line 1042, in main
    pyb.enter_raw_repl_without_soft_reset()
  File "/home/adminpete/.local/lib/python3.8/site-packages/mpremote/main.py", line 593, in enter_raw_repl_without_soft_reset
    self.serial.write(b"\r\x03\x03")
  File "/home/adminpete/.local/lib/python3.8/site-packages/mpremote/main.py", line 582, in write
    self.orig_serial.write(buf)
  File "/usr/lib/python3/dist-packages/serial/serialposix.py", line 571, in write
    raise SerialException('write failed: {}'.format(e))
serial.serialutil.SerialException: write failed: [Errno 5] Input/output error
[adminpete@capybara]: ~
$

CPython version:

[adminpete@capybara]: ~
$ python3
Python 3.8.5 (default, Jan 27 2021, 15:41:15) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

dpgeorge · 2021-05-12T14:13:12Z

Ok, I can reproduce now. There needs to be a mpremote mount, not just mpremote repl.

peterhinch · 2021-05-13T13:33:51Z

Another minor observation: a RemoteFile object lacks readline and readlines methods. File rats.py:

with open('color_setup.py', 'r') as f:
    print(dir(f))
    while l := f.readline():
        print(l, end='')

Files color_setup.py and rats.py are located in the current directory of the PC.

$ mpremote mount .
Local directory . is mounted at /remote
Connected to MicroPython at /dev/ttyACM0
Use Ctrl-] to exit this shell
>
MicroPython v1.15 on 2021-05-13; Raspberry Pi Pico with RP2040
Type "help()" for more information.
>>> import rats
['__class__', '__enter__', '__exit__', '__init__', '__module__', '__qualname__', 'close', 'read', 'readinto', 'write', '__dict__', 'flush', 'ioctl', 'seek', 'cmd', 'fd', 'is_text']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "rats.py", line 4, in <module>
AttributeError: 'RemoteFile' object has no attribute 'readline'
>>>

glenn20 · 2021-05-27T03:56:58Z

Thanks Damien - I like the new macros. I have one more patch to sanitise directory handling in the RemoteVFS.

Without this patch you can "uos.chdir('/remote/mis-typed-directory-name')" and then things get confusing very quickly.

Oh - and the strange use of "if self.path is not None:" below works around a problem that I haven't been able to resolve that causes a lock-up if you perform the stat() call on the first call to uos.chdir('/remote'). That is, I avoid performing the check on first use.

diff --color -u ./pyboardextended.py ../mpremote3/pyboardextended.py
--- ./pyboardextended.py	2021-05-27 08:22:49.628934771 +1000
+++ ../mpremote/pyboardextended.py	2021-05-19 01:27:12.198357862 +1000
@@ -206,10 +206,10 @@
         c.end()
         return n
 
-
 class RemoteFS:
     def __init__(self, cmd):
         self.cmd = cmd
+        self.path = None
 
     def mount(self, readonly, mkfs):
         pass
@@ -218,12 +218,14 @@
         pass
 
     def chdir(self, path):
-        if path.startswith('/'):
-            self.path = path
-        else:
-            self.path += path
-        if not self.path.endswith('/'):
-            self.path += '/'
+        if not path.startswith('/'):
+            path = self.path + path
+        if not path.endswith('/'):
+            path += '/'
+        if self.path is not None:
+            if (self.stat(path)[0] & 0x4000) == 0:
+                raise OSError(-20)
+        self.path = path
 
     def getcwd(self):
         return self.path

peterhinch · 2021-05-27T06:42:40Z

Another observation. If, while a program is running, I press ctrl-d in place of ctrl-c to quit execution, mpremote becomes unresponsive. It takes many ctrl-c presses before mpremote quits to a bash prompt.

UnexpectedMaker · 2021-05-28T23:04:43Z

Excuse my ignorance, but is there a way when my local filesystem is mounted that when I save whatever code I am actually running mpremote mount . run example.py or even main.py, my connected board can auto soft reset and re-execute? Or clear the modules from sys and reload/restart the running code?

Right now if I make a code change, I have to cancel the execution and soft reset myself (if main.py) or re connect and run whatever other code I changed.

I'm not sure if this is beyond the capabilities of what mpr can do, but it would give similar workflow to how in CiruitPython, saving code onto the board reloads and restarts it. Which is a very nice workflow for many projects.

dpgeorge · 2021-05-28T23:14:01Z

If the user pulls the plug on the device while the session is in progress a device disconnected message is output followed by a traceback.

This is now fixed.

Another minor observation: a RemoteFile object lacks readline and readlines methods.

These are now added.

Another observation. If, while a program is running, I press ctrl-d in place of ctrl-c to quit execution, mpremote becomes unresponsive. It takes many ctrl-c presses before mpremote quits to a bash prompt.

Good find! Now fixed.

dpgeorge · 2021-05-29T00:07:47Z

I have one more patch to sanitise directory handling in the RemoteVFS.

Thanks, now integrated, but slightly differently.

Oh - and the strange use of "if self.path is not None:" below works around a problem that I haven't been able to resolve that causes a lock-up if you perform the stat() call on the first call to uos.chdir('/remote'). That is, I avoid performing the check on first use

The first chdir call is chdir("/") and that also sets self.path. I made a simple fix for this.

This has been under development since April 2017. See micropython#3034 and micropython#6375. Signed-off-by: Damien George <damien@micropython.org>

Signed-off-by: Damien George <damien@micropython.org>

This has been under development since April 2017. See micropython#3034 and micropython#6375. Signed-off-by: Damien George <damien@micropython.org>

dpgeorge · 2021-05-29T07:55:23Z

Excuse my ignorance, but is there a way when my local filesystem is mounted that when I save whatever code I am actually running mpremote mount . run example.py or even main.py, my connected board can auto soft reset and re-execute? Or clear the modules from sys and reload/restart the running code?

That could be possible using inotify. But it might make sense to provide such a feature natively in mpremote (please open a new issue for this).

peterhinch · 2021-05-31T11:04:44Z

@dpgeorge Please see #7338 which includes an observation on mpremote.

UnexpectedMaker · 2021-06-02T23:49:15Z

Excuse my ignorance, but is there a way when my local filesystem is mounted that when I save whatever code I am actually running mpremote mount . run example.py or even main.py, my connected board can auto soft reset and re-execute? Or clear the modules from sys and reload/restart the running code?

That could be possible using inotify. But it might make sense to provide such a feature natively in mpremote (please open a new issue for this).

Sorry for the late reply, sure thing, will add one soon!

This has been under development since April 2017. See micropython#3034 and micropython#6375. Signed-off-by: Damien George <damien@micropython.org>

…n-main Translations update from Hosted Weblate

dpgeorge added the tools Relates to tools/ directory in source, or other tooling label Aug 26, 2020

dpgeorge mentioned this pull request Aug 26, 2020

idea: host MPY as front-end for attached board #6345

Closed

dpgeorge force-pushed the tools-add-mpr branch from 89ca246 to 5dde59b Compare August 26, 2020 10:03

dpgeorge mentioned this pull request Sep 3, 2020

WIP adding a new tool to access a MicroPython target with host filesystem mounted on the board #3034

Closed

mattytrentini mentioned this pull request Feb 22, 2021

Roadmap to next release v1.15 #6832

Closed

dpgeorge mentioned this pull request Apr 20, 2021

tools: Add locking to pyboard.py serial port for linux and OSX #2256

Closed

dpgeorge force-pushed the tools-add-mpr branch from f3be554 to a6d01fe Compare May 6, 2021 05:39

peterhinch mentioned this pull request May 26, 2021

ESP32 exception BREAK instr when running with async v3 peterhinch/micropython-async#60

Closed

dpgeorge force-pushed the tools-add-mpr branch from 0aab860 to f4ded20 Compare May 29, 2021 07:16

dpgeorge added 3 commits May 29, 2021 17:17

tools/pyboard.py: Track raw REPL state via in_raw_repl variable.

4982d09

Signed-off-by: Damien George <damien@micropython.org>

tools/pyboard.py: Add "soft_reset" option to Pyboard.enter_raw_repl().

e4ba57c

Signed-off-by: Damien George <damien@micropython.org>

tools/mpremote: Add new CLI utility to interact with remote device.

a60ad33

This has been under development since April 2017. See micropython#3034 and micropython#6375. Signed-off-by: Damien George <damien@micropython.org>

dpgeorge force-pushed the tools-add-mpr branch from f4ded20 to a60ad33 Compare May 29, 2021 07:30

dpgeorge merged commit a60ad33 into micropython:master May 29, 2021

dpgeorge deleted the tools-add-mpr branch May 29, 2021 07:52

tannewt added a commit to tannewt/circuitpython that referenced this pull request May 12, 2022

Merge pull request micropython#6375 from weblate/weblate-circuitpytho…

a7b6b4e

…n-main Translations update from Hosted Weblate

Uh oh!

tools/mpr: Add new command-line utility to interact with remote device. #6375

tools/mpr: Add new command-line utility to interact with remote device. #6375

Uh oh!

Conversation

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Cause:

Fix (mostly):

More Complete Fix:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!