10000 Unable to upload files to ESP8266 board · Issue #947 · adafruit/circuitpython · GitHub
[go: up one dir, main page]
Skip to content

Unable to upload files to ESP8266 board #947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed 8000
papahabla opened this issue Jun 20, 2018 · 14 comments
Closed

Unable to upload files to ESP8266 board #947

papahabla opened this issue Jun 20, 2018 · 14 comments

Comments

@papahabla
Copy link

I'm struggling to upload code to a Feather Huzzah board using the latest 3.0.0 Release Candidate 0. If I rollback to version 2.3.1, everything seems to work as expected.

Based on the recommendation of the CircuitPython for ESP8266 page, I've tried using AMPY in place of using my usual mpfshell utility. Even running the list command ampy --port /dev/cu.SLAB_USBtoUART ls / results in an error. Other commands return the same error.

18:26 $ ampy --port /dev/cu.SLAB_USBtoUART ls /
b'#5 ets_task(40100394, 3, 3fff83f0, 4)\r\nThe FAT filesystem starting at sector 153 with size 866 sectors appears to\r\nbe corrupted. If you had important data there, you may want to make a flash\r\nsnapshot to try to recover it. Otherwise, perform factory reprogramming\r\nof MicroPython firmware (completely erase flash, followed by firmware\r\nprogramming).\r\n\r\nTraceback (most recent call last):\r\n  File "_boot.py", line 12, in <module>\r\n  File "inisetup.py", line 40, in setup\r\n  File "inisetup.py", line 25, in check_bootsec\r\n  File "inisetup.py", line 37, in fs_corrupted\r\nKeyboardInterrupt: \r\n'
Traceback (most recent call last):
  File "/Users/awarner/anaconda2/envs/upy/bin/ampy", line 11, in <module>
    sys.exit(cli())
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/ampy/cli.py", line 147, in ls
    for f in board_files.ls(directory):
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/ampy/files.py", line 94, in ls
    self._pyboard.enter_raw_repl()
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/ampy/pyboard.py", line 210, in enter_raw_repl
    raise PyboardError('could not enter raw repl')
ampy.pyboard.PyboardError: could not enter raw repl

One issue I've noticed which could be the root problem, is that a file cannot be created within CircuitPython 3 as can be done for version 2. For example:

Adafruit CircuitPython 3.0.0-rc.0 on 2018-06-18; ESP module with ESP8266
>>> f = open('temp.txt', 'wb')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 1] Permission denied

I think that the Express boards still need to prevent file system issues with the USB mounted file system and prevent file creation in this way but there is no such issue with the Feather Huzzah boards that I know of.

Also, I tried to upload files using the webrepl but that seems to have a similar issue.

Adafruit CircuitPython 3.0.0-rc.0 on 2018-06-18; ESP module with ESP8266
>>> import webrepl
>>> webrepl.start()
WebREPL is not configured, run 'import webrepl_setup'
>>> import webrepl_setup
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "webrepl_setup.py", line 108, in <module>
  File "webrepl_setup.py", line 76, in main
  File "webrepl_setup.py", line 45, in get_daemon_status
OSError: [Errno 1] Permission denied
@jerryneedell
Copy link
Collaborator

FYI - I have reproduced this - I think it is new with the RC version - it was working as of the Beta releases.
I tried erasing the flash first but that did not help.

@jerryneedell
Copy link
Collaborator

putting in a delay of 1.5 seconds allows ls to work.
ampy -d 1.5 -p /dev/ttyUSB0 ls
unfortunately - I erased the flash so it is empty :-(
put still fails with "permission denied"

@jerryneedell
Copy link
Collaborator
jerryneedell commented Jun 20, 2018 •
edited
Loading

delay does help tried running my boot.py via ampy without delay and with dealy

b'#6 ets_task(40100394, 3, 3fff83f0, 4)\r\nThe FAT filesystem starting at sector 153 with size 866 sectors appears to\r\nbe corrupted. If you had important data there, you may want to make a flash\r\nsnapshot to try to recover it. Otherwise, perform factory reprogramming\r\nof MicroPython firmware (completely erase flash, followed by firmware\r\nprogramming).\r\n\r\nTraceback (most recent call last):\r\n  File "_boot.py", line 12, in <module>\r\n  File "inisetup.py", line 40, in setup\r\n  File "inisetup.py", line 25, in check_bootsec\r\n  File "inisetup.py", line 37, in fs_corrupted\r\nKeyboardInterrupt: \r\n\r\n\r\nPress any key to enter the REPL. Use CTRL-D to soft reset.\r\n'
Traceback (most recent call last):
  File "/usr/local/bin/ampy", line 11, in <module>
    load_entry_point('adafruit-ampy==1.0.4', 'console_scripts', 'ampy')()
  File "/usr/local/lib/python3.5/dist-packages/click-6.7-py3.5.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/click-6.7-py3.5.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.5/dist-packages/click-6.7-py3.5.egg/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.5/dist-packages/click-6.7-py3.5.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.5/dist-packages/click-6.7-py3.5.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/adafruit_ampy-1.0.4-py3.5.egg/ampy/cli.py", line 285, in run
  File "/usr/local/lib/python3.5/dist-packages/adafruit_ampy-1.0.4-py3.5.egg/ampy/files.py", line 244, in run
  File "/usr/local/lib/python3.5/dist-packages/adafruit_ampy-1.0.4-py3.5.egg/ampy/pyboard.py", line 212, in enter_raw_repl
ampy.pyboard.PyboardError: could not enter raw repl

works with 1 second delay

jerryneedell@Ubuntu-Macmini:~/projects/esp8266/ampy$ ampy -d 1  -p /dev/ttyUSB0 run boot.py
network config: ('10.0.0.144', '255.255.255.0', '10.0.0.1', '75.75.75.75')
jerryneedell@Ubuntu-Macmini:~/projects/esp8266/ampy$

something has changed in the soft reboot timing as well as making the FS write protected. At least the delay is an easy workaround.

@jerryneedell
Copy link
Collaborator

Just verified that it was working on 6/14/2018 with the beta version - I don't have the commit handy now. I'll try to reconstruct it this evening.
also note that loading 3.0 RC makes everything in the FS disappear. Not sure yet if it is just not mounted properly or if it was wiped.

@jerryneedell
Copy link
Collaborator
jerryneedell commented Jun 20, 2018 •
edited
Loading

interesting - resetting appears to set up the WiFI connection - I need to figure out where it gets this... I thought it was from boot.py but if teh FS is not mounted yet, it may be something elses. - then it continually reports FS errors when executing _boot.py which mounts the FS

Are we just out of room for the Flash FS?

>>> import os
>>> os.listdir()
[]
>>>  #### pressed RESET

rl��r��c�n����p�<����x��ǒ��p�nn��;�n�����b�cl`�p�n����l���b�n��n���l��8��nn����l`��#�n�rnr���;��b�;p�n���r���pp��<��l�p��nn���r��#�n�$�l`�8`rn|��n��b��l`9~����#�n�$���nn���l`nn���r����pp��<�nj��x��nn���l`��#�n��;nr���;����;r�ےn����l`�l��r��     yr�'b����b�䎂b����l�p�lll��r��  yr�'b��l��b���b��������b���l�l��r��    yr�'b��l�b���b��������b��bl`ľ��8�ǒ���?b����lrb�������l�|��n���l`�>r�x��l`#�r�����8��� 5 (10)
add 0
aid 6
cnt

connected with Needell Airport_2GEXT, channel 11
dhcp client start...
ip:10.0.0.152,mask:255.255.255.0,gw:10.0.0.1
The FAT filesystem starting at sector 153 with size 866 sectors appears to
be corrupted. If you had important data there, you may want to make a flash
snapshot to try to recover it. Otherwise, perform factory reprogramming
of MicroPython firmware (completely erase flash, followed by firmware
programming).

The FAT filesystem starting at sector 153 with size 866 sectors appears to
be corrupted. If you had important data there, you may want to make a flash
snapshot to try to recover it. Otherwise, perform factory reprogramming
of MicroPython firmware (completely erase flash, followed by firmware
programming).

The FAT filesystem starting at sector 153 with size 866 sectors appears to
be corrupted. If you had important data there, you may want to make a flash
snapshot to try to recover it. Otherwise, perform factory reprogramming
of MicroPython firmware (completely erase flash, followed by firmware
programming).

The FAT filesystem starting at sector 153 with size 866 sectors appears to
be corrupted. If you had important data there, you may want to make a flash
snapshot to try to recover it. Otherwise, perform factory reprogramming
of MicroPython firmware (completely erase flash, followed by firmware
programming).

Traceback (most recent call last):
  File "_boot.py", line 12, in <module>
  File "inisetup.py", line 40, in setup
  File "inisetup.py", line 25, in check_bootsec
  File "inisetup.py", line 37, in fs_corrupted
KeyboardInterrupt:


Adafruit CircuitPython 3.0.0-rc.0-4-g5fb4cdc on 2018-06-20; ESP module with ESP8266
>>>

@jerryneedell
Copy link
Collaborator

one more piece of information - I verified that ampy can still load files to a feather 52832 with the current build so this is appears to be only an issue on the esp8266.

@jerryneedell
Copy link
Collaborator

@dhalbert the timing of #932 is suspicious. I'm still trying to understand it but can you think of nay reason this might impact the esp8266 in an odd way?

@dhalbert
Copy link
Collaborator

The code in #932 basically just converts os.listdir() to os.listdir(os.getcwd()). You could try that by hand and see if it breaks. Also I fixed a bug in shared_module/os/__init__.c, line 109: was

         iter.is_str = mp_obj_get_type(path) == &mp_type_str;

which was just wrong, because path is a char *, not a MicroPython object pointer. But I changed it to

         iter.is_str = true;

You could experiment and try false. That change was to fix the occasional b'sd' instead of `'sd'.

@papahabla
Copy link
Author

I tried using 1.5 seconds delay as you did after freshly erasing and flashing, and it failed with the following output. Using a delay of 2 or 3 seconds also fails.

10:16 $ ampy --port /dev/cu.SLAB_USBtoUART --delay 1.5 ls
b'#6 ets_task(40100394, 3, 3fff83f0, 4)\r\nThe FAT filesystem starting at sector 153 with size 866 sectors appears to\r\nbe corrupted. If you had important data there, you may want to make a flash\r\nsnapshot to try to recover it. Otherwise, perform factory reprogramming\r\nof MicroPython firmware (completely erase flash, followed by firmware\r\nprogramming).\r\n\r\nTraceback (most recent call last):\r\n  File "_boot.py", line 12, in <module>\r\n  File "inisetup.py", line 40, in setup\r\n  File "inisetup.py", line 25, in check_bootsec\r\n  File "inisetup.py", line 37, in fs_corrupted\r\nKeyboardInterrupt: \r\n'
Traceback (most recent call last):
  File "/Users/awarner/anaconda2/envs/upy/bin/ampy", line 11, in <module>
    sys.exit(cli())
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/ampy/cli.py", line 147, in ls
    for f in board_files.ls(directory):
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/ampy/files.py", line 94, in ls
    self._pyboard.enter_raw_repl()
  File "/Users/awarner/anaconda2/envs/upy/lib/python3.6/site-packages/ampy/pyboard.py", line 210, in enter_raw_repl
    raise PyboardError('could not enter raw repl')
ampy.pyboard.PyboardError: could not enter raw repl

However I was able to change the mpfshell module code to get repl, and to list files. Basically, the CPY3 changed ubinascii to binascii which mpfshell uses to transfer files using unhexlify.

diff --git a/mp/mpfexp.py b/mp/mpfexp.py
index a3abbb0..ad2cb0f 100644
--- a/mp/mpfexp.py
+++ b/mp/mpfexp.py
@@ -166,7 +166,12 @@ class MpFileExplorer(Pyboard):
     def setup(self):

         self.enter_raw_repl()
-        self.exec_("import os, sys, ubinascii")
+        self.exec_("import os, sys")
+        try:
+            self.exec_("import ubinascii")
+        except:
+            self.exec_("import binascii")
+            self.exec_("import binascii as ubinascii")

         # New version mounts files on /flash so lets set dir based on where we are in
         # filesystem.

Uploading file is still not possible due to the permissions issue preventing open('filename', 'wb') in version 3.

@jerryneedell
Copy link
Collaborator

@dhalbert reverting this commit
f152889
clears up the "corrupted FS" messages.
I still can't write to the FS = Permission Denied, but I get past the corrupted FS issue
on reset I now see

(40100394, 3, 3fff83f0, 4)
Traceback (most recent call last):
  File "_boot.py", line 12, in <module>
  File "_boot.py", line 9, in <module>
TypeError: can't convert NoneType to int


Adafruit CircuitPython 3.0.0-rc.0-4-g5fb4cdc-dirty on 2018-06-20; ESP module with ESP8266
>>>

@jerryneedell
Copy link
Collaborator

it appears to be failing here:
https://github.com/adafruit/circuitpython/blob/master/ports/esp8266/modules/inisetup.py#L43

  • If I do this in the REPL I get the same error
>>> storage.VfsFat.mkfs(bdev)
Traceback (most rec
8000
ent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't convert NoneType to int

not sure if this is a symptom or a cause.

@jerryneedell
Copy link
Collaborator
jerryneedell commented Jun 21, 2018 •
edited
Loading

apparently the usage storage.VfsFat is limited to the esp8266.

jerryneedell@Ubuntu-Macmini:~/projects/adafruit_github/circuitpython_master$ grep -r storage.VfsFat * 
ports/esp8266/modules/inisetup.py:    storage.VfsFat.mkfs(bdev)
ports/esp8266/modules/inisetup.py:    vfs = storage.VfsFat(bdev)
ports/esp8266/modules/_boot.py:        vfs = storage.VfsFat(bdev)
ports/esp8266/build/frozen_mpy/inisetup.py:    storage.VfsFat.mkfs(bdev)
ports/esp8266/build/frozen_mpy/inisetup.py:    vfs = storage.VfsFat(bdev)
Binary file ports/esp8266/build/frozen_mpy/_boot.mpy matches
ports/esp8266/build/frozen_mpy/_boot.py:        vfs = storage.VfsFat(bdev)
Binary file ports/esp8266/build/frozen_mpy/inisetup.mpy matches

did something change to break it?

@jerryneedell
Copy link
Collaborator

@dhalbert just to recap what we tried last night. I modified flashbdev.py to return 0 for readblocks/writeblocks. With only that change I once again was getting the "corrupted FS" on boot.
If I remove the commit f152889 thnen I can get past the previous errors in _boot.py but it is still not working correctly.
os.listdir() gives a long string of odd results:

Adafruit CircuitPython 3.0.0-rc.0-4-g5fb4cdc-dirty on 2018-06-21; ESP module with ESP8266
>>> import os
>>> junk = os.listdir()
>>> junk[0:5]
['\x00\x00\x00\x00\x00\x00\x00\x00.\x00\x00\x00', '\x00\x00\x00\x00\x00\x00\x00\x00.\x00\x00\x00', '\x00\x00\x00\x00\x00\x00\x00\x00.\x00\x00\x00', '\x00\x00\x00\x00\x00\x00\x00\x00.\x00\x00\x00', '\x00\x00\x00\x00\x00\x00\x00\x00.\x00\x00\x00']
>>>

ampy put still results in a Permission denied error
ampy run works OK

FYI
here are the diffs I have applied

jerryneedell@Ubuntu-Macmini:~/projects/adafruit_github/circuitpython_master/ports/esp8266$ git diff
diff --git a/ports/esp8266/modules/flashbdev.py b/ports/esp8266/modules/flashbdev.py
index 40ba655..da4194b 100644
--- a/ports/esp8266/modules/flashbdev.py
+++ b/ports/esp8266/modules/flashbdev.py
@@ -13,12 +13,14 @@ class FlashBdev:
     def readblocks(self, n, buf):
         #print("readblocks(%s, %x(%d))" % (n, id(buf), len(buf)))
         esp.flash_read((n + self.START_SEC) * self.SEC_SIZE, buf)
+        return 0
 
     def writeblocks(self, n, buf):
         #print("writeblocks(%s, %x(%d))" % (n, id(buf), len(buf)))
         #assert len(buf) <= self.SEC_SIZE, len(buf)
         esp.flash_erase(n + self.START_SEC)
         esp.flash_write((n + self.START_SEC) * self.SEC_SIZE, buf)
+        return 0
 
     def ioctl(self, op, arg):
         #print("ioctl(%d, %r)" % (op, arg))
diff --git a/shared-module/storage/__init__.c b/shared-module/storage/__init__.c
index cc10ebd..ee275ca 100644
--- a/shared-module/storage/__init__.c
+++ b/shared-module/storage/__init__.c
@@ -32,7 +32,6 @@
 #include "py/mperrno.h"
 #include "py/obj.h"
 #include "py/runtime.h"
-#include "shared-bindings/os/__init__.h"
 #include "shared-bindings/storage/__init__.h"
 
 STATIC mp_obj_t mp_vfs_proxy_call(mp_vfs_mount_t *vfs, qstr meth_name, size_t n_args, const mp_obj_t *args) {
@@ -64,14 +63,8 @@ void common_hal_storage_mount(mp_obj_t vfs_obj, const char* mount_path, bool rea
     args[0] = readonly ? mp_const_true : mp_const_false;
     args[1] = mp_const_false; // Don't make the file system automatically when mounting.
 
-    // Check that there's no file or directory with the same name as the mount point.
-    nlr_buf_t nlr;
-    if (nlr_push(&nlr) == 0) {
-        common_hal_os_stat(mount_path);
-        nlr_pop();
-        // Something with the same name exists.
-        mp_raise_OSError(MP_EEXIST);
-    }
+    // call the underlying object to do any mounting operation
+    mp_vfs_proxy_call(vfs, MP_QSTR_mount, 2, (mp_obj_t*)&args);
 
     // check that the destination mount point is unused
     const char *path_out;
@@ -85,8 +78,6 @@ void common_hal_storage_mount(mp_obj_t vfs_obj, const char* mount_path, bool rea
         }
     }
 
-    // call the underlying object to do any mounting operation
-    mp_vfs_proxy_call(vfs, MP_QSTR_mount, 2, (mp_obj_t*)&args);
 
     // Insert the vfs into the mount table by pushing it onto the front of the
     // mount table.

@dhalbert dhalbert mentioned this issue Jun 21, 2018
Merged
@jerryneedell
Copy link
Collaborator

fixed by #953

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dhalbert @papahabla @jerryneedell
0