File corruption when transferring files from PyBoard V1 to Windows 11 PC using File Explorer #17218
Replies: 3 comments 14 replies
-
|
The problem is that no two computers can reliably write to the same physical filesystem at the same time without corrupting it. Bulletproof: Avoid writing and reading from two places at one.
If you can eliminate all writing on the MCU, you will have less problems , but you will need frequent MCU restarts so make the MCU 'see' recent updates from your host. |
Beta Was this translation helpful? Give feedback.
-
|
Really? I thought I was at pains to make clear that no two computers are writing to the same physical file system at the same time: Thonny writes the files on the SD card; Windows reads the files when copying them. Unless you are asserting that Thonny does not close the file between saves, which would be a bug. How does "mounting" a remote directory (whatever that means for MicroPython) change any of this? I think you are shooting the messenger. |
Beta Was this translation helpful? Give feedback.
-
|
Is there any MicroPython code running on your mcu that opens files? Computer 1 Tonny is running on windows, so it's not tho NY , but windows writing Computer 2: |
Beta Was this translation helpful? Give feedback.
-
|
As I said in my original post: No |
Beta Was this translation helpful? Give feedback.
-
Still the root cause remains:
As such there is a large body off issues and reports that have been tracked down to the setup you are using. a few examples :
If you prefer the convienence , and and want to improve reliability: |
Beta Was this translation helpful? Give feedback.
-
|
Well ok. As your message came in, I was looking at a Python file that I had copied and pasted from the Thonny code pane to an editor on the PC. I closed Thonny and the editor. The PC file (examined with an Windows editor) contains the right text, but the file on the PyBoard SD card (examined with a Windows editor) is now corrupted (truncated). Sheesh. That tells me that the MicroPython firmware does not properly emulate an SD card as a stand-alone entity. No SD card worth the money would do what it does. I don't understand how mpremote can improve on this, unless it papers over some pretty big cracks, but I'll give it a try. |
Beta Was this translation helpful? Give feedback.
-
|
As said, it's a problem of concurrent writes by two entities and of caching. The PC software assumes to be the only entity having control over the SD card. So it can cache an write as needed for best performance. With MicroPython and SD card emulation there are two entities writing, and in addition Thonny as known source of surprises. |
Beta Was this translation helpful? Give feedback.
-
When you have MSC turned off, you can use Thonny's Files pane (View menu → Files) to access files on the Pyboard. This will avoid file corruption. |
Beta Was this translation helpful? Give feedback.
-
|
I’ll try it. Sorry for this dumb question, but what’s the recommended way of running my program from the REPL? |
Beta Was this translation helpful? Give feedback.
-
|
If you're using Thonny, using Thonny's built-in REPL is usually okay. If you notice output problems, though, mpremote in a terminal windowshould give you a clearer picture of what's happening |
Beta Was this translation helpful? Give feedback.
-
|
Ok. mpremote mount . fs ls run |
Beta Was this translation helpful? Give feedback.
-
|
Use Thonny or mpremote. They're unlikely to work well together. Thonny does unspecified things to REPL output. While it may be convenient to use, it can (inadvertently) hide problems for users. There are lots of threads here where the real problem was Thonny hiding/masking error output. You might not like how mpremote looks, but at least it's honest with what it tells you. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
This is a testing of the waters (a bit lengthy for which I apologise).
PyBoard V1.0 - I have very recent firmware installed (latest before the introduction of ROMfs). I have definitely had file corruption resulting in lost time, but I can’t ID the cause. I’m using Thonny IDE for development of Python files on the PyBoard SD card.
The application has no exit statement, and includes interrupt code, so can only be stopped by using Thonny’s stop button. The application itself does not read or write files.
When I try to back up (copy, using Windows 11 File Explorer) my Python files from the SD card to the Windows PC, and especially when the file name on the PC already exists, resulting in the “Skip or Replace” prompt, both the source and the destination file are corrupted in the same way - usually both comprise an almost-complete snippet of a previous version of the file( (I kid you not!). I try to make sure that my application is not running at the time, but I may slip up occasionally, as there is no output to the REPL pane to indicate that anything is running.
My solution at the moment is to copy and paste source code (text, using an editor) from the PyBoard files to new files on the PC.
I am aware of multiple reports of file copy problems on Windows 11.
It strikes me as odd that Micropython syncs the file system after a restart but not immediately after the “Go” button is pressed in Thonny, after which the source code is saved, and the code is then run.
So the questions are: Is this a bug? Is this kind of corruption likely to happen in certain conditions? What’s a bullet-proof process for copying files from the SD card to the PC? Exclude the MicroPython utility program unless it’s fundamentally necessary for some reason; I prefer GUI tools.
Beta Was this translation helpful? Give feedback.
All reactions