stm32: littlefs support #5330
stm32: littlefs support #5330
Conversation
|
looking forward to this, great. Btw would there be a way to add a Vfs layer on top of littlefs so that it can be exposed as MSC over USB too? or is this not feasable? I‘m having no problem if it is not possible, just curious. |
It may be feasible using the "ghost FAT" scheme https://github.com/microsoft/uf2-nrf5/blob/master/src/ghostfat.c but it's a lot of work and may in the end not be reliable enough. For now I'd suggest looking at ways to actually read a littlefs on your PC, eg https://github.com/ARMmbed/littlefs-fuse |
| // populate the filesystem with factory files | ||
| factory_reset_make_files(&vfs_fat->fatfs); | ||
| #if MICROPY_VFS_LFS2 | ||
| if (memcmp(&buf[8], "littlefs", 8) == 0) { |
There was a problem hiding this comment.
Are these strings at magic locations part of the official lfs1/2 header formats?
If so, this chunk of code the check the format of the block device would be a good candidate to move to common place for all the ports to use!
There was a problem hiding this comment.
I found them out by inspecting the littlefs code, the superblocks.
Can certainly move it to a common location for general use.
|
@andrewleech I know this misses a lot of the things you added in your PRs on littlefs, in particular compile-time configurability of what FS type and partition to use by default (this PR still defaults to FAT FS). But my main aim was to make it simple, and dynamically configurable to start with, so the user can easily select their FS type. Is there anything critical missing from here that you need, or could you work with this PR as-is? |
|
I wholehartedly agree with simplifying the merge changesets, it makes sense to have the smaller changes that keep exising things working. I've been trying to split up my stuff into more sensible commits but not making much progress lately. I've been thinking of ways to simplify my spiflash changes to work on all ports equally and to not need to changes as much, as well as reduce the overheads I've currently added. I'll get back to that at some stage. What you've got here looks good to me, haven't had a chance to test it yet though. Looking through it I think this actually would support all of my current use case. I've got qspi flash as main storage (no internal flash used), with half of it configured as fatfs, the other half as littlefs. |
ports/stm32/main.c
Outdated
| return false; | ||
| } | ||
| uint8_t buf[FLASH_BLOCK_SIZE]; | ||
| storage_read_blocks(buf, 0x100, 1); |
There was a problem hiding this comment.
0x100 should probably be FLASH_PART1_START_BLOCK
|
|
||
| if (ret == -MP_ENODEV && reset_mode != 3) { | ||
| // No filesystem (and didn't already create one), try to create a fresh one | ||
| ret = factory_reset_create_filesystem(); |
There was a problem hiding this comment.
I think there's a case here where the above code can detect LFS, but mount fails. In which case bdev will now be re-initialised by the len != -1 check above, but should now be reset back to the pyb_flash_obj singleton.
There was a problem hiding this comment.
fixed: this code will now only run if bdev==pyb_flash_obj, which reduces the chance that it will accidentally wipe a littlefs filesystem
ports/stm32/storage.c
Outdated
| uint32_t offset = mp_obj_get_int(args[3]); | ||
| #if MICROPY_HW_ENABLE_INTERNAL_FLASH_STORAGE | ||
| block_num += self->start / FLASH_BLOCK_SIZE; | ||
| ret = flash_bdev_readblocks_ext(bufinfo.buf, block_num, offset, bufinfo.len); |
There was a problem hiding this comment.
storage_read_blocks returns unsigned, but flash_bdev_readblocks_ext (and spi_bdev_readblocks_raw) return signed.
There was a problem hiding this comment.
fixed, storage_read_blocks now returns an int
| block_num += FLASH_PART1_START_BLOCK + (int32_t)self->start / FLASH_BLOCK_SIZE; | ||
| ret = storage_read_blocks(bufinfo.buf, block_num, bufinfo.len / FLASH_BLOCK_SIZE); | ||
| } else { | ||
| uint32_t offset = mp_obj_get_int(args[3]); |
There was a problem hiding this comment.
mp_vfs_autodetect will call the 4-arg version of this, so when this happens on the singleton default pyb_flash_obj it ends up calculating an invalid block.
convert_block_to_flash_addr returns -1 (in an unsigned), which then results in this function returning (unsigned)-1 as a small int. But mp_vfs_autodetect ignores the return value anyway, reads the uninitialised buffer, then ends up defaulting to flash.
There was a problem hiding this comment.
fixed by returning an error if 4 args are used with pyb_flash_obj
| mp_int_t n; | ||
| if (self == &pyb_flash_obj) { | ||
| // Get true size | ||
| n = storage_get_block_count(); |
There was a problem hiding this comment.
It would be worth adding a comment where pyb_flash_obj is defined explaining that it has self->len set to zero (because it's explicitly handled here).
|
I'm trying to test this on my qspi-flash board, running into a problem with the start offsets. Starting with After that, I'm getting to This resolves to Later in that function however; where |
|
With the change to use |
This looks exactly like the problem @jimmo flagged above, that the auto-detection acting on the default Probably best if |
I pushed a different fix, which just checks if the object is |
fd382bf to
c070d02
Compare
The pyb.Flash() class can now be used to construct objects which reference sections of the flash storage, starting at a certain offset and going for a certain length. Such objects also support the extended block protocol. The signature for the constructor is: pyb.Flash(start=-1, len=-1).
To hint to the block device that the extended block protocol will be used.
And return -MP_EIO if calling storage_read_block/storage_write_block fails. This lines up with the return type and value (negative for error) of the calls to MICROPY_HW_BDEV_READBLOCKS (and WRITEBLOCKS, and BDEV2 versions).
c070d02 to
6b3404f
Compare
|
This was cleaned up and merged. |
|
Thanks for this! I didn't get a chance to test again but I'll follow up with feedback if needed when I do soon. |
|
@andrewleech probably the next thing to look at adding/merging would be support for 32-bit SPI flash addressing. But no hurry. |
|
Yes I've got a newer cleaner version of that ready for testing (already based on top of a previous verions of this PR). I'll try to slice it into my workload this week. I think that's the only thing blocking me from completely switching over to this new littlefs implementation. |
|
FYI for mass storage use this might help windows users: https://bluscape.blog/2019/10/01/littlefs-explorer-lfse-for-windows/ |
Update the docs for vectorio
This PR aims to add littlefs support to the stm32 port. It builds upon #5299 (generalise flash mounting code in stm32); see #3847, #5167 and #5249 for prior attempts at stm32+littlefs.
The commits here do a few things:
pyb.Flash()class so it can represent sections/partitions of the flash. Egpyb.Flash(start=0, len=1024*1024)creates a 1MiB block device at the start of the flash.pyb.Flash()class to support the extended block protocol.With this PR one can use littlefs on PYBv1.x (using just internal flash, 512 byte blocks), and on PYBD (using external SPI flash, 4k byte blocks). Other boards can also be supported, with internal or external flash, but they don't have the new features enabled.
On PYBv1.x and PYBD to switch the entire storage to a littlefs filesystem do:
On PYBD to create two partitions/filesystems, the first being a FAT the second being a littlefs, do:
The first filesystem will be mounted automatically on boot at
/flash(retains old behaviour), whether it's a FAT or LFS. You can then mount any other filesystems manually, eg for PYBD example just above:There is a little bit of "magic" behind the scenes to select either 512 or 4096 byte block sizes when using FAT vs LFS. All it is is that littlefs passes a flag along to the block device when initialising it, to indicate that it will use the extended block device. The flash block device then configures itself for 4k block sizes. (The point is that for stm32 the flash storage originally had 512 byte blocks, and now it needs to support both 512 byte and 4096 byte blocks to accommodate both FAT and LFS efficiently.)
IMO this is the "simplest" set of changes that makes littlefs "just work" on stm32, with the goals that 1) the user can dynamically select FAT or LFS by simply formatting as desired; 2) 512 byte blocks are still used for FAT, and 4096 byte blocks are used for LFS.