8000 unix, windows: convert all variants to use VFS subsystem and VfsPosix by dpgeorge · Pull Request #8394 · micropython/micropython · GitHub
[go: up one dir, main page]
Skip to content

unix, windows: convert all variants to use VFS subsystem and VfsPosix#8394

Merged
dpgeorge merged 3 commits intomicropython:masterfrom
dpgeorge:unix-use-common-moduos
Mar 9, 2022
Merged

unix, windows: convert all variants to use VFS subsystem and VfsPosix#8394
dpgeorge merged 3 commits intomicropython:masterfrom
dpgeorge:unix-use-common-moduos

Conversation

@dpgeorge
Copy link
Member
@dpgeorge dpgeorge commented Mar 9, 2022

This PR follows on from #8388.

All unix variants, and the windows port, now use extmod/moduos.c as their uos module implementation. In particular this means they all have MICROPY_VFS enabled and use VfsPosix for their filesystem.

As part of this, the available functions in uos become more consistent with other ports:

Reasons for doing this:

  • makes all ports consistent with the uos module
  • fills in gaps in the unix port
  • simplifies the unix port, it no longer needs a separate uos implementation
  • IMO the VFS subsystem is a fundamental feature of MicroPython and should be enabled on all major ports of MicroPython, and here it will be enabled on all unix variants (and windows)

@dpgeorge
Copy link
Member Author
dpgeorge commented Mar 9, 2022

@stinos any idea how to get statvfs working on Windows? All builds are complaining about missing sys/statvfs.h. Alternatively we can just disable this function on Windows (it was not there previously).

@stinos
Copy link
Contributor
stinos commented Mar 9, 2022 •
edited
Loading

Windows doesn't have a true equivalent of that. There are some of the statvfs struct members which could be filled in but I don't think it's worth trying to cram Windows into POSIX like that. This used to be guarded by MICROPY_PY_OS_STATVFS, just put that back? Here's a working patch:

diff --git a/extmod/vfs_posix.c b/extmod/vfs_posix.c
index 719afe28f..b7dcf25ff 100644
--- a/extmod/vfs_posix.c
+++ b/extmod/vfs_posix.c
@@ -37,6 +37,9 @@
 #include <string.h>
 #include <sys/stat.h>
 #include <dirent.h>
+#ifdef _MSC_VER
+#include <direct.h> // For mkdir etc.
+#endif

 typedef struct _mp_obj_vfs_posix_t {
     mp_obj_base_t base;
@@ -254,7 +257,11 @@ STATIC mp_obj_t vfs_posix_mkdir(mp_obj_t self_in, mp_obj_t path_in) {
     mp_obj_vfs_posix_t *self = MP_OBJ_TO_PTR(self_in);
     const char *path = vfs_posix_get_path_str(self, path_in);
     MP_THREAD_GIL_EXIT();
+    #ifdef _MSC_VER
+    int ret = mkdir(path);
+    #else
     int ret = mkdir(path, 0777);
+    #endif
     MP_THREAD_GIL_ENTER();
     if (ret != 0) {
         mp_raise_OSError(errno);
@@ -308,6 +315,7 @@ STATIC mp_obj_t vfs_posix_stat(mp_obj_t self_in, mp_obj_t path_in) {
 }
 STATIC MP_DEFINE_CONST_FUN_OBJ_2(vfs_posix_stat_obj, vfs_posix_stat);

+#if MICROPY_PY_OS_STATVFS
 #ifdef __ANDROID__
 #define USE_STATFS 1
 #endif
@@ -348,6 +356,7 @@ STATIC mp_obj_t vfs_posix_statvfs(mp_obj_t self_in, mp_obj_t path_in) {
     return MP_OBJ_FROM_PTR(t);
 }
 STATIC MP_DEFINE_CONST_FUN_OBJ_2(vfs_posix_statvfs_obj, vfs_posix_statvfs);
+#endif

 STATIC const mp_rom_map_elem_t vfs_posix_locals_dict_table[] = {
     { MP_ROM_QSTR(MP_QSTR_mount), MP_ROM_PTR(&vfs_posix_mount_obj) },
@@ -362,7 +371,9 @@ STATIC const mp_rom_map_elem_t vfs_posix_locals_dict_table[] = {
     { MP_ROM_QSTR(MP_QSTR_rename), MP_ROM_PTR(&vfs_posix_rename_obj) },
     { MP_ROM_QSTR(MP_QSTR_rmdir), MP_ROM_PTR(&vfs_posix_rmdir_obj) },
     { MP_ROM_QSTR(MP_QSTR_stat), MP_ROM_PTR(&vfs_posix_stat_obj) },
+    #if MICROPY_PY_OS_STATVFS
     { MP_ROM_QSTR(MP_QSTR_statvfs), MP_ROM_PTR(&vfs_posix_statvfs_obj) },
+    #endif
 };
 STATIC MP_DEFINE_CONST_DICT(vfs_posix_locals_dict, vfs_posix_locals_dict_table);

diff --git a/ports/unix/mpconfigport.h b/ports/unix/mpconfigport.h
index f96abb942..b9a93baec 100644
--- a/ports/unix/mpconfigport.h
+++ b/ports/unix/mpconfigport.h
@@ -155,6 +155,7 @@
 #define MICROPY_PY_UOS_SEP          (1)
 #define MICROPY_PY_UOS_SYSTEM       (1)
 #define MICROPY_PY_UOS_URANDOM      (1)
+#define MICROPY_PY_UOS_STATVFS      (1)
 #define MICROPY_PY_UTIME            (1)
 #define MICROPY_PY_UTIME_MP_HAL     (1)
 #define MICROPY_PY_UERRNO           (1)
diff --git a/ports/windows/mpconfigport.h b/ports/windows/mpconfigport.h
index e140d5464..3c15b600f 100644
--- a/ports/windows/mpconfigport.h
+++ b/ports/windows/mpconfigport.h
@@ -214,11 +214,9 @@ typedef long mp_off_t;
 #define MICROPY_PORT_BUILTINS \
     { MP_ROM_QSTR(MP_QSTR_open), MP_ROM_PTR(&mp_builtin_open_obj) },

-extern const struct _mp_obj_module_t mp_module_os;
 extern const struct _mp_obj_module_t mp_module_time;
 #define MICROPY_PORT_BUILTIN_MODULES \
     { MP_ROM_QSTR(MP_QSTR_utime), MP_ROM_PTR(&mp_module_time) }, \
-    { MP_ROM_QSTR(MP_QSTR_uos), MP_ROM_PTR(&mp_module_os) }, \

 #if MICROPY_USE_READLINE == 1
 #define MICROPY_PORT_ROOT_POINTERS \
diff --git a/ports/windows/msvc/sources.props b/ports/windows/msvc/sources.props
index 21c46ca92..b755049db 100644
--- a/ports/windows/msvc/sources.props
+++ b/ports/windows/msvc/sources.props
@@ -24,6 +24,7 @@
     <PyExtModSource Include="$(PyBaseDir)extmod\vfs.c" />
     <PyExtModSource Include="$(PyBaseDir)extmod\vfs_posix.c" />
     <PyExtModSource Include="$(PyBaseDir)extmod\vfs_posix_file.c" />
+    <PyExtModSource Include="$(PyBaseDir)extmod\vfs_reader.c" />
   </ItemGroup>
   <ItemGroup>

@dpgeorge
unix/moduos: Convert module to use extmod version.
2b409ef 
All variants now use extmod/moduos.c as their uos module implementation.
In particular this means they all have MICROPY_VFS enabled and use
8000
 VfsPosix
for their filesystem.

As part of this, the available functions in uos become more consistent with
other ports:
- coverage variant gets uos.urandom
- minimal and standard variant get: unlink, chdir, getcwd, listdir

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge force-pushed the unix-use-common-moduos branch from 5f793d8 to 313c575 Compare March 9, 2022 10:16
@dpgeorge
Copy link
Member Author
dpgeorge commented Mar 9, 2022

Thanks, I've applied those patches (except made MICROPY_PY_UOS_STATVFS enabled by default and only disabled explicitly on windows).

@dpgeorge
Copy link
Member Author
dpgeorge commented Mar 9, 2022

It seems mkdir() with 2 args works on appveyor builds but not the windows github action. I'm not sure why, it did previously work with the old ports/unix/modos.c.

@stinos
Copy link
Contributor
stinos commented Mar 9, 2022

My mistake, the original modos.c used

    #ifdef _WIN32
    int r = mkdir(path);
    #else
    int r = mkdir(path, 0777);
    #endif

but I used _MSC_VER in the patch above, which only catches the msvc but not the mingw compiler. So should be _WIN32.

@dpgeorge
windows: Switch to VFS subsystem and use VfsPosix.
0149cd6 
Following the unix port.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge force-pushed the unix-use-common-moduos branch from 313c575 to 0149cd6 Compare March 9, 2022 13:28
@dpgeorge
tests/extmod/vfs_posix.py: Only test statvfs if it exists.
d470c5a 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge merged commit d470c5a into micropython:master Mar 9, 2022
@dpgeorge dpgeorge deleted the unix-use-common-moduos branch March 9, 2022 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

port-unix port-windows

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dpgeorge @stinos
0