8000 unix, windows: convert all variants to use VFS subsystem and VfsPosix by dpgeorge · Pull Request #8394 · micropython/micropython · GitHub
[go: up one dir, main page]
Skip to content

unix, windows: convert all variants to use VFS subsystem and VfsPosix #8394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Mar 9, 2022
Merged

unix, windows: convert all variants to use VFS subsystem and VfsPosix #8394

merged 3 commits into from
Mar 9, 2022

Conversation

dpgeorge
Copy link
Member
@dpgeorge dpgeorge commented Mar 9, 2022

This PR follows on from #8388.

All unix variants, and the windows port, now use extmod/moduos.c as their uos module implementation. In particular this means they all have MICROPY_VFS enabled and use VfsPosix for their filesystem.

As part of this, the available functions in uos become more consistent with other ports:

Reasons for doing this:

  • makes all ports consistent with the uos module
  • fills in gaps in the unix port
  • simplifies the unix port, it no longer needs a separate uos implementation
  • IMO the VFS subsystem is a fundamental feature of MicroPython and should be enabled on all major ports of MicroPython, and here it will be enabled on all unix variants (and windows)

< 8000 path d="M15 8a7.002 7.002 0 00-7-7" stroke="currentColor" stroke-width="2" stroke-linecap="round" vector-effect="non-scaling-stroke" />

@dpgeorge
Copy link
Member Author
dpgeorge commented Mar 9, 2022

@stinos any idea how to get statvfs working on Windows? All builds are complaining about missing sys/statvfs.h. Alternatively we can just disable this function on Windows (it was not there previously).

@stinos
Copy link
Contributor
stinos commented Mar 9, 2022 •
edited
Loading

Windows doesn't have a true equivalent of that. There are some of the statvfs struct members which could be filled in but I don't think it's worth trying to cram Windows into POSIX like that. This used to be guarded by MICROPY_PY_OS_STATVFS, just put that back? Here's a working patch:

diff --git a/extmod/vfs_posix.c b/extmod/vfs_posix.c
index 719afe28f..b7dcf25ff 100644
--- a/extmod/vfs_posix.c
+++ b/extmod/vfs_posix.c
@@ -37,6 +37,9 @@
 #include <string.h>
 #include <sys/stat.h>
 #include <dirent.h>
+#ifdef _MSC_VER
+#include <direct.h> // For mkdir etc.
+#endif

 typedef struct _mp_obj_vfs_posix_t {
     mp_obj_base_t base;
@@ -254,7 +257,11 @@ STATIC mp_obj_t vfs_posix_mkdir(mp_obj_t self_in, mp_obj_t path_in) {
     mp_obj_vfs_posix_t *self = MP_OBJ_TO_PTR(self_in);
     const char *path = vfs_posix_get_path_str(self, path_in);
     MP_THREAD_GIL_EXIT();
+    #ifdef _MSC_VER
+    int ret = mkdir(path);
+    #else
     int ret = mkdir(path, 0777);
+    #endif
     MP_THREAD_GIL_ENTER();
     if (ret != 0) {
         mp_raise_OSError(errno);
@@ -308,6 +315,7 @@ STATIC mp_obj_t vfs_posix_stat(mp_obj_t self_in, mp_obj_t path_in) {
 }
 STATIC MP_DEFINE_CONST_FUN_OBJ_2(vfs_posix_stat_obj, vfs_posix_stat);

+#if MICROPY_PY_OS_STATVFS
 #ifdef __ANDROID__
 #define USE_STATFS 1
 #endif
@@ -348,6 +356,7 @@ STATIC mp_obj_t vfs_posix_statvfs(mp_obj_t self_in, mp_obj_t path_in) {
     return MP_OBJ_FROM_PTR(t);
 }
 STATIC MP_DEFINE_CONST_FUN_OBJ_2(vfs_posix_statvfs_obj, vfs_posix_statvfs);
+#endif

 STATIC const mp_rom_map_elem_t vfs_posix_locals_dict_table[] = {
     { MP_ROM_QSTR(MP_QSTR_mount), MP_ROM_PTR(&vfs_posix_mount_obj) },
@@ -362,7 +371,9 @@ STATIC const mp_rom_map_elem_t vfs_posix_locals_dict_table[] = {
     { MP_ROM_QSTR(MP_QSTR_rename), MP_ROM_PTR(&vfs_posix_rename_obj) },
     { MP_ROM_QSTR(MP_QSTR_rmdir), MP_ROM_PTR(&vfs_posix_rmdir_obj) },
     { MP_ROM_QSTR(MP_QSTR_stat), MP_ROM_PTR(&vfs_posix_stat_obj) },
+    #if MICROPY_PY_OS_STATVFS
     { MP_ROM_QSTR(MP_QSTR_statvfs), MP_ROM_PTR(&vfs_posix_statvfs_obj) },
+    #endif
 };
 STATIC MP_DEFINE_CONST_DICT(vfs_posix_locals_dict, vfs_posix_locals_dict_table);

diff --git a/ports/unix/mpconfigport.h b/ports/unix/mpconfigport.h
index f96abb942..b9a93baec 100644
--- a/ports/unix/mpconfigport.h
+++ b/ports/unix/mpconfigport.h
@@ -155,6 +155,7 @@
 #define MICROPY_PY_UOS_SEP          (1)
 #define MICROPY_PY_UOS_SYSTEM       (1)
 #define MICROPY_PY_UOS_URANDOM      (1)
+#define MICROPY_PY_UOS_STATVFS      (1)
 #define MICROPY_PY_UTIME            (1)
 #define MICROPY_PY_UTIME_MP_HAL     (1)
 #define MICROPY_PY_UERRNO           (1)
diff --git a/ports/windows/mpconfigport.h b/ports/windows/mpconfigport.h
index e140d5464..3c15b600f 100644
--- a/ports/windows/mpconfigport.h
+++ b/ports/windows/mpconfigport.h
@@ -214,11 +214,9 @@ typedef long mp_off_t;
 #define MICROPY_PORT_BUILTINS \
     { MP_ROM_QSTR(MP_QSTR_open), MP_ROM_PTR(&mp_builtin_open_obj) },

-extern const struct _mp_obj_module_t mp_module_os;
 extern const struct _mp_obj_module_t mp_module_time;
 #define MICROPY_PORT_BUILTIN_MODULES \
     { MP_ROM_QSTR(MP_QSTR_utime), MP_ROM_PTR(&mp_module_time) }, \
-    { MP_ROM_QSTR(MP_QSTR_uos), MP_ROM_PTR(&mp_module_os) }, \

 #if MICROPY_USE_READLINE == 1
 #define MICROPY_PORT_ROOT_POINTERS \
diff --git a/ports/windows/msvc/sources.props b/ports/windows/msvc/sources.props
index 21c46ca92..b755049db 100644
--- a/ports/windows/msvc/sources.props
+++ b/ports/windows/msvc/sources.props
@@ -24,6 +24,7 @@
     <PyExtModSource Include="$(PyBaseDir)extmod\vfs.c" />
     <PyExtModSource Include="$(PyBaseDir)extmod\vfs_posix.c" />
     <PyExtModSource Include="$(PyBaseDir)extmod\vfs_posix_file.c" />
+    <PyExtModSource Include="$(PyBaseDir)extmod\vfs_reader.c" />
   </ItemGroup>
   <ItemGroup>

@dpgeorge
unix/moduos: Convert module to use extmod version.
2b409ef 
All variants now use extmod/moduos.c as their uos module implementation.
In particular this means they all have MICROPY_VFS enabled and use VfsPosix
for their filesystem.

As part of this, the available functions in uos become more consistent with
other ports:
- coverage variant gets uos.urandom
- minimal and standard variant get: unlink, chdir, getcwd, listdir

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge force-pushed the unix-use-common-moduos branch from 5f793d8 to 313c575 Compare March 9, 2022 10:16
@dpgeorge
Copy link
Member Author
dpgeorge commented Mar 9, 2022

Thanks, I've applied those patches (except made MICROPY_PY_UOS_STATVFS enabled by default and only disabled explicitly on windows).

@dpgeorge
Copy link
Member Author
dpgeorge commented Mar 9, 2022

It seems mkdir() with 2 args works on appveyor builds but not the windows github action. I'm not sure why, it did previously work with the old ports/unix/modos.c.

@stinos
Copy link
Contributor
stinos commented Mar 9, 2022

My mistake, the original modos.c used

    #ifdef _WIN32
    int r = mkdir(path);
    #else
    int r = mkdir(path, 0777);
    #endif

but I used _MSC_VER in the patch above, which only catches the msvc but not the mingw compiler. So should be _WIN32.

@dpgeorge
windows: Switch to VFS subsystem and use VfsPosix.
0149cd6 
Following the unix port.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge force-pushed the unix-use-common-moduos branch from 313c575 to 0149cd6 Compare March 9, 2022 13:28
@dpgeorge
tests/extmod/vfs_posix.py: Only test statvfs if it exists.
d470c5a 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge merged commit d470c5a into micropython:master Mar 9, 2022
@dpgeorge dpgeorge deleted the unix-use-common-moduos branch March 9, 2022 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
port-unix port-windows
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dpgeorge @stinos
0