unix: can't import frozen modules from a script that was executed from a file #2322
Comments
|
I don't think I like # 2. Too much adhoc magic, which likely will interfere with resource (non-executable) file access. |
|
Note that frozen bytecode (frozen mpy files) can't really be exposed as a read-only file (unlike frozen strings), so they'd anyway need to live in a separate "virtual" filesystem to any frozen resource files. |
|
I'm now being hit by this and eager to fix this ;-). |
So, I can see following way: if sys.path[0] != "", search before it only frozen modules. That at least should give the same search order as for REPL. But what did we decide on overridability of frozen modules? Actually, need to check what CPython does wrt to frozen modules search order and overridability. |
|
The current workaround is to specify empty path component via MICROPYPATH (de48a27 fixes a case when there're multiple path components, first empty). |
I think it's useful and important to be able to control the order (and whether they are even searched) of import of frozen modules. Controlling that via sys.path seems the most obvious way. |
|
It now seems that behavior is different from str vs mpy frozen modules (which is probably rooted in the fact that we have inconsistent, duplicated code to look up each type). |
|
Ok, now frozen mpy's don't seem to work even from repl. |
|
Ok, that's because frozen mpy is disable for unix port. |
|
...or not |
|
Ok, something just glitches in make apparently. |
|
Any updates on this? Having a build of micropython with unittests module frozen: root@e1ec92421b56:/build# micropython
MicroPython v1.9.3-477-g7b0a020 on 2018-03-21; linux version
Use Ctrl-D to exit, Ctrl-E for paste mode
>>> import unittest
>>>Simple script root@e1ec92421b56:/build# cat z.py
import unittestDoesn't work: root@e1ec92421b56:/build# micropython z.py
Traceback (most recent call last):
File "z.py", line 1, in <module>
ImportError: no module named 'unittest' |
This change "moves" frozen modules into a virtual directory called ".frozen". This allows controlling the search order for frozen modules, relative to other directories. Before this change, frozen modules were found via the current directory (i.e. the "" entry in sys.path). However, when a file is executed from the command line the first entry in sys.path (which is normally the empty string ""), is replaced by the base path of the executing file (correct behaviour as per CPython). This made loading frozen modules fail. With this change, frozen modules have their own entry in sys.path, which also allows the user to control the order of search when looking for frozen modules. By default, the virtual ".frozen" directory is the last entry in sys.path (i.e. it gets searched last). Fixes issue micropython#2322.
This change "moves" frozen modules into a virtual directory called "|frozen". This allows controlling the search order for frozen modules, relative to other directories. Before this change, frozen modules were found via the current directory (i.e. the "" entry in sys.path). However, when a file is executed from the command line the first entry in sys.path (which is normally the empty string ""), is replaced by the base path of the executing file (correct behavior as per CPython). This made loading frozen modules fail. With this change, frozen modules have their own entry in sys.path, which also allows the user to control the order of search when looking for frozen modules. By default, the virtual "|frozen" directory is the last entry in sys.path (i.e. it gets searched last). Fixes issue micropython#2322.
…xisting entry. Also applies to windows port. Addresses issue micropython#2322
This allows any frozen code to still be found with existing empty entry. Also applies to windows port. Addresses issue micropython#2322
This allows any frozen code to still be found with existing empty entry. Also applies to windows port. Addresses issue micropython#2322
This changes makemanifest.py & mpy-tool.py to merge string and mpy names into the same list (now mp_frozen_names). The various paths for loading a frozen module (mp_find_frozen_module) and checking existence of a frozen module (mp_frozen_stat) use a common function that searches this list. In addition, the frozen lookup will now only take place if the path starts with ".frozen", which needs to be added to sys.path. This fixes issues micropython#1804, micropython#2322, micropython#3509, micropython#6419. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
|
Fixed by e0bf461 |
This allows any frozen code to still be found with existing empty entry. Also applies to windows port. Addresses issue micropython#2322
This changes makemanifest.py & mpy-tool.py to merge string and mpy names into the same list (now mp_frozen_names). The various paths for loading a frozen module (mp_find_frozen_module) and checking existence of a frozen module (mp_frozen_stat) use a common function that searches this list. In addition, the frozen lookup will now only take place if the path starts with ".frozen", which needs to be added to sys.path. This fixes issues micropython#1804, micropython#2322, micropython#3509, micropython#6419. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The unix port can import frozen modules (str and mpy) from the REPL but it can't import them when executing a file (eg from
./micropython test.py). This is because when a file is executed from the command line the first entry in sys.path (which is originally the empty string '') is replaced by the base path of the executing file. This is the correct (CPython) behaviour.But it means that frozen modules can no longer be found in the path -- they require the empty string in the path in order for them to be found in the import search (note that bare-metal ports always have '' in the path so they are ok).
Some ways to fix:
:frozen:/) and prefix all the filenames in the list of frozen files with this special directory. The benefit of this is that it would allow the user to control the order of search when looking for frozen modules.The text was updated successfully, but these errors were encountered: