Support CPython standard library #1167
Conversation
|
|
||
| #define MICROPY_PORT_BUILTIN_MODULE_WEAK_LINKS \ | ||
| { MP_ROM_QSTR(MP_QSTR_os), MP_ROM_PTR(&os_module) }, \ | ||
| { MP_OBJ_NEW_QSTR(MP_QSTR_time), (mp_obj_t)&time_module }, \ |
There was a problem hiding this comment.
So that import os and import time continues to work if there's no python module with that name. It's a weak alias.
|
Version 2
|
|
Please wait with this, I'm having problems with the workaround for setting properties on functions. |
This sets the __file__ property on MPY modules like how it's done on pure python modules.
This gives access to the function underlying the bound method. Used in the converted CPython stdlib logging.Formatter class to handle overrriding a default converter method bound to a class variable. The method becomes bound when accessed from an instance of that class. I didn't investigate why CircuitPython turns it into a bound method.
Add traceback chain to sys.exec_info()[2]. No actual frame info is added, but just enough to recreate the printed exception traceback. Used by the unittest module which collects errors and failures and prints them at the end.
|
Version 3 The problem with the function attribute workaround has been solve 8000 d using the name as key instead of the hash (#1179). Note: The first patch The reason it has taken so long to spin a new version, is that I've spent time converting some more of the CPython stdlib modules. There are subtle differences between MP and CPython so it has been slow work, and memory is a challenge squeezing the tests in. Hopefully the conversion speed will go up as I get to know the differences. |
|
What differences are you running into? I'd like to be as similar as we can so I wonder if its worth changing CircuitPython. |
|
I haven't written them down since I've been focused on seeing if it was at all possible to convert stdlib. From lib/logging/init.py: import time
class LogRecord(object):
def __init__(self, name, level, pathname, lineno,
msg, args, exc_info, func=None, sinfo=None, **kwargs):
ct = time.time()
self.created = ct
class Formatter(object):
converter = time.localtime
def __init__(self, fmt=None, datefmt=None, style='%'):
pass
def formatTime(self, record, datefmt=None):
print('converter', self.converter)
ct = self.converter(record.created)
record = LogRecord('', 0, '', 0, '', None, None)
fmt = Formatter()
fmt.formatTime(record)CircuitPython turns CPython: CircuitPython workaround (relies on the py/objboundmeth patch): def formatTime(self, record, datefmt=None):
# ct = self.converter(record.created)
try:
ct = self.converter.__func__(record.created)
except AttributeError:
ct = self.converter(record.created)
# <snip>From lib/datetime.py: def _cmp(x, y):
return 0 if x == y else 1 if x > y else -1
class date:
def __new__(cls, year, month, day): ###
self = object.__new__(cls)
self._year = year
self._month = month
self._day = day
return self
def __eq__(self, other):
print('date.__eq__({!r})'.format(other)) ###
if isinstance(other, date):
return self._cmp(other) == 0
return NotImplemented
def __ne__(self, other):
print('date.__ne__({!r})'.format(other)) ###
if isinstance(other, date):
return self._cmp(other) != 0
return NotImplemented
def _cmp(self, other):
assert isinstance(other, date)
y, m, d = self._year, self._month, self._day
y2, m2, d2 = other._year, other._month, other._day
return _cmp((y, m, d), (y2, m2, d2))
class datetime(date):
def __new__(cls, year, month, day, hour=0, minute=0, second=0, ###
microsecond=0, tzinfo=None):
self = date.__new__(cls, year, month, day)
self._hour = hour
self._minute = minute
self._second = second
self._microsecond = microsecond
self._tzinfo = tzinfo
return self
def __eq__(self, other):
print('datetime.__eq__({!r})'.format(other)) ###
if isinstance(other, datetime):
return self._cmp(other, allow_mixed=True) == 0<
8000
/span>
elif not isinstance(other, date):
return NotImplemented
else:
return False
def __ne__(self, other):
print('datetime.__ne__({!r})'.format(other)) ###
if isinstance(other, datetime):
return self._cmp(other, allow_mixed=True) != 0
elif not isinstance(other, date):
return NotImplemented
else:
return True
as_date = date(2018, 10, 6)
as_datetime = datetime(2018, 10, 6)
as_date == as_datetimeCircuitPython calls CPython calls I haven't got a workaround for this one. |
|
Ah interesting! I'd happily merge PRs that align our behavior with C python. Its not on the top of my list of work to do though. |
|
ping |
|
Thanks for the ping! Sorry I didn't do any additional review. I didn't know the state of this PR. I don't want to re-use the u* versions of modules because MicroPython uses them. How about weak linking to the top level and adding a |
|
I think that would be strange. If you don't like it, can we do what CPython does with its C modules and prepend it with an underscore |
|
|
Add alias for uerrno so the user doesn't have to know about the CircuitPython special names for the module. Make os and time weak modules (aliases) making it possible to add functionality to those modules written in python. Example: 'import os' will now look in the path for an os module and if not found it will import the builtin module. An os module written in python will import the builtin module through its name prefixed with an underscore (_os) following the C module naming practice in CPython. Also right align the macro values to increase readability making it easier to compare the values for samd21 and samd51. Even the longest macro from py/mpconfig.h will fit with this alignment.
This enables various things in order to support the CPython standard library.
MICROPY_PY_BUILTINS_NOTIMPLEMENTED:
Support NotImplemented for easy conversion of stdlib.
It doesn't do fallbacks though, only raises TypeError.
MICROPY_PY_COLLECTIONS_ORDEREDDICT:
collections.OrderedDict
MICROPY_PY_FUNCTION_ATTRS:
Support function.__name__ for use as key in the function attribute workaround.
MICROPY_PY_IO:
uio module: BytesIO, FileIO, StringIO, TextIOWrapper
Also add 'io' alias.
MICROPY_PY_REVERSE_SPECIAL_METHODS:
Support the __r*__ special methods.
MICROPY_PY_SYS_EXC_INFO:
sys.exc_info() used by unittest when collecting exceptions.
MICROPY_CPYTHON_COMPAT:
Some of the things it adds:
>>> object.__init__
<function>
>>> object.__new__
<function>
>>> object.__class__
<class 'type'>
>>> object().__class__
<class 'object'>
>>> object.__name__
'object'
>>> 'Hello'.encode()
b'Hello'
>>> b'Hello'.decode()
'Hello'
Named tuple field names from string:
namedtuple('Point', 'x y')
This is necessary in order to run unittest. Heavy tests like those in the stdlib need 12-14k.
|
Version 4
|
|
Thanks @tannewt. |
This is a follow up on #1155 with the CircuitPython changes.
This shows the flash use in bytes on a Metro M4 build:
Here's a grep of the stdlib use of what's provided by MICROPY_CPYTHON_COMPAT: https://gist.github.com/notro/63d716efef0fb12fcc9edcd151c684b1
See commit messages for more info.
I suppose this will overflow on some boards.