8000 all: Implement a module system for external C modules. · micropython/micropython@5f33ccc · GitHub
[go: up one dir, main page]
Skip to content

Commit 5f33ccc

Browse files
aykevlaykevl
committed
all: Implement a module system for external C modules.
This system makes it a lot easier to include external libraries as native modules in MicroPython. Simply pass USER_C_MODULES (like FROZEN_MPY_DIR) as a make parameter.
1 parent a8736e5 commit 5f33ccc

File tree

8 files changed

+143
-2
lines changed

8 files changed

+143
-2
lines changed

‎docs/reference/cmodules.rst

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
Extending MicroPython with C
2+
============================
3+
4+
Some specialized code would be unacceptably slow or needs to access hardware in
5+
a way that cannot be done from MicroPython. Therefore, it supports a way of
6+
extending the language with custom modules written in C. But before you consider
7+
writing a module in C, please take a look at :ref:`speed_python`.
8+
9+
`Unlike CPython <https://docs.python.org/3/extending/building.html>`_, these
10+
modules are (currently) embedded directly in the program image instead of being
11+
dynamically loaded. This requires a `custom build of MicroPython
12+
<https://github.com/micropython/micropython/wiki/Getting-Started>`_.
13+
14+
15+
Writing a module
16+
----------------
17+
18+
A module is a directory with the following files:
19+
20+
* ``micropython.mk``, which contains the Makefile fragment for this module.
21+
* All C files you would like included.
22+
23+
Put the required build commands in ``micropython.mk``. For a simple module, you
24+
will only have to add the file paths to ``SRC_MOD``, which will include these C
25+
files in the build:
26+
27+
.. highlight:: make
28+
.. code::
29+
30+
# Add all C files to SRC_MOD.
31+
SRC_MOD += $(USER_C_MODULES)/example/example.c
32+
33+
This is a very bare bones module named ``example`` that provides
34+
``example.double(x)``. Note that the name of the module must be equal to the
35+
directory name and is also used in the name of the ``mp_obj_module_t`` object at
36+
the bottom.
37+
38+
.. highlight:: c
39+
.. code::
40+
41+
// Include required definitions first.
42+
#include "py/obj.h"
43+
#include "py/runtime.h"
44+
45+
// This is the function you will call using example.double(n).
46+
STATIC mp_obj_t example_double(mp_obj_t x_obj) {
47+
// Check input value and convert it to a C type.
48+
if (!MP_OBJ_IS_SMALL_INT(x_obj)) {
49+
mp_raise_ValueError("x is not a small int");
50+
}
51+
int x = mp_obj_int_get_truncated(x_obj);
52+
53+
// Calculate the double, and convert back to MicroPython object.
54+
return mp_obj_new_int(x + x);
55+
}
56+
STATIC MP_DEFINE_CONST_FUN_OBJ_1(example_double_obj, example_double);
57+
58+
// Define all properties of the example module, which currently are the name (a
59+
// string) and a function.
60+
// All identifiers and strings are written as MP_QSTR_xxx and will be
61+
// optimized to word-sized integers by the build system (interned strings).
62+
STATIC const mp_rom_map_elem_t example_module_globals_table[] = {
63+
{ MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_example) },
64+
{ MP_ROM_QSTR(MP_QSTR_double), MP_ROM_PTR(&example_double_obj) },
65+
};
66+
STATIC MP_DEFINE_CONST_DICT(example_module_globals, example_module_globals_table);
67+
68+
// Define module object.
69+
const mp_obj_module_t example_user_cmodule = {
70+
.base = { &mp_type_module },
71+
.globals = (mp_obj_dict_t*)&example_module_globals,
72+
};
73+
74+
75+
Using a module
76+
--------------
77+
78+
To build such a module, compile MicroPython (see `getting started
79+
<https://github.com/micropython/micropython/wiki/Getting-Started>`_) with an
80+
extra ``make`` flag named ``USER_C_MODULES`` set to the directory containing
81+
all modules you want included (not to the module itself!). For example:
82+
83+
.. highlight:: shell
84+
.. code::
85+
86+
$ make USER_C_MODULES=path-to-modules-folder all

‎docs/reference/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ implementation and the best practices to use them.
2525
speed_python.rst
2626
constrained.rst
2727
packages.rst
28+
cmodules.rst
2829

2930
.. only:: port_pyboard
3031

‎docs/reference/speed_python.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
.. _speed_python:
2+
13
Maximising MicroPython Speed
24
============================
35

‎py/mkenv.mk

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ endif
6161
MAKE_FROZEN = $(PYTHON) $(TOP)/tools/make-frozen.py
6262
MPY_CROSS = $(TOP)/mpy-cross/mpy-cross
6363
MPY_TOOL = $(PYTHON) $(TOP)/tools/mpy-tool.py
64+
GEN_CMODULES = $(PYTHON) $(TOP)/tools/gen-cmodules.py
6465

6566
all:
6667
.PHONY: all

‎py/mkrules.mk

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ endif
105105
ifneq ($(FROZEN_MPY_DIR),)
106106
# to build the MicroPython cross compiler
107107
$(TOP)/mpy-cross/mpy-cross: $(TOP)/py/*.[ch] $(TOP)/mpy-cross/*.[ch] $(TOP)/ports/windows/fmode.c
108-
$(Q)$(MAKE) -C $(TOP)/mpy-cross
108+
$(Q)$(MAKE) -C $(TOP)/mpy-cross USER_C_MODULES=
109109

110110
# make a list of all the .py files that need compiling and freezing
111111
FROZEN_MPY_PY_FILES := $(shell find -L $(FROZEN_MPY_DIR) -type f -name '*.py' | $(SED) -e 's=^$(FROZEN_MPY_DIR)/==')
@@ -123,6 +123,13 @@ $(BUILD)/frozen_mpy.c: $(FROZEN_MPY_MPY_FILES) $(BUILD)/genhdr/qstrdefs.generate
123123
$(Q)$(MPY_TOOL) -f -q $(BUILD)/genhdr/qstrdefs.preprocessed.h $(FROZEN_MPY_MPY_FILES) > $@
124124
endif
125125

126+
# to build a list of modules for py/objmodule.c.
127+
ifneq ($(USER_C_MODULES),)
128+
$(BUILD)/genhdr/cmodules.h: | $(HEADER_BUILD)/mpversion.h
129+
@$(ECHO) "GEN $@"
130+
$(Q)$(GEN_CMODULES) $(USER_C_MODULES) > $@
131+
endif
132+
126133
ifneq ($(PROG),)
127134
# Build a standalone executable (unix does this)
128135

‎py/objmodule.c

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,10 @@
2727
#include <stdlib.h>
2828
#include <assert.h>
2929

30+
#ifdef MICROPY_CMODULES_INCLUDE_H
31+
#include MICROPY_CMODULES_INCLUDE_H
32+
#endif
33+
3034
#include "py/objmodule.h"
3135
#include "py/runtime.h"
3236
#include "py/builtin.h"
@@ -220,6 +224,11 @@ STATIC const mp_rom_map_elem_t mp_builtin_module_table[] = {
220224

221225
// extra builtin modules as defined by a port
222226
MICROPY_PORT_BUILTIN_MODULES
227+
228+
#ifdef MICROPY_EXTRA_BUILTIN_MODULES
229+
// extra builtin modules from USER_C_MODULES
230+
MICROPY_EXTRA_BUILTIN_MODULES
231+
#endif
223232
};
224233

225234
MP_DEFINE_CONST_MAP(mp_builtin_module_map, mp_builtin_module_table);

‎py/py.mk

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,13 @@ $(BUILD)/$(BTREE_DIR)/%.o: CFLAGS += -Wno-old-style-definition -Wno-sign-compare
103103
$(BUILD)/extmod/modbtree.o: CFLAGS += $(BTREE_DEFS)
104104
endif
105105

106+
# External modules written in C.
107+
ifneq ($(USER_C_MODULES),)
108+
CFLAGS_MOD += -DMICROPY_CMODULES_INCLUDE_H='"genhdr/cmodules.h"'
109+
include $(USER_C_MODULES)/*/micropython.mk
110+
SRC_QSTR += $(BUILD)/genhdr/cmodules.h
111+
endif
112+
106113
# py object files
107114
PY_CORE_O_BASENAME = $(addprefix py/,\
108115
mpstate.o \
@@ -274,7 +281,7 @@ endif
274281

275282
# Sources that may contain qstrings
276283
SRC_QSTR_IGNORE = py/nlr%
277-
SRC_QSTR = $(SRC_MOD) $(filter-out $(SRC_QSTR_IGNORE),$(PY_CORE_O_BASENAME:.o=.c)) $(PY_EXTMOD_O_BASENAME:.o=.c)
284+
SRC_QSTR += $(SRC_MOD) $(filter-out $(SRC_QSTR_IGNORE),$(PY_CORE_O_BASENAME:.o=.c)) $(PY_EXTMOD_O_BASENAME:.o=.c)
278285

279286
# Anything that depends on FORCE will be considered out-of-date
280287
FORCE:

‎tools/gen-cmodules.py

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
#!/usr/bin/env python
2+
3+
# Generate genhdr/cmodules.h for inclusion in py/objmodule.c.
4+
5+
from __future__ import print_function
6+
7+
import sys
8+
import os
9+
from glob import glob
10+
11+
def update_modules(path):
12+
modules = []
13+
for module in sorted(os.listdir(path)):
14+
if not os.path.isfile('%s/%s/micropython.mk' % (path, module)):
15+
continue # not a module
16+
modules.append(module)
17+
18+
# Print header file for all external modules.
19+
print('// Automatically generated by genmodules.py.\n')
20+
for module in modules:
21+
print('extern const struct _mp_obj_module_t %s_user_cmodule;' % module)
22+
print('\n#define MICROPY_EXTRA_BUILTIN_MODULES \\')
23+
for module in modules:
24+
print(' { MP_ROM_QSTR(MP_QSTR_%s), MP_ROM_PTR(&%s_user_cmodule) }, \\' % (module, module))
25+
print()
26+
27+
if __name__ == '__main__':
28+
update_modules(sys.argv[1])

0 commit comments

Comments
 (0)
0