8000 OS vendors · adamtheturtle/pythoncapi@a18fa52 · GitHub
[go: up one dir, main page]
Skip to content

Commit a18fa52

Browse files
vstinnervstinner
committed
OS vendors
1 parent 1495342 commit a18fa52

File tree

4 files changed

+97
-6
lines changed

4 files changed

+97
-6
lines changed

‎doc/index.rst

Lines changed: 19 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,24 @@ Design a new better C API for Python
55
Subtitle: "Make the stable API usable".
66

77
To be able to introduce backward incompatible changes to the C API without
8-
breaking too many C extensions, this project proposes two things: design a
9-
:ref:`helper layer <back-compat>` providing :ref:`removed functions
10-
<remove-funcs>`, and a new :ref:`Python runtime <runtimes>` which is only
11-
usable with C extensions compiled with the new stricter and smaller C API (and
12-
the new :ref:`stable ABI
13-
<stable-abi>`).
8+
breaking too many C extensions, this project proposes two things:
9+
10+
* design a :ref:`helper layer <back-compat>` providing :ref:`removed functions
11+
<remove-funcs>`;
12+
* a new :ref:`Python runtime <runtimes>` which is only usable with C extensions
13+
compiled with the new stricter and smaller C API (and the new :ref:`stable
14+
ABI <stable-abi>`) for Python 3.8 and newer, whereas the existing "regular
15+
python" becomes the "regular runtime" which provides maximum backward
16+
compatibility with Python 3.7 and older.
17+
18+
The current C API has multiple issues:
19+
20+
* Optimization attempts like Unladen Swallow and Pyston failed because of the C
21+
API and the lack of an usable :ref:`stable ABI <stable-abi>`.
22+
* "Short" Python lifecycle doesn't fit well with the "long" :ref:`lifecycle of
23+
operating systems <os-vendors>`: how to get the latest Python on an "old" but
24+
stable operating system?
25+
* :ref:`Python debug build <debug-build>` is currently basically unusable.
1426

1527
Pages
1628
=====
@@ -25,6 +37,7 @@ Pages
2537
private_c_api
2638
remove_functions
2739
backward_compatibility
40+
os_vendors
2841
calling_conventions
2942
stable_abi
3043
consumers

‎doc/os_vendors.rst

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
.. _os-vendors:
2+
3+
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
4+
Supporting multiple Python versions per operating system release
5+
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
6+
7+
Supporting multiple minor Python versions, like Python 3.6 and 3.7, requires
8+
more work for operating system vendors, like Linux vendors. To reduce the
9+
maintenance burden, Linux vendors chose to only support one minor Python
10+
version. For example, even if Fedora 28 provides multiple Python binaries (ex:
11+
2.7, 3.5, 3.6 and 3.7), only packages for Python 3.6 are available. Only
12+
providing a binary is easy. Providing the full chain of dependencies to get a
13+
working Django application is something different.
14+
15+
Issues:
16+
17+
* Each Python minor version introduces subtle minor behaviour changes which
18+
requires to sometimes to fix issues in Python modules and applications. This
19+
issue is not solved by the new C API.
20+
* Each C extension must be recompiled once per Python minor version.
21+
* The QA team has to test each Python package: having two packages per Python
22+
module doubles the work.
23+
24+
Time scale:
25+
26+
* A Python release is supported upstream for 5 years.
27+
* A Fedora release is supported for less than one year.
28+
* Ubuntu LTS releases are supported for 5 years.
29+
* Red Hat Entreprise Linux (RHEL) is supported for 10 years, and customers can
30+
subscribe to an extended support up to 15 years.
31+
32+
In 2018, the latest macOS release still only provides Python 2.7 which will
33+
reach its end-of-life (EOL) at January 1, 2020 (in less than 2 years).

‎doc/runtimes.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
.. _runtimes:
2+
13
+++++++++++++++
24
Python runtimes
35
+++++++++++++++
@@ -37,6 +39,8 @@ Example of Python 3.7 code::
3739
The ``if (PyList_Check(op) && (newitem != NULL))`` belongs to the debug runtime
3840
and should be removed from the regular Python.
3941

42+
.. _debug-build:
43+
4044
Debug runtime: /usr/bin/python3-dbg
4145
-----------------------------------
4246

‎doc/stable_abi.rst

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,47 @@
44
Python stable ABI?
55
++++++++++++++++++
66

7+
Relationship between the C API and the ABI
8+
==========================================
9+
10+
Here is a short explanation. For a longer explanation, read `A New C API for
11+
CPython <https://vstinner.github.io/new-python-c-api.html>`_ (September 2017)
12+
by Victor Stinner.
13+
14+
Given the following code::
15+
16+
typedef struct {
17+
PyVarObject ob_base;
18+
PyObject **ob_item; // <-- pointer to the array of list items
19+
Py_ssize_t allocated;
20+
} PyListObject;
21+
22+
#define PyList_GET_ITEM(op, i) ((PyListObject *)op)->ob_item[i]
23+
24+
And the following C code::
25+
26+
PyObject *item = PyList_GET_ITEM(list, 0);
27+
28+
On a 64-bit machine, the machine code of a release build becomes something
29+
like::
30+
31+
PyObject **items = (PyObject **)(((char*)op) + 24);
32+
PyObject *item = items[0];
33+
34+
whereas a debug build uses an offset of **40** instead of **24**, because
35+
``PyVarObject`` contains additional fields for debugging purpose::
36+
37+
PyObject **items = (PyObject **)(((char*)op) + 40);
38+
PyObject *item = items[0];
39+
40+
As a consequence, the compiled C extension is incompatible at the ABI level: a
41+
C extension has to be build twice, once in release mode and once in debug mode.
42+
43+
To reduce the maintaince burden, :ref:`Linux vendors <os-vendors>` only provide
44+
C extensions compiled in release mode, making the :ref:`debug mode
45+
<debug-build>` mostly unusable on Linux in practice.
46+
47+
748
CPython Py_LIMITED_API
849
======================
950

0 commit comments

Comments
 (0)
0