8000 gh-101101: Unstable C API tier (PEP 689) (GH-101102) · python/cpython@6b2d7c0 · GitHub
[go: up one dir, main page]
Skip to content

Commit 6b2d7c0

Browse files
encukouencukou
authored
gh-101101: Unstable C API tier (PEP 689) (GH-101102)
1 parent c41af81 commit 6b2d7c0

File tree

18 files changed

+358
-29
lines changed

18 files changed

+358
-29
lines changed

‎Doc/c-api/code.rst

Lines changed: 92 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -33,28 +33,47 @@ bound into a function.
3333
3434
Return the number of free variables in *co*.
3535
36-
.. c:function:: PyCodeObject* PyCode_New(int argcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable)
36+
.. c:function:: PyCodeObject* PyUnstable_Code_New(int argcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable)
3737
3838
Return a new code object. If you need a dummy code object to create a frame,
39-
use :c:func:`PyCode_NewEmpty` instead. Calling :c:func:`PyCode_New` directly
40-
will bind you to a precise Python version since the definition of the bytecode
41-
changes often. The many arguments of this function are inter-dependent in complex
39+
use :c:func:`PyCode_NewEmpty` instead.
40+
41+
Since the definition of the bytecode changes often, calling
42+
:c:func:`PyCode_New` directly can bind you to a precise Python version.
43+
44+
The many arguments of this function are inter-dependent in complex
4245
ways, meaning F438 that subtle changes to values are likely to result in incorrect
4346
execution or VM crashes. Use this function only with extreme care.
4447
4548
.. versionchanged:: 3.11
4649
Added ``exceptiontable`` parameter.
4750
48-
.. c:function:: PyCodeObject* PyCode_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable)
51+
.. index:: single: PyCode_New
52+
53+
.. versionchanged:: 3.12
54+
55+
Renamed from ``PyCode_New`` as part of :ref:`unstable-c-api`.
56+
The old name is deprecated, but will remain available until the
57+
signature changes again.
58+
59+
.. c:function:: PyCodeObject* PyUnstable_Code_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, int firstlineno, PyObject *linetable, PyObject *exceptiontable)
4960
5061
Similar to :c:func:`PyCode_New`, but with an extra "posonlyargcount" for positional-only arguments.
5162
The same caveats that apply to ``PyCode_New`` also apply to this function.
5263
53-
.. versionadded:: 3.8
64+
.. index:: single: PyCode_NewWithPosOnlyArgs
65+
66+
.. versionadded:: 3.8 as ``PyCode_NewWithPosOnlyArgs``
5467
5568
.. versionchanged:: 3.11
5669
Added ``exceptiontable`` parameter.
5770
71+
.. versionchanged:: 3.12
72+
73+
Renamed to ``PyUnstable_Code_NewWithPosOnlyArgs``.
74+
The old name is deprecated, but will remain available until the
75+
signature changes again.
76+
5877
.. c:function:: PyCodeObject* PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno)
5978
6079
Return a new empty code object with the specified filename,
@@ -165,3 +184,70 @@ bound into a function.
165184
:c:func:`PyErr_WriteUnraisable`. Otherwise it should return ``0``.
166185
167186
.. versionadded:: 3.12
187+
188+
189+
Extra information
190+
-----------------
191+
192+
To support low-level extensions to frame evaluation, such as external
193+
just-in-time compilers, it is possible to attach arbitrary extra data to
194+
code objects.
195+
196+
These functions are part of the unstable C API tier:
197+
this functionality is a CPython implementation detail, and the API
198+
may change without deprecation warnings.
199+
200+
.. c:function:: Py_ssize_t PyUnstable_Eval_RequestCodeExtraIndex(freefunc free)
201+
202+
Return a new an opaque index value used to adding data to code objects.
203+
204+
You generally call this function once (per interpreter) and use the result
205+
with ``PyCode_GetExtra`` and ``PyCode_SetExtra`` to manipulate
206+
data on individual code objects.
207+
208+
If *free* is not ``NULL``: when a code object is deallocated,
209+
*free* will be called on non-``NULL`` data stored under the new index.
210+
Use :c:func:`Py_DecRef` when storing :c:type:`PyObject`.
211+
212+
.. index:: single: _PyEval_RequestCodeExtraIndex
213+
214+
.. versionadded:: 3.6 as ``_PyEval_RequestCodeExtraIndex``
215+
216+
.. versionchanged:: 3.12
217+
218+
Renamed to ``PyUnstable_Eval_RequestCodeExtraIndex``.
219+
The old private name is deprecated, but will be available until the API
220+
changes.
221+
222+
.. c:function:: int PyUnstable_Code_GetExtra(PyObject *code, Py_ssize_t index, void **extra)
223+
224+
Set *extra* to the extra data stored under the given index.
225+
Return 0 on success. Set an exception and return -1 on failure.
226+
227+
If no data was set under the index, set *extra* to ``NULL`` and return
228+
0 without setting an exception.
229+
230+
.. index:: single: _PyCode_GetExtra
231+
232+
.. versionadded:: 3.6 as ``_PyCode_GetExtra``
233+
234+
.. versionchanged:: 3.12
235+
236+
Renamed to ``PyUnstable_Code_GetExtra``.
237+
The old private name is deprecated, but will be available until the API
238+
changes.
239+
240+
.. c:function:: int PyUnstable_Code_SetExtra(PyObject *code, Py_ssize_t index, void *extra)
241+
242+
Set the extra data stored under the given index to *extra*.
243+
Return 0 on success. Set an exception and return -1 on failure.
244+
245+
.. index:: single: _PyCode_SetExtra
246+
247+
.. versionadded:: 3.6 as ``_PyCode_SetExtra``
248+
249+
.. versionchanged:: 3.12
250+
251+
Renamed to ``PyUnstable_Code_SetExtra``.
252+
The old private name is deprecated, but will be available until the API
253+
changes.

‎Doc/c-api/stable.rst

Lines changed: 33 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@
66
C API Stability
77
***************
88

9-
Python's C API is covered by the Backwards Compatibility Policy, :pep:`387`.
10-
While the C API will change with every minor release (e.g. from 3.9 to 3.10),
11-
most changes will be source-compatible, typically by only adding new API.
9+
Unless documented otherwise, Python's C API is covered by the Backwards
10+
Compatibility Policy, :pep:`387`.
11+
Most changes to it are source-compatible (typically by only adding new API).
1212
Changing existing API or removing API is only done after a deprecation period
1313
or to fix serious issues.
1414

@@ -18,8 +18,38 @@ way; see :ref:`stable-abi-platform` below).
1818
So, code compiled for Python 3.10.0 will work on 3.10.8 and vice versa,
1919
but will need to be compiled separately for 3.9.x and 3.10.x.
2020

21+
There are two tiers of C API with different stability exepectations:
22+
23+
- *Unstable API*, may change in minor versions without a deprecation period.
24+
It is marked by the ``PyUnstable`` prefix in names.
25+
- *Limited API*, is compatible across several minor releases.
26+
When :c:macro:`Py_LIMITED_API` is defined, only this subset is exposed
27+
from ``Python.h``.
28+
29+
These are discussed in more detail below.
30+
2131
Names prefixed by an underscore, such as ``_Py_InternalState``,
2232
are private API that can change without notice even in patch releases.
33+
If you need to use this API, consider reaching out to
34+
`CPython developers <https://discuss.python.org/c/core-dev/c-api/30>`_
35+
to discuss adding public API for your use case.
36+
37+
.. _unstable-c-api:
38+
39+
Unstable C API
40+
==============
41+
42+
.. index:: single: PyUnstable
43+
44+
Any API named with the ``PyUnstable`` prefix exposes CPython implementation
45+
details, and may change in every minor release (e.g. from 3.9 to 3.10) without
46+
any deprecation warnings.
47+
However, it will not change in a bugfix release (e.g. from 3.10.0 to 3.10.1).
48+
49+
It is generally intended for specialized, low-level tools like debuggers.
50+
51+
Projects that use this API are expected to follow
52+
CPython development and spend extra effort adjusting to changes.
2353

2454

2555
Stable Application Binary Interface

‎Doc/tools/extensions/c_annotations.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,22 @@ def add_annotations(self, app, doctree):
143143
' (Only some members are part of the stable ABI.)')
144144
node.insert(0, emph_node)
145145

146+
# Unstable API annotation.
147+
if name.startswith('PyUnstable'):
148+
warn_node = nodes.admonition(
149+
classes=['unstable-c-api', 'warning'])
150+
message = 'This is '
151+
emph_node = nodes.emphasis(message, message)
152+
ref_node = addnodes.pending_xref(
153+
'Unstable API', refdomain="std",
154+
reftarget='unstable-c-api',
155+
reftype='ref', refexplicit="False")
156+
ref_node += nodes.Text('Unstable API')
157+
emph_node += ref_node
158+
emph_node += nodes.Text('. It may change without warning in minor releases.')
159+
warn_node += emph_node
160+
node.insert(0, warn_node)
161+
146162
# Return value annotation
147163
if objtype != 'function':
148164
continue

‎Doc/whatsnew/3.12.rst

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -810,6 +810,29 @@ C API Changes
810810
New Features
811811
------------
812812

813+
814+
* :pep:`697`: Introduced the :ref:`Unstable C API tier <unstable-c-api>`,
815+
intended for low-level tools like debuggers and JIT compilers.
816+
This API may change in each minor release of CPython without deprecation
817+
warnings.
818+
Its contents are marked by the ``PyUnstable_`` prefix in names.
819+
820+
Code object constructors:
821+
822+
- ``PyUnstable_Code_New()`` (renamed from ``PyCode_New``)
823+
- ``PyUnstable_Code_NewWithPosOnlyArgs()`` (renamed from ``PyCode_NewWithPosOnlyArgs``)
824+
825+
Extra storage for code objects (:pep:`523`):
826+
827+
- ``PyUnstable_Eval_RequestCodeExtraIndex()`` (renamed from ``_PyEval_RequestCodeExtraIndex``)
828+
- ``PyUnstable_Code_GetExtra()`` (renamed from ``_PyCode_GetExtra``)
829+
- ``PyUnstable_Code_SetExtra()`` (renamed from ``_PyCode_SetExtra``)
830+
831+
The original names will continue to be available until the respective
832+
API changes.
833+
834+
(Contributed by Petr Viktorin in :gh:`101101`.)
835+
813836
* Added the new limited C API function :c:func:`PyType_FromMetaclass`,
814837
which generalizes the existing :c:func:`PyType_FromModuleAndSpec` using
815838
an additional metaclass argument.

‎Include/README.rst

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,13 @@
11
The Python C API
22
================
33

4-
The C API is divided into three sections:
4+
The C API is divided into these sections:
55

66
1. ``Include/``: Limited API
77
2. ``Include/cpython/``: CPython implementation details
8-
3. ``Include/internal/``: The internal API
8+
3. ``Include/cpython/``, names with the ``PyUnstable_`` prefix: API that can
9+
change between minor releases
10+
4. ``Include/internal/``, and any name with ``_`` prefix: The internal API
911

1012
Information on changing the C API is available `in the developer guide`_
1113

‎Include/cpython/ceval.h

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,12 @@ PyAPI_FUNC(PyObject *) _PyEval_EvalFrameDefault(PyThreadState *tstate, struct _P
2222
PyAPI_FUNC(void) _PyEval_SetSwitchInterval(unsigned long microseconds);
2323
PyAPI_FUNC(unsigned long) _PyEval_GetSwitchInterval(void);
2424

25-
PyAPI_FUNC(Py_ssize_t) _PyEval_RequestCodeExtraIndex(freefunc);
25+
PyAPI_FUNC(Py_ssize_t) PyUnstable_Eval_RequestCodeExtraIndex(freefunc);
26+
// Old name -- remove when this API changes:
27+
_Py_DEPRECATED_EXTERNALLY(3.12) static inline Py_ssize_t
28+
_PyEval_RequestCodeExtraIndex(freefunc f) {
29+
return PyUnstable_Eval_RequestCodeExtraIndex(f);
30+
}
2631

2732
PyAPI_FUNC(int) _PyEval_SliceIndex(PyObject *, Py_ssize_t *);
2833
PyAPI_FUNC(int) _PyEval_SliceIndexNotNone(PyObject *, Py_ssize_t *);

‎Include/cpython/code.h

Lines changed: 39 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -178,19 +178,40 @@ static inline int PyCode_GetFirstFree(PyCodeObject *op) {
178178
#define _PyCode_CODE(CO) _Py_RVALUE((_Py_CODEUNIT *)(CO)->co_code_adaptive)
179179
#define _PyCode_NBYTES(CO) (Py_SIZE(CO) * (Py_ssize_t)sizeof(_Py_CODEUNIT))
180180

181-
/* Public interface */
182-
PyAPI_FUNC(PyCodeObject *) PyCode_New(
181+
/* Unstable public interface */
182+
PyAPI_FUNC(PyCodeObject *) PyUnstable_Code_New(
183183
int, int, int, int, int, PyObject *, PyObject *,
184184
PyObject *, PyObject *, PyObject *, PyObject *,
185185
PyObject *, PyObject *, PyObject *, int, PyObject *,
186186
PyObject *);
187187

188-
PyAPI_FUNC(PyCodeObject *) PyCode_NewWithPosOnlyArgs(
188+
PyAPI_FUNC(PyCodeObject *) PyUnstable_Code_NewWithPosOnlyArgs(
189189
int, int, int, int, int, int, PyObject *, PyObject *,
190190
PyObject *, PyObject *, PyObject *, PyObject *,
191191
PyObject *, PyObject *, PyObject *, int, PyObject *,
192192
PyObject *);
193193
/* same as struct above */
194+
// Old names -- remove when this API changes:
195+
_Py_DEPRECATED_EXTERNALLY(3.12) static inline PyCodeObject *
196+
PyCode_New(
197+
int a, int b, int c, int d, int e, PyObject *f, PyObject *g,
198+
PyObject *h, PyObject *i, PyObject *j, PyObject *k,
199+
C2EE PyObject *l, PyObject *m, PyObject *n, int o, PyObject *p,
200+
PyObject *q)
201+
{
202+
return PyUnstable_Code_New(
203+
a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q);
204+
}
205+
_Py_DEPRECATED_EXTERNALLY(3.12) static inline PyCodeObject *
206+
PyCode_NewWithPosOnlyArgs(
207+
int a, int poac, int b, int c, int d, int e, PyObject *f, PyObject *g,
208+
PyObject *h, PyObject *i, PyObject *j, PyObject *k,
209+
PyObject *l, PyObject *m, PyObject *n, int o, PyObject *p,
210+
PyObject *q)
211+
{
212+
return PyUnstable_Code_NewWithPosOnlyArgs(
213+
a, poac, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q);
214+
}
194215

195216
/* Creates a new empty code object with the specified source location. */
196217
PyAPI_FUNC(PyCodeObject *)
@@ -269,11 +290,21 @@ PyAPI_FUNC(PyObject*) _PyCode_ConstantKey(PyObject *obj);
269290
PyAPI_FUNC(PyObject*) PyCode_Optimize(PyObject *code, PyObject* consts,
270291
PyObject *names, PyObject *lnotab);
271292

272-
273-
PyAPI_FUNC(int) _PyCode_GetExtra(PyObject *code, Py_ssize_t index,
274-
void **extra);
275-
PyAPI_FUNC(int) _PyCode_SetExtra(PyObject *code, Py_ssize_t index,
276-
void *extra);
293+
PyAPI_FUNC(int) PyUnstable_Code_GetExtra(
294+
PyObject *code, Py_ssize_t index, void **extra);
295+
PyAPI_FUNC(int) PyUnstable_Code_SetExtra(
296+
PyObject *code, Py_ssize_t index, void *extra);
297+
// Old names -- remove when this API changes:
298+
_Py_DEPRECATED_EXTERNALLY(3.12) static inline int
299+
_PyCode_GetExtra(PyObject *code, Py_ssize_t index, void **extra)
300+
{
301+
return PyUnstable_Code_GetExtra(code, index, extra);
302+
}
303+
_Py_DEPRECATED_EXTERNALLY(3.12) static inline int
304+
_PyCode_SetExtra(PyObject *code, Py_ssize_t index, void *extra)
305+
{
306+
return PyUnstable_Code_SetExtra(code, index, extra);
307+
}
277308

278309
/* Equivalent to getattr(code, 'co_code') in Python.
279310
Returns a strong reference to a bytes object. */

‎Include/pyport.h

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -323,6 +323,15 @@ extern "C" {
323323
#define Py_DEPRECATED(VERSION_UNUSED)
324324
#endif
325325

326+
// _Py_DEPRECATED_EXTERNALLY(version)
327+
// Deprecated outside CPython core.
328+
#ifdef Py_BUILD_CORE
329+
#define _Py_DEPRECATED_EXTERNALLY(VERSION_UNUSED)
330+
#else
331+
#define _Py_DEPRECATED_EXTERNALLY(version) Py_DEPRECATED(version)
332+
#endif
333+
334+
326335
#if defined(__clang__)
327336
#define _Py_COMP_DIAG_PUSH _Pragma("clang diagnostic push")
328337
#define _Py_COMP_DIAG_IGNORE_DEPR_DECLS \

‎Lib/test/test_code.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -752,15 +752,15 @@ def f():
752752
py = ctypes.pythonapi
753753
freefunc = ctypes.CFUNCTYPE(None,ctypes.c_voidp)
754754

755-
RequestCodeExtraIndex = py._PyEval_RequestCodeExtraIndex
755+
RequestCodeExtraIndex = py.PyUnstable_Eval_RequestCodeExtraIndex
756756
RequestCodeExtraIndex.argtypes = (freefunc,)
757757
RequestCodeExtraIndex.restype = ctypes.c_ssize_t
758758

759-
SetExtra = py._PyCode_SetExtra
759+
SetExtra = py.PyUnstable_Code_SetExtra
760760
SetExtra.argtypes = (ctypes.py_object, ctypes.c_ssize_t, ctypes.c_voidp)
761761
SetExtra.restype = ctypes.c_int
762762

763-
GetExtra = py._PyCode_GetExtra
763+
GetExtra = py.PyUnstable_Code_GetExtra
764764
GetExtra.argtypes = (ctypes.py_object, ctypes.c_ssize_t,
765765
ctypes.POINTER(ctypes.c_voidp))
766766
GetExtra.restype = ctypes.c_int

‎Misc/NEWS.d/next/C API/2022-04-21-17-25-22.gh-issue-91744.FgvaMi.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Introduced the *Unstable C API tier*, marking APi that is allowed to change
2+
in minor releases without a deprecation period.
3+
See :pep:`689` for details.

‎Modules/Setup.stdlib.in

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -169,7 +169,7 @@
169169
@MODULE__XXTESTFUZZ_TRUE@_xxtestfuzz _xxtestfuzz/_xxtestfuzz.c _xxtestfuzz/fuzzer.c
170170
@MODULE__TESTBUFFER_TRUE@_testbuffer _testbuffer.c
171171
@MODULE__TESTINTERNALCAPI_TRUE@_testinternalcapi _testinternalcapi.c
172-
@MODULE__TESTCAPI_TRUE@_testcapi _testcapimodule.c _testcapi/vectorcall.c _testcapi/vectorcall_limited.c _testcapi/heaptype.c _testcapi/unicode.c _testcapi/getargs.c _testcapi/pytime.c _testcapi/datetime.c _testcapi/docstring.c _testcapi/mem.c _testcapi/watchers.c _testcapi/long.c _testcapi/float.c _testcapi/structmember.c _testcapi/exceptions.c
172+
@MODULE__TESTCAPI_TRUE@_testcapi _testcapimodule.c _testcapi/vectorcall.c _testcapi/vectorcall_limited.c _testcapi/heaptype.c _testcapi/unicode.c _testcapi/getargs.c _testcapi/pytime.c _testcapi/datetime.c _testcapi/docstring.c _testcapi/mem.c _testcapi/watchers.c _testcapi/long.c _testcapi/float.c _testcapi/structmember.c _testcapi/exceptions.c _testcapi/code.c
173173
@MODULE__TESTCLINIC_TRUE@_testclinic _testclinic.c
174174

175175
# Some testing modules MUST be built as shared libraries.

0 commit comments

Comments
 (0)
0