8000 Merge branch 'main' into optimize_stage · python/cpython@065d97d · GitHub
[go: up one dir, main page]

Skip to content

Commit 065d97d

Browse files
authored
Merge branch 'main' into optimize_stage
2 parents 9c06493 + 0cd33e1 commit 065d97d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

68 files changed

+2113
-473
lines changed

.github/workflows/project-updater.yml

+12-4
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,20 @@ on:
88

99
jobs:
1010
add-to-project:
11-
name: Add to the Release and Deferred Blocker project
11+
name: Add issues to projects
1212
runs-on: ubuntu-latest
13+
strategy:
14+
matrix:
15+
include:
16+
# if an issue has any of these labels, it will be added
17+
# to the corresponding project
18+
- { project: 2, label: "release-blocker, deferred-blocker" }
19+
- { project: 3, label: expert-subinterpreters }
20+
- { project: 29, label: expert-asyncio }
21+
1322
steps:
1423
- uses: actions/add-to-project@v0.1.0
1524
with:
16-
project-url: https://github.com/orgs/python/projects/2
25+
project-url: https://github.com/orgs/python/projects/${{ matrix.project }}
1726
github-token: ${{ secrets.ADD_TO_PROJECT_PAT }}
18-
labeled: release-blocker, deferred-blocker
19-
label-operator: OR
27+
labeled: ${{ matrix.label }}

Doc/c-api/init_config.rst

+14
Original file line numberDiff line numberDiff line change
@@ -1155,6 +1155,20 @@ PyConfig
11551155
11561156
Default: ``-1`` in Python mode, ``0`` in isolated mode.
11571157
1158+
.. c:member:: int perf_profiling
1159+
1160+
Enable compatibility mode with the perf profiler?
1161+
1162+
If non-zero, initialize the perf trampoline. See :ref:`perf_profiling`
1163+
for more information.
1164+
1165+
Set by :option:`-X perf <-X>` command line option and by the
1166+
:envvar:`PYTHONPERFSUPPORT` environment variable.
1167+
1168+
Default: ``-1``.
1169+
1170+
.. versionadded:: 3.12
1171+
11581172
.. c:member:: int use_environment
11591173
11601174
Use :ref:`environment variables <using-on-envvars>`?

Doc/c-api/structures.rst

+8-6
Original file line numberDiff line numberDiff line change
@@ -469,19 +469,21 @@ Accessing attributes of extension types
469469
.. _pymemberdef-offsets:
470470
471471
Heap allocated types (created using :c:func:`PyType_FromSpec` or similar),
472-
``PyMemberDef`` may contain definitions for the special members
473-
``__dictoffset__``, ``__weaklistoffset__`` and ``__vectorcalloffset__``,
474-
corresponding to
475-
:c:member:`~PyTypeObject.tp_dictoffset`,
476-
:c:member:`~PyTypeObject.tp_weaklistoffset` and
472+
``PyMemberDef`` may contain definitions for the special member
473+
``__vectorcalloffset__``, corresponding to
477474
:c:member:`~PyTypeObject.tp_vectorcall_offset` in type objects.
478475
These must be defined with ``T_PYSSIZET`` and ``READONLY``, for example::
479476
480477
static PyMemberDef spam_type_members[] = {
481-
{"__dictoffset__", T_PYSSIZET, offsetof(Spam_object, dict), READONLY},
478+
{"__vectorcalloffset__", T_PYSSIZET, offsetof(Spam_object, vectorcall), READONLY},
482479
{NULL} /* Sentinel */
483480
};
484481
482+
The legacy offsets :c:member:`~PyTypeObject.tp_dictoffset` and
483+
:c:member:`~PyTypeObject.tp_weaklistoffset` are still supported, but extensions are
484+
strongly encouraged to use ``Py_TPFLAGS_MANAGED_DICT`` and
485+
``Py_TPFLAGS_MANAGED_WEAKREF`` instead.
486+
485487
486488
.. c:function:: PyObject* PyMember_GetOne(const char *obj_addr, struct PyMemberDef *m)
487489

Doc/c-api/type.rst

+2-2
Original file line numberDiff line numberDiff line change
@@ -327,9 +327,9 @@ The following functions and structs are used to create
327327
* :c:member:`~PyTypeObject.tp_weaklist`
328328
* :c:member:`~PyTypeObject.tp_vectorcall`
329329
* :c:member:`~PyTypeObject.tp_weaklistoffset`
330-
(see :ref:`PyMemberDef <pymemberdef-offsets>`)
330+
(use :const:`Py_TPFLAGS_MANAGED_WEAKREF` instead)
331331
* :c:member:`~PyTypeObject.tp_dictoffset`
332-
(see :ref:`PyMemberDef <pymemberdef-offsets>`)
332+
(use :const:`Py_TPFLAGS_MANAGED_DICT` instead)
333333
* :c:member:`~PyTypeObject.tp_vectorcall_offset`
334334
(see :ref:`PyMemberDef <pymemberdef-offsets>`)
335335

Doc/c-api/typeobj.rst

+59-47
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ Quick Reference
9696
| | | __gt__, | | | | |
9797
| | | __ge__ | | | | |
9898
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
99-
| :c:member:`~PyTypeObject.tp_weaklistoffset` | :c:type:`Py_ssize_t` | | | X | | ? |
99+
| (:c:member:`~PyTypeObject.tp_weaklistoffset`) | :c:type:`Py_ssize_t` | | | X | | ? |
100100
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
101101
| :c:member:`~PyTypeObject.tp_iter` | :c:type:`getiterfunc` | __iter__ | | | | X |
102102
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
@@ -117,7 +117,7 @@ Quick Reference
117117
| :c:member:`~PyTypeObject.tp_descr_set` | :c:type:`descrsetfunc` | __set__, | | | | X |
118118
| 10000 | | __delete__ | | | | |
119119
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
120-
| :c:member:`~PyTypeObject.tp_dictoffset` | :c:type:`Py_ssize_t` | | | X | | ? |
120+
| (:c:member:`~PyTypeObject.tp_dictoffset`) | :c:type:`Py_ssize_t` | | | X | | ? |
121121
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
122122
| :c:member:`~PyTypeObject.tp_init` | :c:type:`initproc` | __init__ | X | X | | X |
123123
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
@@ -1018,7 +1018,6 @@ and :c:type:`PyType_Type` effectively act as defaults.)
10181018
:const:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the
10191019
:c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist and have
10201020
``NULL`` values.
1021-
10221021
.. XXX are most flag bits *really* inherited individually?
10231022
10241023
**Default:**
@@ -1135,6 +1134,33 @@ and :c:type:`PyType_Type` effectively act as defaults.)
11351134
:const:`Py_TPFLAGS_IMMUTABLETYPE` flag set. For extension types, it is
11361135
inherited whenever :c:member:`~PyTypeObject.tp_descr_get` is inherited.
11371136

1137+
.. data:: Py_TPFLAGS_MANAGED_DICT
1138+
1139+
This bit indicates that instances of the class have a ``__dict___``
1140+
attribute, and that the space for the dictionary is managed by the VM.
1141+
1142+
If this flag is set, :const:`Py_TPFLAGS_HAVE_GC` should also be set.
1143+
1144+
.. versionadded:: 3.12
1145+
1146+
**Inheritance:**
1147+
1148+
This flag is inherited unless the
1149+
:c:member:`~PyTypeObject.tp_dictoffset` field is set in a superclass.
1150+
1151+
1152+
.. data:: Py_TPFLAGS_MANAGED_WEAKREF
1153+
1154+
This bit indicates that instances of the class should 10000 be weakly
1155+
referenceable.
1156+
1157+
.. versionadded:: 3.12
1158+
1159+
**Inheritance:**
1160+
1161+
This flag is inherited unless the
1162+
:c:member:`~PyTypeObject.tp_weaklistoffset` field is set in a superclass.
1163+
11381164

11391165
.. XXX Document more flags here?
11401166
@@ -1487,6 +1513,9 @@ and :c:type:`PyType_Type` effectively act as defaults.)
14871513

14881514
.. c:member:: Py_ssize_t PyTypeObject.tp_weaklistoffset
14891515
1516+
While this field is still supported, :const:`Py_TPFLAGS_MANAGED_WEAKREF`
1517+
should be used instead, if at all possible.
1518+
14901519
If the instances of this type are weakly referenceable, this field is greater
14911520
than zero and contains the offset in the instance structure of the weak
14921521
reference list head (ignoring the GC header, if present); this offset is used by
@@ -1497,26 +1526,22 @@ and :c:type:`PyType_Type` effectively act as defaults.)
14971526
Do not confuse this field with :c:member:`~PyTypeObject.tp_weaklist`; that is the list head for
14981527
weak references to the type object itself.
14991528

1529+
It is an error to set both the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit and
1530+
:c:member:`~PyTypeObject.tp_weaklist`.
1531+
15001532
**Inheritance:**
15011533

15021534
This field is inherited by subtypes, but see the rules listed below. A subtype
15031535
may override this offset; this means that the subtype uses a different weak
15041536
reference list head than the base type. Since the list head is always found via
15051537
:c:member:`~PyTypeObject.tp_weaklistoffset`, this should not be a problem.
15061538

1507-
When a type defined by a class statement has no :attr:`~object.__slots__` declaration,
1508-
and none of its base types are weakly referenceable, the type is made weakly
1509-
referenceable by adding a weak reference list head slot to the instance layout
1510-
and setting the :c:member:`~PyTypeObject.tp_weaklistoffset` of that slot's offset.
1511-
1512-
When a type's :attr:`__slots__` declaration contains a slot named
1513-
:attr:`__weakref__`, that slot becomes the weak reference list head for
1514-
instances of the type, and the slot's offset is stored in the type's
1515-
:c:member:`~PyTypeObject.tp_weaklistoffset`.
1539+
**Default:**
15161540

1517-
When a type's :attr:`__slots__` declaration does not contain a slot named
1518-
:attr:`__weakref__`, the type inherits its :c:member:`~PyTypeObject.tp_weaklistoffset` from its
1519-
base type.
1541+
If the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit is set in the
1542+
:c:member:`~PyTypeObject.tp_dict` field, then
1543+
:c:member:`~PyTypeObject.tp_weaklistoffset` will be set to a negative value,
1544+
to indicate that it is unsafe to use this field.
15201545

15211546

15221547
.. c:member:: getiterfunc PyTypeObject.tp_iter
@@ -1695,6 +1720,9 @@ and :c:type:`PyType_Type` effectively act as defaults.)
16951720

16961721
.. c:member:: Py_ssize_t PyTypeObject.tp_dictoffset
16971722
1723+
While this field is still supported, :const:`Py_TPFLAGS_MANAGED_DICT` should be
1724+
used instead, if at all possible.
1725+
16981726
If the instances of this type have a dictionary containing instance variables,
16991727
this field is non-zero and contains the offset in the instances of the type of
17001728
the instance variable dictionary; this offset is used by
@@ -1703,48 +1731,34 @@ and :c:type:`PyType_Type` effectively act as defaults.)
17031731
Do not confuse this field with :c:member:`~PyTypeObject.tp_dict`; that is the dictionary for
17041732
attributes of the type object itself.
17051733

1706-
If the value of this field is greater than zero, it specifies the offset from
1707-
the start of the instance structure. If the value is less than zero, it
1708-
specifies the offset from the *end* of the instance structure. A negative
1709-
offset is more expensive to use, and should only be used when the instance
1710-
structure contains a variable-length part. This is used for example to add an
1711-
instance variable dictionary to subtypes of :class:`str` or :class:`tuple`. Note
1712-
that the :c:member:`~PyTypeObject.tp_basicsize` field should account for the dictionary added to
1713-
the end in that case, even though the dictionary is not included in the basic
1714-
object layout. On a system with a pointer size of 4 bytes,
1715-
:c:member:`~PyTypeObject.tp_dictoffset` should be set to ``-4`` to indicate that the dictionary is
1716-
at the very end of the structure.
1734+
The value specifies the offset of the dictionary from the start of the instance structure.
17171735

17181736
The :c:member:`~PyTypeObject.tp_dictoffset` should be regarded as write-only.
17191737
To get the pointer to the dictionary call :c:func:`PyObject_GenericGetDict`.
17201738
Calling :c:func:`PyObject_GenericGetDict` may need to allocate memory for the
17211739
dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr`
17221740
when accessing an attribute on the object.
17231741

1724-
**Inheritance:**
1725-
1726-
This field is inherited by subtypes, but see the rules listed below. A subtype
1727-
may override this offset; this means that the subtype instances store the
1728-
dictionary at a difference offset than the base type. Since the dictionary is
1729-
always found via :c:member:`~PyTypeObject.tp_dictoffset`, this should not be a problem.
1742+
It is an error to set both the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit and
1743+
:c:member:`~PyTypeObject.tp_dictoffset`.
17301744

1731-
When a type defined by a class statement has no :attr:`~object.__slots__` declaration,
1732-
and none of its base types has an instance variable dictionary, a dictionary
1733-
slot is added to the instance layout and the :c:member:`~PyTypeObject.tp_dictoffset` is set to
1734-
that slot's offset.
1735-
1736-
When a type defined by a class statement has a :attr:`__slots__` declaration,
1737-
the type inherits its :c:member:`~PyTypeObject.tp_dictoffset` from its base type.
1745+
**Inheritance:**
17381746

1739-
(Adding a slot named :attr:`~object.__dict__` to the :attr:`__slots__` declaration does
1740-
not have the expected effect, it just causes confusion. Maybe this should be
1741-
added as a feature just like :attr:`__weakref__` though.)
1747+
This field is inherited by subtypes. A subtype should not override this offset;
1748+
doing so could be unsafe, if C code tries to access the dictionary at the
1749+
previous offset.
1750+
To properly support inheritance, use :const:`Py_TPFLAGS_MANAGED_DICT`.
17421751

17431752
**Default:**
17441753

17451754
This slot has no default. For :ref:`static types <static-types>`, if the
17461755
field is ``NULL`` then no :attr:`__dict__` gets created for instances.
17471756

1757+
If the :const:`Py_TPFLAGS_MANAGED_DICT` bit is set in the
1758+
:c:member:`~PyTypeObject.tp_dict` field, then
1759+
:c:member:`~PyTypeObject.tp_dictoffset` will be set to ``-1``, to indicate
1760+
that it is unsafe to use this field.
1761+
17481762

17491763
.. c:member:: initproc PyTypeObject.tp_init
17501764
@@ -2663,18 +2677,16 @@ A type that supports weakrefs, instance dicts, and hashing::
26632677
typedef struct {
26642678
PyObject_HEAD
26652679
const char *data;
2666-
PyObject *inst_dict;
2667-
PyObject *weakreflist;
26682680
} MyObject;
26692681

26702682
static PyTypeObject MyObject_Type = {
26712683
PyVarObject_HEAD_INIT(NULL, 0)
26722684
.tp_name = "mymod.MyObject",
26732685
.tp_basicsize = sizeof(MyObject),
26742686
.tp_doc = PyDoc_STR("My objects"),
2675-
.tp_weaklistoffset = offsetof(MyObject, weakreflist),
2676-
.tp_dictoffset = offsetof(MyObject, inst_dict),
2677-
.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC,
2687+
.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE |
2688+
Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_MANAGED_DICT |
2689+
Py_TPFLAGS_MANAGED_WEAKREF,
26782690
.tp_new = myobj_new,
26792691
.tp_traverse = (traverseproc)myobj_traverse,
26802692
.tp_clear = (inquiry)myobj_clear,

Doc/extending/newtypes.rst

+9-24
Original file line numberDiff line numberDiff line change
@@ -570,43 +570,28 @@ performance-critical objects (such as numbers).
570570
.. seealso::
571571
Documentation for the :mod:`weakref` module.
572572

573-
For an object to be weakly referencable, the extension type must do two things:
573+
For an object to be weakly referencable, the extension type must set the
574+
``Py_TPFLAGS_MANAGED_WEAKREF`` bit of the :c:member:`~PyTypeObject.tp_flags`
575+
field. The legacy :c:member:`~PyTypeObject.tp_weaklistoffset` field should
576+
be left as zero.
574577

575-
#. Include a :c:type:`PyObject\*` field in the C object structure dedicated to
576-
the weak reference mechanism. The object's constructor should leave it
577-
``NULL`` (which is automatic when using the default
578-
:c:member:`~PyTypeObject.tp_alloc`).
579-
580-
#. Set the :c:member:`~PyTypeObject.tp_weaklistoffset` type member
581-
to the offset of the aforementioned field in the C object structure,
582-
so that the interpreter knows how to access and modify that field.
583-
584-
Concretely, here is how a trivial object structure would be augmented
585-
with the required field::
586-
587-
typedef struct {
588-
PyObject_HEAD
589-
PyObject *weakreflist; /* List of weak references */
590-
} TrivialObject;
591-
592-
And the corresponding member in the statically declared type object::
578+
Concretely, here is how the statically declared type object would look::
593579

594580
static PyTypeObject TrivialType = {
595581
PyVarObject_HEAD_INIT(NULL, 0)
596582
/* ... other members omitted for brevity ... */
597-
.tp_weaklistoffset = offsetof(TrivialObject, weakreflist),
583+
.tp_flags = Py_TPFLAGS_MANAGED_WEAKREF | ...,
598584
};
599585

586+
600587
The only further addition is that ``tp_dealloc`` needs to clear any weak
601-
references (by calling :c:func:`PyObject_ClearWeakRefs`) if the field is
602-
non-``NULL``::
588+
references (by calling :c:func:`PyObject_ClearWeakRefs`)::
603589

604590
static void
605591
Trivial_dealloc(TrivialObject *self)
606592
{
607593
/* Clear weakrefs first before calling any destructors */
608-
if (self->weakreflist != NULL)
609-
PyObject_ClearWeakRefs((PyObject *) self);
594+
PyObject_ClearWeakRefs((PyObject *) self);
610595
/* ... remainder of destruction code omitted for brevity ... */
611596
Py_TYPE(self)->tp_free((PyObject *) self);
612597
}

Doc/faq/general.rst

+2-2
Original file line numberDiff line numberDiff line change
@@ -113,8 +113,8 @@ to many different classes of problems.
113113

114114
The language comes with a large standard library that covers areas such as
115115
string processing (regular expressions, Unicode, calculating differences between
116-
files), internet protocols (HTTP, FTP, SMTP, XML-RPC, POP, IMAP, CGI
117-
programming), software engineering (unit testing, logging, profiling, parsing
116+
files), internet protocols (HTTP, FTP, SMTP, XML-RPC, POP, IMAP),
117+
software engineering (unit testing, logging, profiling, parsing
118118
Python code), and operating system interfaces (system calls, filesystems, TCP/IP
119119
sockets). Look at the table of contents for :ref:`library-index` to get an idea
120120
of what's available. A wide variety of third-party extensions are also

Doc/howto/index.rst

+1
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ Currently, the HOWTOs are:
3030
ipaddress.rst
3131
clinic.rst
3232
instrumentation.rst
33+
perf_profiling.rst
3334
annotations.rst
3435
isolating-extensions.rst
3536

0 commit comments

Comments
 (0)
0