8000 gh-113993: Don't immortalize in PyUnicode_InternInPlace; keep immorta… · python/cpython@b4aedb2 · GitHub
[go: up one dir, main page]
Skip to content

Commit b4aedb2

Browse files
encukouencukou
authored
gh-113993: Don't immortalize in PyUnicode_InternInPlace; keep immortalizing in other API (#121364)
* Switch PyUnicode_InternInPlace to _PyUnicode_InternMortal, clarify docs * Document immortality in some functions that take `const char *` This is PyUnicode_InternFromString; PyDict_SetItemString, PyObject_SetAttrString; PyObject_DelAttrString; PyUnicode_InternFromString; and the PyModule_Add convenience functions. Always point out a non-immortalizing alternative. * Don't immortalize user-provided attr names in _ctypes
1 parent d7a099d commit b4aedb2

File tree

6 files changed

+82
-10
lines changed

6 files changed

+82
-10
lines changed

‎Doc/c-api/module.rst

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -549,6 +549,14 @@ state:
549549
Note that ``Py_XDECREF()`` should be used instead of ``Py_DECREF()`` in
550550
this case, since *obj* can be ``NULL``.
551551
552+
The number of different *name* strings passed to this function
553+
should be kept small, usually by only using statically allocated strings
554+
as *name*.
555+
For names that aren't known at compile time, prefer calling
556+
:c:func:`PyUnicode_FromString` and F438 :c:func:`PyObject_SetAttr` directly.
557+
For more details, see :c:func:`PyUnicode_InternFromString`, which may be
558+
used internally to create a key object.
559+
552560
.. versionadded:: 3.10
553561
554562
@@ -610,6 +618,9 @@ state:
610618
used from the module's initialization function.
611619
Return ``-1`` with an exception set on error, ``0`` on success.
612620
621+
This is a convenience function that calls :c:func:`PyLong_FromLong` and
622+
:c:func:`PyModule_AddObjectRef`; see their documentation for details.
623+
613624
614625
.. c:function:: int PyModule_AddStringConstant(PyObject *module, const char *name, const char *value)
615626
@@ -618,6 +629,10 @@ state:
618629
``NULL``-terminated.
619630
Return ``-1`` with an exception set on error, ``0`` on success.
620631
632+
This is a convenience function that calls
633+
:c:func:`PyUnicode_InternFromString` and :c:func:`PyModule_AddObjectRef`;
634+
see their documentation for details.
635+
621636
622637
.. c:macro:: PyModule_AddIntMacro(module, macro)
623638

‎Doc/c-api/object.rst

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -206,6 +206,13 @@ Object Protocol
206206
If *v* is ``NULL``, the attribute is deleted, but this feature is
207207
deprecated in favour of using :c:func:`PyObject_DelAttrString`.
208208
209+
The number of different attribute names passed to this function
210+
should be kept small, usually by using a statically allocated string
211+
as *attr_name*.
212+
For attribute names that aren't known at compile time, prefer calling
213+
:c:func:`PyUnicode_FromString` and :c:func:`PyObject_SetAttr` directly.
214+
For more details, see :c:func:`PyUnicode_InternFromString`, which may be
215+
used internally to create a key object.
209216
210217
.. c:function:: int PyObject_GenericSetAttr(PyObject *o, PyObject *name, PyObject *value)
211218
@@ -231,6 +238,14 @@ Object Protocol
231238
specified as a :c:expr:`const char*` UTF-8 encoded bytes string,
232239
rather than a :c:expr:`PyObject*`.
233240
241+
The number of different attribute names passed to this function
242+
should be kept small, usually by using a statically allocated string
243+
as *attr_name*.
244+
For attribute names that aren't known at compile time, prefer calling
245+
:c:func:`PyUnicode_FromString` and :c:func:`PyObject_DelAttr` directly.
246+
For more details, see :c:func:`PyUnicode_InternFromString`, which may be
247+
used internally to create a key object for lookup.
248+
234249
235250
.. c:function:: PyObject* PyObject_GenericGetDict(PyObject *o, void *context)
236251

‎Doc/c-api/unicode.rst

Lines changed: 31 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1490,18 +1490,43 @@ They all return ``NULL`` or ``-1`` if an exception occurs.
14901490
existing interned string that is the same as :c:expr:`*p_unicode`, it sets :c:expr:`*p_unicode` to
14911491
it (releasing the reference to the old string object and creating a new
14921492
:term:`strong reference` to the interned string object), otherwise it leaves
1493-
:c:expr:`*p_unicode` alone and interns it (creating a new :term:`strong reference`).
1493+
:c:expr:`*p_unicode` alone and interns it.
1494+
14941495
(Clarification: even though there is a lot of talk about references, think
1495-
of this function as reference-neutral; you own the object after the call
1496-
if and only if you owned it before the call.)
1496+
of this function as reference-neutral. You must own the object you pass in;
1497+
after the call you no longer own the passed-in reference, but you newly own
1498+
the result.)
1499+
1500+
This function never raises an exception.
1501+
On error, it leaves its argument unchanged without interning it.
1502+
1503+
Instances of subclasses of :py:class:`str` may not be interned, that is,
1504+
:c:expr:`PyUnicode_CheckExact(*p_unicode)` must be true. If it is not,
1505+
then -- as with any other error -- the argument is left unchanged.
1506+
1507+
Note that interned strings are not “immortal”.
1508+
You must keep a reference to the result to benefit from interning.
14971509
14981510
14991511
.. c:function:: PyObject* PyUnicode_InternFromString(const char *str)
15001512
15011513
A combination of :c:func:`PyUnicode_FromString` and
1502-
:c:func:`PyUnicode_InternInPlace`, returning either a new Unicode string
1503-
object that has been interned, or a new ("owned") reference to an earlier
1504-
interned string object with the same value.
1514+
:c:func:`PyUnicode_InternInPlace`, meant for statically allocated strings.
1515+
1516+
Return a new ("owned") reference to either a new Unicode string object
1517+
that has been interned, or an earlier interned string object with the
1518+
same value.
1519+
1520+
Python may keep a reference to the result, or make it :term:`immortal`,
1521+
preventing it from being garbage-collected promptly.
1522+
For interning an unbounded number of different strings, such as ones coming
1523+
from user input, prefer calling :c:func:`PyUnicode_FromString` and
1524+
:c:func:`PyUnicode_InternInPlace` directly.
1525+
F438
1526+
.. impl-detail::
1527+
1528+
Strings interned this way are made :term:`immortal`.
1529+
15051530
15061531
PyUnicodeWriter
15071532
^^^^^^^^^^^^^^^

‎Misc/NEWS.d/next/C API/2024-07-04-15-41-10.gh-issue-113993.cLSiWV.rst

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
:c:func:`PyUnicode_InternInPlace` no longer prevents its argument from being
2+
garbage collected.
3+
4+
Several functions that take ``char *`` are now
5+
documented as possibly preventing string objects from being garbage
6+
collected; refer to their documentation for details:
7+
:c:func:`PyUnicode_InternFromString`,
8+
:c:func:`PyDict_SetItemString`,
9+
:c:func:`PyObject_SetAttrString`,
10+
:c:func:`PyObject_DelAttrString`,
11+
:c:func:`PyUnicode_InternFromString`,
12+
and ``PyModule_Add*`` convenience functions.

‎Modules/_ctypes/_ctypes.c

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2303,9 +2303,14 @@ PyCSimpleType_init(PyObject *self, PyObject *args, PyObject *kwds)
23032303
if (!meth) {
23042304
return -1;
23052305
}
2306-
x = PyDict_SetItemString(((PyTypeObject*)self)->tp_dict,
2307-
ml->ml_name,
2308-
meth);
2306+
PyObject *name = PyUnicode_FromString(ml->ml_name);
2307+
if (name == NULL) {
2308+
Py_DECREF(meth);
2309+
return -1;
2310+
}
2311+
PyUnicode_InternInPlace(&name);
2312+
x = PyDict_SetItem(((PyTypeObject*)self)->tp_dict, name, meth);
2313+
Py_DECREF(name);
23092314
Py_DECREF(meth);
23102315
if (x == -1) {
23112316
return -1;

‎Objects/unicodeobject.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15584,7 +15584,7 @@ void
1558415584
PyUnicode_InternInPlace(PyObject **p)
1558515585
{
1558615586
PyInterpreterState *interp = _PyInterpreterState_GET();
15587-
_PyUnicode_InternImmortal(interp, p);
15587+
_PyUnicode_InternMortal(interp, p);
1558815588
}
1558915589

1559015590
// Public-looking name kept for the stable ABI; user should not call this:

0 commit comments

Comments
 (0)
0