From ad7ee2d0c2f8d80ac5816b157c0af8ddf7bd935b Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:51:13 -0500 Subject: [PATCH 01/73] Document the term 'thread state.' --- Doc/glossary.rst | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index f67f3ecad0bc40..0f40a3c4d85c50 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -622,6 +622,10 @@ Glossary multi-threaded applications and makes it easier to use multi-core CPUs efficiently. For more details, see :pep:`703`. + In Python's C API, a function might declare that it requires (or doesn't require) + the GIL to be held in order to use it. This refers to having an active + :term:`thread state` for the current thread. + hash-based pyc A bytecode cache file that uses the hash rather than the last-modified time of the corresponding source file to determine its validity. See @@ -1281,6 +1285,19 @@ Glossary See also :term:`binary file` for a file object able to read and write :term:`bytes-like objects `. + thread state + In Python's C API, a thread state is a structure that holds + information about the current thread. A thread state must be + active for the current thread in order to call most of the C API. + + On most builds of Python, an attached thread state means that the + caller holds the :term:`GIL` for the current interpreter, but on + :term:`free-threaded ` builds, multiple thread states + can be active at once. + + See :ref:`Thread State and the Global Interpreter Lock ` for more + information. + triple-quoted string A string which is bound by three instances of either a quotation mark (") or an apostrophe ('). While they don't provide any functionality From bea045a2d2d7883aa6570de2f5c9a834e3e00d03 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:52:07 -0500 Subject: [PATCH 02/73] Change term for PyDict_GetItem --- Doc/c-api/dict.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/dict.rst b/Doc/c-api/dict.rst index ce73fa0cc60ebb..6f960556db0ba9 100644 --- a/Doc/c-api/dict.rst +++ b/Doc/c-api/dict.rst @@ -127,7 +127,7 @@ Dictionary Objects Prefer the :c:func:`PyDict_GetItemWithError` function instead. .. versionchanged:: 3.10 - Calling this API without :term:`GIL` held had been allowed for historical + Calling this API without an active :term:`thread state` had been allowed for historical reason. It is no longer allowed. From e286ba54979d6f9d6964efe7c50b50707a6c25a3 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:52:36 -0500 Subject: [PATCH 03/73] Change term for PyErr_Occurred --- Doc/c-api/exceptions.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index c1f0bd750361d6..e0272726987918 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -413,7 +413,7 @@ Querying the error indicator own a reference to the return value, so you do not need to :c:func:`Py_DECREF` it. - The caller must hold the GIL. + The caller must have an active :term:`thread state`. .. note:: From 6006b2273e6a6af8124d4907b38ac34b5163f5e6 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:53:12 -0500 Subject: [PATCH 04/73] Change term for PyErr_SetInterrupt --- Doc/c-api/exceptions.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index e0272726987918..ce7da630227c20 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -675,7 +675,7 @@ Signal Handling .. note:: This function is async-signal-safe. It can be called without - the :term:`GIL` and from a C signal handler. + an active :term:`thread state` and from a C signal handler. .. c:function:: int PyErr_SetInterruptEx(int signum) From 8dc3928b00f8c7537282df48e29bd1285706a1e0 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:53:36 -0500 Subject: [PATCH 05/73] Change term for PyErr_SetInterruptEx --- Doc/c-api/exceptions.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index ce7da630227c20..27b4a88ce889c3 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -702,7 +702,7 @@ Signal Handling .. note:: This function is async-signal-safe. It can be called without - the :term:`GIL` and from a C signal handler. + an active :term:`thread state` and from a C signal handler. .. versionadded:: 3.10 From f46ca3d9b90e31bfe8d7f73bb4efcd508bbc67bd Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:54:25 -0500 Subject: [PATCH 06/73] Change term for PyConfig_Get --- Doc/c-api/init_config.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 6b33d93a9f2af9..ec0a4a3a632096 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -1893,8 +1893,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option * ``list[str]`` * ``dict[str, str]`` - The caller must hold the GIL. The function cannot be called before - Python initialization nor after Python finalization. + The caller must have an active :term:`thread state`. The function cannot + be called before Python initialization nor after Python finalization. .. versionadded:: 3.14 From 555a03c0c72d04b743f50167f30437432284810c Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:54:43 -0500 Subject: [PATCH 07/73] Change term for PyConfig_Names --- Doc/c-api/init_config.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index ec0a4a3a632096..093fcc0ec9642d 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -1916,8 +1916,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option * Return a new reference on success. * Set an exception and return ``NULL`` on error. - The caller must hold the GIL. The function cannot be called before - Python initialization nor after Python finalization. + The caller must have an active :term:`thread state`. The function cannot + be called before Python initialization nor after Python finalization. .. versionadded:: 3.14 From ddb98cd627ca33765eb6506c2e516b2077eb20c7 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:54:56 -0500 Subject: [PATCH 08/73] Change term for PyConfig_Set --- Doc/c-api/init_config.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 093fcc0ec9642d..4b426632f65f29 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -1931,8 +1931,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option * Raise a :exc:`ValueError` if the option is read-only (cannot be set). * Raise a :exc:`TypeError` if *value* has not the proper type. - The caller must hold the GIL. The function cannot be called before - Python initialization nor after Python finalization. + The caller must have an active :term:`thread state`. The function cannot + be called before Python initialization nor after Python finalization. .. versionadded:: 3.14 From 5d423f846ee3514ad07b7f7c4e0ed0a8abcb6945 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:55:23 -0500 Subject: [PATCH 09/73] Change term for PyUnstable_AtExit --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index dd63dd013e32dc..0751599f6c3685 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -573,7 +573,7 @@ Initializing and finalizing the interpreter This is similar to :c:func:`Py_AtExit`, but takes an explicit interpreter and data pointer for the callback. - The :term:`GIL` must be held for *interp*. + A :term:`thread state` must be active for *interp*. .. versionadded:: 3.13 From d2d2f67509aa63dc09d92124e1277f31ada0e0c1 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:58:02 -0500 Subject: [PATCH 10/73] Change term for Py_Initialize note --- Doc/c-api/init.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 0751599f6c3685..c5364bdf532925 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1345,7 +1345,8 @@ Low-level API All of the following functions must be called after :c:func:`Py_Initialize`. .. versionchanged:: 3.7 - :c:func:`Py_Initialize()` now initializes the :term:`GIL`. + :c:func:`Py_Initialize()` now initializes the :term:`GIL` + and actives a :term:`thread state`. .. c:function:: PyInterpreterState* PyInterpreterState_New() From aa9c4c67b4be5d88f6f86421fa668327c61d6ef4 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:58:28 -0500 Subject: [PATCH 11/73] Change term for PyInterpreterState_Get --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index c5364bdf532925..933888c114d34d 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1467,7 +1467,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. Issue a fatal error if there no current Python thread state or no current interpreter. It cannot return NULL. - The caller must hold the GIL. + The caller must have an active :term:`thread state`. .. versionadded:: 3.9 From c05be67ca27671731421fb98b0c846cc5fdddd3d Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:58:38 -0500 Subject: [PATCH 12/73] Change term for PyInterpreterState_GetID --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 933888c114d34d..86771e0a8364b8 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1477,7 +1477,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. Return the interpreter's unique ID. If there was any error in doing so then ``-1`` is returned and an error is set. - The caller must hold the GIL. + The caller must have an active :term:`thread state`. .. versionadded:: 3.7 From 3a69922bfd63d6f1a2b003510bd36e15ce5f0063 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 12:59:16 -0500 Subject: [PATCH 13/73] Change term for PyThreadState_SetAsyncEx --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 86771e0a8364b8..a5850c6849e6b5 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1537,7 +1537,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. Asynchronously raise an exception in a thread. The *id* argument is the thread id of the target thread; *exc* is the exception object to be raised. This function does not steal any references to *exc*. To prevent naive misuse, you - must write your own C extension to call this. Must be called with the GIL held. + must write your own C extension to call this. Must be called with an active :term:`thread state`. Returns the number of thread states modified; this is normally one, but will be zero if the thread id isn't found. If *exc* is ``NULL``, the pending exception (if any) for the thread is cleared. This raises no exceptions. From 3336ad7b40a0df857e0fd9bcf40fc31368d97916 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:01:01 -0500 Subject: [PATCH 14/73] Change term for Py_NewInterpreterFromConfig --- Doc/c-api/init.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index a5850c6849e6b5..df3185f44a424b 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1729,10 +1729,11 @@ function. You can create and destroy them using the following functions: must be held before calling this function and is still held when it returns. Likewise a current thread state must be set on entry. On success, the returned thread state will be set as current. If the - sub-interpreter is created with its own GIL then the GIL of the - calling interpreter will be released. When the function returns, - the new interpreter's GIL will be held by the current thread and - the previously interpreter's GIL will remain released here. + sub-interpreter is created with its own :term:`thread state` then the + :term:`thread state` of the calling interpreter will be detached. + When the function returns, the new interpreter's :term:`thread state` + will be activated for the current thread and the previously interpreter's + :term:`thread state` will remain detached here. .. versionadded:: 3.12 From 399168126d15f53de8184ce14eb2c896afb0f4b1 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:01:19 -0500 Subject: [PATCH 15/73] Change term for Py_FinalizeEx --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index df3185f44a424b..7c99a6ce35c643 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1821,7 +1821,7 @@ function. You can create and destroy them using the following functions: the current thread state is ``NULL``. All thread states associated with this interpreter are destroyed. The global interpreter lock used by the target interpreter must be held before calling this - function. No GIL is held when it returns. + function. No :term:`thread state` is active when it returns. :c:func:`Py_FinalizeEx` will destroy all sub-interpreters that haven't been explicitly destroyed at that point. From 6d397ebb1d7b69102ba6cad002c9193c2998cdce Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:02:12 -0500 Subject: [PATCH 16/73] Change term for Py_AddPendingCall --- Doc/c-api/init.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 7c99a6ce35c643..5a313195d73cdf 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1926,9 +1926,9 @@ pointer and a void pointer argument. This function doesn't need a current thread state to run, and it doesn't need the global interpreter lock. - To call this function in a subinterpreter, the caller must hold the GIL. - Otherwise, the function *func* can be scheduled to be called from the wrong - interpreter. + To call this function in a subinterpreter, the caller must have an active + :term:`thread state`. Otherwise, the function *func* can be scheduled to + be called from the wrong interpreter. .. warning:: This is a low-level function, only useful for very special cases. From 3b7b3ff5b2e3ae71e1b15247881ebc271bb0a757 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:02:59 -0500 Subject: [PATCH 17/73] Change term for PyEval_SetProfile --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 5a313195d73cdf..21bb31c297be9a 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -2069,7 +2069,7 @@ Python-level trace functions in previous versions. See also the :func:`sys.setprofile` function. - The caller must hold the :term:`GIL`. + The caller must have an active :term:`thread state`. .. c:function:: void PyEval_SetProfileAllThreads(Py_tracefunc func, PyObject *obj) From 5358d144b2ca4e34603abce626349e2d29a0d4cc Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:03:08 -0500 Subject: [PATCH 18/73] Change term for PyEval_SetProfileAllThreads --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 21bb31c297be9a..b634781a8f8216 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -2076,7 +2076,7 @@ Python-level trace functions in previous versions. Like :c:func:`PyEval_SetProfile` but sets the profile function in all running threads belonging to the current interpreter instead of the setting it only on the current thread. - The caller must hold the :term:`GIL`. + The caller must have an active :term:`thread state`. As :c:func:`PyEval_SetProfile`, this function ignores any exceptions raised while setting the profile functions in all threads. From 093ba0523c3d4603236058eb5dc627b1679720d8 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:03:19 -0500 Subject: [PATCH 19/73] Change term for PyEval_SetTrace --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index b634781a8f8216..7ac12b6ae17178 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -2095,7 +2095,7 @@ Python-level trace functions in previous versions. See also the :func:`sys.settrace` function. - The caller must hold the :term:`GIL`. + The caller must have an active :term:`thread state`. .. c:function:: void PyEval_SetTraceAllThreads(Py_tracefunc func, PyObject *obj) From 11582fa57ced80326685c0a49fe43688325c2945 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:04:19 -0500 Subject: [PATCH 20/73] Change term for PyEval_SetTraceAllThreads --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 7ac12b6ae17178..324be8d309b7a8 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -2102,7 +2102,7 @@ Python-level trace functions in previous versions. Like :c:func:`PyEval_SetTrace` but sets the tracing function in all running threads belonging to the current interpreter instead of the setting it only on the current thread. - The caller must hold the :term:`GIL`. + The caller must have an active :term:`thread state`. As :c:func:`PyEval_SetTrace`, this function ignores any exceptions raised while setting the trace functions in all threads. From 3d55d2d684799553ad67a91f040bae3f207600b7 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:05:31 -0500 Subject: [PATCH 21/73] Change term for PyRefTracer_SetTracer --- Doc/c-api/init.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 324be8d309b7a8..885e143470f354 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -2144,10 +2144,10 @@ Reference tracing Not that tracer functions **must not** create Python objects inside or otherwise the call will be re-entrant. The tracer also **must not** clear - any existing exception or set an exception. The GIL will be held every time - the tracer function is called. + any existing exception or set an exception. A :term:`thread state` will be active + every time the tracer function is called. - The GIL must be held when calling this function. + The :term:`thread state` must be active when calling this function. .. versionadded:: 3.13 From 2785c1d9b90812bb5f883505215898ebdb0a227b Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:05:50 -0500 Subject: [PATCH 22/73] Change term for PyRefTracer_GetTracer --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 885e143470f354..6ba2a90a801143 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -2158,7 +2158,7 @@ Reference tracing If no tracer was registered this function will return NULL and will set the **data** pointer to NULL. - The GIL must be held when calling this function. + A :term:`thread state` must be active when calling this function. .. versionadded:: 3.13 From 6d80b020aabd23f0eb952d845c3f1884a7001929 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:07:42 -0500 Subject: [PATCH 23/73] Change term for PyMutex_Lock --- Doc/c-api/init.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 6ba2a90a801143..f9fa6569b813bd 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -2215,7 +2215,7 @@ CPython C level APIs are similar to those offered by pthreads and Windows: use a thread key and functions to associate a :c:expr:`void*` value per thread. -The GIL does *not* need to be held when calling these functions; they supply +A :term:`thread state` does *not* need to be active when calling these functions; they supply their own locking. Note that :file:`Python.h` does not include the declaration of the TLS APIs, @@ -2385,7 +2385,7 @@ The C-API provides a basic mutual exclusion lock. Lock mutex *m*. If another thread has already locked it, the calling thread will block until the mutex is unlocked. While blocked, the thread - will temporarily release the :term:`GIL` if it is held. + will temporarily detach the current :term:`thread state` if one is active. .. versionadded:: 3.13 From 94e046b27cdb853c92d35d9a9a8b57550094d9e6 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:08:36 -0500 Subject: [PATCH 24/73] Change term for allocator domains --- Doc/c-api/memory.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index aa2ef499bddaf3..9a207d14c8220f 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -110,12 +110,12 @@ The three allocation domains are: * Raw domain: intended for allocating memory for general-purpose memory buffers where the allocation *must* go to the system allocator or where the - allocator can operate without the :term:`GIL`. The memory is requested directly - from the system. See :ref:`Raw Memory Interface `. + allocator can operate without an active :term:`thread state`. The memory + is requested directly from the system. See :ref:`Raw Memory Interface `. * "Mem" domain: intended for allocating memory for Python buffers and general-purpose memory buffers where the allocation must be performed with - the :term:`GIL` held. The memory is taken from the Python private heap. + an active :term:`thread state`. The memory is taken from the Python private heap. See :ref:`Memory Interface `. * Object domain: intended for allocating memory for Python objects. The From 713beb212892a056c65e42aa9aecf1a101fded22 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:09:30 -0500 Subject: [PATCH 25/73] Change term for raw memory interface --- Doc/c-api/memory.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 9a207d14c8220f..76122ee6c97954 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -139,8 +139,8 @@ Raw Memory Interface ==================== The following function sets are wrappers to the system allocator. These -functions are thread-safe, the :term:`GIL ` does not -need to be held. +functions are thread-safe, so a :term:`thread state` does not +need to be active. The :ref:`default raw memory allocator ` uses the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` From 2140bde5fad7f69a1f5c6d3fe07e15aa413fb1dd Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:09:54 -0500 Subject: [PATCH 26/73] Change term for memory interface --- Doc/c-api/memory.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 76122ee6c97954..35ab89e27d1cce 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -213,7 +213,7 @@ The :ref:`default memory allocator ` uses the .. warning:: - The :term:`GIL ` must be held when using these + A :term:`thread state` must be active when using these functions. .. versionchanged:: 3.6 From f60821127c5432fa1ce3453aa16b53566ff9ee2a Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:10:29 -0500 Subject: [PATCH 27/73] Change term for object allocators --- Doc/c-api/memory.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 35ab89e27d1cce..e355ff90309cab 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -327,8 +327,7 @@ The :ref:`default object allocator ` uses the .. warning:: - The :term:`GIL ` must be held when using these - functions. + A :term:`thread state` must be active when using these functions. .. c:function:: void* PyObject_Malloc(size_t n) From d619387d15ce166de0ba0dc1e18347b25b7cfdcc Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:11:24 -0500 Subject: [PATCH 28/73] Change term for PyMem_SetAllocator --- Doc/c-api/memory.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index e355ff90309cab..5f251ea645a25c 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -484,12 +484,12 @@ Customize Memory Allocators zero bytes. For the :c:macro:`PYMEM_DOMAIN_RAW` domain, the allocator must be - thread-safe: the :term:`GIL ` is not held when the + thread-safe: a :term:`thread state` is not held when the allocator is called. For the remaining domains, the allocator must also be thread-safe: the allocator may be called in different interpreters that do not - share a ``GIL``. + share a :term:`GIL`. If the new allocator is not a hook (does not call the previous allocator), the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the From e7828ed49d8b56ab7841e0dc6b935b0c65cb66bf Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:12:04 -0500 Subject: [PATCH 29/73] Change term for PyMem_SetAllocator (again) --- Doc/c-api/memory.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 5f251ea645a25c..da956a52cb1317 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -506,8 +506,8 @@ Customize Memory Allocators :c:func:`Py_InitializeFromConfig` to install a custom memory allocator. There are no restrictions over the installed allocator other than the ones imposed by the domain (for instance, the Raw - Domain allows the allocator to be called without the GIL held). See - :ref:`the section on allocator domains ` for more + Domain allows the allocator to be called without an active :term:`thread state`). + See :ref:`the section on allocator domains ` for more information. * If called after Python has finish initializing (after From 6b335659c31814d3e93e559982b94a54390a104e Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:12:34 -0500 Subject: [PATCH 30/73] Change term for debug hooks --- Doc/c-api/memory.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index da956a52cb1317..81c46365f39e95 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -554,7 +554,7 @@ Runtime checks: called on a memory block allocated by :c:func:`PyMem_Malloc`. - Detect write before the start of the buffer (buffer underflow). - Detect write after the end of the buffer (buffer overflow). -- Check that the :term:`GIL ` is held when +- Check that a :term:`thread state` is active when allocator functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. From 623017725d84f860f344798287844c81fdeed987 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:30:00 -0500 Subject: [PATCH 31/73] Change term for debug hooks (again) --- Doc/c-api/memory.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 81c46365f39e95..96b2f4a02bd0bc 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -619,7 +619,7 @@ PYMEM_CLEANBYTE (meaning uninitialized memory is getting used). The :c:func:`PyMem_SetupDebugHooks` function now also works on Python compiled in release mode. On error, the debug hooks now use :mod:`tracemalloc` to get the traceback where a memory block was allocated. - The debug hooks now also check if the GIL is held when functions of + The debug hooks now also check if a :term:`thread state` is active when functions of :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are called. From f74832ae09ae50d9923bea6b49dc52543f46eca9 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:30:57 -0500 Subject: [PATCH 32/73] Change term for PyState* --- Doc/c-api/module.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index f82a050ab75de0..6daed8392f584e 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -708,7 +708,7 @@ since multiple such modules can be created from a single definition. mechanisms (either by calling it directly, or by referring to its implementation for details of the required state updates). - The caller must hold the GIL. + The caller must have an active :term:`thread state`. Return ``-1`` with an exception set on error, ``0`` on success. @@ -719,6 +719,6 @@ since multiple such modules can be created from a single definition. Removes the module object created from *def* from the interpreter state. Return ``-1`` with an exception set on error, ``0`` on success. - The caller must hold the GIL. + The caller must have an active :term:`thread state`. .. versionadded:: 3.3 From 7161fc0988fd755a91258122a285395f7896ad81 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:32:28 -0500 Subject: [PATCH 33/73] Change term for various things --- Doc/c-api/perfmaps.rst | 2 +- Doc/c-api/sys.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/perfmaps.rst b/Doc/c-api/perfmaps.rst index 3d44d2eb6bf41d..fef2e8c1edefea 100644 --- a/Doc/c-api/perfmaps.rst +++ b/Doc/c-api/perfmaps.rst @@ -16,7 +16,7 @@ kernel/git/torvalds/linux.git/tree/tools/perf/Documentation/jit-interface.txt>`_ In Python, these helper APIs can be used by libraries and features that rely on generating machine code on the fly. -Note that holding the Global Interpreter Lock (GIL) is not required for these APIs. +Note that holding an active :term:`thread state` is not required for these APIs. .. c:function:: int PyUnstable_PerfMapState_Init(void) diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index c688afdca8231d..441a26fe53505d 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -346,7 +346,7 @@ accessible to C code. They all work with the current interpreter thread's silently abort the operation by raising an error subclassed from :class:`Exception` (other errors will not be silenced). - The hook function is always called with the GIL held by the Python + The hook function is always called with a :term:`thread state` by the Python interpreter that raised the event. See :pep:`578` for a detailed description of auditing. Functions in the From d58b74f785ea33b2f4aeb62fe42404c2315e3c2a Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:32:53 -0500 Subject: [PATCH 34/73] Change term for clock functions --- Doc/c-api/time.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/time.rst b/Doc/c-api/time.rst index 7032cc48aa6913..2e82c11fcd3ea9 100644 --- a/Doc/c-api/time.rst +++ b/Doc/c-api/time.rst @@ -56,7 +56,7 @@ range. system time.) As any other C API (unless otherwise specified), the functions must be called -with the :term:`GIL` held. +with an active :term:`thread state`. .. c:function:: int PyTime_Monotonic(PyTime_t *result) From a761c4a8757b5767e9343b2fa4711d394c5bfcae Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:34:01 -0500 Subject: [PATCH 35/73] Change term for raw clock functions --- Doc/c-api/time.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/c-api/time.rst b/Doc/c-api/time.rst index 2e82c11fcd3ea9..a97ad0bb152b97 100644 --- a/Doc/c-api/time.rst +++ b/Doc/c-api/time.rst @@ -78,29 +78,29 @@ Raw Clock Functions ------------------- Similar to clock functions, but don't set an exception on error and don't -require the caller to hold the GIL. +require the caller to have an active :term:`thread state`. On success, the functions return ``0``. On failure, they set ``*result`` to ``0`` and return ``-1``, *without* setting -an exception. To get the cause of the error, acquire the GIL and call the +an exception. To get the cause of the error, attach a :term:`thread state`, and call the regular (non-``Raw``) function. Note that the regular function may succeed after the ``Raw`` one failed. .. c:function:: int PyTime_MonotonicRaw(PyTime_t *result) Similar to :c:func:`PyTime_Monotonic`, - but don't set an exception on error and don't require holding the GIL. + but don't set an exception on error and don't require an active :term:`thread state`. .. c:function:: int PyTime_PerfCounterRaw(PyTime_t *result) Similar to :c:func:`PyTime_PerfCounter`, - but don't set an exception on error and don't require holding the GIL. + but don't set an exception on error and don't require an active :term:`thread state`. .. c:function:: int PyTime_TimeRaw(PyTime_t *result) Similar to :c:func:`PyTime_Time`, - but don't set an exception on error and don't require holding the GIL. + but don't set an exception on error and don't require an active :term:`thread state`. Conversion functions From de122f5190944a8e2ee889830f7c6b8356e3e1b6 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:34:40 -0500 Subject: [PATCH 36/73] Change term for tp_dealloc warning --- Doc/c-api/typeobj.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index ba58cc1c26c70b..78c1e92cf6142a 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -689,7 +689,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) object becomes part of a refcount cycle, that cycle might be collected by a garbage collection on any thread). This is not a problem for Python API calls, since the thread on which :c:member:`!tp_dealloc` is called - will own the Global Interpreter Lock (GIL). However, if the object being + with an active :term:`thread state`. However, if the object being destroyed in turn destroys objects from some other C or C++ library, care should be taken to ensure that destroying those objects on the thread which called :c:member:`!tp_dealloc` will not violate any assumptions of From 0b0176b243c6eebd947a905e76875c23598eeaaf Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 13:35:20 -0500 Subject: [PATCH 37/73] Change term for the types tutorial --- Doc/extending/newtypes_tutorial.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index bcf938f117d148..c34c90b8307d12 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -403,8 +403,8 @@ the new attribute values. We might be tempted, for example to assign the But this would be risky. Our type doesn't restrict the type of the ``first`` member, so it could be any kind of object. It could have a destructor that causes code to be executed that tries to access the -``first`` member; or that destructor could release the -:term:`Global interpreter Lock ` and let arbitrary code run in other +``first`` member; or that destructor could detach the +:term:`term state` and let arbitrary code run in other threads that accesses and modifies our object. To be paranoid and protect ourselves against this possibility, we almost @@ -413,8 +413,8 @@ don't we have to do this? * when we absolutely know that the reference count is greater than 1; -* when we know that deallocation of the object [#]_ will neither release - the :term:`GIL` nor cause any calls back into our type's code; +* when we know that deallocation of the object [#]_ will neither detach + the :term:`thread state` nor cause any calls back into our type's code; * when decrementing a reference count in a :c:member:`~PyTypeObject.tp_dealloc` handler on a type which doesn't support cyclic garbage collection [#]_. From c15f5c4e3782c4c1a19f0d4f663d8b4e6ccb9905 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 14:06:02 -0500 Subject: [PATCH 38/73] Add more information about thread states. --- Doc/c-api/init.rst | 47 ++++++++++++++++++++++++++++++---------------- 1 file changed, 31 insertions(+), 16 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index f9fa6569b813bd..cc891d25101d1d 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -940,7 +940,8 @@ Thread State and the Global Interpreter Lock single: interpreter lock single: lock, interpreter -The Python interpreter is not fully thread-safe. In order to support +Unless on a :term:`free-threaded ` build of :term:`CPython`, +the Python interpreter is not fully thread-safe. In order to support multi-threaded Python programs, there's a global lock, called the :term:`global interpreter lock` or :term:`GIL`, that must be held by the current thread before it can safely access Python objects. Without the lock, even the simplest @@ -961,14 +962,28 @@ a file, so that other Python threads can run in the meantime. single: PyThreadState (C type) The Python interpreter keeps some thread-specific bookkeeping information -inside a data structure called :c:type:`PyThreadState`. There's also one -global variable pointing to the current :c:type:`PyThreadState`: it can +inside a data structure called :c:type:`PyThreadState`, known as a :term:`thread state`. +There's also one thread-local variable pointing to the current :c:type:`PyThreadState`: it can be retrieved using :c:func:`PyThreadState_Get`. -Releasing the GIL from extension code -------------------------------------- +A thread can only have one attached :term:`thread state` at a time. An attached +:term:`thread state` is typically analogous with holding the :term:`GIL`, except on +:term:`free-threaded ` builds. On builds with the :term:`GIL` enabled, +attaching a thread state will block until the :term:`GIL` can be acquired. +However, even on builds with the :term:`GIL` disabled, it is still required +to have a :term:`thread state` attached to the current thread to call most of +the C API. -Most extension code manipulating the :term:`GIL` has the following simple +In general, a :term:`thread state` will always be active for the current thread +when using Python's C API. Only in some specific cases (such as in a +:c:macro:`Py_BEGIN_ALLOW_THREADS` block) will the thread not have an active +thread state. If uncertain, check if :c:func:`PyThreadState_GetUnchecked` returns +``NULL``. + +Detaching the thread state from extension code +---------------------------------------------- + +Most extension code manipulating the :term:`thread state` has the following simple structure:: Save the thread state in a local variable. @@ -1004,20 +1019,20 @@ The block above expands to the following code:: single: PyEval_SaveThread (C function) Here is how these functions work: the global interpreter lock is used to protect the pointer to the -current thread state. When releasing the lock and saving the thread state, +current :term:`thread state`. When releasing the lock and saving the :term:`thread state`, the current thread state pointer must be retrieved before the lock is released -(since another thread could immediately acquire the lock and store its own thread -state in the global variable). Conversely, when acquiring the lock and restoring -the thread state, the lock must be acquired before storing the thread state +(since another thread could immediately acquire the lock and store its own :term:`thread +state` in the global variable). Conversely, when acquiring the lock and restoring +the :term:`thread state`, the lock must be acquired before storing the :term:`thread state`` pointer. .. note:: - Calling system I/O functions is the most common use case for releasing - the GIL, but it can also be useful before calling long-running computations - which don't need access to Python objects, such as compression or - cryptographic functions operating over memory buffers. For example, the - standard :mod:`zlib` and :mod:`hashlib` modules release the GIL when - compressing or hashing data. + Calling system I/O functions is the most common use case for detaching + the :term:`thread state`, but it can also be useful before calling + long-running computations which don't need access to Python objects, such + as compression or cryptographic functions operating over memory buffers. + For example, the standard :mod:`zlib` and :mod:`hashlib` modules detach the + :term:`thread state` when compressing or hashing data. .. _gilstate: From 0130b20320e253bc863df23375d7a928fb98f78e Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 14:22:56 -0500 Subject: [PATCH 39/73] Change terms for 'cautions regarding runtime finalization' --- Doc/c-api/init.rst | 57 ++++++++++++++++++++++++++-------------------- 1 file changed, 32 insertions(+), 25 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index cc891d25101d1d..5f7fad442dd491 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1018,13 +1018,23 @@ The block above expands to the following code:: single: PyEval_RestoreThread (C function) single: PyEval_SaveThread (C function) -Here is how these functions work: the global interpreter lock is used to protect the pointer to the -current :term:`thread state`. When releasing the lock and saving the :term:`thread state`, -the current thread state pointer must be retrieved before the lock is released -(since another thread could immediately acquire the lock and store its own :term:`thread -state` in the global variable). Conversely, when acquiring the lock and restoring -the :term:`thread state`, the lock must be acquired before storing the :term:`thread state`` -pointer. +Here is how these functions work: + +The :term:`thread state` holds the :term:`GIL` for the entire interpreter. When detaching +the :term:`thread state`, the :term:`GIL` is released, allowing other threads to attach +their own :term:`thread state`, thus getting the :term:`GIL` and can start executing. +The pointer to the now-detached :term:`thread state`` is stored as a local variable. +Upon reaching :c:macro:`Py_END_ALLOW_THREADS`, the :term:`thread state` that was +previously attached is given to :c:func:`PyEval_RestoreThread`. This function will +block until another thread that the :term:`GIL` can be re-acquired, thus allowing +the :term:`thread state` to get re-attached and the C API can be called again. + +For :term:`free-threaded ` builds, the :term:`GIL` is normally +out of the question, but detaching the thread state is still required for blocking I/O +and long operations. The difference is that threads don't have to wait for the :term:`GIL` +to be released to attach their thread state, allowing true multi-core parallelism. + +:c:func:`PyEval_SaveThread` .. note:: Calling system I/O functions is the most common use case for detaching @@ -1044,16 +1054,15 @@ When threads are created using the dedicated Python APIs (such as the :mod:`threading` module), a thread state is automatically associated to them and the code showed above is therefore correct. However, when threads are created from C (for example by a third-party library with its own thread -management), they don't hold the GIL, nor is there a thread state structure -for them. +management), they don't hold the :term:`GIL`, because they don't have a +:term:`thread state`. If you need to call Python code from these threads (often this will be part of a callback API provided by the aforementioned third-party library), you must first register these threads with the interpreter by -creating a thread state data structure, then acquiring the GIL, and finally -storing their thread state pointer, before you can start using the Python/C -API. When you are done, you should reset the thread state pointer, release -the GIL, and finally free the thread state data structure. +creating a :term:`thread state` before you can start using the Python/C +API. When you are done, you should detach the :term:`thread state`, and +finally free it. The :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release` functions do all of the above automatically. The typical idiom for calling into Python @@ -1126,21 +1135,19 @@ is marked as *finalizing*: :c:func:`_Py_IsFinalizing` and thread* that initiated finalization (typically the main thread) is allowed to acquire the :term:`GIL`. -If any thread, other than the finalization thread, attempts to acquire the GIL -during finalization, either explicitly via :c:func:`PyGILState_Ensure`, -:c:macro:`Py_END_ALLOW_THREADS`, :c:func:`PyEval_AcquireThread`, or -:c:func:`PyEval_AcquireLock`, or implicitly when the interpreter attempts to -reacquire it after having yielded it, the thread enters **a permanently blocked -state** where it remains until the program exits. In most cases this is -harmless, but this can result in deadlock if a later stage of finalization -attempts to acquire a lock owned by the blocked thread, or otherwise waits on -the blocked thread. +If any thread, other than the finalization thread, attempts to attach a :term:`thread state` +during finalization, either explicitly via a :term:`thread state` function, or +implicitly when the interpreter attempts yields the :term:`GIL` by detaching the +:term:`thread state`, the thread enters **a permanently blocked state** where it +remains until the program exits. In most cases this is harmless, but this can result +in deadlock if a later stage of finalization attempts to acquire a lock owned by the +blocked thread, or otherwise waits on the blocked thread. Gross? Yes. This prevents random crashes and/or unexpectedly skipped C++ finalizations further up the call stack when such threads were forcibly exited -here in CPython 3.13 and earlier. The CPython runtime GIL acquiring C APIs -have never had any error reporting or handling expectations at GIL acquisition -time that would've allowed for graceful exit from this situation. Changing that +here in CPython 3.13 and earlier. The CPython runtime :term:`thread state` C APIs +have never had any error reporting or handling expectations at :term:`thread state` +attachment time that would've allowed for graceful exit from this situation. Changing that would require new stable C APIs and rewriting the majority of C code in the CPython ecosystem to use those with error handling. From a1e47b7066bb7e3122e727324bb29768b7361de8 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 14:27:02 -0500 Subject: [PATCH 40/73] Simplify PyEval_SaveThread and PyEval_RestoreThread --- Doc/c-api/init.rst | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 5f7fad442dd491..587364e4848b2a 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1210,17 +1210,14 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyEval_SaveThread() - Release the global interpreter lock (if it has been created) and reset the - thread state to ``NULL``, returning the previous thread state (which is not - ``NULL``). If the lock has been created, the current thread must have - acquired it. + Detach the active :term:`thread state` (if it has been created) and + return it. .. c:function:: void PyEval_RestoreThread(PyThreadState *tstate) - Acquire the global interpreter lock (if it has been created) and set the - thread state to *tstate*, which must not be ``NULL``. If the lock has been - created, the current thread must not have acquired it, otherwise deadlock + Set the current :term:`thread state` to *tstate*, which must not be ``NULL``. + The passed :term:`thread state` **should not** be attached, otherwise deadlock ensues. .. note:: From 0173c4679836a08a07af93051ac4217e59273023 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 14:36:31 -0500 Subject: [PATCH 41/73] Some more terminology fixups. --- Doc/c-api/init.rst | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 587364e4848b2a..539cf93f5223e6 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1232,9 +1232,9 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyThreadState_Get() - Return the current thread state. The global interpreter lock must be held. - When the current thread state is ``NULL``, this issues a fatal error (so that - the caller needn't check for ``NULL``). + Return the active :term:`thread state`. If there is no active :term:`thread state` (such + as when inside of :c:macro:`Py_BEGIN_ALLOW_THREADS` block), then this issues a fatal + error (so that the caller needn't check for ``NULL``). See also :c:func:`PyThreadState_GetUnchecked`. @@ -1252,9 +1252,12 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyThreadState_Swap(PyThreadState *tstate) - Swap the current thread state with the thread state given by the argument - *tstate*, which may be ``NULL``. The global interpreter lock must be held - and is not released. + Swap the current :term:`thread state` with *tstate*. + + If there is an attached :term:`thread state` for the current + thread, it will be detached. Upon returning from this function, + *tstate* will become attached instead if it's not ``NULL``. If it + is ``NULL``, then no :term:`thread state` will be attached upon returning. The following functions use thread-local storage, and are not compatible @@ -1263,7 +1266,7 @@ with sub-interpreters: .. c:function:: PyGILState_STATE PyGILState_Ensure() Ensure that the current thread is ready to call the Python C API regardless - of the current state of Python, or of the global interpreter lock. This may + of the current state of Python, or of the global interpreter lock.This may be called as many times as desired by a thread as long as each call is matched with a call to :c:func:`PyGILState_Release`. In general, other thread-related APIs may be used between :c:func:`PyGILState_Ensure` and @@ -1272,15 +1275,15 @@ with sub-interpreters: :c:macro:`Py_BEGIN_ALLOW_THREADS` and :c:macro:`Py_END_ALLOW_THREADS` macros is acceptable. - The return value is an opaque "handle" to the thread state when + The return value is an opaque "handle" to the :term:`thread state`` when :c:func:`PyGILState_Ensure` was called, and must be passed to :c:func:`PyGILState_Release` to ensure Python is left in the same state. Even though recursive calls are allowed, these handles *cannot* be shared - each unique call to :c:func:`PyGILState_Ensure` must save the handle for its call to :c:func:`PyGILState_Release`. - When the function returns, the current thread will hold the GIL and be able - to call arbitrary Python code. Failure is a fatal error. + When the function returns, there will be an active :term:`thread state` + and the thread will be able to call arbitrary Python code. Failure is a fatal error. .. note:: Calling this function from a thread when the runtime is finalizing will @@ -1305,7 +1308,7 @@ with sub-interpreters: .. c:function:: PyThreadState* PyGILState_GetThisThreadState() - Get the current thread state for this thread. May return ``NULL`` if no + Get the current :term:`thread state` for this thread. May return ``NULL`` if no GILState API has been used on the current thread. Note that the main thread always has such a thread-state, even if no auto-thread-state call has been made on the main thread. This is mainly a helper/diagnostic function. @@ -1313,13 +1316,13 @@ with sub-interpreters: .. c:function:: int PyGILState_Check() - Return ``1`` if the current thread is holding the GIL and ``0`` otherwise. + Return ``1`` if the current thread is holding the :term:`GIL` and ``0`` otherwise. This function can be called from any thread at any time. Only if it has had its Python thread state initialized and currently is - holding the GIL will it return ``1``. + holding the :term:`GIL` will it return ``1``. This is mainly a helper/diagnostic function. It can be useful for example in callback contexts or memory allocation functions when - knowing that the GIL is locked can allow the caller to perform sensitive + knowing that the :term:`GIL` is locked can allow the caller to perform sensitive actions or otherwise behave differently. .. versionadded:: 3.4 From 166d970d277a6ece86fef31e9ece00caa865ed94 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 14:41:31 -0500 Subject: [PATCH 42/73] Fix Sphinx build errors. --- Doc/c-api/init.rst | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 539cf93f5223e6..aec381d54b7ee6 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -940,7 +940,7 @@ Thread State and the Global Interpreter Lock single: interpreter lock single: lock, interpreter -Unless on a :term:`free-threaded ` build of :term:`CPython`, +Unless on a :term:`free-threaded ` build of :term:`CPython`, the Python interpreter is not fully thread-safe. In order to support multi-threaded Python programs, there's a global lock, called the :term:`global interpreter lock` or :term:`GIL`, that must be held by the current thread before @@ -968,7 +968,7 @@ be retrieved using :c:func:`PyThreadState_Get`. A thread can only have one attached :term:`thread state` at a time. An attached :term:`thread state` is typically analogous with holding the :term:`GIL`, except on -:term:`free-threaded ` builds. On builds with the :term:`GIL` enabled, +:term:`free-threaded ` builds. On builds with the :term:`GIL` enabled, attaching a thread state will block until the :term:`GIL` can be acquired. However, even on builds with the :term:`GIL` disabled, it is still required to have a :term:`thread state` attached to the current thread to call most of @@ -1023,19 +1023,17 @@ Here is how these functions work: The :term:`thread state` holds the :term:`GIL` for the entire interpreter. When detaching the :term:`thread state`, the :term:`GIL` is released, allowing other threads to attach their own :term:`thread state`, thus getting the :term:`GIL` and can start executing. -The pointer to the now-detached :term:`thread state`` is stored as a local variable. +The pointer to the now-detached :term:`thread state` is stored as a local variable. Upon reaching :c:macro:`Py_END_ALLOW_THREADS`, the :term:`thread state` that was previously attached is given to :c:func:`PyEval_RestoreThread`. This function will block until another thread that the :term:`GIL` can be re-acquired, thus allowing the :term:`thread state` to get re-attached and the C API can be called again. -For :term:`free-threaded ` builds, the :term:`GIL` is normally +For :term:`free-threaded ` builds, the :term:`GIL` is normally out of the question, but detaching the thread state is still required for blocking I/O and long operations. The difference is that threads don't have to wait for the :term:`GIL` to be released to attach their thread state, allowing true multi-core parallelism. -:c:func:`PyEval_SaveThread` - .. note:: Calling system I/O functions is the most common use case for detaching the :term:`thread state`, but it can also be useful before calling @@ -1275,7 +1273,7 @@ with sub-interpreters: :c:macro:`Py_BEGIN_ALLOW_THREADS` and :c:macro:`Py_END_ALLOW_THREADS` macros is acceptable. - The return value is an opaque "handle" to the :term:`thread state`` when + The return value is an opaque "handle" to the :term:`thread state` when :c:func:`PyGILState_Ensure` was called, and must be passed to :c:func:`PyGILState_Release` to ensure Python is left in the same state. Even though recursive calls are allowed, these handles *cannot* be shared - each From adae494e80db873e549b6802572808f5d318d0c8 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 14:49:40 -0500 Subject: [PATCH 43/73] Change some usage of the term 'global interpreter lock' --- Doc/c-api/init.rst | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index aec381d54b7ee6..56c3c54dbb79fc 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -987,9 +987,7 @@ Most extension code manipulating the :term:`thread state` has the following simp structure:: Save the thread state in a local variable. - Release the global interpreter lock. ... Do some blocking I/O operation ... - Reacquire the global interpreter lock. Restore the thread state from the local variable. This is so common that a pair of macros exists to simplify it:: @@ -1371,8 +1369,8 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: PyInterpreterState* PyInterpreterState_New() - Create a new interpreter state object. The global interpreter lock need not - be held, but may be held if it is necessary to serialize calls to this + Create a new interpreter state object. A :term:`thread state` need not + be attached, but may be held if it is necessary to serialize calls to this function. .. audit-event:: cpython.PyInterpreterState_New "" c.PyInterpreterState_New @@ -1380,30 +1378,28 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyInterpreterState_Clear(PyInterpreterState *interp) - Reset all information in an interpreter state object. The global interpreter - lock must be held. + Reset all information in an interpreter state object. A :term:`thread state` + for the interpreter must be attached. .. audit-event:: cpython.PyInterpreterState_Clear "" c.PyInterpreterState_Clear .. c:function:: void PyInterpreterState_Delete(PyInterpreterState *interp) - Destroy an interpreter state object. The global interpreter lock need not be - held. The interpreter state must have been reset with a previous call to + Destroy an interpreter state object. A :term:`thread state` for the interpreter + shouldn't be attached. The interpreter state must have been reset with a previous call to :c:func:`PyInterpreterState_Clear`. .. c:function:: PyThreadState* PyThreadState_New(PyInterpreterState *interp) Create a new thread state object belonging to the given interpreter object. - The global interpreter lock need not be held, but may be held if it is - necessary to serialize calls to this function. - + A :term:`thread state` can be optionally attached. .. c:function:: void PyThreadState_Clear(PyThreadState *tstate) - Reset all information in a thread state object. The global interpreter lock - must be held. + Reset all information in a :term:`thread state` object. The + :term:`thread state` must be active for the current thread. .. versionchanged:: 3.9 This function now calls the :c:member:`PyThreadState.on_delete` callback. @@ -1415,16 +1411,15 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyThreadState_Delete(PyThreadState *tstate) - Destroy a thread state object. The global interpreter lock need not be held. - The thread state must have been reset with a previous call to + Destroy a :term:`thread state`` object. The :term:`thread state` should not + be active. *tstate* must have been reset with a previous call to :c:func:`PyThreadState_Clear`. .. c:function:: void PyThreadState_DeleteCurrent(void) - Destroy the current thread state and release the global interpreter lock. - Like :c:func:`PyThreadState_Delete`, the global interpreter lock must - be held. The thread state must have been reset with a previous call + Destroy the active :term:`thread state` and detach it. + The current :term:`thread state` must have been reset with a previous call to :c:func:`PyThreadState_Clear`. From a70ab115a164f9dd0ff2d1cbe40feb32e01de5fd Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 15:00:14 -0500 Subject: [PATCH 44/73] Add some more terms for disambiguation. --- Doc/c-api/dict.rst | 2 +- Doc/c-api/exceptions.rst | 6 +++--- Doc/c-api/init.rst | 24 ++++++++++++------------ Doc/c-api/init_config.rst | 6 +++--- Doc/c-api/memory.rst | 6 +++--- Doc/c-api/module.rst | 4 ++-- Doc/c-api/perfmaps.rst | 2 +- Doc/c-api/time.rst | 10 +++++----- Doc/c-api/typeobj.rst | 2 +- Doc/glossary.rst | 22 ++++++++++++++++------ 10 files changed, 47 insertions(+), 37 deletions(-) diff --git a/Doc/c-api/dict.rst b/Doc/c-api/dict.rst index 6f960556db0ba9..e55c5c80cb83c0 100644 --- a/Doc/c-api/dict.rst +++ b/Doc/c-api/dict.rst @@ -127,7 +127,7 @@ Dictionary Objects Prefer the :c:func:`PyDict_GetItemWithError` function instead. .. versionchanged:: 3.10 - Calling this API without an active :term:`thread state` had been allowed for historical + Calling this API without an :term:`attached thread state` had been allowed for historical reason. It is no longer allowed. diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 27b4a88ce889c3..eacb24d0bed0c0 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -413,7 +413,7 @@ Querying the error indicator own a reference to the return value, so you do not need to :c:func:`Py_DECREF` it. - The caller must have an active :term:`thread state`. + The caller must have an :term:`attached thread state`. .. note:: @@ -675,7 +675,7 @@ Signal Handling .. note:: This function is async-signal-safe. It can be called without - an active :term:`thread state` and from a C signal handler. + an :term:`attached thread state` and from a C signal handler. .. c:function:: int PyErr_SetInterruptEx(int signum) @@ -702,7 +702,7 @@ Signal Handling .. note:: This function is async-signal-safe. It can be called without - an active :term:`thread state` and from a C signal handler. + an :term:`attached thread state` and from a C signal handler. .. versionadded:: 3.10 diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 56c3c54dbb79fc..20a01220b2c996 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -573,7 +573,7 @@ Initializing and finalizing the interpreter This is similar to :c:func:`Py_AtExit`, but takes an explicit interpreter and data pointer for the callback. - A :term:`thread state` must be active for *interp*. + There must be an :term:`attached thread state` for *interp*. .. versionadded:: 3.13 @@ -1206,7 +1206,7 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyEval_SaveThread() - Detach the active :term:`thread state` (if it has been created) and + Detach the :term:`active thread state` (if it has been created) and return it. @@ -1228,7 +1228,7 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyThreadState_Get() - Return the active :term:`thread state`. If there is no active :term:`thread state` (such + Return the :term:`active thread state`. If there is no :term:`active thread state` (such as when inside of :c:macro:`Py_BEGIN_ALLOW_THREADS` block), then this issues a fatal error (so that the caller needn't check for ``NULL``). @@ -1278,7 +1278,7 @@ with sub-interpreters: unique call to :c:func:`PyGILState_Ensure` must save the handle for its call to :c:func:`PyGILState_Release`. - When the function returns, there will be an active :term:`thread state` + When the function returns, there will be an :term:`attached thread state` and the thread will be able to call arbitrary Python code. Failure is a fatal error. .. note:: @@ -1418,7 +1418,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyThreadState_DeleteCurrent(void) - Destroy the active :term:`thread state` and detach it. + Destroy the :term:`active thread state` and detach it. The current :term:`thread state` must have been reset with a previous call to :c:func:`PyThreadState_Clear`. @@ -1482,7 +1482,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. Issue a fatal error if there no current Python thread state or no current interpreter. It cannot return NULL. - The caller must have an active :term:`thread state`. + The caller must have an :term:`attached thread state`. .. versionadded:: 3.9 @@ -1492,7 +1492,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. Return the interpreter's unique ID. If there was any error in doing so then ``-1`` is returned and an error is set. - The caller must have an active :term:`thread state`. + The caller must have an :term:`attached thread state`. .. versionadded:: 3.7 @@ -1552,7 +1552,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. Asynchronously raise an exception in a thread. The *id* argument is the thread id of the target thread; *exc* is the exception object to be raised. This function does not steal any references to *exc*. To prevent naive misuse, you - must write your own C extension to call this. Must be called with an active :term:`thread state`. + must write your own C extension to call this. Must be called with an :term:`attached thread state`. Returns the number of thread states modified; this is normally one, but will be zero if the thread id isn't found. If *exc* is ``NULL``, the pending exception (if any) for the thread is cleared. This raises no exceptions. @@ -2084,14 +2084,14 @@ Python-level trace functions in previous versions. See also the :func:`sys.setprofile` function. - The caller must have an active :term:`thread state`. + The caller must have an :term:`attached thread state`. .. c:function:: void PyEval_SetProfileAllThreads(Py_tracefunc func, PyObject *obj) Like :c:func:`PyEval_SetProfile` but sets the profile function in all running threads belonging to the current interpreter instead of the setting it only on the current thread. - The caller must have an active :term:`thread state`. + The caller must have an :term:`attached thread state`. As :c:func:`PyEval_SetProfile`, this function ignores any exceptions raised while setting the profile functions in all threads. @@ -2110,14 +2110,14 @@ Python-level trace functions in previous versions. See also the :func:`sys.settrace` function. - The caller must have an active :term:`thread state`. + The caller must have an :term:`attached thread state`. .. c:function:: void PyEval_SetTraceAllThreads(Py_tracefunc func, PyObject *obj) Like :c:func:`PyEval_SetTrace` but sets the tracing function in all running threads belonging to the current interpreter instead of the setting it only on the current thread. - The caller must have an active :term:`thread state`. + The caller must have an :term:`attached thread state`. As :c:func:`PyEval_SetTrace`, this function ignores any exceptions raised while setting the trace functions in all threads. diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 4b426632f65f29..acbe804531a6fa 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -1893,7 +1893,7 @@ Some options are read from the :mod:`sys` attributes. For example, the option * ``list[str]`` * ``dict[str, str]`` - The caller must have an active :term:`thread state`. The function cannot + The caller must have an :term:`attached thread state`. The function cannot be called before Python initialization nor after Python finalization. .. versionadded:: 3.14 @@ -1916,7 +1916,7 @@ Some options are read from the :mod:`sys` attributes. For example, the option * Return a new reference on success. * Set an exception and return ``NULL`` on error. - The caller must have an active :term:`thread state`. The function cannot + The caller must have an :term:`attached thread state`. The function cannot be called before Python initialization nor after Python finalization. .. versionadded:: 3.14 @@ -1931,7 +1931,7 @@ Some options are read from the :mod:`sys` attributes. For example, the option * Raise a :exc:`ValueError` if the option is read-only (cannot be set). * Raise a :exc:`TypeError` if *value* has not the proper type. - The caller must have an active :term:`thread state`. The function cannot + The caller must have an :term:`attached thread state`. The function cannot be called before Python initialization nor after Python finalization. .. versionadded:: 3.14 diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 96b2f4a02bd0bc..71b71fa4e56615 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -110,12 +110,12 @@ The three allocation domains are: * Raw domain: intended for allocating memory for general-purpose memory buffers where the allocation *must* go to the system allocator or where the - allocator can operate without an active :term:`thread state`. The memory + allocator can operate without an :term:`attached thread state`. The memory is requested directly from the system. See :ref:`Raw Memory Interface `. * "Mem" domain: intended for allocating memory for Python buffers and general-purpose memory buffers where the allocation must be performed with - an active :term:`thread state`. The memory is taken from the Python private heap. + an :term:`attached thread state`. The memory is taken from the Python private heap. See :ref:`Memory Interface `. * Object domain: intended for allocating memory for Python objects. The @@ -506,7 +506,7 @@ Customize Memory Allocators :c:func:`Py_InitializeFromConfig` to install a custom memory allocator. There are no restrictions over the installed allocator other than the ones imposed by the domain (for instance, the Raw - Domain allows the allocator to be called without an active :term:`thread state`). + Domain allows the allocator to be called without an :term:`attached thread state`). See :ref:`the section on allocator domains ` for more information. diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst index 6daed8392f584e..76eb8af24ea6ff 100644 --- a/Doc/c-api/module.rst +++ b/Doc/c-api/module.rst @@ -708,7 +708,7 @@ since multiple such modules can be created from a single definition. mechanisms (either by calling it directly, or by referring to its implementation for details of the required state updates). - The caller must have an active :term:`thread state`. + The caller must have an :term:`attached thread state`. Return ``-1`` with an exception set on error, ``0`` on success. @@ -719,6 +719,6 @@ since multiple such modules can be created from a single definition. Removes the module object created from *def* from the interpreter state. Return ``-1`` with an exception set on error, ``0`` on success. - The caller must have an active :term:`thread state`. + The caller must have an :term:`attached thread state`. .. versionadded:: 3.3 diff --git a/Doc/c-api/perfmaps.rst b/Doc/c-api/perfmaps.rst index fef2e8c1edefea..77b5e3c0876bbb 100644 --- a/Doc/c-api/perfmaps.rst +++ b/Doc/c-api/perfmaps.rst @@ -16,7 +16,7 @@ kernel/git/torvalds/linux.git/tree/tools/perf/Documentation/jit-interface.txt>`_ In Python, these helper APIs can be used by libraries and features that rely on generating machine code on the fly. -Note that holding an active :term:`thread state` is not required for these APIs. +Note that holding an :term:`attached thread state` is not required for these APIs. .. c:function:: int PyUnstable_PerfMapState_Init(void) diff --git a/Doc/c-api/time.rst b/Doc/c-api/time.rst index a97ad0bb152b97..f258082c0af70e 100644 --- a/Doc/c-api/time.rst +++ b/Doc/c-api/time.rst @@ -56,7 +56,7 @@ range. system time.) As any other C API (unless otherwise specified), the functions must be called -with an active :term:`thread state`. +with an :term:`attached thread state`. .. c:function:: int PyTime_Monotonic(PyTime_t *result) @@ -78,7 +78,7 @@ Raw Clock Functions ------------------- Similar to clock functions, but don't set an exception on error and don't -require the caller to have an active :term:`thread state`. +require the caller to have an :term:`attached thread state`. On success, the functions return ``0``. @@ -90,17 +90,17 @@ the ``Raw`` one failed. .. c:function:: int PyTime_MonotonicRaw(PyTime_t *result) Similar to :c:func:`PyTime_Monotonic`, - but don't set an exception on error and don't require an active :term:`thread state`. + but don't set an exception on error and don't require an :term:`attached thread state`. .. c:function:: int PyTime_PerfCounterRaw(PyTime_t *result) Similar to :c:func:`PyTime_PerfCounter`, - but don't set an exception on error and don't require an active :term:`thread state`. + but don't set an exception on error and don't require an :term:`attached thread state`. .. c:function:: int PyTime_TimeRaw(PyTime_t *result) Similar to :c:func:`PyTime_Time`, - but don't set an exception on error and don't require an active :term:`thread state`. + but don't set an exception on error and don't require an :term:`attached thread state`. Conversion functions diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 78c1e92cf6142a..9a983603518a8b 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -689,7 +689,7 @@ and :c:data:`PyType_Type` effectively act as defaults.) object becomes part of a refcount cycle, that cycle might be collected by a garbage collection on any thread). This is not a problem for Python API calls, since the thread on which :c:member:`!tp_dealloc` is called - with an active :term:`thread state`. However, if the object being + with an :term:`attached thread state`. However, if the object being destroyed in turn destroys objects from some other C or C++ library, care should be taken to ensure that destroying those objects on the thread which called :c:member:`!tp_dealloc` will not violate any assumptions of diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 0f40a3c4d85c50..abdc7a7e2196c0 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -36,6 +36,13 @@ Glossary and loaders (in the :mod:`importlib.abc` module). You can create your own ABCs with the :mod:`abc` module. + active thread state + + The :c:data:`PyThreadState` pointer to a :term:`thread state` for the current thread. + The per-thread pointer might be ``NULL``, in which case Python code should not + get executed. If the active thread state is non-``NULL``, then the :term:`thread state` + that it points to is considered to be :term:`attached `. + annotate function A function that can be called to retrieve the :term:`annotations ` of an object. This function is accessible as the :attr:`~object.__annotate__` @@ -132,6 +139,14 @@ Glossary iterator's :meth:`~object.__anext__` method until it raises a :exc:`StopAsyncIteration` exception. Introduced by :pep:`492`. + attached thread state + + A :term:`thread state` that is :term:`active ` + for the current thread. + + On most builds of Python, an attached thread state means that the + caller holds the :term:`GIL` for the current interpreter. + attribute A value associated with an object which is usually referenced by name using dotted expressions. @@ -1288,12 +1303,7 @@ Glossary thread state In Python's C API, a thread state is a structure that holds information about the current thread. A thread state must be - active for the current thread in order to call most of the C API. - - On most builds of Python, an attached thread state means that the - caller holds the :term:`GIL` for the current interpreter, but on - :term:`free-threaded ` builds, multiple thread states - can be active at once. + :term:`active ` for the current thread in order to call most of the C API. See :ref:`Thread State and the Global Interpreter Lock ` for more information. From 8584c867a1fe75c7a890adac8ad9dbdf121d2c74 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 15:06:36 -0500 Subject: [PATCH 45/73] Change usage of some terms. --- Doc/c-api/init.rst | 44 ++++++++++++++++++++++---------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 20a01220b2c996..78e15071f02f0c 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -966,18 +966,17 @@ inside a data structure called :c:type:`PyThreadState`, known as a :term:`thread There's also one thread-local variable pointing to the current :c:type:`PyThreadState`: it can be retrieved using :c:func:`PyThreadState_Get`. -A thread can only have one attached :term:`thread state` at a time. An attached -:term:`thread state` is typically analogous with holding the :term:`GIL`, except on +A thread can only have one :term:`attached thread state` at a time. An attached +thread state is typically analogous with holding the :term:`GIL`, except on :term:`free-threaded ` builds. On builds with the :term:`GIL` enabled, attaching a thread state will block until the :term:`GIL` can be acquired. However, even on builds with the :term:`GIL` disabled, it is still required -to have a :term:`thread state` attached to the current thread to call most of +to have a thread state attached to the current thread to call most of the C API. -In general, a :term:`thread state` will always be active for the current thread -when using Python's C API. Only in some specific cases (such as in a -:c:macro:`Py_BEGIN_ALLOW_THREADS` block) will the thread not have an active -thread state. If uncertain, check if :c:func:`PyThreadState_GetUnchecked` returns +In general, there will always be an :term:`attached thread state` when using Python's C API. +Only in some specific cases (such as in a :c:macro:`Py_BEGIN_ALLOW_THREADS` block) will the +thread not have an attached thread state. If uncertain, check if :c:func:`PyThreadState_GetUnchecked` returns ``NULL``. Detaching the thread state from extension code @@ -1018,14 +1017,15 @@ The block above expands to the following code:: Here is how these functions work: -The :term:`thread state` holds the :term:`GIL` for the entire interpreter. When detaching -the :term:`thread state`, the :term:`GIL` is released, allowing other threads to attach -their own :term:`thread state`, thus getting the :term:`GIL` and can start executing. -The pointer to the now-detached :term:`thread state` is stored as a local variable. -Upon reaching :c:macro:`Py_END_ALLOW_THREADS`, the :term:`thread state` that was -previously attached is given to :c:func:`PyEval_RestoreThread`. This function will -block until another thread that the :term:`GIL` can be re-acquired, thus allowing -the :term:`thread state` to get re-attached and the C API can be called again. +The :term:`attached thread state` holds the :term:`GIL` for the entire interpreter. When detaching +the :term:`attached thread state`, the :term:`GIL` is released, allowing other threads to attach +a thread state to their own thread, thus getting the :term:`GIL` and can start executing. +The pointer to the prior :term:`attached thread state` is stored as a local variable. +Upon reaching :c:macro:`Py_END_ALLOW_THREADS`, the thread state that was +previously :term:`attached ` is passed to :c:func:`PyEval_RestoreThread`. +This function will block until another releases its :term:`thread state `, +thus allowing the old :term:`thread state ` to get re-attached and the +C API can be called again. For :term:`free-threaded ` builds, the :term:`GIL` is normally out of the question, but detaching the thread state is still required for blocking I/O @@ -1038,7 +1038,7 @@ to be released to attach their thread state, allowing true multi-core parallelis long-running computations which don't need access to Python objects, such as compression or cryptographic functions operating over memory buffers. For example, the standard :mod:`zlib` and :mod:`hashlib` modules detach the - :term:`thread state` when compressing or hashing data. + :term:`attached thread state` when compressing or hashing data. .. _gilstate: @@ -1050,14 +1050,14 @@ When threads are created using the dedicated Python APIs (such as the :mod:`threading` module), a thread state is automatically associated to them and the code showed above is therefore correct. However, when threads are created from C (for example by a third-party library with its own thread -management), they don't hold the :term:`GIL`, because they don't have a -:term:`thread state`. +management), they don't hold the :term:`GIL`, because they don't have an +:term:`attached thread state`. If you need to call Python code from these threads (often this will be part of a callback API provided by the aforementioned third-party library), you must first register these threads with the interpreter by -creating a :term:`thread state` before you can start using the Python/C -API. When you are done, you should detach the :term:`thread state`, and +creating an :term:`attached thread state` before you can start using the Python/C +API. When you are done, you should detach the :term:`thread state `, and finally free it. The :c:func:`PyGILState_Ensure` and :c:func:`PyGILState_Release` functions do @@ -1134,8 +1134,8 @@ acquire the :term:`GIL`. If any thread, other than the finalization thread, attempts to attach a :term:`thread state` during finalization, either explicitly via a :term:`thread state` function, or implicitly when the interpreter attempts yields the :term:`GIL` by detaching the -:term:`thread state`, the thread enters **a permanently blocked state** where it -remains until the program exits. In most cases this is harmless, but this can result +:term:`thread state `, the thread enters **a permanently blocked state** +where it remains until the program exits. In most cases this is harmless, but this can result in deadlock if a later stage of finalization attempts to acquire a lock owned by the blocked thread, or otherwise waits on the blocked thread. From 48b145aad670222909e69750126280425cd5fe42 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 15:22:51 -0500 Subject: [PATCH 46/73] Play around with terminology. --- Doc/c-api/init.rst | 98 +++++++++++++++-------------- Doc/c-api/memory.rst | 17 +++-- Doc/extending/newtypes_tutorial.rst | 4 +- 3 files changed, 60 insertions(+), 59 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 78e15071f02f0c..bc84fbbc50394f 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -963,16 +963,15 @@ a file, so that other Python threads can run in the meantime. The Python interpreter keeps some thread-specific bookkeeping information inside a data structure called :c:type:`PyThreadState`, known as a :term:`thread state`. -There's also one thread-local variable pointing to the current :c:type:`PyThreadState`: it can -be retrieved using :c:func:`PyThreadState_Get`. +There's also one :term:`thread-local variable ` pointing to the +current :c:type:`PyThreadState`: it can be retrieved using :c:func:`PyThreadState_Get`. A thread can only have one :term:`attached thread state` at a time. An attached thread state is typically analogous with holding the :term:`GIL`, except on :term:`free-threaded ` builds. On builds with the :term:`GIL` enabled, -attaching a thread state will block until the :term:`GIL` can be acquired. -However, even on builds with the :term:`GIL` disabled, it is still required -to have a thread state attached to the current thread to call most of -the C API. +:term:`attaching ` a thread state will block until the :term:`GIL` +can be acquired. However, even on builds with the :term:`GIL` disabled, it is still required +to have an attached thread state to call most of the C API. In general, there will always be an :term:`attached thread state` when using Python's C API. Only in some specific cases (such as in a :c:macro:`Py_BEGIN_ALLOW_THREADS` block) will the @@ -1132,9 +1131,8 @@ thread* that initiated finalization (typically the main thread) is allowed to acquire the :term:`GIL`. If any thread, other than the finalization thread, attempts to attach a :term:`thread state` -during finalization, either explicitly via a :term:`thread state` function, or -implicitly when the interpreter attempts yields the :term:`GIL` by detaching the -:term:`thread state `, the thread enters **a permanently blocked state** +during finalization, either explicitly or +implicitly, the thread enters **a permanently blocked state** where it remains until the program exits. In most cases this is harmless, but this can result in deadlock if a later stage of finalization attempts to acquire a lock owned by the blocked thread, or otherwise waits on the blocked thread. @@ -1212,9 +1210,9 @@ code, or when embedding the Python interpreter: .. c:function:: void PyEval_RestoreThread(PyThreadState *tstate) - Set the current :term:`thread state` to *tstate*, which must not be ``NULL``. - The passed :term:`thread state` **should not** be attached, otherwise deadlock - ensues. + Set the :term:`active thread state` to *tstate*, which must not be ``NULL``. + The passed :term:`thread state` **should not** be :term:`attached `, + otherwise deadlock ensues. .. note:: Calling this function from a thread when the runtime is finalizing will @@ -1228,8 +1226,8 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyThreadState_Get() - Return the :term:`active thread state`. If there is no :term:`active thread state` (such - as when inside of :c:macro:`Py_BEGIN_ALLOW_THREADS` block), then this issues a fatal + Return the :term:`active thread state`. If the :term:`active thread state` is ``NULL`` + (such as when inside of :c:macro:`Py_BEGIN_ALLOW_THREADS` block), then this issues a fatal error (so that the caller needn't check for ``NULL``). See also :c:func:`PyThreadState_GetUnchecked`. @@ -1248,12 +1246,14 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyThreadState_Swap(PyThreadState *tstate) - Swap the current :term:`thread state` with *tstate*. + Set the :term:`active thread state` to *tstate*, and return + the old value. - If there is an attached :term:`thread state` for the current + If there is an :term:`attached thread state` for the current thread, it will be detached. Upon returning from this function, - *tstate* will become attached instead if it's not ``NULL``. If it - is ``NULL``, then no :term:`thread state` will be attached upon returning. + *tstate* will become :term:`attached ` instead + if it's not ``NULL``. If it is ``NULL``, then the :term:`active thread state` + will be ``NULL``. The following functions use thread-local storage, and are not compatible @@ -1271,7 +1271,7 @@ with sub-interpreters: :c:macro:`Py_BEGIN_ALLOW_THREADS` and :c:macro:`Py_END_ALLOW_THREADS` macros is acceptable. - The return value is an opaque "handle" to the :term:`thread state` when + The return value is an opaque "handle" to the :term:`attached thread state` when :c:func:`PyGILState_Ensure` was called, and must be passed to :c:func:`PyGILState_Release` to ensure Python is left in the same state. Even though recursive calls are allowed, these handles *cannot* be shared - each @@ -1304,7 +1304,7 @@ with sub-interpreters: .. c:function:: PyThreadState* PyGILState_GetThisThreadState() - Get the current :term:`thread state` for this thread. May return ``NULL`` if no + Get the :term:`active thread state` for this thread. May return ``NULL`` if no GILState API has been used on the current thread. Note that the main thread always has such a thread-state, even if no auto-thread-state call has been made on the main thread. This is mainly a helper/diagnostic function. @@ -1364,13 +1364,13 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. versionchanged:: 3.7 :c:func:`Py_Initialize()` now initializes the :term:`GIL` - and actives a :term:`thread state`. + and sets an :term:`attached thread state`. .. c:function:: PyInterpreterState* PyInterpreterState_New() - Create a new interpreter state object. A :term:`thread state` need not - be attached, but may be held if it is necessary to serialize calls to this + Create a new interpreter state object. An :term:`attached thread state` is not needed, + but may optionally exist if it is necessary to serialize calls to this function. .. audit-event:: cpython.PyInterpreterState_New "" c.PyInterpreterState_New @@ -1378,28 +1378,28 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyInterpreterState_Clear(PyInterpreterState *interp) - Reset all information in an interpreter state object. A :term:`thread state` - for the interpreter must be attached. + Reset all information in an interpreter state object. There must be + an :term:`attached thread state` for the the interpreter. .. audit-event:: cpython.PyInterpreterState_Clear "" c.PyInterpreterState_Clear .. c:function:: void PyInterpreterState_Delete(PyInterpreterState *interp) - Destroy an interpreter state object. A :term:`thread state` for the interpreter - shouldn't be attached. The interpreter state must have been reset with a previous call to - :c:func:`PyInterpreterState_Clear`. + Destroy an interpreter state object. There **should not** be an + :term:`attached thread state` for the target interpreter. The interpreter + state must have been reset with a previous call to :c:func:`PyInterpreterState_Clear`. .. c:function:: PyThreadState* PyThreadState_New(PyInterpreterState *interp) Create a new thread state object belonging to the given interpreter object. - A :term:`thread state` can be optionally attached. + An :term:`attached thread state` is not needed. .. c:function:: void PyThreadState_Clear(PyThreadState *tstate) - Reset all information in a :term:`thread state` object. The - :term:`thread state` must be active for the current thread. + Reset all information in a :term:`thread state` object. *tstate* + must be :term:`attached ` .. versionchanged:: 3.9 This function now calls the :c:member:`PyThreadState.on_delete` callback. @@ -1411,16 +1411,17 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyThreadState_Delete(PyThreadState *tstate) - Destroy a :term:`thread state`` object. The :term:`thread state` should not - be active. *tstate* must have been reset with a previous call to + Destroy a :term:`thread state`` object. *tstate* should not + be :term:`attached ` to any thread. + *tstate* must have been reset with a previous call to :c:func:`PyThreadState_Clear`. .. c:function:: void PyThreadState_DeleteCurrent(void) - Destroy the :term:`active thread state` and detach it. - The current :term:`thread state` must have been reset with a previous call - to :c:func:`PyThreadState_Clear`. + Destroy the :term:`attached thread state` and set the :term:`active thread state` + to ``NULL``. The :term:`thread state ` must have been reset + with a previous call to :c:func:`PyThreadState_Clear`. .. c:function:: PyFrameObject* PyThreadState_GetFrame(PyThreadState *tstate) @@ -1744,11 +1745,11 @@ function. You can create and destroy them using the following functions: must be held before calling this function and is still held when it returns. Likewise a current thread state must be set on entry. On success, the returned thread state will be set as current. If the - sub-interpreter is created with its own :term:`thread state` then the - :term:`thread state` of the calling interpreter will be detached. + sub-interpreter is created with its own :term:`GIL` then the + :term:`attached thread state` of the calling interpreter will be detached. When the function returns, the new interpreter's :term:`thread state` - will be activated for the current thread and the previously interpreter's - :term:`thread state` will remain detached here. + will be :term:`attached ` to the current thread and + the previous interpreter's :term:`attached thread state` will remain detached. .. versionadded:: 3.12 @@ -1836,7 +1837,8 @@ function. You can create and destroy them using the following functions: the current thread state is ``NULL``. All thread states associated with this interpreter are destroyed. The global interpreter lock used by the target interpreter must be held before calling this - function. No :term:`thread state` is active when it returns. + function. No :term:`thread state` is :term:`attached ` + when it returns. :c:func:`Py_FinalizeEx` will destroy all sub-interpreters that haven't been explicitly destroyed at that point. @@ -1941,8 +1943,8 @@ pointer and a void pointer argument. This function doesn't need a current thread state to run, and it doesn't need the global interpreter lock. - To call this function in a subinterpreter, the caller must have an active - :term:`thread state`. Otherwise, the function *func* can be scheduled to + To call this function in a subinterpreter, the caller must have an + :term:`attached thread state`. Otherwise, the function *func* can be scheduled to be called from the wrong interpreter. .. warning:: @@ -2162,7 +2164,7 @@ Reference tracing any existing exception or set an exception. A :term:`thread state` will be active every time the tracer function is called. - The :term:`thread state` must be active when calling this function. + There must be an :term:`attached thread state` when calling this function. .. versionadded:: 3.13 @@ -2173,7 +2175,7 @@ Reference tracing If no tracer was registered this function will return NULL and will set the **data** pointer to NULL. - A :term:`thread state` must be active when calling this function. + There must be an :term:`attached thread state` when calling this function. .. versionadded:: 3.13 @@ -2230,8 +2232,8 @@ CPython C level APIs are similar to those offered by pthreads and Windows: use a thread key and functions to associate a :c:expr:`void*` value per thread. -A :term:`thread state` does *not* need to be active when calling these functions; they supply -their own locking. +A :term:`thread state` does *not* need to be :term:`attached ` +when calling these functions; they suppl their own locking. Note that :file:`Python.h` does not include the declaration of the TLS APIs, you need to include :file:`pythread.h` to use thread-local storage. @@ -2400,7 +2402,7 @@ The C-API provides a basic mutual exclusion lock. Lock mutex *m*. If another thread has already locked it, the calling thread will block until the mutex is unlocked. While blocked, the thread - will temporarily detach the current :term:`thread state` if one is active. + will temporarily detach the :term:`attached thread state` if one exists. .. versionadded:: 3.13 diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 71b71fa4e56615..64ae35daa703b8 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -140,7 +140,7 @@ Raw Memory Interface The following function sets are wrappers to the system allocator. These functions are thread-safe, so a :term:`thread state` does not -need to be active. +need to be :term:`attached `. The :ref:`default raw memory allocator ` uses the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` @@ -213,8 +213,7 @@ The :ref:`default memory allocator ` uses the .. warning:: - A :term:`thread state` must be active when using these - functions. + There must be an :term:`attached thread state` when using these functions. .. versionchanged:: 3.6 @@ -327,7 +326,7 @@ The :ref:`default object allocator ` uses the .. warning:: - A :term:`thread state` must be active when using these functions. + There must be an :term:`attached thread state` when using these functions. .. c:function:: void* PyObject_Malloc(size_t n) @@ -484,8 +483,8 @@ Customize Memory Allocators zero bytes. For the :c:macro:`PYMEM_DOMAIN_RAW` domain, the allocator must be - thread-safe: a :term:`thread state` is not held when the - allocator is called. + thread-safe: a :term:`thread state` is not :term:`attached ` + when the allocator is called. For the remaining domains, the allocator must also be thread-safe: the allocator may be called in different interpreters that do not @@ -554,7 +553,7 @@ Runtime checks: called on a memory block allocated by :c:func:`PyMem_Malloc`. - Detect write before the start of the buffer (buffer underflow). - Detect write after the end of the buffer (buffer overflow). -- Check that a :term:`thread state` is active when +- Check that there is an :term:`attached thread state` when allocator functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. @@ -619,8 +618,8 @@ PYMEM_CLEANBYTE (meaning uninitialized memory is getting used). The :c:func:`PyMem_SetupDebugHooks` function now also works on Python compiled in release mode. On error, the debug hooks now use :mod:`tracemalloc` to get the traceback where a memory block was allocated. - The debug hooks now also check if a :term:`thread state` is active when functions of - :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are + The debug hooks now also check if there is an :term:`attached thread state` when + functions of :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are called. .. versionchanged:: 3.8 diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index c34c90b8307d12..e76d8ef7278827 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -404,7 +404,7 @@ But this would be risky. Our type doesn't restrict the type of the ``first`` member, so it could be any kind of object. It could have a destructor that causes code to be executed that tries to access the ``first`` member; or that destructor could detach the -:term:`term state` and let arbitrary code run in other +:term:`attached thread state` and let arbitrary code run in other threads that accesses and modifies our object. To be paranoid and protect ourselves against this possibility, we almost @@ -414,7 +414,7 @@ don't we have to do this? * when we absolutely know that the reference count is greater than 1; * when we know that deallocation of the object [#]_ will neither detach - the :term:`thread state` nor cause any calls back into our type's code; + the :term:`attached thread state` nor cause any calls back into our type's code; * when decrementing a reference count in a :c:member:`~PyTypeObject.tp_dealloc` handler on a type which doesn't support cyclic garbage collection [#]_. From 8ce3d7dee28bda6baeed286fa4954d9d7e82a4fa Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 15:28:47 -0500 Subject: [PATCH 47/73] Various terminology changes. --- Doc/c-api/init.rst | 16 ++++++------- Doc/c-api/sys.rst | 4 ++-- Doc/c-api/time.rst | 4 ++-- Doc/glossary.rst | 23 ++++++++++--------- ...-11-26-22-06-10.gh-issue-127314.SsRrIu.rst | 2 +- 5 files changed, 25 insertions(+), 24 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index bc84fbbc50394f..57bd13ffb3c285 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -963,7 +963,7 @@ a file, so that other Python threads can run in the meantime. The Python interpreter keeps some thread-specific bookkeeping information inside a data structure called :c:type:`PyThreadState`, known as a :term:`thread state`. -There's also one :term:`thread-local variable ` pointing to the +There's also one :term:`thread-local variable ` pointing to the current :c:type:`PyThreadState`: it can be retrieved using :c:func:`PyThreadState_Get`. A thread can only have one :term:`attached thread state` at a time. An attached @@ -1204,13 +1204,13 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyEval_SaveThread() - Detach the :term:`active thread state` (if it has been created) and + Detach the :term:`current thread state` (if it has been created) and return it. .. c:function:: void PyEval_RestoreThread(PyThreadState *tstate) - Set the :term:`active thread state` to *tstate*, which must not be ``NULL``. + Set the :term:`current thread state` to *tstate*, which must not be ``NULL``. The passed :term:`thread state` **should not** be :term:`attached `, otherwise deadlock ensues. @@ -1226,7 +1226,7 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyThreadState_Get() - Return the :term:`active thread state`. If the :term:`active thread state` is ``NULL`` + Return the :term:`current thread state`. If the :term:`current thread state` is ``NULL`` (such as when inside of :c:macro:`Py_BEGIN_ALLOW_THREADS` block), then this issues a fatal error (so that the caller needn't check for ``NULL``). @@ -1246,13 +1246,13 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyThreadState_Swap(PyThreadState *tstate) - Set the :term:`active thread state` to *tstate*, and return + Set the :term:`current thread state` to *tstate*, and return the old value. If there is an :term:`attached thread state` for the current thread, it will be detached. Upon returning from this function, *tstate* will become :term:`attached ` instead - if it's not ``NULL``. If it is ``NULL``, then the :term:`active thread state` + if it's not ``NULL``. If it is ``NULL``, then the :term:`current thread state` will be ``NULL``. @@ -1304,7 +1304,7 @@ with sub-interpreters: .. c:function:: PyThreadState* PyGILState_GetThisThreadState() - Get the :term:`active thread state` for this thread. May return ``NULL`` if no + Get the :term:`current thread state` for this thread. May return ``NULL`` if no GILState API has been used on the current thread. Note that the main thread always has such a thread-state, even if no auto-thread-state call has been made on the main thread. This is mainly a helper/diagnostic function. @@ -1419,7 +1419,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyThreadState_DeleteCurrent(void) - Destroy the :term:`attached thread state` and set the :term:`active thread state` + Destroy the :term:`attached thread state` and set the :term:`current thread state` to ``NULL``. The :term:`thread state ` must have been reset with a previous call to :c:func:`PyThreadState_Clear`. diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 441a26fe53505d..27553e2642d9c9 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -346,8 +346,8 @@ accessible to C code. They all work with the current interpreter thread's silently abort the operation by raising an error subclassed from :class:`Exception` (other errors will not be silenced). - The hook function is always called with a :term:`thread state` by the Python - interpreter that raised the event. + The hook function is always called with an :term:`attached thread state` by + the Python interpreter that raised the event. See :pep:`578` for a detailed description of auditing. Functions in the runtime and standard library that raise events are listed in the diff --git a/Doc/c-api/time.rst b/Doc/c-api/time.rst index f258082c0af70e..31d6adbc225439 100644 --- a/Doc/c-api/time.rst +++ b/Doc/c-api/time.rst @@ -83,8 +83,8 @@ require the caller to have an :term:`attached thread state`. On success, the functions return ``0``. On failure, they set ``*result`` to ``0`` and return ``-1``, *without* setting -an exception. To get the cause of the error, attach a :term:`thread state`, and call the -regular (non-``Raw``) function. Note that the regular function may succeed after +an exception. To get the cause of the error, :term:`attach ` a :term:`thread state`, +and call the regular (non-``Raw``) function. Note that the regular function may succeed after the ``Raw`` one failed. .. c:function:: int PyTime_MonotonicRaw(PyTime_t *result) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index abdc7a7e2196c0..c755a0597233f0 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -36,13 +36,6 @@ Glossary and loaders (in the :mod:`importlib.abc` module). You can create your own ABCs with the :mod:`abc` module. - active thread state - - The :c:data:`PyThreadState` pointer to a :term:`thread state` for the current thread. - The per-thread pointer might be ``NULL``, in which case Python code should not - get executed. If the active thread state is non-``NULL``, then the :term:`thread state` - that it points to is considered to be :term:`attached `. - annotate function A function that can be called to retrieve the :term:`annotations ` of an object. This function is accessible as the :attr:`~object.__annotate__` @@ -141,10 +134,11 @@ Glossary attached thread state - A :term:`thread state` that is :term:`active ` - for the current thread. + A :term:`thread state` that is :term:`active ` + for the current thread. If no thread state is attached, then the + :term:`current thread state` is ``NULL``. - On most builds of Python, an attached thread state means that the + On most builds of Python, having an attached thread state means that the caller holds the :term:`GIL` for the current interpreter. attribute @@ -348,6 +342,13 @@ Glossary tasks (see :mod:`asyncio`) associate each task with a context which becomes the current context whenever the task starts or resumes execution. + current thread state + + The :c:data:`PyThreadState` pointer to a :term:`thread state` for the current thread. + The per-thread pointer might be ``NULL``, in which case Python code should not + get executed. If the current thread state is non-``NULL``, then the :term:`thread state` + that it points to is considered to be :term:`attached `. + decorator A function returning another function, usually applied as a function transformation using the ``@wrapper`` syntax. Common examples for @@ -1303,7 +1304,7 @@ Glossary thread state In Python's C API, a thread state is a structure that holds information about the current thread. A thread state must be - :term:`active ` for the current thread in order to call most of the C API. + :term:`active ` for the current thread in order to call most of the C API. See :ref:`Thread State and the Global Interpreter Lock ` for more information. diff --git a/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst b/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst index 8ea3c4ee2a2c53..0f812f5980ac05 100644 --- a/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst +++ b/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst @@ -1,2 +1,2 @@ -Improve error message when calling the C API without an active thread state +Improve error message when calling the C API without an current thread state on the :term:`free-threaded ` build. From e5f1d18f5fb371ee157b19c17eae210885277df2 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 15:33:13 -0500 Subject: [PATCH 48/73] Change the glossary definition a little bit. --- Doc/glossary.rst | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index c755a0597233f0..7f8cae184ab68b 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -136,7 +136,11 @@ Glossary A :term:`thread state` that is :term:`active ` for the current thread. If no thread state is attached, then the - :term:`current thread state` is ``NULL``. + :term:`current thread state` is ``NULL``. Attempting to call Python + without an attached thread state will generally result in a fatal error. + + A thread state can be attached and detached explicitly by the user, or + implicitly by the interpreter. On most builds of Python, having an attached thread state means that the caller holds the :term:`GIL` for the current interpreter. @@ -1303,8 +1307,10 @@ Glossary thread state In Python's C API, a thread state is a structure that holds - information about the current thread. A thread state must be - :term:`active ` for the current thread in order to call most of the C API. + information about the current thread. A thread state can be + attached or detached. An :term:`attached thread state` is required + to call most of the C API, unless a function explicitly documents + otherwise. See :ref:`Thread State and the Global Interpreter Lock ` for more information. From 905ca5bf62c6ba3bd80d499e4f5150034562b35b Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 15:39:15 -0500 Subject: [PATCH 49/73] Change some phrasing. --- Doc/c-api/init.rst | 2 +- Doc/glossary.rst | 19 +++++++++++++------ 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 57bd13ffb3c285..621b3dfc9ab768 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1411,7 +1411,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyThreadState_Delete(PyThreadState *tstate) - Destroy a :term:`thread state`` object. *tstate* should not + Destroy a :term:`thread state` object. *tstate* should not be :term:`attached ` to any thread. *tstate* must have been reset with a previous call to :c:func:`PyThreadState_Clear`. diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 7f8cae184ab68b..d153f8c42952e2 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -137,10 +137,12 @@ Glossary A :term:`thread state` that is :term:`active ` for the current thread. If no thread state is attached, then the :term:`current thread state` is ``NULL``. Attempting to call Python - without an attached thread state will generally result in a fatal error. + without an attached thread state will result in a fatal error. A thread state can be attached and detached explicitly by the user, or - implicitly by the interpreter. + implicitly by the interpreter in between calls. For example, an attached + thread state is detached upon entering a :c:macro:`Py_BEGIN_ALLOW_THREADS` + block, and then re-attached when :c:macro:`Py_END_ALLOW_THREADS` is reached. On most builds of Python, having an attached thread state means that the caller holds the :term:`GIL` for the current interpreter. @@ -348,10 +350,15 @@ Glossary current thread state - The :c:data:`PyThreadState` pointer to a :term:`thread state` for the current thread. - The per-thread pointer might be ``NULL``, in which case Python code should not - get executed. If the current thread state is non-``NULL``, then the :term:`thread state` - that it points to is considered to be :term:`attached `. + A per-thread :c:data:`PyThreadState` pointer for the current thread. + The pointer might be ``NULL``, in which case Python code should not + get executed. + + If the current thread state is non-``NULL``, then the :term:`thread state` + that it points to is considered to be :term:`attached `. + + The per-thread pointer can be acquired via :c:func:`PyThreadState_Get` or + :c:func:`PyThreadState_GetUnchecked` if it might be ``NULL``. decorator A function returning another function, usually applied as a function From f2826c470b62dd7813a0cce1c9c9befb3667e2e4 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sat, 14 Dec 2024 15:50:52 -0500 Subject: [PATCH 50/73] Fix some usage of terms. --- Doc/c-api/init.rst | 60 ++++++++++++++++++++-------------------------- 1 file changed, 26 insertions(+), 34 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 621b3dfc9ab768..d80b566557354e 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1433,16 +1433,16 @@ All of the following functions must be called after :c:func:`Py_Initialize`. See also :c:func:`PyEval_GetFrame`. - *tstate* must not be ``NULL``. + *tstate* must not be ``NULL``, and must be :term:`attached `. .. versionadded:: 3.9 .. c:function:: uint64_t PyThreadState_GetID(PyThreadState *tstate) - Get the unique thread state identifier of the Python thread state *tstate*. + Get the unique :term:`thread state` identifier of the Python thread state *tstate*. - *tstate* must not be ``NULL``. + *tstate* must not be ``NULL``, and must be :term:`attached `. .. versionadded:: 3.9 @@ -1451,7 +1451,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. Get the interpreter of the Python thread state *tstate*. - *tstate* must not be ``NULL``. + *tstate* must not be ``NULL``, and must be :term:`attached `. .. versionadded:: 3.9 @@ -1480,11 +1480,9 @@ All of the following functions must be called after :c:func:`Py_Initialize`. Get the current interpreter. - Issue a fatal error if there no current Python thread state or no current + Issue a fatal error if there no :term:`attached thread state` or no current interpreter. It cannot return NULL. - The caller must have an :term:`attached thread state`. - .. versionadded:: 3.9 @@ -1543,9 +1541,10 @@ All of the following functions must be called after :c:func:`Py_Initialize`. Return a dictionary in which extensions can store thread-specific state information. Each extension should use a unique key to use to store state in - the dictionary. It is okay to call this function when no current thread state - is available. If this function returns ``NULL``, no exception has been raised and - the caller should assume no current thread state is available. + the dictionary. It is okay to call this function when no :term:`thread state` + is :term:`attached `. If this function returns + ``NULL``, no exception has been raised and the caller should assume no + thread state is attached. .. c:function:: int PyThreadState_SetAsyncExc(unsigned long id, PyObject *exc) @@ -1564,9 +1563,8 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyEval_AcquireThread(PyThreadState *tstate) - Acquire the global interpreter lock and set the current thread state to - *tstate*, which must not be ``NULL``. The lock must have been created earlier. - If this thread already has the lock, deadlock ensues. + Set the :term:`current thread state` to *tstate*, which must not be ``NULL`` or + :term:`attached `. .. note:: Calling this function from a thread when the runtime is finalizing will @@ -1589,10 +1587,9 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyEval_ReleaseThread(PyThreadState *tstate) - Reset the current thread state to ``NULL`` and release the global interpreter - lock. The lock must have been created earlier and must be held by the current - thread. The *tstate* argument, which must not be ``NULL``, is only used to check - that it represents the current thread state --- if it isn't, a fatal error is + Reset the :term:`current thread state` to ``NULL``. + The *tstate* argument, which must not be ``NULL``, is only used to check + that it represents the :term:`attached thread state` --- if it isn't, a fatal error is reported. :c:func:`PyEval_SaveThread` is a higher-level function which is always @@ -1732,20 +1729,19 @@ function. You can create and destroy them using the following functions: The given *config* controls the options with which the interpreter is initialized. - Upon success, *tstate_p* will be set to the first thread state - created in the new - sub-interpreter. This thread state is made in the current thread state. + Upon success, *tstate_p* will be set to the first :term:`thread state` + created in the new sub-interpreter. This thread state is + :term:`attached `. Note that no actual thread is created; see the discussion of thread states below. If creation of the new interpreter is unsuccessful, *tstate_p* is set to ``NULL``; no exception is set since the exception state is stored in the - current thread state and there may not be a current thread state. + :term:`attached thread state`, which might not exist. - Like all other Python/C API functions, the global interpreter lock - must be held before calling this function and is still held when it - returns. Likewise a current thread state must be set on entry. On - success, the returned thread state will be set as current. If the - sub-interpreter is created with its own :term:`GIL` then the + Like all other Python/C API functions, an :term:`attached thread state` + must be present before calling this function, but it might be detached upon + returning. On success, the returned thread state will be :term:`attached `. + If the sub-interpreter is created with its own :term:`GIL` then the :term:`attached thread state` of the calling interpreter will be detached. When the function returns, the new interpreter's :term:`thread state` will be :term:`attached ` to the current thread and @@ -1831,14 +1827,10 @@ function. You can create and destroy them using the following functions: .. index:: single: Py_FinalizeEx (C function) - Destroy the (sub-)interpreter represented by the given thread state. - The given thread state must be the current thread state. See the - discussion of thread states below. When the call returns, - the current thread state is ``NULL``. All thread states associated - with this interpreter are destroyed. The global interpreter lock - used by the target interpreter must be held before calling this - function. No :term:`thread state` is :term:`attached ` - when it returns. + Destroy the (sub-)interpreter represented by the given :term:`thread state`. + The given thread state must be :term:`attached `. + When the call returns, the :term:`current thread state` is ``NULL``. + All thread states associated with this interpreter are destroyed. :c:func:`Py_FinalizeEx` will destroy all sub-interpreters that haven't been explicitly destroyed at that point. From 5c4439181da4fdf007078cc5f4b1a10ac36e0fdf Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sun, 15 Dec 2024 10:30:21 -0500 Subject: [PATCH 51/73] Change some usage of 'global interpreter lock' --- Doc/c-api/init.rst | 8 ++++---- Doc/glossary.rst | 6 +++--- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index d80b566557354e..840329b92c8dc7 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1262,7 +1262,7 @@ with sub-interpreters: .. c:function:: PyGILState_STATE PyGILState_Ensure() Ensure that the current thread is ready to call the Python C API regardless - of the current state of Python, or of the global interpreter lock.This may + of the current state of Python, or of the :term:`current thread state`. This may be called as many times as desired by a thread as long as each call is matched with a call to :c:func:`PyGILState_Release`. In general, other thread-related APIs may be used between :c:func:`PyGILState_Ensure` and @@ -1924,16 +1924,16 @@ pointer and a void pointer argument. both these conditions met: * on a :term:`bytecode` boundary; - * with the main thread holding the :term:`global interpreter lock` + * with the main thread holding an :term:`attached thread state` (*func* can therefore use the full C API). *func* must return ``0`` on success, or ``-1`` on failure with an exception set. *func* won't be interrupted to perform another asynchronous notification recursively, but it can still be interrupted to switch - threads if the global interpreter lock is released. + threads if the :term:`thread state ` is detached. This function doesn't need a current thread state to run, and it doesn't - need the global interpreter lock. + need an :term:`attached thread state`. To call this function in a subinterpreter, the caller must have an :term:`attached thread state`. Otherwise, the function *func* can be scheduled to diff --git a/Doc/glossary.rst b/Doc/glossary.rst index d153f8c42952e2..5d292973147246 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -649,9 +649,9 @@ Glossary multi-threaded applications and makes it easier to use multi-core CPUs efficiently. For more details, see :pep:`703`. - In Python's C API, a function might declare that it requires (or doesn't require) - the GIL to be held in order to use it. This refers to having an active - :term:`thread state` for the current thread. + In prior versions of Python's C API, a function might declare that it + requires the GIL to be held in order to use it. This refers to having an + :term:`attached thread state`. hash-based pyc A bytecode cache file that uses the hash rather than the last-modified From 7d5877f01b532cecc3fad5036996f4b270fe4e6f Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Sun, 15 Dec 2024 10:37:29 -0500 Subject: [PATCH 52/73] Fix pre-commit lint. --- Doc/c-api/init.rst | 4 ++-- Doc/glossary.rst | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 840329b92c8dc7..17385711ef947c 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1248,9 +1248,9 @@ code, or when embedding the Python interpreter: Set the :term:`current thread state` to *tstate*, and return the old value. - + If there is an :term:`attached thread state` for the current - thread, it will be detached. Upon returning from this function, + thread, it will be detached. Upon returning from this function, *tstate* will become :term:`attached ` instead if it's not ``NULL``. If it is ``NULL``, then the :term:`current thread state` will be ``NULL``. diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 5d292973147246..e94c7e32aa40a1 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -145,7 +145,7 @@ Glossary block, and then re-attached when :c:macro:`Py_END_ALLOW_THREADS` is reached. On most builds of Python, having an attached thread state means that the - caller holds the :term:`GIL` for the current interpreter. + caller holds the :term:`GIL` for the current interpreter. attribute A value associated with an object which is usually referenced by name @@ -353,7 +353,7 @@ Glossary A per-thread :c:data:`PyThreadState` pointer for the current thread. The pointer might be ``NULL``, in which case Python code should not get executed. - + If the current thread state is non-``NULL``, then the :term:`thread state` that it points to is considered to be :term:`attached `. From 283fc92fe09a3f5f159e3cb9ff73803c035b832b Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 16 Dec 2024 09:28:08 -0500 Subject: [PATCH 53/73] Delete redundant file --- .../next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst | 2 -- 1 file changed, 2 deletions(-) delete mode 100644 Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst diff --git a/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst b/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst deleted file mode 100644 index 0f812f5980ac05..00000000000000 --- a/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst +++ /dev/null @@ -1,2 +0,0 @@ -Improve error message when calling the C API without an current thread state -on the :term:`free-threaded ` build. From a7488ab330771d5ee086c4b042dbeab968afff1f Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 16 Dec 2024 14:29:31 +0000 Subject: [PATCH 54/73] Revert "Delete redundant file" This reverts commit 283fc92fe09a3f5f159e3cb9ff73803c035b832b. --- .../next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst diff --git a/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst b/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst new file mode 100644 index 00000000000000..0f812f5980ac05 --- /dev/null +++ b/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst @@ -0,0 +1,2 @@ +Improve error message when calling the C API without an current thread state +on the :term:`free-threaded ` build. From 4c532b6f7e5c29802c455128a03f88f632c2721b Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 16 Dec 2024 14:30:46 +0000 Subject: [PATCH 55/73] Remove accidental find-and-replace change. --- .../next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst b/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst index 0f812f5980ac05..8ea3c4ee2a2c53 100644 --- a/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst +++ b/Misc/NEWS.d/next/C_API/2024-11-26-22-06-10.gh-issue-127314.SsRrIu.rst @@ -1,2 +1,2 @@ -Improve error message when calling the C API without an current thread state +Improve error message when calling the C API without an active thread state on the :term:`free-threaded ` build. From 352acc542112b17699ce8c30533ba46703ae1048 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 16 Dec 2024 16:18:48 -0500 Subject: [PATCH 56/73] Switch to a reference --- Doc/c-api/init.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 17385711ef947c..758111f0c1d119 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1027,8 +1027,8 @@ thus allowing the old :term:`thread state ` to get re-att C API can be called again. For :term:`free-threaded ` builds, the :term:`GIL` is normally -out of the question, but detaching the thread state is still required for blocking I/O -and long operations. The difference is that threads don't have to wait for the :term:`GIL` +out of the question, but detaching the :term:`thread state ` is still required +for blocking I/O and long operations. The difference is that threads don't have to wait for the :term:`GIL` to be released to attach their thread state, allowing true multi-core parallelism. .. note:: From ed550bc6bb68fee4ac11547cc5e48ce276e08059 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 16 Dec 2024 16:20:17 -0500 Subject: [PATCH 57/73] Use a less wordy phrase for detaching. --- Doc/c-api/init.rst | 6 +++--- Doc/extending/newtypes_tutorial.rst | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 758111f0c1d119..6c00b051310bff 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1033,11 +1033,11 @@ to be released to attach their thread state, allowing true multi-core parallelis .. note:: Calling system I/O functions is the most common use case for detaching - the :term:`thread state`, but it can also be useful before calling + the :term:`thread state `, but it can also be useful before calling long-running computations which don't need access to Python objects, such as compression or cryptographic functions operating over memory buffers. For example, the standard :mod:`zlib` and :mod:`hashlib` modules detach the - :term:`attached thread state` when compressing or hashing data. + :term:`thread state ` when compressing or hashing data. .. _gilstate: @@ -2394,7 +2394,7 @@ The C-API provides a basic mutual exclusion lock. Lock mutex *m*. If another thread has already locked it, the calling thread will block until the mutex is unlocked. While blocked, the thread - will temporarily detach the :term:`attached thread state` if one exists. + will temporarily detach the :term:`thread state ` if one exists. .. versionadded:: 3.13 diff --git a/Doc/extending/newtypes_tutorial.rst b/Doc/extending/newtypes_tutorial.rst index e76d8ef7278827..b2ae18d75c3831 100644 --- a/Doc/extending/newtypes_tutorial.rst +++ b/Doc/extending/newtypes_tutorial.rst @@ -404,7 +404,7 @@ But this would be risky. Our type doesn't restrict the type of the ``first`` member, so it could be any kind of object. It could have a destructor that causes code to be executed that tries to access the ``first`` member; or that destructor could detach the -:term:`attached thread state` and let arbitrary code run in other +:term:`thread state ` and let arbitrary code run in other threads that accesses and modifies our object. To be paranoid and protect ourselves against this possibility, we almost @@ -414,7 +414,7 @@ don't we have to do this? * when we absolutely know that the reference count is greater than 1; * when we know that deallocation of the object [#]_ will neither detach - the :term:`attached thread state` nor cause any calls back into our type's code; + the :term:`thread state ` nor cause any calls back into our type's code; * when decrementing a reference count in a :c:member:`~PyTypeObject.tp_dealloc` handler on a type which doesn't support cyclic garbage collection [#]_. From d7cf403b909e7c54a9d2acfe3be338d826c945c8 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 16 Dec 2024 16:22:41 -0500 Subject: [PATCH 58/73] Fix a typo in PyInterpreterState_Get --- Doc/c-api/init.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 6c00b051310bff..29ffc10bcf2fea 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1480,8 +1480,8 @@ All of the following functions must be called after :c:func:`Py_Initialize`. Get the current interpreter. - Issue a fatal error if there no :term:`attached thread state` or no current - interpreter. It cannot return NULL. + Issue a fatal error if there no :term:`attached thread state`. + It cannot return NULL. .. versionadded:: 3.9 From f61a6daa9b38e435eacb430c0afcac7ec9da96d5 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 16 Dec 2024 16:27:37 -0500 Subject: [PATCH 59/73] Change some wording in PyEval_SaveThread and PyEval_RestoreThread --- Doc/c-api/init.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 29ffc10bcf2fea..a699e71c9ce50f 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1204,15 +1204,16 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyEval_SaveThread() - Detach the :term:`current thread state` (if it has been created) and - return it. + Detach the :term:`thread state ` and + return it. The :term:`current thread state` will be ``NULL`` upon + returning. .. c:function:: void PyEval_RestoreThread(PyThreadState *tstate) Set the :term:`current thread state` to *tstate*, which must not be ``NULL``. The passed :term:`thread state` **should not** be :term:`attached `, - otherwise deadlock ensues. + otherwise deadlock ensues. *tstate* will be attached upon returning. .. note:: Calling this function from a thread when the runtime is finalizing will From 36af9417f98c4a85fffc546c202b55b291775fb2 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 16 Dec 2024 16:28:51 -0500 Subject: [PATCH 60/73] Change wording for PyThreadState_Swap --- Doc/c-api/init.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index a699e71c9ce50f..79b9332c14c590 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1247,13 +1247,13 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyThreadState_Swap(PyThreadState *tstate) - Set the :term:`current thread state` to *tstate*, and return + Set the :term:`current thread state` to *tstate* (which may be ``NULL``), and return the old value. If there is an :term:`attached thread state` for the current thread, it will be detached. Upon returning from this function, *tstate* will become :term:`attached ` instead - if it's not ``NULL``. If it is ``NULL``, then the :term:`current thread state` + (if it's not ``NULL``). If it is ``NULL``, then the :term:`current thread state` will be ``NULL``. From 0b23daa9a3951bf13e78c4929a4818722a5cc877 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 16 Dec 2024 16:32:09 -0500 Subject: [PATCH 61/73] Make glossary terms more clear. --- Doc/glossary.rst | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index e94c7e32aa40a1..1a6a3f3f039a91 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -350,15 +350,16 @@ Glossary current thread state - A per-thread :c:data:`PyThreadState` pointer for the current thread. + A per-thread :c:data:`PyThreadState` pointer. + The pointer might be ``NULL``, in which case Python code should not get executed. If the current thread state is non-``NULL``, then the :term:`thread state` that it points to is considered to be :term:`attached `. - The per-thread pointer can be acquired via :c:func:`PyThreadState_Get` or - :c:func:`PyThreadState_GetUnchecked` if it might be ``NULL``. + The pointer for the calling thread can be acquired via :c:func:`PyThreadState_Get` or + :c:func:`PyThreadState_GetUnchecked`, if it might be ``NULL``. decorator A function returning another function, usually applied as a function @@ -1314,9 +1315,9 @@ Glossary thread state In Python's C API, a thread state is a structure that holds - information about the current thread. A thread state can be - attached or detached. An :term:`attached thread state` is required - to call most of the C API, unless a function explicitly documents + information about the current thread, typically in the :term:`current thread state` + pointer. A thread state can be attached or detached. An :term:`attached thread state` + is required to call most of the C API, unless a function explicitly documents otherwise. See :ref:`Thread State and the Global Interpreter Lock ` for more From 163b0b8291b52c932375926a0223e7b99094a7b7 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 16 Dec 2024 16:33:52 -0500 Subject: [PATCH 62/73] Change up wording for 'attached thread state' --- Doc/glossary.rst | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 1a6a3f3f039a91..50c3e4ceb5a061 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -134,10 +134,9 @@ Glossary attached thread state - A :term:`thread state` that is :term:`active ` - for the current thread. If no thread state is attached, then the - :term:`current thread state` is ``NULL``. Attempting to call Python - without an attached thread state will result in a fatal error. + A :term:`thread state` that is stored in the :term:`current thread state`. + If no thread state is attached, then the :term:`current thread state` is ``NULL``. + Attempting to call Python without an attached thread state will result in a fatal error. A thread state can be attached and detached explicitly by the user, or implicitly by the interpreter in between calls. For example, an attached From 3b2bed8a77fd468fd4592658f2e6eb0e91034c68 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 16 Dec 2024 16:39:46 -0500 Subject: [PATCH 63/73] Clarify that the C API is what needs a thread state. --- Doc/glossary.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 50c3e4ceb5a061..37313b08d1ce49 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -136,7 +136,7 @@ Glossary A :term:`thread state` that is stored in the :term:`current thread state`. If no thread state is attached, then the :term:`current thread state` is ``NULL``. - Attempting to call Python without an attached thread state will result in a fatal error. + Attempting to call Python's C API without an attached thread state will result in a fatal error. A thread state can be attached and detached explicitly by the user, or implicitly by the interpreter in between calls. For example, an attached From 80ae46b7cdc212b4582c9e46b118b102dce691c8 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Tue, 17 Dec 2024 09:52:04 -0500 Subject: [PATCH 64/73] Update Doc/glossary.rst Co-authored-by: Victor Stinner --- Doc/glossary.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 37313b08d1ce49..0287cf0858cf91 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -136,7 +136,8 @@ Glossary A :term:`thread state` that is stored in the :term:`current thread state`. If no thread state is attached, then the :term:`current thread state` is ``NULL``. - Attempting to call Python's C API without an attached thread state will result in a fatal error. + Attempting to call Python's C API without an attached thread state will result + in a fatal error or an undefined behavior. A thread state can be attached and detached explicitly by the user, or implicitly by the interpreter in between calls. For example, an attached From 3dee63faed59f89786df74fb16c0167528c5cefe Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Tue, 17 Dec 2024 09:52:11 -0500 Subject: [PATCH 65/73] Update Doc/glossary.rst Co-authored-by: Victor Stinner --- Doc/glossary.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 0287cf0858cf91..66983e0ecc1a2b 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -352,7 +352,7 @@ Glossary A per-thread :c:data:`PyThreadState` pointer. - The pointer might be ``NULL``, in which case Python code should not + The pointer might be ``NULL``, in which case Python code must not get executed. If the current thread state is non-``NULL``, then the :term:`thread state` From 9ed3a0b376066129879af38fd2671a32386bc763 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Wed, 12 Feb 2025 11:50:03 -0500 Subject: [PATCH 66/73] Drop the phrase 'current thread state' and only use 'attached thread state' in its place --- Doc/c-api/init.rst | 62 +++++++++++++++++++--------------------- Doc/c-api/reflection.rst | 2 +- Doc/glossary.rst | 62 ++++++++++++++++++++++------------------ 3 files changed, 65 insertions(+), 61 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 79b9332c14c590..05082bb15d22e7 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -963,8 +963,8 @@ a file, so that other Python threads can run in the meantime. The Python interpreter keeps some thread-specific bookkeeping information inside a data structure called :c:type:`PyThreadState`, known as a :term:`thread state`. -There's also one :term:`thread-local variable ` pointing to the -current :c:type:`PyThreadState`: it can be retrieved using :c:func:`PyThreadState_Get`. +Each OS thread has a thread-local pointer to a :c:type:`PyThreadState`; a thread state +referenced by this pointer is considered to be :term:`attached `. A thread can only have one :term:`attached thread state` at a time. An attached thread state is typically analogous with holding the :term:`GIL`, except on @@ -1204,14 +1204,13 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyEval_SaveThread() - Detach the :term:`thread state ` and - return it. The :term:`current thread state` will be ``NULL`` upon - returning. + Detach the :term:`attached thread state` and return it. + The thread will have no :term:`thread state` upon returning. .. c:function:: void PyEval_RestoreThread(PyThreadState *tstate) - Set the :term:`current thread state` to *tstate*, which must not be ``NULL``. + Set the :term:`attached thread state` to *tstate*. The passed :term:`thread state` **should not** be :term:`attached `, otherwise deadlock ensues. *tstate* will be attached upon returning. @@ -1227,13 +1226,13 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyThreadState_Get() - Return the :term:`current thread state`. If the :term:`current thread state` is ``NULL`` - (such as when inside of :c:macro:`Py_BEGIN_ALLOW_THREADS` block), then this issues a fatal - error (so that the caller needn't check for ``NULL``). + Return the :term:`attached thread state`. If the thread has no attached + thread state, (such as when inside of :c:macro:`Py_BEGIN_ALLOW_THREADS` + block), then this issues a fatal error (so that the caller needn't check + for ``NULL``). See also :c:func:`PyThreadState_GetUnchecked`. - .. c:function:: PyThreadState* PyThreadState_GetUnchecked() Similar to :c:func:`PyThreadState_Get`, but don't kill the process with a @@ -1247,14 +1246,11 @@ code, or when embedding the Python interpreter: .. c:function:: PyThreadState* PyThreadState_Swap(PyThreadState *tstate) - Set the :term:`current thread state` to *tstate* (which may be ``NULL``), and return - the old value. + Set the :term:`attached thread state` to *tstate*, and return the + :term:`thread state` that was attached prior to calling. - If there is an :term:`attached thread state` for the current - thread, it will be detached. Upon returning from this function, - *tstate* will become :term:`attached ` instead - (if it's not ``NULL``). If it is ``NULL``, then the :term:`current thread state` - will be ``NULL``. + This function is safe to call without an :term:`attached thread state`; it + will simply return ``NULL`` indicating that there was no prior thread state. The following functions use thread-local storage, and are not compatible @@ -1263,7 +1259,7 @@ with sub-interpreters: .. c:function:: PyGILState_STATE PyGILState_Ensure() Ensure that the current thread is ready to call the Python C API regardless - of the current state of Python, or of the :term:`current thread state`. This may + of the current state of Python, or of the :term:`attached thread state`. This may be called as many times as desired by a thread as long as each call is matched with a call to :c:func:`PyGILState_Release`. In general, other thread-related APIs may be used between :c:func:`PyGILState_Ensure` and @@ -1305,11 +1301,13 @@ with sub-interpreters: .. c:function:: PyThreadState* PyGILState_GetThisThreadState() - Get the :term:`current thread state` for this thread. May return ``NULL`` if no + Get the :term:`attached thread state` for this thread. May return ``NULL`` if no GILState API has been used on the current thread. Note that the main thread always has such a thread-state, even if no auto-thread-state call has been made on the main thread. This is mainly a helper/diagnostic function. + .. seealso: PyThreadState_Get + .. c:function:: int PyGILState_Check() @@ -1420,10 +1418,11 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyThreadState_DeleteCurrent(void) - Destroy the :term:`attached thread state` and set the :term:`current thread state` - to ``NULL``. The :term:`thread state ` must have been reset - with a previous call to :c:func:`PyThreadState_Clear`. + Detach the :term:`attached thread state` (which must have been reset + with a previous call to :c:func:`PyThreadState_Clear`) and then destroy it. + No :term:`thread state` will be :term:`attached ` upon + returning. .. c:function:: PyFrameObject* PyThreadState_GetFrame(PyThreadState *tstate) @@ -1564,8 +1563,10 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyEval_AcquireThread(PyThreadState *tstate) - Set the :term:`current thread state` to *tstate*, which must not be ``NULL`` or - :term:`attached `. + :term:`Attach ` *tstate* to the current thread, + which must not be ``NULL`` or already :term:`attached `. + + The calling thread must not already have an :term:`attached thread state`. .. note:: Calling this function from a thread when the runtime is finalizing will @@ -1588,7 +1589,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. .. c:function:: void PyEval_ReleaseThread(PyThreadState *tstate) - Reset the :term:`current thread state` to ``NULL``. + Detach the :term:`attached thread state`. The *tstate* argument, which must not be ``NULL``, is only used to check that it represents the :term:`attached thread state` --- if it isn't, a fatal error is reported. @@ -1830,7 +1831,7 @@ function. You can create and destroy them using the following functions: Destroy the (sub-)interpreter represented by the given :term:`thread state`. The given thread state must be :term:`attached `. - When the call returns, the :term:`current thread state` is ``NULL``. + When the call returns, there will be no :term:`attached thread state`. All thread states associated with this interpreter are destroyed. :c:func:`Py_FinalizeEx` will destroy all sub-interpreters that @@ -1933,12 +1934,9 @@ pointer and a void pointer argument. notification recursively, but it can still be interrupted to switch threads if the :term:`thread state ` is detached. - This function doesn't need a current thread state to run, and it doesn't - need an :term:`attached thread state`. - - To call this function in a subinterpreter, the caller must have an - :term:`attached thread state`. Otherwise, the function *func* can be scheduled to - be called from the wrong interpreter. + This function doesn't need an :term:`attached thread state`. However, to call this + function in a subinterpreter, the caller must have an :term:`attached thread state`. + Otherwise, the function *func* can be scheduled to be called from the wrong interpreter. .. warning:: This is a low-level function, only useful for very special cases. diff --git a/Doc/c-api/reflection.rst b/Doc/c-api/reflection.rst index 038e6977104560..54fd5a064aa2ac 100644 --- a/Doc/c-api/reflection.rst +++ b/Doc/c-api/reflection.rst @@ -55,7 +55,7 @@ Reflection .. c:function:: PyFrameObject* PyEval_GetFrame(void) - Return the current thread state's frame, which is ``NULL`` if no frame is + Return the :term:`attached thread state`'s frame, which is ``NULL`` if no frame is currently executing. See also :c:func:`PyThreadState_GetFrame`. diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 66983e0ecc1a2b..4fba7bc2a8e55e 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -134,18 +134,24 @@ Glossary attached thread state - A :term:`thread state` that is stored in the :term:`current thread state`. - If no thread state is attached, then the :term:`current thread state` is ``NULL``. - Attempting to call Python's C API without an attached thread state will result - in a fatal error or an undefined behavior. + A :term:`thread state` that is active for the current OS thread. - A thread state can be attached and detached explicitly by the user, or - implicitly by the interpreter in between calls. For example, an attached - thread state is detached upon entering a :c:macro:`Py_BEGIN_ALLOW_THREADS` - block, and then re-attached when :c:macro:`Py_END_ALLOW_THREADS` is reached. + When a :term:`thread state` is attached, the OS thread has + access to the full Python C API and can safely invoke the + bytecode interpreter. - On most builds of Python, having an attached thread state means that the - caller holds the :term:`GIL` for the current interpreter. + Unless a function explicitly notes otherwise, attempting to call + the C API without an attached thread state will result in a fatal + error or undefined behavior. A thread state can be attached and detached + explicitly by the user through the C API, or implicitly by the runtime, + including during blocking C calls and by the bytecode interpreter in between + calls. + + On most builds of Python, having an attached context means that the + caller holds the :term:`GIL` for the current interpreter, so only + one OS thread can have an attached thread state at a given moment. In + :term:`free-threaded ` builds of Python, threads can concurrently + hold an attached thread state, allowing for true parallelism. attribute A value associated with an object which is usually referenced by name @@ -348,19 +354,6 @@ Glossary tasks (see :mod:`asyncio`) associate each task with a context which becomes the current context whenever the task starts or resumes execution. - current thread state - - A per-thread :c:data:`PyThreadState` pointer. - - The pointer might be ``NULL``, in which case Python code must not - get executed. - - If the current thread state is non-``NULL``, then the :term:`thread state` - that it points to is considered to be :term:`attached `. - - The pointer for the calling thread can be acquired via :c:func:`PyThreadState_Get` or - :c:func:`PyThreadState_GetUnchecked`, if it might be ``NULL``. - decorator A function returning another function, usually applied as a function transformation using the ``@wrapper`` syntax. Common examples for @@ -1314,11 +1307,24 @@ Glossary :term:`bytes-like objects `. thread state - In Python's C API, a thread state is a structure that holds - information about the current thread, typically in the :term:`current thread state` - pointer. A thread state can be attached or detached. An :term:`attached thread state` - is required to call most of the C API, unless a function explicitly documents - otherwise. + + The information used by the :term:`CPython` runtime to run in an OS thread. + For example, this includes the current exception, if any, and the + state of the bytecode interpreter. + + Each thread state is bound to a single OS thread, but threads may have + many thread states available. At most, one of them may be + :term:`attached ` at once. + + An :term:`attached thread state` is required to call most + of Python's C API, unless a function explicitly documents otherwise. + The bytecode interpreter only runs under an attached thread state. + + Each thread state belongs to a single interpreter, but each interpreter + may have many thread states, including multiple for the same OS thread. + Thread states from multiple interpreters may be bound to the same + thread, but only one can be :term:`attached ` in + that thread at any given moment. See :ref:`Thread State and the Global Interpreter Lock ` for more information. From b3cbbf889eea67dc313e4dc43e92564f7ee9d7f1 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Wed, 12 Feb 2025 11:50:57 -0500 Subject: [PATCH 67/73] Add extra note about free-threading. --- Doc/glossary.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 4fba7bc2a8e55e..78293700fe6125 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -151,7 +151,8 @@ Glossary caller holds the :term:`GIL` for the current interpreter, so only one OS thread can have an attached thread state at a given moment. In :term:`free-threaded ` builds of Python, threads can concurrently - hold an attached thread state, allowing for true parallelism. + hold an attached thread state, allowing for true parallelism of the bytecode + interpreter. attribute A value associated with an object which is usually referenced by name From 1c37d59020819e1965d03ac0f394036724a6ddd9 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Wed, 12 Feb 2025 11:52:19 -0500 Subject: [PATCH 68/73] Add a seealso. --- Doc/c-api/init.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 05082bb15d22e7..2f7b839219570c 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1252,6 +1252,9 @@ code, or when embedding the Python interpreter: This function is safe to call without an :term:`attached thread state`; it will simply return ``NULL`` indicating that there was no prior thread state. + .. seealso: + :c:func:`PyEval_ReleaseThread` + The following functions use thread-local storage, and are not compatible with sub-interpreters: @@ -1306,7 +1309,7 @@ with sub-interpreters: always has such a thread-state, even if no auto-thread-state call has been made on the main thread. This is mainly a helper/diagnostic function. - .. seealso: PyThreadState_Get + .. seealso: :c:func:`PyThreadState_Get`` .. c:function:: int PyGILState_Check() From 2fba67c013f1635e7e62380592dc4270b24a5ff1 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Wed, 12 Feb 2025 11:58:46 -0500 Subject: [PATCH 69/73] Fix weird merge artifact. --- Doc/c-api/init_config.rst | 318 -------------------------------------- 1 file changed, 318 deletions(-) diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index f322d5a4c97444..b791d3cdc5d95c 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -2266,324 +2266,6 @@ The ``__PYVENV_LAUNCHER__`` environment variable is used to set set to the ``pyvenv.cfg`` directory. This was previously done by :mod:`site`, therefore affected by :option:`-S`. -.. _pyinitconfig_api: - -PyInitConfig C API -================== - -C API to configure the Python initialization (:pep:`741`). - -.. versionadded:: 3.14 - -Create Config -------------- - -.. c:struct:: PyInitConfig - - Opaque structure to configure the Python initialization. - - -.. c:function:: PyInitConfig* PyInitConfig_Create(void) - - Create a new initialization configuration using :ref:`Isolated Configuration - ` default values. - - It must be freed by :c:func:`PyInitConfig_Free`. - - Return ``NULL`` on memory allocation failure. - - -.. c:function:: void PyInitConfig_Free(PyInitConfig *config) - - Free memory of the initialization configuration *config*. - - If *config* is ``NULL``, no operation is performed. - - -Error Handling --------------- - -.. c:function:: int PyInitConfig_GetError(PyInitConfig* config, const char **err_msg) - - Get the *config* error message. - - * Set *\*err_msg* and return ``1`` if an error is set. - * Set *\*err_msg* to ``NULL`` and return ``0`` otherwise. - - An error message is an UTF-8 encoded string. - - If *config* has an exit code, format the exit code as an error - message. - - The error message remains valid until another ``PyInitConfig`` - function is called with *config*. The caller doesn't have to free the - error message. - - -.. c:function:: int PyInitConfig_GetExitCode(PyInitConfig* config, int *exitcode) - - Get the *config* exit code. - - * Set *\*exitcode* and return ``1`` if *config* has an exit code set. - * Return ``0`` if *config* has no exit code set. - - Only the ``Py_InitializeFromInitConfig()`` function can set an exit - code if the ``parse_argv`` option is non-zero. - - An exit code can be set when parsing the command line failed (exit - code ``2``) or when a command line option asks to display the command - line help (exit code ``0``). - - -Get Options ------------ - -The configuration option *name* parameter must be a non-NULL -null-terminated UTF-8 encoded string. - -.. c:function:: int PyInitConfig_HasOption(PyInitConfig *config, const char *name) - - Test if the configuration has an option called *name*. - - Return ``1`` if the option exists, or return ``0`` otherwise. - - -.. c:function:: int PyInitConfig_GetInt(PyInitConfig *config, const char *name, int64_t *value) - - Get an integer configuration option. - - * Set *\*value*, and return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - -.. c:function:: int PyInitConfig_GetStr(PyInitConfig *config, const char *name, char **value) - - Get a string configuration option as a null-terminated UTF-8 - encoded string. - - * Set *\*value*, and return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - *\*value* can be set to ``NULL`` if the option is an optional string and the - option is unset. - - On success, the string must be released with ``free(value)`` if it's not - ``NULL``. - - -.. c:function:: int PyInitConfig_GetStrList(PyInitConfig *config, const char *name, size_t *length, char ***items) - - Get a string list configuration option as an array of - null-terminated UTF-8 encoded strings. - - * Set *\*length* and *\*value*, and return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - On success, the string list must be released with - ``PyInitConfig_FreeStrList(length, items)``. - - -.. c:function:: void PyInitConfig_FreeStrList(size_t length, char **items) - - Free memory of a string list created by - ``PyInitConfig_GetStrList()``. - - -Set Options ------------ - -The configuration option *name* parameter must be a non-NULL null-terminated -UTF-8 encoded string. - -Some configuration options have side effects on other options. This logic is -only implemented when ``Py_InitializeFromInitConfig()`` is called, not by the -"Set" functions below. For example, setting ``dev_mode`` to ``1`` does not set -``faulthandler`` to ``1``. - -.. c:function:: int PyInitConfig_SetInt(PyInitConfig *config, const char *name, int64_t value) - - Set an integer configuration option. - - * Return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - -.. c:function:: int PyInitConfig_SetStr(PyInitConfig *config, const char *name, const char *value) - - Set a string configuration option from a null-terminated UTF-8 - encoded string. The string is copied. - - * Return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - -.. c:function:: int PyInitConfig_SetStrList(PyInitConfig *config, const char *name, size_t length, char * const *items) - - Set a string list configuration option from an array of - null-terminated UTF-8 encoded strings. The string list is copied. - - * Return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - -Module ------- - -.. c:function:: int PyInitConfig_AddModule(PyInitConfig *config, const char *name, PyObject* (*initfunc)(void)) - - Add a built-in extension module to the table of built-in modules. - - The new module can be imported by the name *name*, and uses the function - *initfunc* as the initialization function called on the first attempted - import. - - * Return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - - If Python is initialized multiple times, ``PyInitConfig_AddModule()`` must - be called at each Python initialization. - - Similar to the :c:func:`PyImport_AppendInittab` function. - - -Initialize Python ------------------ - -.. c:function:: int Py_InitializeFromInitConfig(PyInitConfig *config) - - Initialize Python from the initialization configuration. - - * Return ``0`` on success. - * Set an error in *config* and return ``-1`` on error. - * Set an exit code in *config* and return ``-1`` if Python wants to - exit. - - See ``PyInitConfig_GetExitcode()`` for the exit code case. - - -Example -------- - -Example initializing Python, set configuration options of various types, -return ``-1`` on error: - -.. code-block:: c - - int init_python(void) - { - PyInitConfig *config = PyInitConfig_Create(); - if (config == NULL) { - printf("PYTHON INIT ERROR: memory allocation failed\n"); - return -1; - } - - // Set an integer (dev mode) - if (PyInitConfig_SetInt(config, "dev_mode", 1) < 0) { - goto error; - } - - // Set a list of UTF-8 strings (argv) - char *argv[] = {"my_program", "-c", "pass"}; - if (PyInitConfig_SetStrList(config, "argv", - Py_ARRAY_LENGTH(argv), argv) < 0) { - goto error; - } - - // Set a UTF-8 string (program name) - if (PyInitConfig_SetStr(config, "program_name", L"my_program") < 0) { - goto error; - } - - // Initialize Python with the configuration - if (Py_InitializeFromInitConfig(config) < 0) { - goto error; - } - PyInitConfig_Free(config); - return 0; - - error: - { - // Display the error message - // This uncommon braces style is used, because you cannot make - // goto targets point to variable declarations. - const char *err_msg; - (void)PyInitConfig_GetError(config, &err_msg); - printf("PYTHON INIT ERROR: %s\n", err_msg); - PyInitConfig_Free(config); - - return -1; - } - } - - -Runtime Python configuration API -================================ - -The configuration option *name* parameter must be a non-NULL null-terminated -UTF-8 encoded string. - -Some options are read from the :mod:`sys` attributes. For example, the option -``"argv"`` is read from :data:`sys.argv`. - - -.. c:function:: PyObject* PyConfig_Get(const char *name) - - Get the current runtime value of a configuration option as a Python object. - - * Return a new reference on success. - * Set an exception and return ``NULL`` on error. - - The object type depends on the configuration option. It can be: - - * ``bool`` - * ``int`` - * ``str`` - * ``list[str]`` - * ``dict[str, str]`` - - The caller must have an :term:`attached thread state`. The function cannot - be called before Python initialization nor after Python finalization. - - .. versionadded:: 3.14 - - -.. c:function:: int PyConfig_GetInt(const char *name, int *value) - - Similar to :c:func:`PyConfig_Get`, but get the value as a C int. - - * Return ``0`` on success. - * Set an exception and return ``-1`` on error. - - .. versionadded:: 3.14 - - -.. c:function:: PyObject* PyConfig_Names(void) - - Get all configuration option names as a ``frozenset``. - - * Return a new reference on success. - * Set an exception and return ``NULL`` on error. - - The caller must have an :term:`attached thread state`. The function cannot - be called before Python initialization nor after Python finalization. - - .. versionadded:: 3.14 - - -.. c:function:: int PyConfig_Set(const char *name, PyObject *value) - - Set the current runtime value of a configuration option. - - * Raise a :exc:`ValueError` if there is no option *name*. - * Raise a :exc:`ValueError` if *value* is an invalid value. - * Raise a :exc:`ValueError` if the option is read-only (cannot be set). - * Raise a :exc:`TypeError` if *value* has not the proper type. - - The caller must have an :term:`attached thread state`. The function cannot - be called before Python initialization nor after Python finalization. - - .. versionadded:: 3.14 - Py_GetArgcArgv() ================ From 94619f601f9cbf91108cda13e8be77ead14ffd5c Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Thu, 27 Feb 2025 07:32:58 -0500 Subject: [PATCH 70/73] Change term for new PyConfig* APIs --- Doc/c-api/init_config.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index b791d3cdc5d95c..7aa74213f4d0c9 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -574,8 +574,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option * ``list[str]`` * ``dict[str, str]`` - The caller must hold the GIL. The function cannot be called before - Python initialization nor after Python finalization. + The caller must have an :term:`attached thread state`. The function cannot + be called before Python initialization nor after Python finalization. .. versionadded:: 3.14 @@ -597,8 +597,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option * Return a new reference on success. * Set an exception and return ``NULL`` on error. - The caller must hold the GIL. The function cannot be called before - Python initialization nor after Python finalization. + The caller must have an :term:`attached thread state`. The function cannot + be called before Python initialization nor after Python finalization. .. versionadded:: 3.14 @@ -612,8 +612,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option * Raise a :exc:`ValueError` if the option is read-only (cannot be set). * Raise a :exc:`TypeError` if *value* has not the proper type. - The caller must hold the GIL. The function cannot be called before - Python initialization nor after Python finalization. + The caller must have an :term:`attached thread state`. The function cannot + be called before Python initialization nor after Python finalization. .. versionadded:: 3.14 From e6f1f2876834a378cc295bd8c2c029eec8c15d6f Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Thu, 27 Feb 2025 07:33:19 -0500 Subject: [PATCH 71/73] Change term for PyUnstable_InterpreterState_GetMainModule --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index ea85d1fa1dbe37..3597f35e0a2656 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1522,7 +1522,7 @@ All of the following functions must be called after :c:func:`Py_Initialize`. Return a :term:`strong reference` to the ``__main__`` :ref:`module object ` for the given interpreter. - The caller must hold the GIL. + The caller must have an :term:`attached thread state`. .. versionadded:: 3.13 From 828d24bdbfd18799eaebee662418edd36a2cdcef Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Thu, 27 Feb 2025 07:33:59 -0500 Subject: [PATCH 72/73] Change term for Py_fopen --- Doc/c-api/sys.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/sys.rst b/Doc/c-api/sys.rst index 44cd9943d76c7b..b3c89800e386ff 100644 --- a/Doc/c-api/sys.rst +++ b/Doc/c-api/sys.rst @@ -232,7 +232,7 @@ Operating System Utilities The file descriptor is created non-inheritable (:pep:`446`). - The caller must hold the GIL. + The caller must have an :term:`attached thread state`. .. versionadded:: 3.14 From 788d735989f18452e0467b2fdd696675dfc37a06 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Thu, 20 Mar 2025 06:23:18 -0400 Subject: [PATCH 73/73] Change artifact from 'runtime context' to 'thread state' --- Doc/glossary.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/glossary.rst b/Doc/glossary.rst index a9b6d5d2996dc0..0b26e18efd7f1b 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -147,7 +147,7 @@ Glossary including during blocking C calls and by the bytecode interpreter in between calls. - On most builds of Python, having an attached context means that the + On most builds of Python, having an attached thread state implies that the caller holds the :term:`GIL` for the current interpreter, so only one OS thread can have an attached thread state at a given moment. In :term:`free-threaded ` builds of Python, threads can concurrently