8000 gh-87135: Raise PythonFinalizationError when joining a blocked daemon… · python/cpython@4ebbfcf · GitHub
[go: up one dir, main page]
Skip to content

    • Commit 4ebbfcf

    Browse files
    encukouencukou
    authored
    gh-87135: Raise PythonFinalizationError when joining a blocked daemon thread (gh-130402)
    If `Py_IsFinalizing()` is true, non-daemon threads (other than the current one) are done, and daemon threads are prevented from running, so they cannot finalize themselves and become done. Joining them (without timeout) would block forever. Raise PythonFinalizationError instead of hanging. Raise even when a timeout is given, for consistency with trying to join your own thread. See gh-123940 for a use case: calling `join()` from `__del__`. This is ill-advised, but an exception should at least make it easier to diagnose.
    1 parent 995b1a7 commit 4ebbfcf

    File tree

    6 files changed

    +101
    -6
    lines changed

    6 files changed

    +101
    -6
    lines changed

    ‎Doc/c-api/init.rst

    Lines changed: 1 addition & 1 deletion
    Original file line numberDiff line numberDiff line change
    @@ -1131,7 +1131,7 @@ Cautions regarding runtime finalization
    11311131
    In the late stage of :term:`interpreter shutdown`, after attempting to wait for
    11321132
    non-daemon threads to exit (though this can be interrupted by
    11331133
    :class:`KeyboardInterrupt`) and running the :mod:`atexit` functions, the runtime
    1134-
    is marked as *finalizing*: :c:func:`_Py_IsFinalizing` and
    1134+
    is marked as *finalizing*: :c:func:`Py_IsFinalizing` and
    11351135
    :func:`sys.is_finalizing` return true. At this point, only the *finalization
    11361136
    thread* that initiated finalization (typically the main thread) is allowed to
    11371137
    acquire the :term:`GIL`.

    ‎Doc/library/exceptions.rst

    Lines changed: 4 additions & 0 deletions
    Original file line numberDiff line numberDiff line change
    @@ -428,13 +428,17 @@ The following exceptions are the exceptions that are usually raised.
    428428
    :exc:`PythonFinalizationError` during the Python finalization:
    429429

    430430
    * Creating a new Python thread.
    431+
    * :meth:`Joining <threading.Thread.join>` a running daemon thread.
    431432
    * :func:`os.fork`.
    432433

    433434
    See also the :func:`sys.is_finalizing` function.
    434435

    435436
    .. versionadded:: 3.13
    436437
    Previously, a plain :exc:`RuntimeError` was raised.
    437438

    439+
    .. versionchanged:: next
    440+
    441+
    :meth:`threading.Thread.join` can now raise this exception.
    438442

    439443
    .. exception:: RecursionError
    440444

    ‎Doc/library/threading.rst

    Lines changed: 8 additions & 0 deletions
    Original file line numberDiff line numberDiff line change
    @@ -448,6 +448,14 @@ since it is impossible to detect the termination of alien threads.
    448448
    an error to :meth:`~Thread.join` a thread before it has been started
    449449
    and attempts to do so raise the same exception.
    450450

    451+
    If an attempt is made to join a running daemonic thread in in late stages
    452+
    of :term:`Python finalization <interpreter shutdown>` :meth:`!join`
    453+
    raises a :exc:`PythonFinalizationError`.
    454+
    455+
    .. versionchanged:: next
    456+
    457+
    May raise :exc:`PythonFinalizationError`.
    458+
    451459
    .. attribute:: name
    452460

    453461
    A string used for identification purposes only. It has no semantics.

    ‎Lib/test/test_threading.py

    Lines changed: 71 additions & 0 deletions
    Original file line numberDiff line numberDiff line change
    @@ -1171,6 +1171,77 @@ def __del__(self):
    11711171
    self.assertEqual(out.strip(), b"OK")
    11721172
    self.assertIn(b"can't create new thread at interpreter shutdown", err)
    11731173

    1174+
    def test_join_daemon_thread_in_finalization(self):
    1175+
    # gh-123940: Py_Finalize() prevents other threads from running Python
    1176+
    # code, so join() can not succeed unless the thread is already done.
    1177+
    # (Non-Python threads, that is `threading._DummyThread`, can't be
    1178+
    # joined at all.)
    1179+
    # We raise an exception rather than hang.
    1180+
    for timeout in (None, 10):
    1181+
    with self.subTest(timeout=timeout):
    1182+
    code = textwrap.dedent(f"""
    1183+
    import threading
    1184+
    1185+
    1186+
    def loop():
    1187+
    while True:
    1188+
    pass
    1189+
    1190+
    1191+
    class Cycle:
    1192+
    def __init__(self):
    1193+
    self.self_ref = self
    1194+
    self.thr = threading.Thread(
    1195+
    target=loop, daemon=True)
    1196+
    self.thr.start()
    1197+
    1198+
    def __del__(self):
    1199+
    assert self.thr.is_alive()
    1200+
    try:
    1201+
    self.thr.join(timeout={timeout})
    1202+
    except PythonFinalizationError:
    1203+
    assert self.thr.is_alive()
    1204+
    print('got the correct exception!')
    1205+
    1206+
    # Cycle holds a reference to itself, which ensures it is
    1207+
    # cleaned up during the GC that runs after daemon threads
    1208+
    # have been forced to exit during finalization.
    1209+
    Cycle()
    1210+
    """)
    1211+
    rc, out, err = assert_python_ok("-c", code)
    1212+
    self.assertEqual(err, b"")
    1213+
    self.assertIn(b"got the correct exception", out)
    1214+
    1215+
    def test_join_finished_daemon_thread_in_finalization(self):
    1216+
    # (see previous test)
    1217+
    # If the thread is already finished, join() succeeds.
    1218+
    code = textwrap.dedent("""
    1219+
    import threading
    1220+
    done = threading.Event()
    1221+
    1222+
    def loop():
    1223+
    done.set()
    1224+
    1225+
    1226+
    class Cycle:
    1227+
    def __init__(self):
    1228+
    self.self_ref = self
    1229+
    self.thr = threading.Thread(target=loop, daemon=True)
    1230+
    self.thr.start()
    1231+
    done.wait()
    1232+
    1233+
    def __del__(self):
    1234+
    assert not self.thr.is_alive()
    1235+
    self.thr.join()
    1236+
    assert not self.thr.is_alive()
    1237+
    print('all clear!')
    1238+
    1239+
    Cycle()
    1240+
    """)
    1241+
    rc, out, err = assert_python_ok("-c", code)
    1242+
    self.assertEqual(err, b"")
    1243+
    self.assertIn(b"all clear", out)
    1244+
    11741245
    def test_start_new_thread_failed(self):
    11751246
    # gh-109746: if Python fails to start newly created thread
    11761247
    # due to failure of underlying PyThread_start_new_thread() call,

    ‎Misc/NEWS.d/next/Library/2025-02-21-15-46-43.gh-issue-130402.Rwu_KK.rst

    Lines changed: 2 additions & 0 deletions
    Original file line numberDiff line numberDiff line change
    @@ -0,0 +1,2 @@
    1+
    Joining running daemon threads during interpreter shutdown
    2+
    now raises :exc:`PythonFinalizationError`.

    ‎Modules/_threadmodule.c

    Lines changed: 15 additions & 5 deletions
    Original file line numberDiff line numberDiff line change
    @@ -511,11 +511,21 @@ ThreadHandle_join(ThreadHandle *self, PyTime_t timeout_ns)
    511511
    // To work around this, we set `thread_is_exiting` immediately before
    512512
    // `thread_run` returns. We can be sure that we are not attempting to join
    513513
    // ourselves if the handle's thread is about to exit.
    514-
    if (!_PyEvent_IsSet(&self->thread_is_exiting) &&
    515-
    ThreadHandle_ident(self) == PyThread_get_thread_ident_ex()) {
    516-
    // PyThread_join_thread() would deadlock or error out.
    517-
    PyErr_SetString(ThreadError, "Cannot join current thread");
    518-
    return -1;
    514+
    if (!_PyEvent_IsSet(&self->thread_is_exiting)) {
    515+
    if (ThreadHandle_ident(self) == PyThread_get_thread_ident_ex()) {
    516+
    // PyThread_join_thread() would deadlock or error out.
    517+
    PyErr_SetString(ThreadError, "Cannot join current thread");
    518+
    return -1;
    519+
    }
    520+
    if (Py_IsFinalizing()) {
    521+
    // gh-123940: On finalization, other threads are prevented from
    522+
    // running Python code. They cannot finalize themselves,
    523+
    // so join() would hang forever (or until timeout).
    524+
    // We raise instead.
    525+
    PyErr_SetString(PyExc_PythonFinalizationError,
    526+
    "cannot join thread at interpreter shutdown");
    527+
    return -1;
    528+
    }
    519529
    }
    520530

    521531
    // Wait until the deadline for the thread to exit.

    0 commit comments

    Comments
     (0)
    0