8000 gh-87135: Raise PythonFinalizationError when joining a blocked daemon… · python/cpython@4ebbfcf · GitHub
[go: up one dir, main page]
Skip to content

Commit 4ebbfcf

Browse files
encukouencukou
authored
gh-87135: Raise PythonFinalizationError when joining a blocked daemon thread (gh-130402)
If `Py_IsFinalizing()` is true, non-daemon threads (other than the current one) are done, and daemon threads are prevented from running, so they cannot finalize themselves and become done. Joining them (without timeout) would block forever. Raise PythonFinalizationError instead of hanging. Raise even when a timeout is given, for consistency with trying to join your own thread. See gh-123940 for a use case: calling `join()` from `__del__`. This is ill-advised, but an exception should at least make it easier to diagnose.
1 parent 995b1a7 commit 4ebbfcf

File tree

6 files changed

+101
-6
lines changed

6 files changed

+101
-6
lines changed

‎Doc/c-api/init.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1131,7 +1131,7 @@ Cautions regarding runtime finalization
11311131
In the late stage of :term:`interpreter shutdown`, after attempting to wait for
11321132
non-daemon threads to exit (though this can be interrupted by
11331133
:class:`KeyboardInterrupt`) and running the :mod:`atexit` functions, the runtime
1134-
is marked as *finalizing*: :c:func:`_Py_IsFinalizing` and
1134+
is marked as *finalizing*: :c:func:`Py_IsFinalizing` and
11351135
:func:`sys.is_finalizing` return true. At this point, only the *finalization
11361136
thread* that initiated finalization (typically the main thread) is allowed to
11371137
acquire the :term:`GIL`.

‎Doc/library/exceptions.rst‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -428,13 +428,17 @@ The following exceptions are the exceptions that are usually raised.
428428
:exc:`PythonFinalizationError` during the Python finalization:
429429

430430
* Creating a new Python thread.
431+
* :meth:`Joining <threading.Thread.join>` a running daemon thread.
431432
* :func:`os.fork`.
432433

433434
See also the :func:`sys.is_finalizing` function.
434435

435436
.. versionadded:: 3.13
436437
Previously, a plain :exc:`RuntimeError` was raised.
437438

439+
.. versionchanged:: next
440+
441+
:meth:`threading.Thread.join` can now raise this exception.
438442

439443
.. exception:: RecursionError
440444

‎Doc/library/threading.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -448,6 +448,14 @@ since it is impossible to detect the termination of alien threads.
448448
an error to :meth:`~Thread.join` a thread before it has been started
449449
and attempts to do so raise the same exception.
450450

451+
If an attempt is made to join a running daemonic thread in in late stages
452+
of :term:`Python finalization <interpreter shutdown>` :meth:`!join`
453+
raises a :exc:`PythonFinalizationError`.
454+
455+
.. versionchanged:: next
456+
457+
May raise :exc:`PythonFinalizationError`.
458+
451459
.. attribute:: name
452460

453461
A string used for identification purposes only. It has no semantics.

‎Lib/test/test_threading.py‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1171,6 +1171,77 @@ def __del__(self):
11711171
self.assertEqual(out.strip(), b"OK")
11721172
self.assertIn(b"can't create new thread at interpreter shutdown", err)
11731173

1174+
def test_join_daemon_thread_in_finalization(self):
1175+
# gh-123940: Py_Finalize() prevents other threads from running Python
1176+
# code, so join() can not succeed unless the thread is already done.
1177+
# (Non-Python threads, that is `threading._DummyThread`, can't be
1178+
# joined at all.)
1179+
# We raise an exception rather than hang.
1180+
for timeout in (None, 10):
1181+
with self.subTest(timeout=timeout):
1182+
code = textwrap.dedent(f"""
1183+
import threading
1184+
1185+
1186+
def loop():
1187+
while True:
1188+
pass
1189+
1190+
1191+
class Cycle:
1192+
def __init__(self):
1193+
self.self_ref = self
1194+
self.thr = threading.Thread(
1195+
target=loop, daemon=True)
1196+
self.thr.start()
1197+
1198+
def __del__(self):
1199+
assert self.thr.is_alive()
1200+
try:
1201+
self.thr.join(timeout={timeout})
1202+
except PythonFinalizationError:
1203+
assert self.thr.is_alive()
1204+
print('got the correct exception!')
1205+
1206+
# Cycle holds a reference to itself, which ensures it is
1207+
# cleaned up during the GC that runs after daemon threads
1208+
# have been forced to exit during finalization.
1209+
Cycle()
1210+
""")
1211+
rc, out, err = assert_python_ok("-c", code)
1212+
self.assertEqual(err, b"")
1213+
self.assertIn(b"got the correct exception", out)
1214+
1215+
def test_join_finished_daemon_thread_in_finalization(self):
1216+
# (see previous test)
1217+
# If the thread is already finished, join() succeeds.
1218+
code = textwrap.dedent("""
1219+
import threading
1220+
done = threading.Event()
1221+
1222+
def loop():
1223+
done.set()
1224+
1225+
1226+
class Cycle:
1227+
def __init__(self):
1228+
self.self_ref = self
1229+
self.thr = threading.Thread(target=loop, daemon=True)
1230+
self.thr.start()
1231+
done.wait()
1232+
1233+
def __del__(self):
1234+
assert not self.thr.is_alive()
1235+
self.thr.join()
1236+
assert not self.thr.is_alive()
1237+
print('all clear!')
1238+
1239+
Cycle()
1240+
""")
1241+
rc, out, err = assert_python_ok("-c", code)
1242+
self.assertEqual(err, b"")
1243+
self.assertIn(b"all clear", out)
1244+
11741245
def test_start_new_thread_failed(self):
11751246
# gh-109746: if Python fails to start newly created thread
11761247
# due to failure of underlying PyThread_start_new_thread() call,

‎Misc/NEWS.d/next/Library/2025-02-21-15-46-43.gh-issue-130402.Rwu_KK.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Joining running daemon threads during interpreter shutdown
2+
now raises :exc:`PythonFinalizationError`.

‎Modules/_threadmodule.c‎

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -511,11 +511,21 @@ ThreadHandle_join(ThreadHandle *self, PyTime_t timeout_ns)
511511
// To work around this, we set `thread_is_exiting` immediately before
512512
// `thread_run` returns. We can be sure that we are not attempting to join
513513
// ourselves if the handle's thread is about to exit.
514-
if (!_PyEvent_IsSet(&self->thread_is_exiting) &&
515-
ThreadHandle_ident(self) == PyThread_get_thread_ident_ex()) {
516-
// PyThread_join_thread() would deadlock or error out.
517-
PyErr_SetString(ThreadError, "Cannot join current thread");
518-
return -1;
514+
if (!_PyEvent_IsSet(&self->thread_is_exiting)) {
515+
if (ThreadHandle_ident(self) == PyThread_get_thread_ident_ex()) {
516+
// PyThread_join_thread() would deadlock or error out.
517+
PyErr_SetString(ThreadError, "Cannot join current thread");
518+
return -1;
519+
}
520+
if (Py_IsFinalizing()) {
521+
// gh-123940: On finalization, other threads are prevented from
522+
// running Python code. They cannot finalize themselves,
523+
// so join() would hang forever (or until timeout).
524+
// We raise instead.
525+
PyErr_SetString(PyExc_PythonFinalizationError,
526+
"cannot join thread at interpreter shutdown");
527+
return -1;
528+
}
519529
}
520530

521531
// Wait until the deadline for the thread to exit.

0 commit comments

Comments
 (0)
0