8000 gh-87135: Raise PythonFinalizationError when joining a blocked daemon thread by encukou · Pull Request #130402 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

gh-87135: Raise PythonFinalizationError when joining a blocked daemon thread #130402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Apr 28, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Raise for calls with timeout too
  • Loading branch information
encukou committed Feb 27, 2025
commit 4aecdf5ffdfac56ae2c9520add06af721812a81f
3 changes: 1 addition & 2 deletions Doc/library/exceptions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -426,8 +426,7 @@ The following exceptions are the exceptions that are usually raised.
:exc:`PythonFinalizationError` during the Python finalization:

* Creating a new Python thread.
* :meth:`Joining <threading.Thread.join>` a running daemon thread
without a timeout.
* :meth:`Joining <threading.Thread.join>` a running daemon thread.
* :func:`os.fork`.

See also the :func:`sys.is_finalizing` function.
Expand Down
9 changes: 3 additions & 6 deletions Doc/library/threading.rst
Original file line number Diff line number Diff line change
Expand Up @@ -435,12 +435,9 @@ since it is impossible to detect the termination of alien threads.
an error to :meth:`~Thread.join` a thread before it has been started
and attempts to do so raise the same exception.

In late stages of :term:`Python finalization <interpreter shutdown>`,
if *timeout* is ``None`` and an attempt is made to join a running
daemonic thread, :meth:`!join` raises a :exc:`PythonFinalizationError`.
(Such a join would block forever: at this point, threads other than the
current one are prevented from running Python code and so they cannot
finalize themselves.)
If an attempt is made to join a running daemonic thread in in late stages
of :term:`Python finalization <interpreter shutdown>` :meth:`!join`
raises a :exc:`PythonFinalizationError`.

.. versionchanged:: next

Expand Down
90 changes: 32 additions & 58 deletions Lib/test/test_threading.py
Original file line number Diff line number Diff line change
Expand Up @@ -1177,35 +1177,38 @@ def test_join_daemon_thread_in_finalization(self):
# (Non-Python threads, that is `threading._DummyThread`, can't be
# joined at all.)
# We raise an exception rather than hang.
code = textwrap.dedent("""
import threading


def loop():
while True:
pass


class Cycle:
def __init__(self):
self.self_ref = self
self.thr = threading.Thread(target=loop, daemon=True)
self.thr.start()

def __del__(self):
try:
self.thr.join()
except PythonFinalizationError:
print('got the correct exception!')

# Cycle holds a reference to itself, which ensures it is cleaned
# up during the GC that runs after daemon threads have been
# forced to exit during finalization.
Cycle()
""")
rc, out, err = assert_python_ok("-c", code)
self.assertEqual(err, b"")
self.assertIn(b"got the correct exception", out)
for timeout in (None, 10):
with self.subTest(timeout=timeout):
code = textwrap.dedent(f"""
import threading


def loop():
while True:
pass


class Cycle:
def __init__(self):
self.self_ref = self
self.thr = threading.Thread(
target=loop, daemon=True)
self.thr.start()

def __del__(self):
try:
self.thr.join(timeout={timeout})
except PythonFinalizationError:
print('got the correct exception!')

# Cycle holds a reference to itself, which ensures it is
# cleaned up during the GC that runs after daemon threads
# have been forced to exit during finalization.
Cycle()
""")
rc, out, err = assert_python_ok("-c", code)
self.assertEqual(err, b"")
self.assertIn(b"got the correct exception", out)

def test_join_finished_daemon_thread_in_finalization(self):
# (see previous test)
Expand Down Expand Up @@ -1235,35 +1238,6 @@ def __del__(self):
self.assertEqual(err, b"")
self.assertIn(b"all clear", out)

def test_timed_join_daemon_thread_in_finalization(self):
# (see previous test)
# When called with timeout, no error is raised.
code = textwrap.dedent("""
import threading
done = threading.Event()

def loop():
done.set()
while True:
pass

class Cycle:
def __init__(self):
self.self_ref = self
self.thr = threading.Thread(target=loop, daemon=True)
self.thr.start()
done.wait()

def __del__(self):
self.thr.join(timeout=0.01)
print('alive:', self.thr.is_alive())

Cycle()
""")
rc, out, err = assert_python_ok("-c", code)
self.assertEqual(err, b"")
self.assertIn(b"alive: True", out)

def test_start_new_thread_failed(self):
# gh-109746: if Python fails to start newly created thread
# due to failure of underlying PyThread_start_new_thread() call,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
Joining running daemon threads during interpreter shutdown without timeout
Joining running daemon threads during interpreter shutdown
now raises :exc:`PythonFinalizationError`.
29 changes: 11 additions & 18 deletions Modules/_threadmodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -517,27 +517,20 @@ ThreadHandle_join(ThreadHandle *self, PyTime_t timeout_ns)
PyErr_SetString(ThreadError, "Cannot join current thread");
return -1;
}

PyTime_t deadline = 0;

if (timeout_ns == -1) {
if (Py_IsFinalizing()) {
// gh-123940: On finalization, other threads are prevented from
// running Python code. They cannot finalize themselves,
// so join() would hang forever.
// We raise instead.
// (We only do this if no timeout is given: otherwise
// we assume the caller can handle a hung thread.)
PyErr_SetString(PyExc_PythonFinalizationError,
"cannot join thread at interpreter shutdown");
return -1;
}
}
else {
deadline = _PyDeadline_Init(timeout_ns);
if (Py_IsFinalizing()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this code path taken by all threads, or only daemon threads?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's taken by the thread that called Py_FinalizeEx.
When Py_IsFinalizing is true, all other threads than the one that called Py_FinalizeEx are daemonic and they cannot call Python API (including ThreadHandle_join).
So, self must be a daemon thread.

// gh-123940: On finalization, other threads are prevented from
// running Python code. They cannot finalize themselves,
// so join() would hang forever.
// We raise instead.
// (We only do this if no timeout is given: otherwise
// we assume the caller can handle a hung thread.)
PyErr_SetString(PyExc_PythonFinalizationError,
"cannot join thread at interpreter shutdown");
return -1;
}

// Wait until the deadline for the thread to exit.
PyTime_t deadline = timeout_ns != -1 ? _PyDeadline_Init(timeout_ns) : 0;
int detach = 1;
while (!PyEvent_WaitTimed(is_exiting, timeout_ns, detach)) {
if (deadline) {
Expand Down
0