8000 gh-116522: Stop the world before fork() and during shutdown by colesbury · Pull Request #116607 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

gh-116522: Stop the world before fork() and during shutdown #116607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 21, 2024
Merged

gh-116522: Stop the world before fork() and during shutdown #116607

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c2ebedb
gh-116522: Stop the world before fork() and during shutdown
colesbury Mar 8, 2024
b13d1e5
Move stop-the-world call before setting finalized
colesbury Mar 12, 2024
4772a69
Add comment and assertion to warn_about_fork_with_threads
colesbury Mar 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
gh-116522: Stop the world before fork() and during shutdown 
This changes the free-threaded build to perform a stop-the-world pause
before deleting other thread states when forking and during shutdown.
This fixes some crashes when using multiprocessing and during shutdown
when running with `PYTHON_GIL=0`.

This also chan
8000
ges `PyOS_BeforeFork` to acquire the runtime lock
(i.e., `HEAD_LOCK(&_PyRuntime)`) before forking to ensure that data
protected by the runtime lock (and not just the GIL or stop-the-world)
is in a consistent state before forking.
  • Loading branch information
@colesbury
colesbury committed Mar 11, 2024
commit c2ebedbaf05bb4065f962ac14758ccfdebf8a0bf
12 changes: 9 additions & 3 deletions Modules/posixmodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -613,11 +613,16 @@ PyOS_BeforeFork(void)
run_at_forkers(interp->before_forkers, 1);

_PyImport_AcquireLock(interp);
_PyEval_StopTheWorldAll(&_PyRuntime);
HEAD_LOCK(&_PyRuntime);
}

void
PyOS_AfterFork_Parent(void)
{
HEAD_UNLOCK(&_PyRuntime);
_PyEval_StartTheWorldAll(&_PyRuntime);

PyInterpreterState *interp = _PyInterpreterState_GET();
if (_PyImport_ReleaseLock(interp) <= 0) {
Py_FatalError("failed releasing import lock after fork");
Expand All @@ -632,6 +637,7 @@ PyOS_AfterFork_Child(void)
PyStatus status;
_PyRuntimeState *runtime = &_PyRuntime;

// re-creates runtime->interpreters.mutex (HEAD_UNLOCK)
status = _PyRuntimeState_ReInitThreads(runtime);
if (_PyStatus_EXCEPTION(status)) {
goto fatal_error;
Expand Down Expand Up @@ -7858,9 +7864,9 @@ os_fork1_impl(PyObject *module)
/* child: this clobbers and resets the import lock. */
PyOS_AfterFork_Child();
} else {
warn_about_fork_with_threads("fork1");
/* parent: release the import lock. */
PyOS_AfterFork_Parent();
warn_about_fork_with_threads("fork1");
}
if (pid == -1) {
errno = saved_errno;
Expand Down Expand Up @@ -7906,9 +7912,9 @@ os_fork_impl(PyObject *module)
/* child: this clobbers and resets the import lock. */
PyOS_AfterFork_Child();
} else {
warn_about_fork_with_threads("fork");
/* parent: release the import lock. */
PyOS_AfterFork_Parent();
warn_about_fork_with_threads("fork");
}
if (pid == -1) {
errno = saved_errno;
Expand Down Expand Up @@ -8737,9 +8743,9 @@ os_forkpty_impl(PyObject *module)
/* child: this clobbers and resets the import lock. */
PyOS_AfterFork_Child();
} else {
warn_about_fork_with_threads("forkpty");
/* parent: release the import lock. */
PyOS_AfterFork_Parent();
warn_about_fork_with_threads("forkpty");
}
if (pid == -1) {
return posix_error();
Expand Down
3 changes: 3 additions & 0 deletions Python/pylifecycle.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1918,6 +1918,9 @@ Py_FinalizeEx(void)
runtime->initialized = 0;
runtime->core_initialized = 0;

/* Ensure that remaining threads are detached */
_PyEval_StopTheWorldAll(runtime);

// XXX Call something like _PyImport_Disable() here?

/* Destroy the state of all threads of the interpreter, except of the
Expand Down
6 changes: 6 additions & 0 deletions Python/pystate.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1715,6 +1715,10 @@ _PyThreadState_DeleteExcept(PyThreadState *tstate)
PyInterpreterState *interp = tstate->interp;
_PyRuntimeState *runtime = interp->runtime;

#ifdef Py_GIL_DISABLED
assert(runtime->stoptheworld.world_stopped);
#endif

HEAD_LOCK(runtime);
/* Remove all thread states, except tstate, from the linked list of
thread states. This will allow calling PyThreadState_Clear()
Expand All @@ -1733,6 +1737,8 @@ _PyThreadState_DeleteExcept(PyThreadState *tstate)
interp->threads.head = tstate;
HEAD_UNLOCK(runtime);

_PyEval_StartTheWorldAll(runtime);

/* Clear and deallocate all stale thread states. Even if this
executes Python code, we should be safe since it executes
in the current thread, not one of the stale threads. */
Expand Down
0