8000 [3.7] bpo-38546: Backport multiprocessing tests fixes from master (GH… · python/cpython@fd32a0e · GitHub
[go: up one dir, main page]
Skip to content

Commit fd32a0e

Browse files
vstinnermiss-islington
vstinner
and
miss-islington
authored
[3.7] bpo-38546: Backport multiprocessing tests fixes from master (GH-19689)
* bpo-37421: multiprocessing tests call _run_finalizers() (GH-14527) multiprocessing tests now call explicitly _run_finalizers() to remove immediately temporary directories created by multiprocessing.util.get_temp_dir(). (cherry picked from commit 039fb49) Co-authored-by: Victor Stinner <vstinner@redhat.com> (cherry picked from commit 632cb36) * bpo-37421: multiprocessing tests now stop ForkServer (GH-14601) multiprocessing tests now stop the ForkServer instance if it's running: close the "alive" file descriptor to ask the server to stop and then remove its UNIX address. (cherry picked from commit 8fbeb14) Co-authored-by: Victor Stinner <vstinner@redhat.com> (cherry picked from commit 229f6e8) * bpo-38546: multiprocessing tests stop the resource tracker (GH-17641) (GH-17647) Multiprocessing and concurrent.futures tests now stop the resource tracker process when tests complete. Add ResourceTracker._stop() method to multiprocessing.resource_tracker. Add _cleanup_tests() helper function to multiprocessing.util: share code between multiprocessing and concurrent.futures tests. (cherry picked from commit 9707e8e) (cherry picked from commit 35acb35) * Remove NEWS about resource tracker Python 3.7 multiprocessing does not have resource tracker. Co-authored-by: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com>
1 parent 857d573 commit fd32a0e

File tree

6 files changed

+49
-2
lines changed

6 files changed

+49
-2
lines changed

‎Lib/multiprocessing/forkserver.py

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,25 @@ def __init__(self):
3939
self._lock = threading.Lock()
4040
self._preload_modules = ['__main__']
4141

42+
def _stop(self):
43+
# Method used by unit tests to stop the server
44+
with self._lock:
45+
self._stop_unlocked()
46+
47+
def _stop_unlocked(self):
48+
if self._forkserver_pid is None:
49+
return
50+
51+
# close the "alive" file descriptor asks the server to stop
52+
os.close(self._forkserver_alive_fd)
53+
self._forkserver_alive_fd = None
54+
55+
os.waitpid(self._forkserver_pid, 0)
56+
self._forkserver_pid = None
57+
58+
os.unlink(self._forkserver_address)
59+
self._forkserver_address = None
60+
4261
def set_forkserver_preload(self, modules_names):
4362
'''Set list of module names to try to load in forkserver process.'''
4463
if not all(type(mod) is str for mod in self._preload_modules):

‎Lib/multiprocessing/util.py

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -456,3 +456,24 @@ def spawnv_passfds(path, args, passfds):
456456
finally:
457457
os.close(errpipe_read)
458458
os.close(errpipe_write)
459+
460+
461+
def _cleanup_tests():
462+
"""Cleanup multiprocessing resources when multiprocessing tests
463+
completed."""
464+
465+
from test import support
466+
467+
# cleanup multiprocessing
468+
process._cleanup()
469+
470+
# Stop the ForkServer process if it's running
471+
from multiprocessing import forkserver
472+
forkserver._forkserver._stop()
473+
474+
# bpo-37421: Explicitly call _run_finalizers() to remove immediately
475+
# temporary directories created by multiprocessing.util.get_temp_dir().
476+
_run_finalizers()
477+
support.gc_collect()
478+
479+
support.reap_children()

‎Lib/test/_test_multiprocessing.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5059,8 +5059,8 @@ def tearDownModule():
50595059
# Sleep 500 ms to give time to child processes to complete.
50605060
if need_sleep:
50615061
time.sleep(0.5)
5062-
multiprocessing.process._cleanup()
5063-
test.support.gc_collect()
5062+
5063+
multiprocessing.util._cleanup_tests()
50645064

50655065
remote_globs['setUpModule'] = setUpModule
50665066
remote_globs['tearDownModule'] = tearDownModule

‎Lib/test/test_concurrent_futures.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626
BrokenExecutor)
2727
from concurrent.futures.process import BrokenProcessPool
2828
from multiprocessing import get_context
29+
import multiprocessing.util
2930

3031

3132
def create_future(state=PENDING, exception=None, result=None):
@@ -1253,6 +1254,7 @@ def test_main():
12531254
test.support.run_unittest(__name__)
12541255
finally:
12551256
test.support.reap_children()
1257+
multiprocessing.util._cleanup_tests()
12561258

12571259
if __name__ == "__main__":
12581260
test_main()

‎Misc/NEWS.d/next/Tests/2019-07-01-19-57-26.bpo-37421.NFH1f0.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
multiprocessing tests now explicitly call ``_run_finalizers()`` to
2+
immediately remove temporary directories created by tests.

‎Misc/NEWS.d/next/Tests/2019-07-05-14-47-55.bpo-37421.n8o2to.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
multiprocessing tests now stop the ForkServer instance if it's running: close
2+
the "alive" file descriptor to ask the server to stop and then remove its UNIX
3+
address.

0 commit comments

Comments
 (0)
0