8000 Restify PEP 3145 (#143) · python/peps@c6fbead · GitHub
[go: up one dir, main page]
Skip to content

Commit c6fbead

Browse files
Mariattaberkerpeksag
Mariatta
authored and
berkerpeksag
committed
Restify PEP 3145 (#143)
1 parent 84de20b commit c6fbead

File tree

1 file changed

+114
-100
lines changed

1 file changed

+114
-100
lines changed

‎pep-3145.txt

Lines changed: 114 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -5,136 +5,150 @@ Last-Modified: $Date$
55
Author: (James) Eric Pruitt, Charles R. McCreary, Josiah Carlson
66
Status: Withdrawn
77
Type: Standards Track
8-
Content-Type: text/plain
8+
Content-Type: text/x-rst
99
Created: 04-Aug-2009
1010
Python-Version: 3.2
1111
Post-History:
1212

13-
Abstract:
1413

15-
In its present form, the subprocess.Popen implementation is prone to
16-
dead-locking and blocking of the parent Python script while waiting on data
17-
from the child process. This PEP proposes to make
18-
subprocess.Popen more asynchronous to help alleviate these
19-
problems.
14+
Abstract
15+
========
2016

17+
In its present form, the subprocess.Popen implementation is prone to
18+
dead-locking and blocking of the parent Python script while waiting on data
19+
from the child process. This PEP proposes to make
20+
subprocess.Popen more asynchronous to help alleviate these
21+
problems.
2122

22-
PEP Deferral:
2323

24-
Further exploration of the concepts covered in this PEP has been deferred
25-
at least until after PEP 3156 has been resolved.
24+
PEP Deferral
25+
============
2626

27+
Further exploration of the concepts covered in this PEP has been deferred
28+
at least until after PEP 3156 has been resolved.
2729

28-
PEP Withdrawal:
2930

30-
This can be dealt with in the bug tracker.
31-
A specific proposal is attached to http://bugs.python.org/issue18823
31+
PEP Withdrawal
32+
==============
3233

34+
This can be dealt with in the bug tracker. A specific proposal is
35+
attached to [11]_.
3336

34-
Motivation:
3537

36-
A search for "python asynchronous subprocess" will turn up numerous
37-
accounts of people wanting to execute a child process and communicate with
38-
it from time to time reading only the data that is available instead of
39-
blocking to wait for the program to produce data [1] [2] [3]. The current
40-
behavior of the subprocess module is that when a user sends or receives
41-
data via the stdin, stderr and stdout file objects, dead locks are common
42-
and documented [4] [5]. While communicate can be used to alleviate some of
43-
the buffering issues, it will still cause the parent process to block while
44-
attempting to read data when none is available to be read from the child
45-
process.
38+
Motivation
39+
==========
4640

47-
Rationale:
41+
A search for "python asynchronous subprocess" will turn up numerous
42+
accounts of people wanting to execute a child process and communicate with
43+
it from time to time reading only the data that is available instead of
44+
blocking to wait for the program to produce data [1]_ [2]_ [3]_. The current
45+
behavior of the subprocess module is that when a user sends or receives
46+
data via the stdin, stderr and stdout file objects, dead locks are common
47+
and documented [4]_ [5]_. While communicate can be used to alleviate some of
48+
the buffering issues, it will still cause the parent process to block while
49+
attempting to read data when none is available to be read from the child
50+
process.
4851

49-
There is a documented need for asynchronous, non-blocking functionality in
50-
subprocess.Popen [6] [7] [2] [3]. Inclusion of the code would improve the
51-
utility of the Python standard library that can be used on Unix based and
52-
Windows builds of Python. Practically every I/O object in Python has a
53-
file-like wrapper of some sort. Sockets already act as such and for
54-
strings there is StringIO. Popen can be made to act like a file by simply
55-
using the methods attached to the subprocess.Popen.stderr, stdout and
56-
stdin file-like objects. But when using the read and write methods of
57-
those options, you do not have the benefit of asynchronous I/O. In the
58-
proposed solution the wrapper wraps the asynchronous methods to mimic a
59-
file object.
6052

61-
Reference Implementation:
53+
Rationale
54+
=========
6255

63-
I have been maintaining a Google Code repository that contains all of my
64-
changes including tests and documentation [9] as well as blog detailing
65-
the problems I have come across in the development process [10].
56+
There is a documented need for asynchronous, non-blocking functionality in
57+
subprocess.Popen [6]_ [7]_ [2]_ [3]_. Inclusion of the code would improve the
58+
utility of the Python standard library that can be used on Unix based and
59+
Windows builds of Python. Practically every I/O object in Python has a
60+
file-like wrapper of some sort. Sockets already act as such and for
61+
strings there is StringIO. Popen can be made to act like a file by simply
62+
using the methods attached to the subprocess.Popen.stderr, stdout and
63+
stdin file-like objects. But when using the read and write methods of
64+
those options, you do not have the benefit of asynchronous I/O. In the
65+
proposed solution the wrapper wraps the asynchronous methods to mimic a
66+
file object.
6667

67-
I have been working on implementing non-blocking asynchronous I/O in the
68-
subprocess.Popen module as well as a wrapper class for subprocess.Popen
69-
that makes it so that an executed process can take the place of a file by
70-
duplicating all of the methods and attributes that file objects have.
7168

72-
There are two base functions that have been added to the subprocess.Popen
73-
class: Popen.send and Popen._recv, each with two separate implementations,
74-
one for Windows and one for Unix-based systems. The Windows
75-
implementation uses ctypes to access the functions needed to control pipes
76-
in the kernel 32 DLL in an asynchronous manner. On Unix based systems,
77-
the Python interface for file control serves the same purpose. The
78-
different implementations of Popen.send and Popen._recv have identical
79-
arguments to make code that uses these functions work across multiple
80-
platforms.
69+
Reference Implementation
70+
========================
8171

82-
When calling the Popen._recv function, it requires the pipe name be
83-
passed as an argument so there exists the Popen.recv function that passes
84-
selects stdout as the pipe for Popen._recv by default. Popen.recv_err
85-
selects stderr as the pipe by default. Popen.recv and Popen.recv_err
86-
are much easier to read and understand than Popen._recv('stdout' ...) and
87-
Popen._recv('stderr' ...) respectively.
72+
I have been maintaining a Google Code repository that contains all of my
73+
changes including tests and documentation [9]_ as well as blog detailing
74+
the problems I have come across in the development process [10]_.
8875

89-
Since the Popen._recv function does not wait on data to be produced
90-
before returning a value, it may return empty bytes. Popen.asyncread
91-
handles this issue by returning all data read over a given time
92-
interval.
76+
I have been working on implementing non-blocking asynchronous I/O in the
77+
subprocess.Popen module as well as a wrapper class for subprocess.Popen
78+
that makes it so that an executed process can take the place of a file by
79+
duplicating all of the methods and attributes that file objects have.
9380

94-
The ProcessIOWrapper class uses the asyncread and asyncwrite functions to
95-
allow a process to act like a file so that there are no blocking issues
96-
that can arise from using the stdout and stdin file objects produced from
97-
a subprocess.Popen call.
98-
81+
There are two base functions that have been added to the subprocess.Popen
82+
class: Popen.send and Popen._recv, each with two separate implementations,
83+
one for Windows and one for Unix-based systems. The Windows
84+
implementation uses ctypes to access the functions needed to control pipes
85+
in the kernel 32 DLL in an asynchronous manner. On Unix based systems,
86+
the Python interface for file control serves the same purpose. The
87+
different implementations of Popen.send and Popen._recv have identical
88+
arguments to make code that uses these functions work across multiple
89+
platforms.
9990

100-
References:
91+
When calling the Popen._recv function, it requires the pipe name be
92+
passed as an argument so there exists the Popen.recv function that passes
93+
selects stdout as the pipe for Popen._recv by default. Popen.recv_err
94+
selects stderr as the pipe by default. Popen.recv and Popen.recv_err
95+
are much easier to read and understand than Popen._recv('stdout' ...) and
96+
Popen._recv('stderr' ...) respectively.
10197

102-
[1] [ python-Feature Requests-1191964 ] asynchronous Subprocess
103-
http://mail.python.org/pipermail/python-bugs-list/2006-December/
104-
036524.html
98+
Since the Popen._recv function does not wait on data to be produced
99+
before returning a value, it may return empty bytes. Popen.asyncread
100+
handles this issue by returning all data read over a given time
101+
interval.
105102

106-
[2] Daily Life in an Ivory Basement : /feb-07/problems-with-subprocess
107-
http://ivory.idyll.org/blog/feb-07/problems-with-subprocess
103+
The ``ProcessIOWrapper`` class uses the ``asyncread`` and ``asyncwrite`` functions to
104+
allow a process to act like a file so that there are no blocking issues
105+
that can arise from using the stdout and stdin file objects produced from
106+
a subprocess.Popen call.
108107

109-
[3] How can I run an external command asynchronously from Python? - Stack
110-
Overflow
111-
http://stackoverflow.com/questions/636561/how-can-i-run-an-external-
112-
command-asynchronously-from-python
113108

114-
[4] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation
115-
http://docs.python.org/library/subprocess.html#subprocess.Popen.wait
109+
References
110+
==========
116111

117-
[5] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation
118-
http://docs.python.org/library/subprocess.html#subprocess.Popen.kill
112+
.. [1] [ python-Feature Requests-1191964 ] asynchronous Subprocess
113+
http://mail.python.org/pipermail/python-bugs-list/2006-December/
114+
036524.html
119115

120-
[6] Issue 1191964: asynchronous Subprocess - Python tracker
121-
http://bugs.python.org/issue1191964
116+
.. [2] Daily Life in an Ivory Basement : /feb-07/problems-with-subprocess
117+
http://ivory.idyll.org/blog/feb-07/problems-with-subprocess
122118

123-
[7] Module to allow Asynchronous subprocess use on Windows and Posix
124-
platforms - ActiveState Code
125-
http://code.activestate.com/recipes/440554/
119+
.. [3] How can I run an external command asynchronously from Python? - Stack
120+
Overflow
121+
http://stackoverflow.com/questions/636561/how-can-i-run-an-external-
122+
command-asynchronously-from-python
126123

127-
[8] subprocess.rst - subprocdev - Project Hosting on Google Code
128-
http://code.google.com/p/subprocdev/source/browse/doc/subprocess.rst?spec=svn2c925e935cad0166d5da85e37c742d8e7f609de5&r=2c925e935cad0166d5da85e37c742d8e7f609de5#437
129-
130-
[9] subprocdev - Project Hosting on Google Code
131-
http://code.google.com/p/subprocdev
132-
133-
[10] Python Subprocess Dev
134-
http://subdev.blogspot.com/
135-
136-
Copyright:
137-
138-
This P.E.P. is licensed under the Open Publication License;
139-
http://www.opencontent.org/openpub/.
124+
.. [4] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation
125+
http://docs.python.org/library/subprocess.html#subprocess.Popen.wait
126+
127+
.. [5] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation
128+
http://docs.python.org/library/subprocess.html#subprocess.Popen.kill
129+
130+
.. [6] Issue 1191964: asynchronous Subprocess - Python tracker
131+
http://bugs.python.org/issue1191964
132+
133+
.. [7] Module to allow Asynchronous subprocess use on Windows and Posix
134+
platforms - ActiveState Code
135+
http://code.activestate.com/recipes/440554/
136+
137+
.. [8] subprocess.rst - subprocdev - Project Hosting on Google Code
138+
http://code.google.com/p/subprocdev/source/browse/doc/subprocess.rst?spec=svn2c925e935cad0166d5da85e37c742d8e7f609de5&r=2c925e935cad0166d5da85e37c742d8e7f609de5#437
139+
140+
.. [9] subprocdev - Project Hosting on Google Code
141+
http://code.google.com/p/subprocdev
142+
143+
.. [10] Python Subprocess Dev
144+
http://subdev.blogspot.com/
145+
146+
.. [11] https://bugs.python.org/issue18823 -- Idle: use pipes instead of
147+
sockets to talk with user subprocess
148+
149+
Copyright
150+
=========
151+
152+
This P.E.P. is licensed under the Open Publication License;
153+
http://www.opencontent.org/openpub/.
140154

0 commit comments

Comments
 (0)
0