-
-
Notifications
You must be signed in to change notification settings - Fork 32k
gh-119506: fix _io.TextIOWrapper.write()
write during flush
#119507
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 8 commits
5b924dc
f8fd659
4809c1a
e551b86
2e72b28
43628ce
a3fe68e
c246d7f
156940d
2392d20
02cf53a
483975c
01327f7
26eadbe
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Fix bug in :meth:`!io.TextIOWrapper.write` method, which causes wrapper's internal buffer to be rewritten during flushing of large data. |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1719,21 +1719,31 @@ _io_TextIOWrapper_write_impl(textio *self, PyObject *text) | |
bytes_len = PyBytes_GET_SIZE(b); | ||
} | ||
|
||
if (self->pending_bytes == NULL) { | ||
self->pending_bytes_count = 0; | ||
self->pending_bytes = b; | ||
} | ||
else if (self->pending_bytes_count + bytes_len > self->chunk_size) { | ||
PyObject *pending = self->pending_bytes; | ||
|
||
if (self->pending_bytes_count + bytes_len > self->chunk_size) { | ||
// Prevent to concatenate more than chunk_size data. | ||
if (_textiowrapper_writeflush(self) < 0) { | ||
Py_DECREF(b); | ||
return NULL; | ||
} | ||
if (self->pending_bytes) { | ||
// gh-119506: call to _textiowrapper_writeflush() | ||
// can write new data to pending_bytes | ||
pending = self->pending_bytes; | ||
self->pending_bytes = b; | ||
b = pending; | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why do you swap data here?
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you look at the test case, where flushing the stream triggers a write, it's not a threading thing, it's because of write being called recursively. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Even single thread case, it is not possble to guarantee "first called, first written." |
||
} | ||
|
||
if (self->pending_bytes == NULL) { | ||
self->pending_bytes_count = 0; | ||
self->pending_bytes = b; | ||
} | ||
else if (!PyList_CheckExact(self->pending_bytes)) { | ||
PyObject *list = PyList_New(2); | ||
if (list == NULL) { | ||
self->pending_bytes = pending; | ||
Py_DECREF(b); | ||
return NULL; | ||
} | ||
|
@@ -1743,6 +1753,7 @@ _io_TextIOWrapper_write_impl(textio *self, PyObject *text) | |
} | ||
else { | ||
if (PyList_Append(self->pending_bytes, b) < 0) { | ||
self->pending_bytes = pending; | ||
Py_DECREF(b); | ||
return NULL; | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about
while
instead ofif
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That would also need a check to make sure that
bytes_len > self->chunk_size
isn't true.