Description
Bug report
Bug description:
The full name of the failed test case is:
test.test_asyncio.test_sendfile.ProactorEventLoopTests.test_sendfile_close_peer_in_the_middle_of_receiving
Test output:
======================================================================
FAIL: test_sendfile_close_peer_in_the_middle_of_receiving (test.test_asyncio.test_sendfile.ProactorEventLoopTests.test_sendfile_close_peer_in_the_middle_of_receiving)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\cpython\Lib\test\test_asyncio\test_sendfile.py", line 473, in test_sendfile_close_peer_in_the_middle_of_receiving
self.assertTrue(1024 <= self.file.tell() < len(self.DATA),
AssertionError: False is not true : 0
----------------------------------------------------------------------
Based on my comprehension, I re-wrote and simplified the test case, as shown below.
import asyncio
import logging
import socket
class MySendFileProto(asyncio.Protocol):
def __init__(self, file_size=0, loop=None):
self.transport = None
self.nbytes = 0
self.data = bytearray()
self.file_size = file_size
if loop:
self.done = loop.create_future()
def connection_made(self, transport):
self.transport = transport
def data_received(self, data):
self.transport.close()
self.nbytes += len(data)
self.data.extend(data)
print('RECV: {}, TOTAL: {}'.format(len(data), self.nbytes))
super().data_received(data)
def connection_lost(self, exc):
if not self.done.done():
self.done.set_result(self.nbytes)
def main():
logging.basicConfig(level='DEBUG')
port = 8075
size = 1024000
file_name = 'tmp.txt'
loop = asyncio.ProactorEventLoop()
srv_proto = MySendFileProto(file_size=size, loop=loop)
srv_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
srv_sock.bind(('localhost', port))
server_creation = loop.create_server(lambda: srv_proto, sock=srv_sock, ssl=None)
server = loop.run_until_complete(server_creation)
cli_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
cli_sock.connect(('localhost', port))
cli_proto = MySendFileProto(loop=loop)
client_creation = loop.create_connection(lambda: cli_proto, sock=cli_sock, ssl=None, server_hostname=None)
loop.run_until_complete(client_creation)
with open(file_name, 'wb') as file:
file.write(b'x' * size)
with open(file_name, 'rb') as file:
sendfile_future = loop.sendfile(cli_proto.transport, file)
try:
loop.run_until_complete(sendfile_future)
except ConnectionError as e:
print('ConnectionError happy: {}'.format(e))
loop.run_until_complete(srv_proto.done)
print(file.tell())
srv_proto.transport.close()
cli_proto.transport.close()
loop.run_until_complete(srv_proto.done)
loop.run_until_complete(cli_proto.done)
server.close()
if __name__ == '__main__':
main()Output:
- NTFS +
loop = asyncio.SelectorEventLoop()
DEBUG:asyncio:Using selector: SelectSelector
RECV: 16384, TOTAL: 16384
ConnectionError happy: Connection closed by peer
49152
- ReFS +
loop = asyncio.SelectorEventLoop()
DEBUG:asyncio:Using selector: SelectSelector
RECV: 16384, TOTAL: 16384
ConnectionError happy: Connection closed by peer
49152
- NTFS +
loop = asyncio.ProactorEventLoop()
DEBUG:asyncio:Using proactor: IocpProactor
RECV: 32768, TOTAL: 32768
RECV: 65536, TOTAL: 98304
ConnectionError happy: [WinError 64] 指定的网络名不再可用。
32768
ERROR:asyncio:Task was destroyed but it is pending!
task: <Task pending name='Task-4' coro=<IocpProactor.accept.<locals>.accept_coro() running at C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.240.0_x64__qbz5n2kfra8p0\Lib\asyncio\windows_events.py:561> wait_for=<_OverlappedFuture cancelled>>
- ReFS +
loop = asyncio.ProactorEventLoop()
DEBUG:asyncio:Using proactor: IocpProactor
RECV: 32768, TOTAL: 32768
RECV: 65536, TOTAL: 98304
ConnectionError happy: [WinError 64] 指定的网络名不再可用。
0
ERROR:asyncio:Task was destroyed but it is pending!
task: <Task pending name='Task-4' coro=<IocpProactor.accept.<locals>.accept_coro() running at C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.240.0_x64__qbz5n2kfra8p0\Lib\asyncio\windows_events.py:561> wait_for=<_OverlappedFuture cancelled>>
Because file.tell() is 0, the test case fails. However, even though the test case passes on NTFS, it does not mean correctness.
I guess that the main purpose of test_sendfile.py:473 is to check the number of bytes sent by the client. When SelectorEventLoop is used on Windows, Python will firstly try sendfile syscall, which will fail, and then fallback to _sendfile_fallback, which uses POSIX-read function in a loop. After each read, the file offset is sure to increase.
When ProactorEventLoop is used on Windows, Python will call transmitFile function. However, this function does not tell us how it will modify the file offset. In fact, on NTFS, file.tell() is 32768, but at least 98304 bytes have been sent and received.
In conclusion, the statement on test_sendfile.py:473 does not work on Windows. Either another method needs to be found to check the number of bytes sent, or the line needs to be removed (test_sendfile.py:471 may be enough, I think).
Windows Version: 22631.2500 with Windows Feature Experience Pack 1000.22677.1000.0
CPython versions tested on:
3.12, CPython main branch
Operating systems tested on:
Windows
Linked PRs
Metadata
Metadata
Assignees
Projects
Status