Description
Python documentation for the open() builtin indicates that open() returns types from the io.IOBase hierarchy:
The type of file object returned by the open() function depends on the mode. When open() is used to open a file in a text mode ('w', 'r', 'wt', 'rt', etc.), it returns a subclass of
io.TextIOBase(specificallyio.TextIOWrapper). When used to open a file in a binary mode with buffering, the returned class is a subclass ofio.BufferedIOBase. The exact class varies: in read binary mode, it returns anio.BufferedReader; in write binary and append binary modes, it returns anio.BufferedWriter, and in read/write mode, it returns anio.BufferedRandom. When buffering is disabled, the raw stream, a subclass ofio.RawIOBase,io.FileIO, is returned.
Python documentation for typing.IO[...] suggests that objects returned by the open() builtin are of type typing.IO[...].
class typing.IO
class typing.TextIO
class typing.BinaryIOGeneric type
IO[AnyStr]and its subclassesTextIO(IO[str])andBinaryIO(IO[bytes])represent the types of I/O streams such as returned by open().
From this, I deduce that if I type a function as accepting either typing.IO[typing.AnyStr] or io.IOBase, it should be able to accept any object returned from open(). And following from this, I would assume that it should be possible to have an object that is of both typing.IO and io.IOBase type, because if objects that are of typing.IO type could not be of typing.IOBase type, and objects of typing.IOBase type could not be of io.IOBase type, then objects returned from open() could not be in line with the documentation I quoted above.
Given this, I would expect the following code to pass type checking without error:
import io
import typing
def accept_text_io(obj: typing.TextIO) -> None:
assert isinstance(obj, io.TextIOBase)
def accept_binary_io(obj: typing.BinaryIO) -> None:
assert isinstance(obj, io.RawIOBase) or isinstance(obj, io.BufferedIOBase)However, when I run mypy on it, I get the following errors.
$ poetry run mypy --show-error-codes --show-error-context --strict --warn-unreachable --warn-unused-configs --python-version 3.7 test_iotypes_000.py
test_iotypes_000.py: note: In function "accept_text_io":
test_iotypes_000.py:6: error: Subclass of "TextIO" and "TextIOBase" cannot exist: would have incompatible method signatures [unreachable]
test_iotypes_000.py: note: In function "accept_binary_io":
test_iotypes_000.py:10: error: Subclass of "BinaryIO" and "RawIOBase" cannot exist: would have incompatible method signatures [unreachable]
test_iotypes_000.py:10: error: Subclass of "BinaryIO" and "BufferedIOBase" cannot exist: would have incompatible method signatures [unreachable]
Found 3 errors in 1 file (checked 1 source file)To further determine what is going wrong here, why something can't be of both typing.IO types and io.IOBase types, I ran the following code through mypy, which again I would expect to pass without any errors:
import io
import typing
class DummyBaseIO0(typing.IO[typing.AnyStr], io.IOBase):
pass
class DummyBaseIO1(typing.IO[typing.Any], io.IOBase):
pass
class DummyTextIO(typing.TextIO, io.TextIOBase):
pass
class DummyRawIO(typing.BinaryIO, io.RawIOBase):
pass
class DummyRawIO(typing.BinaryIO, io.BufferedIOBase):
passAnd for this code mypy reports the following errors, and specifically highlights the incompatibilities:
$ poetry run mypy --show-error-codes --show-error-context --strict --warn-unreachable --warn-unused-configs --python-version 3.7 test_iotypes_010.py
test_iotypes_010.py: note: In class "DummyBaseIO0":
test_iotypes_010.py:5: error: Definition of "readline" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:5: error: Definition of "__iter__" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:5: error: Definition of "readlines" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:5: error: Definition of "__enter__" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:5: error: Definition of "writelines" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:5: error: Definition of "__next__" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:5: error: Definition of "__iter__" in base class "Iterator" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:5: error: Definition of "__next__" in base class "Iterator" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:5: error: Definition of "__iter__" in base class "Iterable" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py: note: In class "DummyBaseIO1":
test_iotypes_010.py:9: error: Definition of "readline" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:9: error: Definition of "__enter__" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py: note: In class "DummyTextIO":
test_iotypes_010.py:13: error: Definition of "__enter__" in base class "TextIO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:13: error: Definition of "newlines" in base class "TextIO" is incompatible with definition in base class "TextIOBase" [misc]
test_iotypes_010.py:13: error: Definition of "errors" in base class "TextIO" is incompatible with definition in base class "TextIOBase" [misc]
test_iotypes_010.py:13: error: Definition of "encoding" in base class "TextIO" is incompatible with definition in base class "TextIOBase" [misc]
test_iotypes_010.py:13: error: Definition of "readline" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:13: error: Definition of "__iter__" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:13: error: Definition of "readlines" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:13: error: Definition of "__enter__" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:13: error: Definition of "writelines" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:13: error: Definition of "read" in base class "IO" is incompatible with definition in base class "TextIOBase" [misc]
test_iotypes_010.py:13: error: Definition of "__next__" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:13: error: Definition of "__iter__" in base class "Iterator" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:13: error: Definition of "__next__" in base class "Iterator" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:13: error: Definition of "__iter__" in base class "Iterable" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py: note: In class "DummyRawIO":
test_iotypes_010.py:17: error: Definition of "__enter__" in base class "BinaryIO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:17: error: Definition of "readline" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:17: error: Definition of "write" in base class "IO" is incompatible with definition in base class "RawIOBase" [misc]
test_iotypes_010.py:17: error: Definition of "__enter__" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:17: error: Definition of "writelines" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py: note: At top level:
test_iotypes_010.py:21: error: Name "DummyRawIO" already defined on line 17 [no-redef]
test_iotypes_010.py: note: In class "DummyRawIO":
test_iotypes_010.py:21: error: Definition of "__enter__" in base class "BinaryIO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:21: error: Definition of "readline" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:21: error: Definition of "write" in base class "IO" is incompatible with definition in base class "BufferedIOBase" [misc]
test_iotypes_010.py:21: error: Definition of "__enter__" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:21: error: Definition of "writelines" in base class "IO" is incompatible with definition in base class "IOBase" [misc]
test_iotypes_010.py:21: error: Definition of "read" in base class "IO" is incompatible with definition in base class "BufferedIOBase" [misc]
Found 37 errors in 1 file (checked 1 source file)Summary
So in summary, I would expect the io.IOBase and typing.IO[...] hierarchies to be compatible, and all the code in here to pass mypy validation, but it does not. I'm not sure if there is a good explanation for why the two type hierarchies are incompatible.
Practically, to work around this, I have to now type variables as typing.Enum[typing.TextIO,io.TextIOBase] if it should work with TextIO like things. This is maybe partly due to some other issues (see python/mypy#11193, #6061, #6076), but definitely these incompatibilities do make things harder, and seem wrong to me.
Versions
This issue is written for:
- Python 3.7
- mypy 0.910
- Typeshed from mypy 0.910
All code can be found here.