What are typing.IO, TextIO, and BinaryIO good for?
#829
Replies: 3 comments 2 replies
-
|
My take: import io
class MyFileWrapper(io.StringIO):
...
def my_opener() -> MyFileWrapper: # not typing.IO[str] or typing.TextIO
return MyFileWrapper()Until the use of protocols for I/O arguments is widespread, these concrete classes often still need to derive from When returning an object returned by For argument types, protocols should be used. These should be fairly tight, according to the actual needs of the function. They can either use predefined protocols, custom protocols, or an intersection of those: import typing
if typing.TYPE_CHECKING:
from _typeshed import SupportsRead
class NamedReader(SupportsRead[str]):
@property
def filename(self) -> str: ...
def read_and_print(stream: NamedReader) -> None:
print(f"Reading {stream.filename}")
print(stream.read())Addendum (after reading Jelle's post): When defining protocols or base classes, it often makes sense to use a protocol for the return type. This documents what callers can expect the actual implementation to return, but doesn't constrain sub-classes to use a specific implementation. Historically, typing's I/O types have proven to be quite problematic. The difference between |
Beta Was this translation helpful? Give feedback.
-
|
These classes were meant to model the concept of a "file object", but when you start looking at how "file objects" are really used across the Python library system, you quickly realize that there's a ton of variation in what exactly you can do with them. There's certainly a Text and binary IO
But we also have None of these classes are currently Protocols in typing. I think this is because they are so big that it is hardly practical to fully implement them without inheritance. typeshed conventionsIn typeshed, where we provide type stubs for the standard library and for many third-party packages, our policy is now to avoid the For example, if an API calls for a "file object" in an argument but actually just calls For return types, we try to use concrete types that match exactly what gets returned at runtime. For example, we created a complicated set of overloads for builtins.open that enables type checkers to pick the exact return type 8000 that will be used at runtime. This makes sure type checkers get to work with the most precise types and avoids incorrect errors when users perform some operation that works only with the concrete types. I would recommend also using typeshed's conventions if you are providing types or stubs for a library. However, it may make sense to also use Protocol return types if you want to have more freedom in how to change your return types in the future. Application conventionsIf you are writing application code, typeshed's conventions can also make sense, but I think it is more defensible to use |
Beta Was this translation helpful? Give feedback.
-
This outlines another problem with the I/O classes: They are all both read and writer classes, but clients are usually only interested in either reading or writing. Providing a dummy |
Beta Was this translation helpful? Give feedback.
-
|
I agree that protocols are the way to go. I think this was an area of PEP 484 where we knew something was needed but we didn't quite know what to do and we just modeled what the runtime had. A big blocker was the lack of protocols at the time. There's one niggling issue with using the minimal protocol: it constrains how a library may evolve. For example, suppose version 1.0 of a library has a function that reads a file using IOW, specifying one of the IO classes constrains the callers but allows the library to evolve, while specifying a minimal protocol gives callers maximal freedom while constraining the future evolution of the library. It's not always clear from reading the docs whether the library intends to evolve or not. Of course, duck typing means that users write code that works with the current version rather than code that is future-proof anyway, so my concern is mostly theoretical (hence my agreement in the first paragraph). |
Beta Was this translation helpful? Give feedback.
-
I think that's one symptom of a larger issue: Should type annotations and stubs represent (documented) API or implementation? At the moment, at least in typeshed, we are leaning heavily on the "implementation" side. This is also the reason why we include undocumented or "private" (underscored) members on request. Personally, I wouldn't mind going a bit further in the "documented API" direction, and for non-typeshed projects my recommendation is to include methods you might reasonable start to use during the current major version of the library in the protocol. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
When should I use
typing.IO,typing.TextIO, andtyping.BinaryIO? Should I use Protocols instead? What is the difference betweenIO[str]andTextIO?I'm starting this topic to get the ball rolling on this new forum and because I think it will give some illuminating insights about the type system. I'll write up some of my own thoughts below and I'd be happy to see insights from other people involved in typing.
Beta Was this translation helpful? Give feedback.
All reactions