8000 PEP 544: Protocols by ilevkivskyi · Pull Request #224 · python/peps · GitHub
[go: up one dir, main page]
Skip to content

PEP 544: Protocols #224

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 40 commits into from
Mar 18, 2017
Merged

PEP 544: Protocols #224

Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
3fa3e48
Collect various ideas
ilevkivskyi Mar 5, 2017
51b0ca0
Some formatting and reordering
ilevkivskyi Mar 5, 2017
993f7b3
Add some examples
ilevkivskyi Mar 5, 2017
e384336
Add planned link templates
ilevkivskyi Mar 6, 2017
7ea5d41
Add links + minor changes
Mar 6, 2017
cdcf62f
Polishing rationale
Mar 6, 2017
1ffed9b
Some more reshuffling and formatting
Mar 6, 2017
72ceae6
Add more examples
Mar 6, 2017
6bea2e8
Add more examples to existing approaches
Mar 6, 2017
d5972c3
Typos, reordering, and few more details (backport)
ilevkivskyi Mar 6, 2017
57d375f
Update list of protocols in typing
ilevkivskyi Mar 6, 2017
9d4d685
Defining protocols plus minor changes and formatting
ilevkivskyi Mar 7, 2017
82258d5
Explicitly declaring implementation and other changes
ilevkivskyi Mar 7, 2017
5d9fb7c
More polishing
ilevkivskyi Mar 7, 2017
a6e6d9e
Edit rejected/postponed ideas
ilevkivskyi Mar 7, 2017
3175013
Runtime things, reorder links
ilevkivskyi Mar 7, 2017
cbff669
Runtime decorator
ilevkivskyi Mar 7, 2017
dfccd06
Backward compatible part and last bits
Mar 8, 2017
60f4d52
Some clarifications
ilevkivskyi Mar 9, 2017
60e7f7f
Add links in text
ilevkivskyi Mar 9, 2017
c90aa1c
Caption style, add cross-refs
Mar 9, 2017
b008de1
Remove redundant links; + minor changes
ilevkivskyi Mar 10, 2017
02cca5c
One more tiny change
ilevkivskyi Mar 10, 2017
7d89b6b
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Mar 10, 2017
0f3732a
Copyediting changes
JelleZijlstra Mar 10, 2017
95fbf58
Merge pull request #1 from JelleZijlstra/patch-2
ilevkivskyi Mar 10, 2017
cb65bff
Rename PEP with a valid number to get the build running
ilevkivskyi Mar 10, 2017
817bf2f
Reflow to 79 characters
ilevkivskyi Mar 10, 2017
2d89ba9
fix typo
JelleZijlstra Mar 10, 2017
0efcbff
Some grammar tweaks
brettcannon Mar 10, 2017
ebd4b17
Merge pull request #3 from brettcannon/patch-1
ilevkivskyi Mar 10, 2017
0de36be
Implement Guido's idea of EIBTI plus minor comments
ilevkivskyi Mar 11, 2017
767c58b
Fix typo
ilevkivskyi Mar 11, 2017
efc3154
Make implementation enforcement optional; fix order of Protocolbase
ilevkivskyi Mar 12, 2017
7d714c3
Add missing @abstractmethod decorators
ilevkivskyi Mar 13, 2017
d4ab050
Minor clarification
ilevkivskyi Mar 13, 2017
d9d21c2
Implement Jukka's and David's comments; few more minor things
ilevkivskyi Mar 16, 2017
4dfbfb2
Implement most comments by Łukasz; few more to do
ilevkivskyi Mar 17, 2017
d51420e
More changes in response to comments
Mar 18, 2017
f6240c8
Remove one reamining 'All'
ilevkivskyi Mar 18, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Reflow to 79 characters
  • Loading branch information
@ilevkivskyi
ilevkivskyi committed Mar 10, 2017
commit 817bf2f95d6437df5bdf1a4c76ea9d49f7e42d2c
44 changes: 23 additions & 21 deletions pep-0544.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,8 @@ approaches related to structural subtyping in Python and other languages:

As mentioned in `rationale`_, we want to avoid such necessity, especially
in static context. However, in runtime context, ABCs are good candidates for
protocol classes and they are already used extensively in the ``typing`` module.
protocol classes and they are already used extensively in
the ``typing`` module.

* Abstract classes defined in ``collections.abc`` module [collections-abc]_
are slightly more advanced since they implement a custom
Expand Down Expand Up @@ -348,15 +349,15 @@ Having an empty method body in a protocol type makes the method
implicitly abstract only statically, and there won't be any runtime checking.
The rationale for this is that most methods in a protocol will be abstract,
and having to always use ``@abstractmethod`` is a little verbose and ugly.
Leaving the method body empty is the recommended way of defining protocol members
that don't have a default implementation. Such syntax mirrors how methods are
defined in stub files. Abstract staticmethods and abstract classmethods
are equally supported.
Leaving the method body empty is the recommended way of defining protocol
members that don't have a default implementation. Such syntax mirrors how
methods are defined in stub files. Abstract staticmethods and abstract
classmethods are equally supported.

To define a protocol variable, one should use PEP 526 variable
annotations in the class body. Attributes defined in the body of a method
by assignment via ``self`` are considered non-protocol members. The rationale for
this is that the protocol class implementation is often not shared by
by assignment via ``self`` are considered non-protocol members. The rationale
for this is that the protocol class implementation is often not shared by
subtypes, so the interface should not depend on the default implementation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well in some sense that same rationale would apply to default (i.e. non-abstract) implementations.

A defensible position could be that as long as a type checker can verify that an implementation that doesn't explicitly inherit from the protocol class still defines the variable, there's no great reason to disallow variables, since they could just be interpreted as a shorthand for a setter and a getter method.

Also note that mypy (to take one example) doesn't actually check whether an attribute is always set -- it only checks that if it is used it has the right type.

All this gets pretty messy though and I am leaning towards not allowing protocol variables at all, other than read-only properties (possibly abstract). I don't recall having seen real-world examples of them (but if there are examples that might sway me).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this point needs some clarifications. First, I don't see why we need to force people to write:

class Point2D(Protocol):
    @property
    def x(self) -> int:
        ...
    @property
    def y(self) -> int:
        ...

instead of just

class Point2D(Protocol):
    x: int
    y: int

I think almost every function parameter annotated with a named tuple or a similar struct-like class could be replaced with a protocol annotation. (Also, this will be a natural counterpart of TypedDict.) Second, I don't think we need to specify how "smart" type checkers should be. Maybe we could only mention that it is reasonable to expect that a type checker will recognize

class Coordinates:
    def __init__(self, x: int, y: int) -> None:
        self.x = x
        self.y = y

as implicitly implementing Point2D.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this gets pretty messy though and I am leaning towards not allowing protocol variables at all, other than read-only properties

Disallowing protocol variables entirely significantly limits the scope of structural typing. There's many examples in the stdlib alone (like the stdout.buffer field, named tuples and enums used structurally, etc.).

Only allowing read-only properties is less limiting but still so. More importantly, it is confusing to the user, like Ivan's example shows above:

  • if we specify Point2D as a read-only property, should the type checker reject a writable implementation?
  • if not, it feels strange to use a property to define something that is just an attribute in the implementation.

I can imagine users of this rewriting their attributes to be properties just to satisfy the protocol specification.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not adamantly against protocol variables, but I think we should at least settle what to do about read-only attributes. When you use @property (not using the @f.setter syntax) or use PEP 526 syntax inside aNamedTuple you get a read-only attribute; otherwise PEP 526 syntax gets you a writable attribute. Which should be the default for Protocols, and how should you be able to declare the other kind?

Note that this is not about immutability -- that's in the programmer's head (or perhaps guided by whether __hash__ is defined). This is purely about whether attribute assignment should be allowed.

An exception from these rules is an attribute starting with double
underscores and not ending with double underscores always considered
Expand Down Expand Up @@ -422,8 +423,8 @@ can be used as regular base classes. In this case:
represent(BadColor()) # Error, 'add_int' is not implemented implicitly

* Type checkers will statically enforce that the class
actually implements the protocol correctly. In particular, type checkers will
always require members without default implementation to be defined in
actually implements the protocol correctly. In particular, type checkers
will always require members without default implementation to be defined in
subclasses that explicitly subclass the protocol::

from typing import Protocol, Tuple
Expand All @@ -447,9 +448,10 @@ can be used as regular base classes. In this case:
The general philosophy is that protocols are mostly like regular ABCs,
but a static type checker will handle them specially.
In particular, a class can explicitly inherit from multiple protocols.
In this case methods are resolved using normal MRO and the type checker verifies
that all subtyping are correct. Note that a concrete method with appropriate
type signature can override a protocol method but not vice versa. Examples::
In this case methods are resolved using normal MRO and the type checker
verifies that all subtyping are correct. Note that a concrete method with
appropriate type signature can override a protocol method but not vice versa.
Examples::

class A(Protocol):
def m(self) -> None: ...
Expand Down Expand Up @@ -559,8 +561,8 @@ relationships are subject to the following rules:

Generic protocol types follow the same rules of variance as non-protocol
types. Protocols can be used in all special type constructors provided
by the ``typing`` module and follow the corresponding subtyping rules. For example,
protocol ``PInt`` is a subtype of ``Union[PInt, int]``.
by the ``typing`` module and follow the corresponding subtyping rules.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be rephrased as something like "Protocol types can be used in all contexts where any other types can used, such as in (examples). Generic protocols follow the rules for generic abstract classes, except for using structural compatibility instead of compatibility defined by inheritance relationships."

For example, protocol ``PInt`` is a subtype of ``Union[PInt, int]``.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unclear. What is PInt?



``Union[]`` and ``All[]``
Expand Down Expand Up @@ -746,10 +748,10 @@ runtime, which is unfortunate::
print(b.x) # 1
print(B.x) # Ellipsis

For this reason we propose to make the ``Protocol`` metaclass recognize ``...``
initializers and translate them to a special property that can't be read.
The ``typing`` module changes proposed in this PEP will be also backported to
earlier versions via the backport currently available on PyPI.
For this reason we propose to make the ``Protocol`` metaclass recognize
``...`` initializers and translate them to a special property that can't
be read. The ``typing`` module changes proposed in this PEP will be also
backported to earlier versions via the backport currently available on PyPI.


Runtime Implementation of Protocol Classes
Expand Down Expand Up @@ -818,9 +820,9 @@ Introspection

The existing class introspection machinery (``dir``, ``__annotations__`` etc)
can be used with protocols. In addition, all introspection tools implemented
in the ``typing`` module will support protocols. Since all attributes need to be
defined in the class body based on this proposal, protocol classes will have
even better perspective for introspection than regular classes where
in the ``typing`` module will support protocols. Since all attributes need
to be defined in the class body based on this proposal, protocol classes will
have even better perspective for introspection than regular classes where
attributes can be defined implicitly -- protocol attributes can't be
initialized in ways that are not visible to introspection
(using ``setattr()``, assignment via ``self``, etc.). Still, some things like
Expand Down
0