Initial round of revisions

python · JelleZijlstra · Mar 14, 2023 · Nov 7, 2022 · Jan 12, 2023 · Jan 12, 2023
commit 6a7a3bd94f682585a2f884aa25c16858a1934c4d
diff --git a/pep-9999.rst b/pep-9999.rst
@@ -1,7 +1,6 @@
 PEP: 9999
 Title: TypedMapping: Type Hints for Mappings with a Fixed Set of Keys
-Author: Alice Purcell <alicederyn@gmail.com>,
-        Daniel Moisset <dfmoisset@gmail.com>
+Author: Alice Purcell <alicederyn@gmail.com>
 Sponsor: Pablo Galindo <pablogsal@gmail.com>
 Discussions-To: https://mail.python.org/archives/list/python-dev@python.org/thread/2P26R4VH2ZCNNNOQCBZWEM4RNF35OXOW/
 Status: Draft
@@ -18,9 +17,9 @@ Post-History: `30-Sep-2022 <https://mail.python.org/archives/list/typing-sig@pyt
 Abstract
 ========
 
-:pep:`589` defines the structural type ``TypedDict``, for dictionaries with a fixed set of keys.
-As it defines a mutable type, it is difficult to correctly type methods which accept readonly parameters, without inadvertently preventing valid inputs.
-This pep proposes a type constructor ``typing.TypedMapping`` to support this use case.
+:pep:`589` defines the structural type ``TypedDict`` for dictionaries with a fixed set of keys.
+As it defines a mutable type, it is difficult to correctly annotate methods which accept read-only parameters in a way that doesn't prevent valid inputs.
+This PEP proposes a type constructor ``typing.TypedMapping`` to support this use case.
 
 For example, given the following definition::
 
@@ -36,7 +35,7 @@ For example, given the following definition::
         else:
             return f'{movie["name"]} ({movie["year"]})'
 
-A type checker should accept this code::
+A type checker should then consider this code type safe::
 
     from typing import TypedDict
 
@@ -48,32 +47,40 @@ A type checker should accept this code::
                           'year': 1982}
     print(movie_string(movie))
 
+Note that currently the code above will fail to type check if we define ``Movie`` based on ``TypedDict``, and there is no replacement.
 
 Motivation
 ==========
 
-Representing an object or structured data using (potentially nested) dictionaries with string keys is a common pattern in Python programs. :pep:`589` allows these to be type checked when the exact type is known up-front, but it is hard to write generic readonly code; for instance, where fields may be optional, or take a range of possible types. This is an especially common issue when writing APIs for services, which may support a wide range of input structures, and will not need to modify its input.
+Representing structured data using (potentially nested) dictionaries with string keys is a common pattern in Python programs. :pep:`589` allows these values to be type checked when the exact type is known up-front, but it is hard to write read-only code code that accepts more specific variants: for instance, where fields may be subtypes or restrict a union of possible types. This is an especially common issue when writing APIs for services, which may support a wide range of input structures, and typically do not need to modify their input.
 
-Take for example the function ``movie_string`` defined above. Suppose its its type signature is defined using a ``TypedDict``::
+Take for example the function ``movie_string`` defined above. Suppose its type signature is defined using a ``TypedDict``::
 
     from typing import NotRequired, TypedDict
 
     class Movie(TypedDict):
         name: str
         year: NotRequired[int | None]
 
+    class MovieRecord(TypedDict):
+        name: str
+        year: int
+
     def movie_string(movie: Movie) -> str:
         if movie.get("year") is None:
             return movie["name"]
         else:
             return f'{movie["name"]} ({movie["year"]})'
 
-Attempting to pass a ``MovieRecord`` into this function gives (in mypy) the error::
+Attempting to pass a ``MovieRecord`` into ``movie_string`` results (using mypy) in the error::
 
     Argument 1 to "movie_string" has incompatible type "MovieRecord"; expected "Movie"
 
-This incompatibility is due to the unused mutate methods generated on ``Movie``.
-This could be avoided by using a :pep:`544` protocol::
+This particular use case should be type-safe, but the type checker correctly stops the
+user from passing a ``MovieRecord`` into a ``Movie`` argument in the general case because
+the ``Movie`` class has mutator methods that could potentially allow the function to break
+the type constraints in ``MovieRecord`` by setting ``movie["year"] = None`` or ``del movie["year"]``.
+The problem disappears if we don't have mutator methods in ``Movie``. This could be achieved by defining an immutable interface
10000
 using a :pep:`544` Protocol::
 
     from typing import Literal, Protocol, overload
 
@@ -90,15 +97,19 @@ This could be avoided by using a :pep:`544` protocol::
         @overload
         def __getitem__(self, key: Literal["year"]) -> int | None: ...
 
-This is very repetitive, easy to get wrong, and is still missing important definitions like ``in`` and ``keys()``.
+This is very repetitive, easy to get wrong, and is still missing important method definitions like ``__contains__()`` and ``keys()``.
+
+The proposed ``TypedMapping`` type allows a straightforward way of defining these types that should be familiar to existing users of ``TypedDict`` and support the cases exemplified above.
+In addition to those benefits, by flagging arguments of a function as ``TypedMapping``, it makes explicit not just to typecheckers but also to users that the function is not going to modify its inputs, which is usually a desirable property of a function interface.
+Finally, this allows bringing the benefits of ``TypedDict`` to other mapping types that are unrelated to ``dict``.
 
 Specification
 =============
 
-The ``TypedMapping`` type is a protocol behaving identically to ``TypedDict``, except:
+The ``TypedMapping`` type is a protocol behaving almost identically to ``TypedDict``, except:
 
-1. instances need not be subclasses of dict
-2. no mutate methods will be generated
+1. The runtime type of a TypedMapping object is not constrained to be a ``dict``
+2. no mutator methods (``__setitem__``, ``__delitem__``, ``update``, etc.) will be generated
 3. subclasses can narrow field types, in the same manner as other protocols
 
 All current and future features of TypedDict are applicable to TypedMapping, including class-based and alternative syntax, totality, and ``Required`` and ``NotRequired`` from :pep:`655`.
@@ -113,7 +124,7 @@ A type that inherits from a TypedMapping subclass and from TypedDict (either dir
 
 4. is the structural intersection of its parents, or invalid if no such intersection exists
 5. instances must be a dict subclass
-6. adds mutate methods only for fields it explicitly (re)declares
+6. adds mutator methods only for fields it explicitly (re)declares
 
 For example::
 
@@ -127,16 +138,17 @@ For example::
     movie: MovieRecord = { "name": "Blade Runner",
                            "year": 1982 }
 
-    movie["year"] = 1985  # Fine, mutate methods added in definition
-    movie["name"] = "Terminator"  # Error, name mutator not declared
+    movie["year"] = 1985  # Fine, mutator methods added in definition
+    movie["name"] = "Terminator"  # Type check error, "name" mutator not declared
 
 
-Type Consistency
-----------------
+Type Consistency Rules
+----------------------
 
 Informally speaking, *type consistency* is a generalization of the is-subtype-of relation to support the ``Any`` type. It is defined more formally in :pep:`483`. This section introduces the new, non-trivial rules needed to support type consistency for TypedMapping types.
 
-First, any TypedMapping type is consistent with ``Mapping[str, object]``. Second, a TypedMapping or TypedDict type ``A`` is consistent with TypedMapping ``B`` if ``A`` is structurally compatibly with ``B``. This is true if and only if both of these conditions are satisfied:
+First, any TypedMapping type is consistent with ``Mapping[str, object]``.
+Second, a TypedMapping or TypedDict type ``A`` is consistent with TypedMapping ``B`` if ``A`` is structurally compatible with ``B``. This is true if and only if both of these conditions are satisfied:
 
 * For each key in ``A``, ``B`` has the corresponding key and the corresponding value type in ``B`` is consistent with the value type in ``A``.
 
@@ -147,7 +159,7 @@ Discussion:
 * Value types behave covariantly, since TypedMapping objects have no mutator methods. This is similar to container types such as ``Mapping``, and different from relationships between two TypedDicts. Example::
 
     class A(TypedMapping):
-        x: Optional[int]
+        x: int | None
 
     class B(TypedDict):
         x: int
@@ -207,7 +219,7 @@ Discussion:
         return sum(m.values())
 
     def f(a: A) -> None:
-        sum_values(a)  # Error: 'A' incompatible with Mapping[str, int]
+        sum_values(a)  # Type check error: 'A' incompatible with Mapping[str, int]
 
     b: B = {'x': 0, 'y': 'foo'}
     f(b)  # Runtime error: int + str
@@ -219,6 +231,6 @@ Rejected Alternatives
 Several variations were considered and discarded:
 
 * A ``readonly`` parameter to ``TypedDict``, behaving much like TypedMapping but with the additional constraint that instances must be dictionaries at runtime. This was discarded as less flexible due to the extra constraint; additionally, the new type nicely mirrors the existing ``Mapping``/``Dict`` types.
-* Inheriting from a ``TypedMapping`` subclass and ``TypedDict`` resulting in mutate methods being added for all fields, not just those actively (re)declared in the class body. Discarded as less flexible, and not matching how inheritance works in other cases for TypedDict (e.g. total=False and total=True do not affect fields not specified in the class body).
-* A generic type that removes mutate methods from its parameter, e.g. ``Readonly[MovieRecord]``. This would naturally want to be defined for a wider set of types than just ``TypedDict`` subclasses, and also raises questions about whether and how it applies to nested types. We decided to keep 
4271
the scope of this PEP narrower.
+* Inheriting from a ``TypedMapping`` subclass and ``TypedDict`` resulting in mutator methods being added for all fields, not just those actively (re)declared in the class body. Discarded as less flexible, and not matching how inheritance works in other cases for TypedDict (e.g. total=False and total=True do not affect fields not specified in the class body).
+* A generic type that removes mutator methods from its parameter, e.g. ``Readonly[MovieRecord]``. This would naturally want to be defined for a wider set of types than just ``TypedDict`` subclasses, and also raises questions about whether and how it applies to nested types. We decided to keep the scope of this PEP narrower.