python · gvanrossum · Aug 23, 2022 · Jul 24, 2022 · Jul 25, 2022 · Jul 25, 2022
@@ -2,13 +2,15 @@ PEP: 692
 Title: Using TypedDict for more precise \*\*kwargs typing
 Author: Franek Magiera <framagie@gmail.com>
 Sponsor: Jelle Zijlstra <jelle.zijlstra@gmail.com>
-Discussions-To: https://mail.python.org/archives/list/typing-sig@python.org/thread/U42MJE6QZYWPVIFHJIGIT7OE52ZGIQV3/
+Discussions-To: https://discuss.python.org/t/pep-692-using-typeddict-for-more-precise-kwargs-typing/17314
 Status: Draft
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 29-May-2022
 Python-Version: 3.12
 Post-History: `29-May-2022 <https://mail.python.org/archives/list/typing-sig@python.org/thread/U42MJE6QZYWPVIFHJIGIT7OE52ZGIQV3/>`__,
+              `12-Jul-2022 <https://mail.python.org/archives/list/python-dev@python.org/thread/PLCNW2XR4OOKAKHEZQM7R2AYVYUXPZGW/>`__,
+              `12-Jul-2022 <https://discuss.python.org/t/pep-692-using-typeddict-for-more-precise-kwargs-typing/17314>`__,
 
 
 Abstract
@@ -20,7 +22,9 @@ be very limiting. Therefore, in this PEP we propose a new way to enable more
 precise ``**kwargs`` typing. The new approach revolves around using
 ``TypedDict`` to type ``**kwargs`` that comprise keyword arguments of different
 types. It also involves introducing a grammar change and a new dunder
-``__unpack__``.
+``__typing_kwargs_unpack__``.
+
+.. _pep-692-motivation:
 
 Motivation
 ==========
@@ -39,9 +43,32 @@ type annotating ``**kwargs`` is not possible. This is especially a problem for
 already existing codebases where the need of refactoring the code in order to
 introduce proper type annotations may be considered not worth the effort. This
 in turn prevents the project from getting all of the benefits that type hinting
-can provide. As a consequence, there has been a `lot of discussion <mypyIssue4441_>`__
-around supporting more precise ``**kwargs`` typing and it became a
-feature that would be valuable for a large part of the Python community.
+can provide.
+
+Moreover, ``**kwargs`` can be used to reduce the amount of code needed in
+cases when there is a top-level function that is a part of a public API and it
+calls a bunch of helper functions, all of which expect the same keyword
+arguments. Unfortunately, if those helper functions were to use ``**kwargs``,
+there is no way to properly type hint them if the keyword arguments they expect
+are of different types. In addition, even if the keyword arguments are of the
+same type, there is no way to check whether the function is being called with
+keyword names that it actually expects.
+
+As described in the :ref:`Intended Usage <pep-692-intended-usage>` section,
+using ``**kwargs`` is not always the best tool for the job. Despite that, it is
+still a widely used pattern. As a consequence, there has been a lot of
+discussion around supporting more precise ``**kwargs`` typing and it became a
+feature that would be valuable for a large part of the Python community. This
+is best illustrated by the `mypy GitHub issue 4441 <mypyIssue4441_>`__ which
+contains a lot of real world cases that could benefit from this propsal.
+
+One more use case worth mentioning for which ``**kwargs`` are also convenient,
+is when a function should accommodate optional keyword-only arguments that
+don't have default values. A need for a pattern like that can arise when values
+that are usually used as defaults to indicate no user input, such as ``None``,
+can be passed in by a user and should result in a valid, non-default behavior.
+A part of a popular httpx library can serve as an `example <httpxIssue1384_>`__
+where an issue like this may come up.
 
 Rationale
 =========
@@ -343,28 +370,19 @@ to perform the function call::
         name = kwargs["name"]
         takes_name(name)
 
+.. _pep-692-intended-usage:
+
 Intended Usage
 --------------
-
-This proposal will bring a large benefit to the codebases that already use
-``**kwargs`` because of the flexibility that they provided in the initial
-phases of the development, but now are mature enough to use a stricter
-contract via type hints. 
-
-Adding type hints directly in the source code as opposed to the ``*.pyi``
-stubs benefits anyone who reads the code as it is easier to understand. Given
-that currently precise ``**kwargs`` type hinting is impossible in that case the
-choices are to either not type hint ``**kwargs`` at all, which isn't ideal, or
-to refactor the function to use explicit keyword arguments, which often exceeds
-the scope of time and effort allocated to adding type hinting and, as any code
-change, introduces risk for both project maintainers and users. In that case
-hinting ``**kwargs`` using a ``TypedDict`` as described in this PEP will not
-require refactoring and function body and function invocations could be
-appropriately type checked.
-
-Another useful pattern that justifies using and typing ``**kwargs`` as proposed
-is when the function's API should allow for optional keyword arguments that
-don't have default values.
+The intended use cases for this proposal are described in the :ref:`Motivation
+<pep-692-motivation>` section. In summary, more precise ``**kwargs`` typing
+can bring benefits to already existing codebases that used ``**kwargs``
+initially, but now are mature enough to use a stricter contract via type hints.
+Using ``**kwargs`` can also help in reducing code duplication and the amount of
+copy-pasting needed when there is a bunch of functions that require the same
+set of keyword arguments. Finally, ``**kwargs`` are useful for cases when a
+function needs to facilitate optional keyword arguments that don't have obvious
+default values.
 
 However, it has to be pointed out that in some cases there are better tools
 for the job than using ``TypedDict`` to type ``**kwargs`` as proposed in this
@@ -397,6 +415,9 @@ explicitly as::
 
     def foo(name: str, year: int): ...
 
+Also, for the benefit of IDEs and documentation pages, functions that are part
+of the public API should prefer explicit keyword arguments whenever possible.
+
 Grammar Changes
 ===============
 
@@ -420,7 +441,7 @@ After:
         | '**' param_no_default
 
     param_no_default_double_star_annotation:
-        | param_double_star_annotation & ')'
+        | param_double_star_annotation ','? &')'
 
     param_double_star_annotation: NAME double_star_annotation
 
@@ -463,13 +484,15 @@ previous example::
     >>> def foo(**kwargs: **Movie): ...
     ...
     >>> foo.__annotations__
-    {'kwargs': **Movie}
+    {'kwargs': Unpack[Movie]}
 
-The double asterisk syntax should call the ``__unpack__`` special method on
-the object it was used on. This means that ``def foo(**kwargs: **T): ...`` is
-equivalent to ``def foo(**kwargs: T.__unpack__()): ...``. In addition,
-``**Movie`` in the example above is the ``repr`` of the object that
-``__unpack__()`` returns.
+To accomplish this, we propose a new dunder called ``__typing_kwargs_unpack__``.
+The double asterisk syntax should result in a call to the
+``__typing_kwargs_unpack__`` special method on an object it was used on.
+This means that at runtime, ``def foo(**kwargs: **T): ...`` is equivalent to
+``def foo(**kwargs: T.__typing_kwargs_unpack__()): ...``.
+``TypedDict`` is the only type in the standard library that is expected to
+implement ``__typing_kwargs_unpack__``, which should return ``Unpack[self]``.
 
 Backwards Compatibility
 -----------------------
@@ -558,6 +581,7 @@ overloaded::
 References
 ==========
 
+.. _httpxIssue1384: https://github.com/encode/httpx/issues/1384
 .. _mypyIssue4441: https://github.com/python/mypy/issues/4441
 .. _mypyPull10576: https://github.com/python/mypy/pull/10576
 .. _mypyExtensionsPull22: https://github.com/python/mypy_extensions/pull/22/files