10000 Implement support for returning TypedDict for dataclasses.asdict by syastrov · Pull Request #8583 · python/mypy · GitHub
[go: up one dir, main page]

Skip to content

Implement support for returning TypedDict for dataclasses.asdict #8583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 46 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
bb9f051
Implement support for returning TypedDict for dataclasses.asdict
syastrov Mar 26, 2020
e2f9f06
Remove redundant test. Fix comment typo.
syastrov Mar 26, 2020
2f6ec2d
Test for cases where dataclasses.asdict is called on non-dataclass in…
syastrov Mar 26, 2020
a9779e2
Clean up tests, and test more edge-cases.
syastrov Mar 26, 2020
c5d0a15
Remove no-longer-needed type: ignore on CheckerPluginInterface.module…
syastrov Mar 26, 2020
454431d
Make typeddicts non-total.
syastrov Mar 26, 2020
d809e8b
Address some of review comments (formatting, docs, code nitpicks, rem…
syastrov Mar 30, 2020
4d195cc
Simplify: Remove _transform_type_args and remove unneeded None check …
syastrov Mar 30, 2020
b33798c
Fix unused import
syastrov Mar 30, 2020
fe19bc9
Add fine-grained test for dataclasses.asdict.
syastrov Mar 30, 2020
6328a7c
Oops, add forgotten fine-grained dataclasses test. And remove redunda…
syastrov Mar 30, 2020
4694299
Only import the module containing TypedDict fallback if dataclasses i…
syastrov Apr 1, 2020
9c29081
Only enable TypedDict for Python >= 3.8.
syastrov Apr 2, 2020
d7df77a
Refactor asdict implementation to use TypeTranslator instead of recur…
syastrov Apr 2, 2020
e9a56ba
Made TypedDicts returned by asdict total again.
syastrov Apr 2, 2020
2e5240e
Fixed test after total change.
syastrov Apr 2, 2020
52a1c27
Make code a bit more readable, and a bit more robust.
syastrov Apr 2, 2020
43f174c
Fix typo
syastrov Apr 2, 2020
227ba90
After refactoring to use TypeTranslator, ensure Callable and Type[..]…
syastrov Apr 2, 2020
d12c665
Address second review comments.
syastrov Apr 8, 2020
45e72d7
Fix return type
syastrov Apr 8, 2020
a03f033
Try to address more review comments and fix flake8
syastrov Jun 3, 2020
b4d7e15
Add fine grained deps test to help debug asdict dependencies.
syastrov Jun 3, 2020
d96d977
Fix some asdict tests missing tuple dependency
syastrov Jun 3, 2020
441b665
Revert "Fix some asdict tests missing tuple dependency"
syastrov Jun 4, 2020
344ca6a
Don't need dep on typing_extensions
syastrov Jun 4, 2020
c8858fa
Checker lookup_fully_qualified_or_none: Don't raise KeyError, return …
syastrov Jun 9, 2020
20d7716
Add dependencies for asdict on the referenced dataclasses and its att…
syastrov Jun 9, 2020
5fec41b
Fix fine-grained no-cache test by adding correct dep on dataclass attrs.
syastrov Aug 18, 2020
5862c16
remove unused imports
syastrov Aug 18, 2020
26b7393
Merge branch 'master' into dataclasses-asdict
syastrov Feb 17, 2021
d7e0310
Remove error when passing a "non-dataclass" to asdict to reduce false…
syastrov Feb 17, 2021
080c00c
Fix flake8
syastrov Feb 17, 2021
9e45f8f
Fix asdict tests (require using python version 3.7 minimum).
syastrov Feb 17, 2021
38b466a
Merge branch 'master' into dataclasses-asdict
syastrov Aug 19, 2021
74ebc6f
Fix tests for quoting changes
syastrov Aug 19, 2021
aef274a
Merge branch 'master' into dataclasses-asdict
97littleleaf11 Nov 17, 2021
79f25db
Merge
97littleleaf11 Jan 18, 2022
f54e503
Fix
97littleleaf11 Jan 18, 2022
9820cfc
Add fixture for tests
97littleleaf11 Jan 18, 2022
9f49cac
Add fixture for tests
97littleleaf11 Jan 18, 2022
fcd1ff5
Add fixture for tests
97littleleaf11 Jan 18, 2022
9fa6a6b
Merge from master
97littleleaf11 Jan 18, 2022
6e1585d
Fix
97littleleaf11 Jan 18, 2022
d6f9170
Merge branch 'master' of https://github.com/python/mypy into HEAD
97littleleaf11 Jan 19, 2022
e226562
Test for a workaround
97littleleaf11 Jan 19, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Address some of review comments (formatting, docs, code nitpicks, rem…
…ove recursion error).
  • Loading branch information
syastrov committed Aug 18, 2020
commit d809e8b6bb44c105ca0576503b696b4583f603ba
13 changes: 2 additions & 11 deletions docs/source/additional_features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -76,17 +76,8 @@ Some functions in the :py:mod:`dataclasses` module, such as :py:func:`~dataclass
have imprecise (too permissive) types. This will be fixed in future releases.

Calls to :py:func:`~dataclasses.asdict` will return a ``TypedDict`` based on the original dataclass
definition, transforming it recursively. There are, however, some limitations:

* Subclasses of ``List``, ``Dict``, and ``Tuple`` appearing within dataclasses are transformed into reparameterized
versions of the respective base class, rather than a transformed version of the original subclass.

* Recursion (e.g. dataclasses which reference each other) is not supported and results in an error.

* ``NamedTuples`` appearing within dataclasses are transformed to ``Any``

* A more precise return type cannot be inferred for calls where ``dict_factory`` is set.

definition, transforming it recursively. There are, however, some limitations. In particular, a precise return type
cannot be inferred for recursive dataclasses, and for calls where ``dict_factory`` is set.

Mypy does not yet recognize aliases of :py:func:`dataclasses.dataclass <dataclasses.dataclass>`, and will
probably never recognize dynamically computed decorators. The following examples
Expand Down
3 changes: 2 additions & 1 deletion mypy/plugins/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@
from mypy.semanal import set_callable_name
from mypy.types import (
CallableType, Overloaded, Type, TypeVarDef, deserialize_type, get_proper_type,
TypedDictType, Instance, TPDICT_FB_NAMES)
TypedDictType, Instance, TPDICT_FB_NAMES
)
from mypy.typevars import fill_typevars
from mypy.util import get_unique_redefinition_name
from mypy.typeops import try_getting_str_literals # noqa: F401 # Part of public API
Expand Down
27 changes: 13 additions & 14 deletions mypy/plugins/dataclasses.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,15 @@
)
from mypy.plugin import ClassDefContext, FunctionContext, CheckerPluginInterface
from mypy.plugin import SemanticAnalyzerPluginInterface
from mypy.plugins.common import (add_method, _get_decorator_bool_argument,
make_anonymous_typeddict, deserialize_and_fixup_type)
from mypy.plugins.common import (
add_method, _get_decorator_bool_argument, make_anonymous_typeddict,
deserialize_and_fixup_type
)
from mypy.server.trigger import make_wildcard_trigger
from mypy.types import (Instance, NoneType, TypeVarDef, TypeVarType, get_proper_type, Type,
TupleType, UnionType, AnyType, TypeOfAny)
from mypy.types import (
Instance, NoneType, TypeVarDef, TypeVarType, get_proper_type, Type, TupleType, UnionType,
AnyType, TypeOfAny
)

# The set of decorators that generate dataclasses.
dataclass_makers = {
Expand Down Expand Up @@ -380,11 +384,9 @@ def asdict_callback(ctx: FunctionContext) -> Type:
positional_arg_types = ctx.arg_types[0]

if positional_arg_types:
dataclass_instance = positional_arg_types[0]
dataclass_instance = get_proper_type(dataclass_instance)
dataclass_instance = get_proper_type(positional_arg_types[0])
if isinstance(dataclass_instance, Instance):
info = dataclass_instance.type
if is_type_dataclass(info):
if is_type_dataclass(dataclass_instance.type):
if len(ctx.arg_types) == 1:
return _asdictify(ctx.api, ctx.context, dataclass_instance)
else:
Expand Down Expand Up @@ -424,10 +426,7 @@ def _asdictify_inner(typ: Type, seen_dataclasses: FrozenSet[str]) -> Type:
info = typ.type
if is_type_dataclass(info):
if info.fullname in seen_dataclasses:
api.fail(
"Recursive types are not supported in call to asdict, so falling back to "
"Dict[str, Any]",
context)
# Recursive types not supported, so fall back to Dict[str, Any]
# Note: Would be nicer to fallback to default_return_type, but that is Any
# (due to overloads?)
return api.named_generic_type('builtins.dict',
Expand All @@ -446,14 +445,14 @@ def _asdictify_inner(typ: Type, seen_dataclasses: FrozenSet[str]) -> Type:
required_keys=set())
elif info.has_base('builtins.list'):
supertype_instance = map_instance_to_supertype(typ, api.named_generic_type(
'builtins.list', []).type)
'builtins.list', [AnyType(TypeOfAny.implementation_artifact)]).type)
new_args = _transform_type_args(
typ=supertype_instance,
transform=lambda arg: _asdictify_inner(arg, seen_dataclasses))
return api.named_generic_type('builtins.list', new_args)
elif info.has_base('builtins.dict'):
supertype_instance = map_instance_to_supertype(typ, api.named_generic_type(
'builtins.dict', []).type)
'builtins.dict', [AnyType(TypeOfAny.implementation_artifact), AnyType(TypeOfAny.implementation_artifact)]).type)
new_args = _transform_type_args(
typ=supertype_instance,
transform=lambda arg: _asdictify_inner(arg, seen_dataclasses))
Expand Down
2 changes: 1 addition & 1 deletion test-data/unit/check-dataclasses.test
Original file line number Diff line number Diff line change
Expand Up @@ -1035,7 +1035,7 @@ class A:
b: Optional[B] = None

# Recursion is not supported, so fall back
result = asdict(A(B(C(A())))) # E: Recursive types are not supported in call to asdict, so falling back to Dict[str, Any]
result = asdict(A(B(C(A()))))
reveal_type(result) # N: Revealed type is 'TypedDict({'b'?: Union[TypedDict({'c'?: TypedDict({'a'?: builtins.dict[builtins.str, Any]})}), None]})'

[typing fixtures/typing-full.pyi]
Expand Down
0