[dataclass_transform] support field_specifiers #14667
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
posting this as a draft because i want to tidy up the error messages & consolidate some duplicated code, but the functionality should be covered AFAICT |
This comment has been minimized.
This comment has been minimized.
…s-transform-field-specifiers
|
I want to consolidate some error message generation/literal parsing code, but that touches enough locations that I think it's better off as a separate PR. Removing the draft status on this one. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
The mypy_primer diff is just an extension of the similar diff from #14657. These fields were not previously detected before |
JukkaL
reviewed
Feb 14, 2023
| @@ -6497,6 +6510,18 @@ def parse_dataclass_transform_spec(self, call: CallExpr) -> DataclassTransformSp | |||
|
|
|||
| return parameters | |||
|
|
|||
| def parse_dataclass_transform_field_specifiers(self, arg: Expression) -> tuple[str, ...]: | |||
| if not isinstance(arg, TupleExpr): | |||
| return tuple() | |||
There was a problem hiding this comment.
Should we generate an error here?
There was a problem hiding this comment.
good catch; i was going to let typeshed handle it, but i overlooked that we need to enforce that a literal is passed
| from typing import dataclass_transform | ||
|
|
||
| def field(extra1, *, kw_only=False, extra2=0): ... | ||
| @dataclass_transform(field_specifiers=(field,)) |
There was a problem hiding this comment.
Also test more than one field specifiers.
| a: int = field(alias='a_') | ||
| b: int = field(alias=B) | ||
| # cannot be passed as a positional | ||
| kwonly: int = field(kw_only=True, default=0) |
|
Diff from mypy_primer, showing the effect of this PR on open source code: artigraph (https://github.com/artigraph/artigraph)
+ src/arti/formats/__init__.py:35: error: Missing named argument "type" for "JSON" [call-arg]
+ src/arti/storage/__init__.py:138: error: Missing named argument "type" for "StringLiteral" [call-arg]
+ src/arti/storage/__init__.py:138: error: Missing named argument "format" for "StringLiteral" [call-arg]
+ src/arti/artifacts/__init__.py:105: error: Missing named argument "producer_output" for "Artifact" [call-arg]
+ src/arti/artifacts/__init__.py:107: error: Missing named argument "type" for "JSON" [call-arg]
+ src/arti/artifacts/__init__.py:108: error: Missing named argument "type" for "StringLiteral" [call-arg]
+ src/arti/artifacts/__init__.py:108: error: Missing named argument "format" for "StringLiteral" [call-arg]
+ src/arti/producers/__init__.py:365: error: Missing named argument "producer_output" for "Artifact" [call-arg]
+ tests/arti/dummies.py:30: error: Missing named argument "type" for "JSON" [call-arg]
+ tests/arti/dummies.py:93: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/dummies.py:94: error: Missing named argument "type" for "DummyStorage" [call-arg]
+ tests/arti/dummies.py:94: error: Missing named argument "format" for "DummyStorage" [call-arg]
+ tests/arti/dummies.py:99: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/dummies.py:100: error: Missing named argument "type" for "DummyStorage" [call-arg]
+ tests/arti/dummies.py:100: error: Missing named argument "format" for "DummyStorage" [call-arg]
+ tests/arti/dummies.py:105: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/dummies.py:106: error: Missing named argument "type" for "DummyStorage" [call-arg]
+ tests/arti/dummies.py:106: error: Missing named argument "format" for "DummyStorage" [call-arg]
+ tests/arti/dummies.py:111: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/dummies.py:112: error: Missing named argument "type" for "DummyStorage" [call-arg]
+ tests/arti/dummies.py:112: error: Missing named argument "format" for "DummyStorage" [call-arg]
+ tests/arti/dummies.py:117: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/dummies.py:118: error: Missing named argument "type" for "DummyStorage" [call-arg]
+ tests/arti/dummies.py:118: error: Missing named argument "format" for "DummyStorage" [call-arg]
+ src/arti/storage/local.py:59: error: Missing named argument "type" for "LocalFile" [call-arg]
+ src/arti/storage/local.py:59: error: Missing named argument "format" for "LocalFile" [call-arg]
+ tests/arti/views/test_python.py:27: error: Missing named argument "type" for "Pickle" [call-arg]
+ tests/arti/storage/test_gcs_storage.py:14: error: Missing named argument "type" for "GCSFile" [call-arg]
+ tests/arti/storage/test_gcs_storage.py:14: error: Missing named argument "format" for "GCSFile" [call-arg]
+ tests/arti/storage/test_gcs_storage.py:22: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ docs/examples/spend/demo.py:33: error: Missing named argument "type" for "JSON" [call-arg]
+ docs/examples/spend/demo.py:34: error: Missing named argument "type" for "LocalFile" [call-arg]
+ docs/examples/spend/demo.py:34: error: Missing named argument "format" for "LocalFile" [call-arg]
+ docs/examples/spend/demo.py:45: error: Missing named argument "producer_output" for "Transactions" [call-arg]
+ docs/examples/spend/demo.py:47: error: Missing named argument "type" for "JSON" [call-arg]
+ docs/examples/spend/demo.py:48: error: Missing named argument "type" for "LocalFile" [call-arg]
+ docs/examples/spend/demo.py:48: error: Missing named argument "format" for "LocalFile" [call-arg]
+ tests/arti/storage/test_storage.py:47: error: Missing named argument "type" for "MockStoragePartition" [call-arg]
+ tests/arti/storage/test_storage.py:47: error: Missing named argument "format" for "MockStoragePartition" [call-arg]
+ tests/arti/storage/test_storage.py:52: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/storage/test_storage.py:58: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/storage/test_storage.py:66: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/storage/test_storage.py:100: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:100: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:107: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:107: error: Missing named argument "type" for "JSON" [call-arg]
+ tests/arti/storage/test_storage.py:108: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:108: error: Missing named argument "type" for "JSON" [call-arg]
+ tests/arti/storage/test_storage.py:112: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:112: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:113: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:113: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:115: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:115: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:119: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:119: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:120: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:120: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:123: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:123: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:127: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:127: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:128: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:128: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:130: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:130: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:131: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:131: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:132: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:132: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:161: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:165: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:168: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:170: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:170: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:195: error: Missing named argument "format" for "Table" [call-arg]
+ tests/arti/storage/test_storage.py:201: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:201: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:202: error: Missing named argument "type" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:202: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:207: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/storage/test_storage.py:222: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/storage/test_storage.py:259: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:263: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_storage.py:268: error: Missing named argument "format" for "MockStorage" [call-arg]
+ tests/arti/storage/test_local_storage.py:42: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/storage/test_local_storage.py:61: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/storage/test_local_storage.py:90: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/storage/test_local_storage.py:108: error: Missing named argument "format" for "LocalFile" [call-arg]
+ tests/arti/storage/test_local_storage.py:126: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/storage/test_literal_storage.py:12: error: Missing named argument "type" for "DummyFormat" [call-arg]
+ tests/arti/storage/test_literal_storage.py:42: error: Missing named argument "format" for "StringLiteral" [call-arg]
+ tests/arti/storage/test_literal_storage.py:49: error: Missing named argument "format" for "StringLiteral" [call-arg]
... (truncated 112 lines) ...
pydantic (https://github.com/samuelcolvin/pydantic)
- pydantic/main.py:48: error: "field_specifiers" support is currently unimplemented [misc]
- pydantic/dataclasses.py:108: error: "field_specifiers" support is currently unimplemented [misc]
- pydantic/dataclasses.py:124: error: "field_specifiers" support is currently unimplemented [misc]
- pydantic/dataclasses.py:175: error: "field_specifiers" support is currently unimplemented [misc]
|
JukkaL
approved these changes
Feb 15, 2023
JukkaL
pushed a commit
that referenced
this pull request
Feb 23, 2023
…es (#14752) `dataclasses` uses a `__dataclass_fields__` attribute on each class to mark that it is a dataclass, and Typeshed checks for this attribute in its stubs for functions like `dataclasses.is_dataclass` and `dataclasses.asdict`. In #14667, I mistakenly removed this attribute for classes transformed by a `dataclass_transform`. This was due to a misinterpretation of PEP 681 on my part; after rereading the [section on dataclass semantics](https://peps.python.org/pep-0681/#dataclass-semantics), it says: > Except where stated otherwise in this PEP, classes impacted by `dataclass_transform`, either by inheriting from a class that is decorated with `dataclass_transform` or by being decorated with a function decorated with `dataclass_transform`, are assumed to behave like stdlib dataclass. The PEP doesn't seem to state anything about `__dataclass_fields__` or the related functions as far as I can tell, so we should assume that transforms should match the behavior of `dataclasses.dataclass` in this regard and include the attribute. This also matches the behavior of Pyright, which the PEP defines as the reference implementation.
cdce8p
pushed a commit
to cdce8p/mypy
that referenced
this pull request
Feb 23, 2023
…transformed types (python#14752) `dataclasses` uses a `__dataclass_fields__` attribute on each class to mark that it is a dataclass, and Typeshed checks for this attribute in its stubs for functions like `dataclasses.is_dataclass` and `dataclasses.asdict`. In python#14667, I mistakenly removed this attribute for classes transformed by a `dataclass_transform`. This was due to a misinterpretation of PEP 681 on my part; after rereading the [section on dataclass semantics](https://peps.python.org/pep-0681/#dataclass-semantics), it says: > Except where stated otherwise in this PEP, classes impacted by `dataclass_transform`, either by inheriting from a class that is decorated with `dataclass_transform` or by being decorated with a function decorated with `dataclass_transform`, are assumed to behave like stdlib dataclass. The PEP doesn't seem to state anything about `__dataclass_fields__` or the related functions as far as I can tell, so we should assume that transforms should match the behavior of `dataclasses.dataclass` in this regard and include the attribute. This also matches the behavior of Pyright, which the PEP defines as the reference implementation. (cherry picked from commit 54635de)
hauntsaninja
pushed a commit
that referenced
this pull request
Feb 23, 2023
…transformed types (#14752) (#14769) `dataclasses` uses a `__dataclass_fields__` attribute on each class to mark that it is a dataclass, and Typeshed checks for this attribute in its stubs for functions like `dataclasses.is_dataclass` and `dataclasses.asdict`. In #14667, I mistakenly removed this attribute for classes transformed by a `dataclass_transform`. This was due to a misinterpretation of PEP 681 on my part; after rereading the [section on dataclass semantics](https://peps.python.org/pep-0681/#dataclass-semantics), it says: > Except where stated otherwise in this PEP, classes impacted by `dataclass_transform`, either by inheriting from a class that is decorated with `dataclass_transform` or by being decorated with a function decorated with `dataclass_transform`, are assumed to behave like stdlib dataclass. The PEP doesn't seem to state anything about `__dataclass_fields__` or the related functions as far as I can tell, so we should assume that transforms should match the behavior of `dataclasses.dataclass` in this regard and include the attribute. This also matches the behavior of Pyright, which the PEP defines as the reference implementation. (cherry picked from commit 54635de) Co-authored-by: Wesley Collin Wright <wesleyw@dropbox.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These are analogous to
dataclasses.field/dataclasses.Field.Like most dataclass_transform features so far, this commit mostly just plumbs through the necessary metadata so that we can re-use the existing
dataclassesplugin logic. It also adds support for thealias=andfactory=kwargs for fields, which are small; we rely on typeshed to enforce that these aren't used withdataclasses.field.