8000 Bring `_collections_abc` closer to runtime definition by AlexWaygood · Pull Request #6312 · python/typeshed · GitHub
[go: up one dir, main page]
Skip to content

Bring _collections_abc closer to runtime definition #6312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Nov 16, 2021
Merged

Bring _collections_abc closer to runtime definition #6312

merged 7 commits into from
Nov 16, 2021

Conversation

AlexWaygood
Copy link
Member
@AlexWaygood AlexWaygood commented Nov 16, 2021 •
edited
Loading

Currently, the following code succeeds at runtime, but fails mypy:

from _collections_abc import dict_keys

This PR proposes bringing the _collections_abc stub closer to its runtime definition by moving the definitions of dict_keys, dict_values and dict_items from builtins to _collections_abc.

I have also added @final to these classes, as they cannot be subclassed at runtime, and added # type: ignore comments to _OrderedDictKeysView/_OrderedDictValuesView/_OrderedDictItemsView. (These are indeed subclasses of dict_keys/dict_values/dict_items, even though the three classes cannot be subclassed at runtime; see #6299.)

@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
1 similar comment
@github-actions

This comment has been minimized.

Sign in to view
@AlexWaygood
Copy link
Member Author

@srittau, it looks like #6252 might be causing CI failures on unrelated PRs? 🙂

@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@srittau
Copy link
Collaborator
srittau commented Nov 16, 2021

The problem wasn't #6252, but a new release of fpdf2, which added arguments. Anyway, #6313 will fix CI.

@github-actions

This comment has been minimized.

Sign in to view
@AlexWaygood
Copy link
Member Author

The problem wasn't #6252, but a new release of fpdf2, which added arguments. Anyway, #6313 will fix CI.

Thanks! 😀

@github-actions

This comment has been minimized.

Sign in to view
@srittau srittau closed this Nov 16, 2021
@srittau srittau reopened this Nov 16, 2021
@github-actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

dedupe (https://github.com/dedupeio/dedupe.git)
- dedupe/training.py:80: error: Argument 1 to "union" of "frozenset" has incompatible type "*_dict_values[Predicate, FrozenSet[int]]"; expected "FrozenSet[<nothing>]"
+ dedupe/training.py:80: error: Argument 1 to "union" of "frozenset" has incompatible type "*dict_values[Predicate, FrozenSet[int]]"; expected "FrozenSet[<nothing>]"
- dedupe/training.py:80: error: Argument 1 to "union" of "frozenset" has incompatible type "*_dict_values[Predicate, FrozenSet[int]]"; expected "FrozenSet[Any]"
+ dedupe/training.py:80: error: Argument 1 to "union" of "frozenset" has incompatible type "*dict_values[Predicate, FrozenSet[int]]"; expected "FrozenSet[Any]"
- dedupe/training.py:297: error: Argument 1 to "union" of "frozenset" has incompatible type "*_dict_values[Predicate, FrozenSet[int]]"; expected "FrozenSet[<nothing>]"
+ dedupe/training.py:297: error: Argument 1 to "union" of "frozenset" has incompatible type "*dict_values[Predicate, FrozenSet[int]]"; expected "FrozenSet[<nothing>]"

SinbadCogs (https://github.com/mikeshardmind/SinbadCogs.git)
- relays/relays.py:114: error: Argument 1 to <tuple> has incompatible type "*_dict_values[str, OnewayRelay]"; expected "Union[NwayRelay, OnewayRelay]"
+ relays/relays.py:114: error: Argument 1 to <tuple> has incompatible type "*dict_values[str, OnewayRelay]"; expected "Union[NwayRelay, OnewayRelay]"
- relays/relays.py:114: error: Argument 2 to <tuple> has incompatible type "*_dict_values[str, NwayRelay]"; expected "Union[NwayRelay, OnewayRelay]"
+ relays/relays.py:114: error: Argument 2 to <tuple> has incompatible type "*dict_values[str, NwayRelay]"; expected "Union[NwayRelay, OnewayRelay]"

@JelleZijlstra JelleZijlstra merged commit fd48026 into python:master Nov 16, 2021
@AlexWaygood AlexWaygood deleted the dict-iterators branch November 16, 2021 17:42
@KotlinIsland
Copy link
Contributor
KotlinIsland commented Nov 22, 2021 •
edited
Loading

_collections_abc.dict_keys is generic in the stubs here, but it's not Generic in reality, such that:

from _collections_abc import dict_keys

dict_keys[int, int]

Will type check fine, but raise errors at runtime.

I think this is true for a lot of the Generics in the stubs, especially for Python < 3.9

@AlexWaygood
Copy link
Member Author

_collections_abc.dict_keys is generic in the stubs here, but it's not Generic in reality

I agree that this is unfortunate, but I'm not entirely sure what your point is. dict_keys was generic before this PR, so this isn't a new change.

Yes it's not great that the stub differs from the runtime implementation in this case, and yes the stub should generally be as close to the runtime implementation as possible. But having these classes be generic in the stub file allows type checkers to perform far more sophisticated type inferences.

If you have a valid use case for wanting to parameterize dict_keys at runtime, like it is in the stub, you can always make a feature request at bugs.python.org! Personally, I can't think of a case where I'd want or need to, however 🙂

@KotlinIsland
Copy link
Contributor

Previously this definition was _dict_keys in builtins so it was a lot harder to get at, but I completely agree with you. Maybe something like a @StubOnlyGeneric annotation could be added.

@KotlinIsland KotlinIsland mentioned this pull request Nov 23, 2021
Open
@JukkaL JukkaL mentioned this pull request Nov 23, 2021
Merged
This was referenced Nov 27, 2021
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@srittau srittau srittau left review comments

@JelleZijlstra JelleZijlstra JelleZijlstra left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@AlexWaygood @srittau @KotlinIsland @JelleZijlstra
0