`QuerySet` class no longer derives from `Collection` (django-stubs 5.0.0) #2095

intgr · 2024-04-30T09:04:20Z

intgr
Apr 30, 2024
Maintainer

Starting in django-stubs version 5.0.0, the QuerySet class is no longer compatible with Collection[T] (PR: #1925). This can require some changes to your code, if you get errors like the following:

Argument 1 to "accepts_collection" has incompatible type "_QuerySet[User, User]"; expected "Collection[User]" [arg-type]

The previous behavior was incorrect and could possibly cause bugs, because Python's runtime isinstance(queryset, Collection) check returns False, where type checkers previously assumed True.

QuerySet class continues to derive from Iterable[T] and Sized.

Example issue

However, this will cause new type checker errors where a QuerySet is passed into a Collection-typed argument, e.g.

def accepts_collection(users: Collection[User]) -> None:
    pass

accepts_collection(User.objects.all())  # ❌
# mypy error: Argument 1 to "accepts_collection" has incompatible type "_QuerySet[User, User]"; expected "Collection[User]"  [arg-type]

Solution 1: Iterable

In most cases functions only want to iterate over the collection. In these cases, it's simple to just replace Collection with Iterable

def accepts_collection(users: Iterable[User]) -> None:
#                             ^^^^^^^^

Solution 2: SizedIterable

The Collection base class includes __len__() method, but that is lost when using Iterable. Thus if the called function wants to call len(users) as well as iterate over it, it's possible to define a new protocol (e.g. SizedIterable) that ensures both are implemented.

# Copy this verbatim to your code
from typing import Iterable, TypeVar, Protocol, Sized

T = TypeVar('T', covariant=True)

class SizedIterable(Protocol, Iterable[T], Sized):
    pass
# End copy

def accepts_collection(users: SizedIterable[User]) -> None:
    len(users)  # ✅

accepts_collection(User.objects.all())  # ✅

Solution 3: Union of concrete classes

Usually when QuerySet was downcast into a Collection, it's because the code needs to handle mixed input of QuerySets as well as other types. It's also possible to use union types (| operator) instead of abstract base classes in such cases.

def accepts_collection(users: QuerySet[User] | list[User] | tuple[User, ...]) -> None:
    len(users)  # ✅

accepts_collection(User.objects.all())  # ✅
accepts_collection(list(User.objects.all()))  # ✅

Other work-arounds

If the called function is not under your control and you cannot change its signature, but you know that the code works correctly, it's also possible to just override the type checker.

# Explicit cast
accepts_collection(cast(Collection[User], User.objects.all()))
# Suppresion comment
accepts_collection(User.objects.all())  # type: ignore[arg-type]

Also you can change the runtime behavior of your code to convert the results of a QuerySet into a list.

accepts_collection(list(User.objects.all()))

martinlehoux · 2024-05-16T14:47:53Z

martinlehoux
May 16, 2024

I understand it's the correct way to go.
As a side note, I had just removed accepting Iterable[Model] in my codebase, because it accepts Generators that can be iterated over only once, and I used instead Collection[Model] to prevent this (I made the mistake several times to iterate twice and it was hard to debug).
This was discussed in python/mypy#2942, and Guido suggested to use Collection.
So now that I can't do that, I think I'll stick with a QuerySet[Model] | Collection[Model].
Just wanted to prevent this issue to happen to other people ^^

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

`QuerySet` class no longer derives from `Collection` (django-stubs 5.0.0) #2095

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

QuerySet class no longer derives from Collection (django-stubs 5.0.0) #2095

Uh oh!

Uh oh!

intgr Apr 30, 2024 Maintainer

Example issue

Solution 1: Iterable

Solution 2: SizedIterable

Solution 3: Union of concrete classes

Other work-arounds

Replies: 1 comment

Uh oh!

martinlehoux May 16, 2024

`QuerySet` class no longer derives from `Collection` (django-stubs 5.0.0) #2095

intgr
Apr 30, 2024
Maintainer

martinlehoux
May 16, 2024