Stealth breaking change 2.9 -> 2.10 for type sequences #11853

JanSobus · 2025-05-09T01:43:24Z

Initial Checks

I confirm that I'm using Pydantic V2

Description

I wrote a code envisioned to be a generic interface to Data Science libraries, where the basic unit of input output is a DATA_MODEL -> BaseModel, Sequence of BaseModels or Pandera DataFrame model (last one irrelevant for this example).

Library can have 1+ endpoints, where each endpoint is a callable that has arguments that are DATA_MODELs (some of them optional) and output that is a single DATA_MODEL, or tuple of them.

It required some custom validators, but the whole setup shown below worked well in 2.9. Starting from 2.10, Endpoint Class definition throws an error:

TypeError: Expected a class, got typing.Sequence[pydantic.main.BaseModel]

already at the Endpoint model definition.
From my understanding of the error, the Type[Sequence[BaseModel]] trips it.

Is this intended behaviour, how can I work around that error or improve my code to not trigger it?

Thank you for all the great work!

Example Code

from __future__ import annotations
from datetime import datetime
from types import NoneType, UnionType
from typing import Callable, Sequence, Type, get_args, get_origin, Union

from pandera import DataFrameModel
from pandera.typing import Series
from pydantic import BaseModel, field_validator, SkipValidation

DATA_MODEL = Type[BaseModel] | Type[DataFrameModel] | Type[Sequence[BaseModel]]
INPUT_SCHEMA = dict[str, DATA_MODEL | UnionType]
OUTPUT_SCHEMA = tuple[DATA_MODEL, ...] | DATA_MODEL

def is_data_model(annotation: type) -> bool:
    if not isinstance(annotation, type):
        raise ValueError("annotation must be a type")
    if len(get_args(annotation)) > 0:
        origin = get_origin(annotation)
        if origin and issubclass(origin, Sequence):
            return is_data_model(get_args(annotation)[0])
        return False
    return issubclass(annotation, (BaseModel, DataFrameModel))

class Endpoint(BaseModel, arbitrary_types_allowed=True):
    endpoint_func: Callable
    input_schema: SkipValidation[INPUT_SCHEMA]
    output_schema: SkipValidation[OUTPUT_SCHEMA]

    @field_validator("output_schema")
    @classmethod
    def validate_output_schema(cls, v: OUTPUT_SCHEMA) -> OUTPUT_SCHEMA:
        if isinstance(v, tuple):
            for i, arg in enumerate(v):
                if not is_data_model(arg):
                    raise ValueError(f"output_schema[{i}] must be a data model")
        elif not is_data_model(v):
            raise ValueError("output_schema must be a data model or a tuple of data models")
        return v

    @field_validator("input_schema")
    @classmethod
    def validate_input_schema(cls, v: INPUT_SCHEMA) -> INPUT_SCHEMA:
        if not isinstance(v, dict):
            raise ValueError("input_schema must be a dictionary")
        for key, value in v.items():
            if not isinstance(key, str):
                raise ValueError("input_schema keys must be strings")
            if type(value) == UnionType or (get_origin(value) == Union):
                args = get_args(value)
                if is_data_model(args[0]) and args[1] == NoneType:
                    continue
                raise ValueError("Wrong Union type, it should only be DATA_MODEL | None")
            elif is_data_model(value):
                continue
            else:
                raise ValueError(f"input_schema[{key}] must be a data model or a Union of a data model and None")
        return v


### Example

class Point(BaseModel):
    x: float
    y: float


points1: DATA_MODEL = dict[str, Point] # invalid
points2: DATA_MODEL = list[Point] # valid

class GPSData(DataFrameModel):
    timestamp: Series[datetime]
    x: Series[float]
    y: Series[float]


def midpoint(point1: Point, point2: Point | None = None) -> Point:
    if point2 is None:
        point2 = Point(x=0, y=0)
    return Point(x=(point1.x + point2.x) / 2, y=(point1.y + point2.y) / 2)

def furthest_point(points: list[Point]) -> Point:
    return max(points, key=lambda p: p.x**2 + p.
8000
y**2)

input_dict1: INPUT_SCHEMA = {
    "point1": Point,
    "point2": Point | None
}

input_dict2: INPUT_SCHEMA = {
    "points": list[Point]
}


e1 = Endpoint(
    endpoint_func=midpoint,
    input_schema=input_dict1,
    output_schema=Point
)

e2 = Endpoint(
    endpoint_func=furthest_point,
    input_schema=input_dict2,
    output_schema=Point
)

Python, Pydantic & OS Version

pydantic version: 2.10.0
pydantic-core version: 2.27.0
pydantic-core build: profile=release pgo=false
install path: C:\Users\Jan\projects\pydantic_test\.venv\Lib\site-packages\pydantic
python version: 3.11.11 (main, Feb 12 2025, 14:49:02) [MSC v.1942 64 bit (AMD64)]
platform: Windows-10-10.0.19045-SP0
related packages: typing_extensions-4.13.2
                       commit: unknown

The text was updated successfully, but these errors were encountered:

Viicos · 2025-05-09T08:22:48Z

An extra check was introduced in 2.10 by #10621. In 2.11, you get a proper error:

pydantic.errors.PydanticUserError: Subscripting `type[]` with an already parametrized type is not supported. Instead of using type[typing.Sequence[pydantic.main.BaseModel]], use type[Sequence].

The code you were using in 2.9 was somewhat unsafe. First, you probably expected e2 to validate fine, but it doesn't as list[Point] isn't a type, but a types.GenericAlias instance and so it raises in is_data_model().

Anyway, instead of using SkipValidation, you should instead use a plain validator, which will not make use of the type hint:

class Endpoint(BaseModel):
    endpoint_func: Callable
    input_schema: INPUT_SCHEMA
    output_schema: OUTPUT_SCHEMA

    @field_validator("output_schema", mode="plain")
    @classmethod
    def validate_output_schema(cls, v: OUTPUT_SCHEMA) -> OUTPUT_SCHEMA:
        ...

Currently, your validator is an after validator. What happens is that Pydantic first validates against the type hint. Because SkipValidation is applied, no validation is performed and your after validator is then triggered. However, this will probably change in the future: SkipValidation will skip any validators attached to the field (see #11807).

JanSobus · 2025-05-11T22:58:59Z

862E

@Viicos

Thanks for the quick reply!

I applied your suggestions - removed SkipValidation and changed the validators to "plain".

With 2.11.4 I get the error you mentioned:
pydantic.errors.PydanticUserError: Subscripting type[] with an already parametrized type is not supported. Instead of using type[typing.Sequence[pydantic.main.BaseModel]], use type[Sequence].

However, applying the suggestion from error message -> changing Type[Sequence[BaseModel]] to Type[Sequence] results in the similar error to before:
TypeError: Expected a class, got typing.Sequence

Is there a way around it?

Thanks for pointing out types.GenericAlias too, that should be easy to fix

Viicos · 2025-05-15T09:34:30Z

typing.Sequence is a deprecated alias to collections.abc.Sequence. This alias is implemented as an instance of an internal typing construct and as such doesn't pass the inspect.isclass() test. Using collections.abc.Sequence will work as expected.

JanSobus added bug V2 Bug related to Pydantic V2 pending Is unconfirmed labels May 9, 2025

Viicos mentioned this issue May 9, 2025

Allow SkipValidation to be applied in any order #11807

Open

1 task

Viicos closed this as not planned Won't fix, can't repro, duplicate, stale May 9, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Stealth breaking change 2.9 -> 2.10 for type sequences #11853

Stealth breaking change 2.9 -> 2.10 for type sequences #11853

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Stealth breaking change 2.9 -> 2.10 for type sequences #11853

Stealth breaking change 2.9 -> 2.10 for type sequences #11853

Comments

Initial Checks

Description

Example Code

Python, Pydantic & OS Version

Uh oh!

Uh oh!

Uh oh!

Uh oh!