8000 Fix JSON Schema generation of fields with plain validators in serialization mode by Viicos · Pull Request #10167 · pydantic/pydantic · GitHub
[go: up one dir, main page]
Skip to content

Fix JSON Schema generation of fields with plain validators in serialization mode #10167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Viicos merged 3 commits into from
Aug 19, 2024
Merged

Fix JSON Schema generation of fields with plain validators in serialization mode #10167

Viicos merged 3 commits into from
Aug 19, 2024

Conversation

@Viicos
Copy link
Member
@Viicos Viicos commented Aug 18, 2024

As per #10094 (comment).

Change Summary

Related issue number

Checklist

  • The pull request title is a good summary of the changes - it will be used in the changelog
  • Unit tests for the changes exist
  • Tests pass on CI
  • Documentation reflects the changes where applicable
  • My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

@github-actions github-actions bot added the relnotes-fix Used for bugfixes. label Aug 18, 2024
@cloudflare-workers-and-pages
Copy link
cloudflare-workers-and-pages bot commented Aug 18, 2024 •
edited
Loading

Deploying pydantic-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 7ae6890
Status: ✅  Deploy successful!
Preview URL: https://f68dc72e.pydantic-docs.pages.dev
Branch Preview URL: https://plain-val-ser.pydantic-docs.pages.dev

View logs

@codspeed-hq
Copy link
codspeed-hq bot commented Aug 18, 2024 •
edited
Loading

CodSpeed Performance Report

Merging #10167 will not alter performance

Comparing plain-val-ser (7ae6890) with main (061e62c)

Summary

✅ 17 untouched benchmarks

@Viicos
Copy link
Member Author
Viicos commented Aug 18, 2024 •
edited
Loading

This changes the behavior of the added test in #10155.

Consider the following example:

from typing import Annotated

from pydantic import BaseModel, PlainValidator

class Model1(BaseModel):
    pass

class Model2(BaseModel):
    model1: Annotated[Model1, PlainValidator(lambda _: Model1())]

PlainValidator internally creates a wrap serializer core schema:

pydantic/pydantic/functional_validators.py

Lines 163 to 178 in 752cdc9

try:
schema = handler(source_type)
serialization = core_schema.wrap_serializer_function_ser_schema(
function=lambda v, h: h(v),
schema=schema,
return_schema=schema,
)
except PydanticSchemaGenerationError:
serialization = None
info_arg = _inspect_validator(self.func, 'plain')
if info_arg:
func = cast(core_schema.WithInfoValidatorFunction, self.func)
return core_schema.with_info_plain_validator_function(
func, field_name=handler.field_name, serialization=serialization
)

With the addition of the return_schema key from this PR, we end up with a 'serialization' core schema looking like this:

{
    'type': 'function-wrap',
    'function': <function PlainValidator.__get_pydantic_core_schema__.<locals>.<lambda> at 0x712713441300>,
    'schema': {
        'type': 'definition-ref',
        'schema_ref': '[...].Model1:101540811979568'
    },
    'return_schema': {
        'type': 'definition-ref',
        'schema_ref': '[...].Model1:101540811979568'
    }
}

Now when counting the refs of a schema, we (wrongly?) assume that having a reference count >= 2 means the model is involved in recursion:

pydantic/pydantic/_internal/_core_utils.py

Lines 446 to 454 in 823d51f

def count_refs(s: core_schema.CoreSchema, recurse: Recurse) -> core_schema.CoreSchema:
if s['type'] != 'definition-ref':
return recurse(s, count_refs)
ref = s['schema_ref']
ref_counts[ref] += 1
if ref_counts[ref] >= 2:
# If this model is involved in a recursion this should be detected
# on its second encounter, we can safely stop the walk here.

And it thus prevents the refs to be inlined. This changes the resulting core schema, to be a 'definitions' core schema:

pydantic/pydantic/_internal/_core_utils.py

Lines 509 to 512 in 823d51f

def_values = [v for v in definitions.values() if ref_counts[v['ref']] > 0] # type: ignore
if def_values:
schema = core_schema.definitions_schema(schema=schema, definitions=def_values)

And the original issue disappears. I adapted the test to reproduce the original behavior (i.e. do not add the return_schema key).

@Viicos
Adapt existing test
8a2e69a 
Thanks to the newly added `return_schema`, the core schema
of `Model2` becomes a `'definitions'` schema, which no longer
reproduces the issue.
@github-actions
Copy link
Contributor
github-actions bot commented Aug 19, 2024 •
edited
Loading

Coverage report

This PR does not seem to contain any modification to coverable code.

sydney-runkle
sydney-runkle approved these changes Aug 19, 2024
View reviewed changes
Copy link
Contributor
@sydney-runkle sydney-runkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great - thanks for breaking this change out into a smaller PR!

@Viicos Viicos enabled auto-merge (squash) August 19, 2024 16:12
@Viicos Viicos merged commit 0d3b0c3 into main Aug 19, 2024
@Viicos Viicos deleted the plain-val-ser branch August 19, 2024 16:28
@ngnpope ngnpope mentioned this pull request Sep 11, 2024
Closed
1 task
This was referenced Sep 12, 2024
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@sydney-runkle sydney-runkle sydney-runkle approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

relnotes-fix Used for bugfixes.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Viicos @sydney-runkle
0