8000 Mirrored modules in `v1` namespace to fix typing and object resolution in python>3.11 by rx-dwoodward · Pull Request #9660 · pydantic/pydantic · GitHub
[go: up one dir, main page]

Skip to content

Mirrored modules in v1 namespace to fix typing and object resolution in python>3.11 #9660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

rx-dwoodward
Copy link
Contributor
@rx-dwoodward rx-dwoodward commented Jun 14, 2024

Change Summary

Attempt to fix the typing issue with .v1.py and the python 3.12 issue regarding objects imported through __getattr__ not being the same object as when imported directly.

This PR:

  • Adds mirrored modules in a v1/ subpackage, that mirrors the modules in pydantic where each module simply imports all symbols from the corresponding parent.
  • Adds a test to ensure that imported objects are the exact same (tested on 3.12),

my mypy seems to be happy with imports from pydantic.v1.fields.ModelField now as well.

Note that the one downside to this is that the modules are no longer the same, i.e. pydantic.fields is pydantic.v1.fields # == False whereas it was True with the previous PR.

I'm not entirely sure there is a way round both issues ..

Related issue number

fix #9656

skip change file check

Checklist

  • The pull request title is a good summary of the changes - it will be used in the changelog
  • Unit tests for the changes exist
  • Tests pass on CI
  • Documentation reflects the changes where applicable
  • My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Selected Reviewer: @samuelcolvin

@rx-dwoodward rx-dwoodward force-pushed the fix-typing-and-py-3.12 branch from 6a73698 to 003273b Compare June 14, 2024 16:35
@rx-dwoodward
Copy link
Contributor Author

please review

@rx-dwoodward
Copy link
Contributor Author

@sydney-runkle attempted a fix for the typing issue / python 3.12 issue

@bdraco
Copy link
bdraco commented Jun 15, 2024

I tried installing this but v1 path isn't listed as a package so it didn't get included

suggested fix

diff --git a/setup.py b/setup.py
index 8e5f82e9..63f16f87 100644
--- a/setup.py
+++ b/setup.py
@@ -132,7 +132,7 @@ setup(
     author_email='s@muelcolvin.com',
     url='https://github.com/pydantic/pydantic',
     license='MIT',
-    packages=['pydantic'],
+    packages=['pydantic', 'pydantic.v1'],
     package_data={'pydantic': ['py.typed']},
     python_requires='>=3.7, <3.13',
     zip_safe=False,  # https://mypy.readthedocs.io/en/latest/installed_packages.html

@bdraco
Copy link
bdraco commented Jun 15, 2024

This does appear to resolve the issues noted in home-assistant/core#119430 and uilibs/uiprotect#39 once the setup.py was modified, make clean was run, cython downgraded to 0.29, and reinstalled. The test suites both pass

@rx-dwoodward
Copy link
Contributor Author
rx-dwoodward commented Jun 17, 2024

Nice, thanks for the check - its pretty frustrating trying to make a subpackage appear like the package itself, bound to have some issues associated with it. Will add this change 👍

Kinda surprised that the CI tests pass without this though. Somewhat worrying 😓

from pydantic.typing import *
else:
# explicit importing
from pydantic.typing import (
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note I'm a bit worried about these, for example if a module uses a __all__ it does not stop someone using from pydantic.<module> import <SymbolNotProvidedInAllButPresent> whereas with these imports from pydantic.v1.<module> import <SymbolNotProvidedInAllButPresent> will not be found.

I'm not sure how common this will actually be (I'm sure some people will definitely import objects from a module that does not have them defined in __all__ 🙃) but I'm also not too sure whether we want to support that in this either.

Personally I'm kinda fine with this, but happy to take opinions. Also, this will likely need a migration guide update too right?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd argue that from pydantic.<module> import <SymbolNotProvidedInAllButPresent> amounts to depending on an undocumented implementation detail in the old API, and that you should be free to avoid replicating that in the "new" pydantic.v1.* API.

@sydney-runkle
Copy link
Contributor

So I think once we decide on a fix here, we'll want to do another v1 release, then pull that into v2 as well.

We'll probably have to modify the logic to update v2 with v1 as well, and I definitely want to think about that ahead of time.

I think we'll want to discard the v1 module in v1, and then keep the changes re absolute import patching that I introduced here: https://github.com/pydantic/pydantic/pull/9639/files#diff-124a316e9ef143df925ee70f53238acb2bbe6d9d2b158818073a8d85c5316656

We can probably remove the v1 module before making those changes.

@rx-dwoodward
Copy link
Contributor Author

yep I agree, when pushing the pydantic.v1 into v2 we can just remove the v1 sub directory / not include it to avoid pydantic.v1.v1 type imports 😨

We'd likely have to update the migration guide for v2 in the docs as well right? with pydantic>=1.10.17 and pydantic<1.10.17 type tabs?

@sydney-runkle
Copy link
Contributor

Ah also one thing that this complicates is new development in v1 requires updates to these files potentially. Hopefully that's not too much of an issue bc we're trying not to add much new stuff in v1.

@sydney-runkle
Copy link
Contributor

We'd likely have to update the migration guide for v2 in the docs as well right? with pydantic>=1.10.17 and pydantic<1.10.17 type tabs?

Yeah, that'd be great. @exs-dwoodward, any interest in opening a docs update PR with these changes? Thanks for all of your help with these fixes thus far 🚀 !

@rx-dwoodward
Copy link
Contributor Author

Yeah I can take a look at it - any links you provide to point me in the right direction? First time looking at pydantic's docs compliation so would probably be quicker with some pointers 👀

@rx-dwoodward
Copy link
Contributor Author

this complicates is new development in v1 requires updates to these files potentially

Worth me adding something to any Contribution part of the v1 docs?

@sydney-runkle
Copy link
Contributor

@exs-dwoodward,

Great!

You'll want to edit docs/migration.md. I don't think it merits a note in the contribution docs, but we should just be careful with reviews.

You can run mkdocs serve to get things running locally!

Copy link
Contributor
@sydney-runkle sydney-runkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @exs-dwoodward,

Looks good. Going to merge this, then will work on a patch for v1 and v2 soon.
I'll wait on the docs update, plus we're also going to remove the version cap, and I'll have to do some juggling for the v2 update as mentioned above.

@sydney-runkle sydney-runkle merged commit 545ac6e into pydantic:1.10.X-fixes Jun 18, 2024
61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0