-
Notifications
You must be signed in to change notification settings - Fork 234
feat: dynamic class creation #1179
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
f746c0d
feat: add func create Doc from dict
AnneYang720 f49f518
test: add test for create_from_dict
AnneYang720 cc6cf68
feat: mirror func from pydantic
AnneYang720 51bf952
fix: mypy
AnneYang720 22cceff
Merge branch 'feat-rewrite-v2' into feat-dynamic-creation
AnneYang720 6839dc0
docs: add example usage
AnneYang720 e637acd
refactor: add return type hint
AnneYang720
8000
Feb 28, 2023
1a665ea
refactor: rename typevar
AnneYang720 56e798e
Merge branch 'feat-rewrite-v2' into feat-dynamic-creation
AnneYang720 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,120 @@ | ||
| from typing import TYPE_CHECKING, Any, Dict, Optional, Tuple, Type, TypeVar | ||
|
|
||
| from pydantic import create_model, create_model_from_typeddict | ||
| from pydantic.config import BaseConfig | ||
| from typing_extensions import TypedDict | ||
|
|
||
| from docarray import BaseDocument | ||
|
|
||
| if TYPE_CHECKING: | ||
| from pydantic.typing import AnyClassMethod | ||
|
|
||
| T_doc = TypeVar('T_doc', bound=BaseDocument) | ||
|
|
||
|
|
||
| def create_doc( | ||
| __model_name: str, | ||
| *, | ||
| __config__: Optional[Type[BaseConfig]] = None, | ||
| __base__: Type['T_doc'] = BaseDocument, # type: ignore | ||
| __module__: str = __name__, | ||
| __validators__: Dict[str, 'AnyClassMethod'] = None, # type: ignore | ||
| __cls_kwargs__: Dict[str, Any] = None, # type: ignore | ||
| __slots__: Optional[Tuple[str, ...]] = None, | ||
| **field_definitions: Any, | ||
| ) -> Type['T_doc']: | ||
| """ | ||
| Dynamically create a subclass of BaseDocument. This is a wrapper around pydantic's create_model. | ||
| :param __model_name: name of the created model | ||
| :param __config__: config class to use for the new model | ||
| :param __base__: base class for the new model to inherit from, must be BaseDocument or its subclass | ||
| :param __module__: module of the created model | ||
| :param __validators__: a dict of method names and @validator class methods | ||
| :param __cls_kwargs__: a dict for class creation | ||
| :param __slots__: Deprecated, `__slots__` should not be passed to `create_model` | ||
| :param field_definitions: fields of the model (or extra fields if a base is supplied) | ||
| in the format `<name>=(<type>, <default default>)` or `<name>=<default value>` | ||
| :return: the new Document class | ||
|
|
||
| EXAMPLE USAGE | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| from docarray.documents import Audio | ||
| from docarray.documents.helper import create_doc | ||
| from docarray.typing.tensor.audio import AudioNdArray | ||
|
|
||
| MyAudio = create_doc( | ||
| 'MyAudio', | ||
| __base__=Audio, | ||
| title=(str, ...), | ||
| tensor=(AudioNdArray, ...), | ||
| ) | ||
|
|
||
| assert issubclass(MyAudio, BaseDocument) | ||
| assert issubclass(MyAudio, Audio) | ||
|
|
||
| """ | ||
|
|
||
| if not issubclass(__base__, BaseDocument): | ||
| raise ValueError(f'{type(__base__)} is not a BaseDocument or its subclass') | ||
|
|
||
| doc = create_model( | ||
| __model_name, | ||
| __config__=__config__, | ||
| __base__=__base__, | ||
| __module__=__module__, | ||
| __validators__=__validators__, | ||
| __cls_kwargs__=__cls_kwargs__, | ||
| __slots__=__slots__, | ||
| **field_definitions, | ||
| ) | ||
|
|
||
| return doc | ||
|
|
||
|
|
||
| def create_from_typeddict( | ||
| typeddict_cls: Type['TypedDict'], # type: ignore | ||
| **kwargs: Any, | ||
| ): | ||
| """ | ||
| Create a subclass of BaseDocument based on the fields of a `TypedDict`. This is a wrapper around pydantic's create_model_from_typeddict. | ||
AnneYang720 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| :param typeddict_cls: TypedDict class to use for the new Document class | ||
| :param kwargs: extra arguments to pass to `create_model_from_typeddict` | ||
| :return: the new Document class | ||
|
|
||
| EXAMPLE USAGE | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| from typing_extensions import TypedDict | ||
|
|
||
| from docarray import BaseDocument | ||
| from docarray.documents import Audio | ||
| from docarray.documents.helper import create_from_typeddict | ||
| from docarray.typing.tensor.audio import AudioNdArray | ||
|
|
||
|
|
||
| class MyAudio(TypedDict): | ||
| title: str | ||
| tensor: AudioNdArray | ||
|
|
||
|
|
||
| Doc = create_from_typeddict(MyAudio, __base__=Audio) | ||
|
|
||
| assert issubclass(Doc, BaseDocument) | ||
| assert issubclass(Doc, Audio) | ||
|
|
||
| """ | ||
|
|
||
| if '__base__' in kwargs: | ||
| if not issubclass(kwargs['__base__'], BaseDocument): | ||
| raise ValueError( | ||
| f'{kwargs["__base__"]} is not a BaseDocument or its subclass' | ||
| ) | ||
| else: | ||
| kwargs['__base__'] = BaseDocument | ||
|
|
||
| doc = create_model_from_typeddict(typeddict_cls, **kwargs) | ||
|
|
||
| return doc | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.