8000 feat: add nested access for document array by anna-charlotte · Pull Request #956 · docarray/docarray · GitHub
[go: up one dir, main page]
Skip to content

feat: add nested access for document array #956

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
JoanFM merged 26 commits into from
Dec 23, 2022
Merged

feat: add nested access for document array #956

JoanFM merged 26 commits into from
Dec 23, 2022

Conversation

@anna-charlotte
Copy link
Contributor
@anna-charlotte anna-charlotte commented Dec 19, 2022 •
edited
Loading

Signed-off-by: anna-charlotte charlotte.gerhaher@jina.ai

Nested access in V2 is possible by calling the nested attributes you want to access on a DocumentArray, e.g. da.image.title. This does not work for nested DocumentArrays though:

class Author(Document):
    name: str


class Book(Document):
    authors: Optional[DocumentArray[Author]]
    content: str


da = DocumentArray[Book](
    [
        Book(
            authors=DocumentArray[Author]([Author(name='Jenny'), Author(name='Ben')]),
            content=f'hello{i}'
        ) for i in range(10)
    ]
)

authors = da.authors.name  # does not work since da.authors is a list of DocumentArrays and does not have

# instead do this:
authors = da.traverse_flat(access_path='authors.name')  # returns List of strings, with len = 20

traverse_flat takes the access path and returns a list of those attributes, flattened on the first level.
da.traverse_flat(access_path='authors.name') therefore returns a list of 20 strings instead of list of ten lists of 2 names each.

  • add traverse_flat + test
  • check and update documentation, if required. See guide

@anna-charlotte anna-charlotte linked an issue Dec 20, 2022 that may be closed by this pull request
Investigate nested access from DocumentArray ("access paths") #957
Closed
@anna-charlotte anna-charlotte mentioned this pull request Dec 20, 2022
Closed
47 tasks
anna-charlotte added 9 commits December 20, 2022 15:00
test: add test for traverse flat
47f0837 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
test: update test for traverse flat
434ab27 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
chore: change ruff line length from 88 to 100
e4f8486 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
feat: add traverse mixin with traverse flat
6cbacb4 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
chore: revert pyproject toml ruff
1f00d9f 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
docs: fix docs in traverse mixin
9ed50c9 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
c651fc8 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
a2a18b6 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
fd0f954 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
anna-charlotte added 9 commits December 20, 2022 15:14
fix: add allmixins
6a52969 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
8c7e84a 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
bf0e617 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
fe3fd8f 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
refactor: move traverse flat to abstract array
85c69bf 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: traverse flat for stack mode
e0d9f1a 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
docs: add docs for stacked da
130dea2 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
feat: traversal flat for stacked and unstacked in abstract array
201a42d 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
feat: move traverse_flat to array and stacked array instead of abstra…
af74fb7 
…ct array

Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
anna-charlotte added 4 commits December 22, 2022 18:00
fix: remove size for instance check
cdb49f5 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: change instance check
c4038cb 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: mypy type hints
ee2a0e3 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: torchtensor and ndarray instead of only tensor
aab4c6a 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
JoanFM
JoanFM requested changes Dec 23, 2022
View reviewed changes
fix: add type hint and len check in flatten
710b927 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
JoanFM
JoanFM reviewed Dec 23, 2022
View reviewed changes
anna-charlotte added 3 commits December 23, 2022 11:42
refactor: rename flatten to flatten one level
fe8bf3c 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
test: add test for flatten one level
baa1c31 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
fix: revert change in array get item
459c4dd 
Signed-off-by: anna-charlotte <charlotte.gerhaher@jina.ai>
@anna-charlotte anna-charlotte marked this pull request as ready for review December 23, 2022 10:47
@github-actions
Copy link
github-actions bot commented Dec 23, 2022

📝 Docs are deployed on https://ft-feat-nested-access--jina-docs.netlify.app 🎉

@JoanFM
Copy link
Member
JoanFM commented Dec 23, 2022

In the v1, we decided to hide traverse flat behind the [@] accessors, are we expecting to expose this now?

@anna-charlotte
Copy link
Contributor Author
anna-charlotte commented Dec 23, 2022

@Joan We haven't made the final decision for this yet. For now, we decided on exposing it, but want to reevaluate this when Sami and Johannes are back.

@JoanFM JoanFM merged commit 6e66c31 into feat-rewrite-v2 Dec 23, 2022
@JoanFM JoanFM deleted the feat-nested-access branch December 23, 2022 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JoanFM JoanFM JoanFM approved these changes

Assignees

No one assigned

Labels

area/core area/testing component/array size/m

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Investigate nested access from DocumentArray ("access paths")

3 participants

@anna-charlotte @JoanFM
0