-
Notifications
You must be signed in to change notification settings - Fork 6.6k
feat(contentwarehouse): Document AI Warehouse - Add Manage Documents code samples #10578
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
85 commits
Select commit
Hold shift + click to select a range
6fa3d06
create ACL sample code
5214e96
Fix snippet-bot check
efb41b4
changes as suggested
e73fef2
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 420c547
fetch acl parametrize tests
9a35248
fetch acl parameterize tests
5a0083f
Merge branch 'main' into main
holtskinner cb2d4eb
Update noxfile_config.py
holtskinner d98dcfb
Ran Black Formatter for lint
holtskinner f4ce078
Update Request body for Document vs Project ACL
holtskinner 755db68
Addressed linter issue
holtskinner 8e4ed76
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 906b0b8
changes as suggested
1d2f433
Update contentwarehouse/snippets/fetch_acl_sample.py
dmoonat 8477d58
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 6aa7eb4
correct user_id format
2dee431
test: Update exception handling to account for product issue
holtskinner 675f428
Merge branch 'main' into main
holtskinner 64e3e82
Merge branch 'main' into main
holtskinner fea318c
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 845b148
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 414d1ee
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 7af5ee7
Document AI Warehouse: create Set ACL sample code
c6b2c04
Add exception handling
2e6a2f0
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 81afcbb
Resolve lint and permission issue
200fe7d
Merge branch 'main' into main
holtskinner 825ba8e
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 09627c8
changes as suggested
a222ca5
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat d7a66ea
add policy type
7d0c13b
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 7852cf9
resolve lint error
7600128
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 25267df
Document AI Warehouse: document schema sample code
31dc801
snippet-bot-check resolve
7f86d2a
Merge branch 'main' into main
holtskinner fc21769
chore: Ran black formatting to fix some lint errors
holtskinner fc78c21
chore: Added Auto-label for `contentwarehouse`
holtskinner 4cc438f
Merge branch 'main' into main
engelke 5664a7d
Merge branch 'main' into main
engelke 93830ea
Merge branch 'main' into main
holtskinner 114bd2b
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 728e945
fix lint
18b0436
Ran black format
holtskinner b1f7df9
Merge branch 'main' into main
holtskinner 689dc98
Merge branch 'main' into main
holtskinner 8ac0546
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 19982bb
Changes as suggested
9952dfc
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat ef1684d
Fix lint
1e26a7a
Fix lint
792b28b
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 685291d
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 1112b76
Update search sample
a258d0e
Update search test
81a814b
Ran nox -s blacken
b5a9ae8
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat aa8978e
Replace metadata dictionary to RequestMetadata instance
141781b
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 62cf123
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 07c1c61
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 5aa4390
create, delete document sample
45d29af
Modify samples
3e33128
Add get_document sample
91e6c32
Add test case
89918b8
Update delete_document
f319747
Add update_document sample
bf8b998
Ran blacken lint
60f8570
Add return types
bc6a797
update update_document sample
dc8061d
update Update_document sample and test
d57c6bb
Rename document test file
1776169
Update user_id to get input from environment variable
3f1df02
correct pylint issue
28269cc
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat a5f547b
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 93d501a
fix lint
3c05e11
Changes suggested
8b5304c
Ran nox
20e0f5e
Fix import order
a2ad678
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat 2a8cc48
changes suggested
87fbad9
Merge branch 'GoogleCloudPlatform:main' into main
dmoonat ce06ef7
changes suggested
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed
8000
to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,85 @@ | ||
# Copyright 2023 Google LLC | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
|
||
# [START contentwarehouse_create_document] | ||
|
||
from typing import Optional | ||
|
||
from google.cloud import contentwarehouse | ||
|
||
|
||
def sample_create_document( | ||
project_number: str, | ||
location: str, | ||
raw_document_path: str, | ||
raw_document_file_type: contentwarehouse.RawDocumentFileType, | ||
document_schema_id: str, | ||
user_id: str, | ||
reference_id: Optional[str] = None, | ||
) -> contentwarehouse.CreateDocumentResponse: | ||
"""Creates a document. | ||
|
||
Args: | ||
project_number: Google Cloud project number. | ||
location: Google Cloud project location. | ||
raw_document_path: Raw document file in Cloud Storage path. | ||
raw_document_file_type: Document file type | ||
https://cloud.google.com/python/docs/ | ||
reference/contentwarehouse/latest/ | ||
google.cloud.contentwarehouse_v1.types.RawDocumentFileType. | ||
document_schema_id: Unique identifier for document schema. | ||
user_id: user:YOUR_SERVICE_ACCOUNT_ID or user:USER_EMAIL. | ||
reference_id: Identifier, must be unique per project and location. | ||
Returns: | ||
Response object. | ||
""" | ||
# Create a Document Service client | ||
client = contentwarehouse.DocumentServiceClient() | ||
|
||
# Get document schema name | ||
document_schema_name = client.document_schema_path( | ||
project=project_number, location=location, document_schema=document_schema_id | ||
) | ||
|
||
# Initialize request argument(s) | ||
document = contentwarehouse.Document( | ||
raw_document_path=raw_document_path, | ||
display_name="Order Invoice", | ||
plain_text="Sample Invoice Document", | ||
raw_document_file_type=raw_document_file_type, | ||
document_schema_name=document_schema_name, | ||
reference_id=reference_id, | ||
) | ||
|
||
request_metadata = contentwarehouse.RequestMetadata( | ||
user_info=contentwarehouse.UserInfo(id=user_id) | ||
) | ||
|
||
parent = client.common_location_path(project=project_number, location=location) | ||
|
||
request = contentwarehouse.CreateDocumentRequest( | ||
parent=parent, | ||
request_metadata=request_metadata, | ||
document=document, | ||
) | ||
|
||
# Make the request | ||
response = client.create_document(request=request) | ||
|
||
return response | ||
|
||
|
||
# [END contentwarehouse_create_document] |
134 changes: 134 additions & 0 deletions
134
contentwarehouse/snippets/create_get_update_delete_document_test.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,134 @@ | ||
# # Copyright 2023 Google LLC | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
import os | ||
|
||
from contentwarehouse.snippets import create_document_sample | ||
from contentwarehouse.snippets import create_document_schema_sample | ||
from contentwarehouse.snippets import delete_document_sample | ||
from contentwarehouse.snippets import delete_document_schema_sample | ||
from contentwarehouse.snippets import get_document_sample | ||
from contentwarehouse.snippets import test_utilities | ||
from contentwarehouse.snippets import update_document_sample | ||
from google.cloud import contentwarehouse | ||
from google.cloud.contentwarehouse import Document | ||
|
||
import pytest | ||
|
||
project_id = os.environ["GOOGLE_CLOUD_PROJECT"] | ||
location = "us" | ||
raw_document_path = ( | ||
"gs://cloud-samples-data/documentai/codelabs/warehouse/order-invoice.pdf" | ||
) | ||
raw_document_file_type = contentwarehouse.RawDocumentFileType.RAW_DOCUMENT_FILE_TYPE_PDF | ||
user_id = os.environ["user_id"] | ||
reference_id = "001" | ||
|
||
|
||
@pytest.mark.dependency(name="create_schema") | ||
def test_create_document_schema(request: pytest.fixture) -> None: | ||
project_number = test_utilities.get_project_number(project_id) | ||
|
||
response = create_document_schema_sample.sample_create_document_schema( | ||
project_number=project_number, location=location | ||
) | ||
|
||
assert "display_name" in response | ||
|
||
document_schema_id = response.name.split("/")[-1] | ||
|
||
request.config.cache.set("document_schema_id", document_schema_id) | ||
|
||
|
||
@pytest.mark.dependency(name="create_doc", depends=["create_schema"]) | ||
def test_create_document(request: pytest.fixture) -> None: | ||
project_number = test_utilities.get_project_number(project_id) | ||
|
||
document_schema_id = request.config.cache.get("document_schema_id", None) | ||
|
||
response = create_document_sample.sample_create_document( | ||
project_number=project_number, | ||
location=location, | ||
raw_document_path=raw_document_path, | ||
raw_document_file_type=raw_document_file_type, | ||
document_schema_id=document_schema_id, | ||
user_id=user_id, | ||
reference_id=reference_id, | ||
) | ||
|
||
assert "document" in response | ||
|
||
request.config.cache.set("document_name", response.document.name) | ||
|
||
|
||
@pytest.mark.dependency(name="get_doc", depends=["create_doc"]) | ||
def test_get_document(request: pytest.fixture) -> None: | ||
document_name = request.config.cache.get("document_name", None) | ||
|
||
response = get_document_sample.sample_get_document( | ||
document_name=document_name, user_id=user_id | ||
) | ||
|
||
assert "name" in response | ||
request.config.cache.set( | ||
"document", | ||
Document.to_json( | ||
response, | ||
including_default_value_fields=False, | ||
preserving_proto_field_name=False, | ||
), | ||
) | ||
|
||
|
||
@pytest.mark.dependency(name="update_doc", depends=["get_doc"]) | ||
def test_update_document(request: pytest.fixture) -> None: | ||
document_name = request.config.cache.get("document_name", None) | ||
document_json = request.config.cache.get("document", None) | ||
|
||
document = Document.from_json(document_json) | ||
|
||
response = update_document_sample.sample_update_document( | ||
document_name=document_name, | ||
document=document, | ||
user_id=user_id, | ||
) | ||
|
||
assert "document" in response | ||
|
||
|
||
@pytest.mark.dependency(name="delete_doc", depends=["update_doc"]) | ||
def test_delete_document(request: pytest.fixture) -> None: | ||
document_name = request.config.cache.get("document_name", None) | ||
|
||
response = delete_document_sample.sample_delete_document( | ||
document_name=document_name, user_id=user_id | ||
) | ||
|
||
assert response is None | ||
|
||
|
||
@pytest.mark.dependency(name="delete_schema", depends=["delete_doc"]) | ||
def test_delete_document_schema(request: pytest.fixture) -> None: | ||
project_number = test_utilities.get_project_number(project_id) | ||
|
||
document_schema_id = request.config.cache.get("document_schema_id", None) | ||
|
||
response = delete_document_schema_sample.sample_delete_document_schema( | ||
project_number=project_number, | ||
location=location, | ||
document_schema_id=document_schema_id, | ||
) | ||
|
||
assert response is None |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
# # Copyright 2023 Google LLC | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
|
||
# [START contentwarehouse_delete_document] | ||
|
||
from google.cloud import contentwarehouse | ||
|
||
|
||
def sample_delete_document(document_name: str, user_id: str) -> None: | ||
"""Deletes a document. | ||
|
||
Args: | ||
document_name: The resource name of the document. | ||
Format: 'projects/{project_number}/ | ||
locations/{location}/documents/{document_id}'. | ||
user_id: user:YOUR_SERVICE_ACCOUNT_ID or user:USER_EMAIL. | ||
Returns: | ||
None, if operation is successful. | ||
""" | ||
|
||
# Create a client | ||
client = contentwarehouse.DocumentServiceClient() | ||
|
||
request_metadata = contentwarehouse.RequestMetadata( | ||
user_info=contentwarehouse.UserInfo(id=user_id) | ||
) | ||
|
||
# Initialize request argument(s) | ||
request = contentwarehouse.DeleteDocumentRequest( | ||
name=document_name, request_metadata=request_metadata | ||
) | ||
|
||
# Make the request | ||
response = client.delete_document(request=request) | ||
|
||
return response | ||
|
||
|
||
# [END contentwarehouse_delete_document] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
# # Copyright 2023 Google LLC | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
|
||
# [START contentwarehouse_get_document] | ||
|
||
from google.cloud import contentwarehouse | ||
|
||
|
||
def sample_get_document(document_name: str, user_id: str) -> contentwarehouse.Document: | ||
"""Gets a document. | ||
|
||
Args: | ||
document_name: The resource name of the document. | ||
Format: 'projects/{project_number}/ | ||
locations/{location}/documents/{document_id}'. | ||
user_id: user:YOUR_SERVICE_ACCOUNT_ID or user:USER_EMAIL. | ||
Returns: | ||
Response object | ||
""" | ||
# Create a client | ||
client = contentwarehouse.DocumentServiceClient() | ||
|
||
request_metadata = contentwarehouse.RequestMetadata( | ||
user_info=contentwarehouse.UserInfo(id=user_id) | ||
) | ||
|
||
# Initialize request argument(s) | ||
request = contentwarehouse.GetDocumentRequest( | ||
name=document_name, request_metadata=request_metadata | ||
) | ||
|
||
# Make the request | ||
response = client.get_document(request=request) | ||
|
||
return response | ||
|
||
|
||
# [END contentwarehouse_get_document] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
# Copyright 2023 Google LLC | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
|
||
# [START contentwarehouse_update_document] | ||
|
||
from google.cloud import contentwarehouse | ||
|
||
|
||
def sample_update_document( | ||
document_name: str, document: contentwarehouse.Document, user_id: str | ||
) -> contentwarehouse.CreateDocumentResponse: | ||
"""Updates a document. | ||
|
||
Args: | ||
document_name: The resource name of the document. | ||
Format: 'projects/{project_number}/ | ||
locations/{location}/documents/{document_id}'. | ||
document: Document AI Warehouse Document object.. | ||
user_id: user_id: user:YOUR_SERVICE_ACCOUNT_ID or user:USER_EMAIL. | ||
Returns: | ||
Response object. | ||
""" | ||
# Create a client | ||
client = contentwarehouse.DocumentServiceClient() | ||
|
||
# Update document fields | ||
# For fields which can be updated, refer | ||
# https://cloud.google.com/python/docs/reference/contentwarehouse/ | ||
# latest/google.cloud.contentwarehouse_v1.types.Document | ||
document.display_name = "Updated Order Invoice" | ||
holtskinner marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
request_metadata = contentwarehouse.RequestMetadata( | ||
user_info=contentwarehouse.UserInfo(id=user_id) | ||
) | ||
|
||
request = contentwarehouse.UpdateDocumentRequest( | ||
name=document_name, document=document, request_metadata=request_metadata | ||
) | ||
|
||
# Make the request | ||
response = client.update_document(request=request) | ||
|
||
return response | ||
|
||
|
||
# [END contentwarehouse_update_document] |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.