8000 BigQuery - REST samples for adding labels. (#764) · dcaballerod/python-docs-samples@1822e4e · GitHub
[go: up one dir, main page]
Skip to content

Commit 1822e4e

Browse files
tswastJon Wayne Parrott
tswast
authored and
Jon Wayne Parrott
committed
BigQuery - REST samples for adding labels. (GoogleCloudPlatform#764)
See docs: https://cloud.google.com/bigquery/docs/labeling-datasets
1 parent bcc95af commit 1822e4e

File tree

5 files changed

+301
-0
lines changed

5 files changed

+301
-0
lines changed

‎bigquery/rest/README.rst

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
.. This file is automatically generated. Do not edit this file directly.
2+
3+
Google BigQuery Python Samples
4+
===============================================================================
5+
6+
This directory contains samples for Google BigQuery. `Google BigQuery`_ is Google's fully managed, petabyte scale, low cost analytics data warehouse. BigQuery is NoOps—there is no infrastructure to manage and you don't need a database administrator—so you can focus on analyzing data to find meaningful insights, use familiar SQL, and take advantage of our pay-as-you-go model.
7+
8+
9+
10+
11+
.. _Google BigQuery: https://cloud.google.com/bigquery/docs
12+
13+
Setup
14+
-------------------------------------------------------------------------------
15+
16+
17+
Authentication
18+
++++++++++++++
19+
20+
Authentication is typically done through `Application Default Credentials`_,
21+
which means you do not have to change the code to authenticate as long as
22+
your environment has credentials. You have a few options for setting up
23+
authentication:
24+
25+
#. When running locally, use the `Google Cloud SDK`_
26+
27+
.. code-block:: bash
28+
29+
gcloud beta auth application-default login
30+
31+
32+
#. When running on App Engine or Compute Engine, credentials are already
33+
set-up. However, you may need to configure your Compute Engine instance
34+
with `additional scopes`_.
35+
36+
#. You can create a `Service Account key file`_. This file can be used to
37+
authenticate to Google Cloud Platform services from any environment. To use
38+
the file, set the ``GOOGLE_APPLICATION_CREDENTIALS`` environment variable to
39+
the path to the key file, for example:
40+
41+
.. code-block:: bash
42+
43+
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service_account.json
44+
45+
.. _Application Default Credentials: https://cloud.google.com/docs/authentication#getting_credentials_for_server-centric_flow
46+
.. _additional scopes: https://cloud.google.com/compute/docs/authentication#using
47+
.. _Service Account key file: https://developers.google.com/identity/protocols/OAuth2ServiceAccount#creatinganaccount
48+
49+
Install Dependencies
50+
++++++++++++++++++++
51+
52+
#. Install `pip`_ and `virtualenv`_ if you do not already have them.
53+
54+
#. Create a virtualenv. Samples are compatible with Python 2.7 and 3.4+.
55+
56+
.. code-block:: bash
57+
58+
$ virtualenv env
59+
$ source env/bin/activate
60+
61+
#. Install the dependencies needed to run the samples.
62+
63+
.. code-block:: bash
64+
65+
$ pip install -r requirements.txt
66+
67+
.. _pip: https://pip.pypa.io/
68+
.. _virtualenv: https://virtualenv.pypa.io/
69+
70+
Samples
71+
-------------------------------------------------------------------------------
72+
73+
Label a dataset or table
74+
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
75+
76+
77+
78+
To run this sample:
79+
80+
.. code-block:: bash
81+
82+
$ python labels.py
83+
84+
usage: labels.py [-h] [--project_id PROJECT_ID] [--table_id TABLE_ID]
85+
dataset_id label_key label_value
86+
87+
Application to add or modify a label on a BigQuery dataset or table.
88+
89+
positional arguments:
90+
dataset_id BigQuery dataset ID.
91+
label_key Key for new/modified label.
92+
label_value Value for new/modified label.
93+
94+
optional arguments:
95+
-h, --help show this help message and exit
96+
--project_id PROJECT_ID
97+
Google Cloud project ID. If not set, uses a default
98+
value from the environment.
99+
--table_id TABLE_ID BigQuery table ID. If present, a label is added to the
100+
specified table instead of the dataset.
101+
102+
103+
104+
105+
.. _Google Cloud SDK: https://cloud.google.com/sdk/

‎bigquery/rest/README.rst.in

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# This file is used to generate README.rst
2+
3+
product:
4+
name: Google BigQuery
5+
short_name: BigQuery
6+
url: https://cloud.google.com/bigquery/docs
7+
description: >
8+
`Google BigQuery`_ is Google's fully managed, petabyte scale, low cost
9+
analytics data warehouse. BigQuery is NoOps—there is no infrastructure to
10+
manage and you don't need a database administrat 42E6 or—so you can focus on
11+
analyzing data to find meaningful insights, use familiar SQL, and take
12+
advantage of our pay-as-you-go model.
13+
14+
setup:
15+
- auth
16+
- install_deps
17+
18+
samples:
19+
- name: Label a dataset or table
20+
show_help: true
21+
file: labels.py
22+
23+
cloud_client_library: false

‎bigquery/rest/labels.py

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
#!/usr/bin/env python
2+
3+
# Copyright 2016, Google, Inc.
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
"""Application to add or modify a label on a BigQuery dataset or table."""
17+
18+
import argparse
19+
20+
import google.auth
21+
import google.auth.transport.requests
22+
23+
24+
def label_dataset(dataset_id, label_key, label_value, project_id=None):
25+
"""Add or modify a label on a dataset."""
26+
# Authenticate requests using Google Application Default credentials.
27+
credentials, default_project_id = google.auth.default(
28+
scopes=['https://www.googleapis.com/auth/bigquery'])
29+
session = google.auth.transport.requests.AuthorizedSession(credentials)
30+
31+
if project_id is None:
32+
project_id = default_project_id
33+
34+
# Send a PATCH request to add or modify a label.
35+
url_format = (
36+
'https://www.googleapis.com/bigquery/v2/'
37+
'projects/{project_id}/datasets/{dataset_id}')
38+
response = session.patch(
39+
url_format.format(project_id=project_id, dataset_id=dataset_id),
40+
params={'fields': 'labels'},
41+
json={
42+
'labels': {
43+
label_key: label_value,
44+
}
45+
})
46+
47+
# Check the response for errors.
48+
response.raise_for_status()
49+
50+
# Print the new label value from the response.
51+
labels = response.json()['labels']
52+
print(
53+
'Updated label "{}" with value "{}"'.format(
54+
label_key,
55+
labels[label_key]))
56+
57+
58+
def label_table(dataset_id, table_id, label_key, label_value, project_id=None):
59+
"""Add or modify a label on a table."""
60+
# Authenticate requests using Google Application Default credentials.
61+
credentials, default_project_id = google.auth.default(
62+
scopes=['https://www.googleapis.com/auth/bigquery'])
63+
session = google.auth.transport.requests.AuthorizedSession(credentials)
64+
65+
if project_id is None:
66+
project_id = default_project_id
67+
68+
# Send a PATCH request to add or modify a label.
69+
url_format = (
70+
'https://www.googleapis.com/bigquery/v2/'
71+
'projects/{project_id}/datasets/{dataset_id}/tables/{table_id}')
72+
response = session.patch(
73+
url_format.format(
74+
project_id=project_id,
75+
dataset_id=dataset_id,
76+
table_id=table_id),
77+
params={'fields': 'labels'},
78+
json={
79+
'labels': {
80+
label_key: label_value,
81+
}
82+
})
83+
84+
# Check the response for errors.
85+
response.raise_for_status()
86+
87+
# Print the new label value from the response.
88+
labels = response.json()['labels']
89+
print(
90+
'Updated label "{}" with value "{}"'.format(
91+
label_key,
92+
labels[label_key]))
93+
94+
95+
if __name__ == '__main__':
96+
parser = argparse.ArgumentParser(
97+
description=__doc__,
98+
formatter_class=argparse.RawDescriptionHelpFormatter)
99+
parser.add_argument('dataset_id', help='BigQuery dataset ID.')
100+
parser.add_argument('label_key', help='Key for new/modified label.')
101+
parser.add_argument('label_value', help='Value for new/modified label.')
102+
parser.add_argument(
103+
'--project_id',
104+
help=(
105+
'Google Cloud project ID. '
106+
'If not set, uses a default value from the environment.'),
107+
default=None)
108+
parser.add_argument(
109+
'--table_id',
110+
help=(
111+
'BigQuery table ID. '
112+
'If present, a label is added to the specified table instead of '
113+
'the dataset.'),
114+
default=None)
115+
116+
args = parser.parse_args()
117+
118+
if args.table_id is None:
119+
label_dataset(
120+
args.dataset_id,
121+
args.label_key,
122+
args.label_value,
123+
project_id=args.project_id)
124+
else:
125+
label_table(
126+
args.dataset_id,
127+
args.table_id,
128+
args.label_key,
129+
args.label_value,
130+
project_id=args.project_id)

‎bigquery/rest/labels_test.py

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Copyright 2016, Google, Inc.
2+
# Licensed under the Apache License, Version 2.0 (the "License");
3+
# you may not use this file except in compliance with the License.
4+
# You may obtain a copy of the License at
5+
#
6+
# http://www.apache.org/licenses/LICENSE-2.0
7+
#
8+
# Unless required by applicable law or agreed to in writing, software
9+
# distributed under the License is distributed on an "AS IS" BASIS,
10+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
# See the License for the specific language governing permissions and
12+
# limitations under the License.
13+
14+
from labels import label_dataset, label_table
15+
16+
17+
def test_label_dataset(cloud_config, capsys):
18+
label_dataset(
19+
'test_dataset',
20+
'environment',
21+
'test',
22+
project_id=cloud_config.project)
23+
24+
out, _ = capsys.readouterr()
25+
result = out.split('\n')[0]
26+
27+
assert 'Updated label "environment" with value "test"' in result
28+
29+
30+
def test_label_table(cloud_config, capsys):
31+
label_table(
32+
'test_dataset',
33+
'test_table',
34+
'data-owner',
35+
'my-team',
36+
project_id=cloud_config.project)
37+
38+
out, _ = capsys.readouterr()
39+
result = out.split('\n')[0]
40+
41+
assert 'Updated label "data-owner" with value "my-team"' in result

‎bigquery/rest/requirements.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
google-auth==0.5.0
2+
requests==2.12.5

0 commit comments

Comments
 (0)
0