10000 storage: S3 SDK sample to list buckets (#2022) · devlance/python-docs-samples@6ea559f · GitHub
[go: up one dir, main page]
Skip to content

Commit 6ea559f

Browse files
frankynengelke
frankyn
authored and
engelke
committed
storage: S3 SDK sample to list buckets (GoogleCloudPlatform#2022)
* Add canonical boto3 sample for review. * Update path, add README, and add system test to sample. * Add newline * Fix wording * Fix typos * Fix region tag names * Update variable name for interop client * Update README to not include Google Auth * Fix quotes and update environment variable * Address feedback * Update readme * Add HMAC credentials to secrets.tar.enc * Fix lint issue
1 parent 4e994b0 commit 6ea559f

File tree

6 files changed

+191
-0
lines changed

6 files changed

+191
-0
lines changed

‎storage/s3-sdk/README.rst

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
.. This file is automatically generated. Do not edit this file directly.
2+
3+
Google Cloud Storage Python Samples
4+
===============================================================================
5+
6+
.. image:: https://gstatic.com/cloudssh/images/open-btn.png
7+
:target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=storage/s3-sdk/README.rst
8+
9+
10+
This directory contains samples for Google Cloud Storage. `Google Cloud Storage`_ provides support for S3 API users through the GCS XML API.
11+
Learn more about migrating data from S3 to GCS at https://cloud.google.com/storage/docs/migrating.
12+
13+
14+
15+
16+
.. _Google Cloud Storage: https://cloud.google.com/storage/docs
17+
18+
Setup
19+
-------------------------------------------------------------------------------
20+
21+
22+
Install Dependencies
23+
++++++++++++++++++++
24+
25+
#. Clone python-docs-samples and change directory to the sample directory you want to use.
26+
27+
.. code-block:: bash
28+
29+
$ git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
30+
31+
#. Install `pip`_ and `virtualenv`_ if you do not already have them. You may want to refer to the `Python Development Environment Setup Guide`_ for Google Cloud Platform for instructions.
32+
33+
.. _Python Development Environment Setup Guide:
34+
https://cloud.google.com/python/setup
35+
36+
#. Create a virtualenv. Samples are compatible with Python 2.7 and 3.4+.
37+
38+
.. code-block:: bash
39+
40+
$ virtualenv env
41+
$ source env/bin/activate
42+
43+
#. Install the dependencies needed to run the samples.
44+
45+
.. code-block:: bash
46+
47+
$ pip install -r requirements.txt
48+
49+
.. _pip: https://pip.pypa.io/
50+
.. _virtualenv: https://virtualenv.pypa.io/
51+
52+
Samples
53+
-------------------------------------------------------------------------------
54+
55+
List GCS Buckets using S3 SDK
56+
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
57+
58+
.. image:: https://gstatic.com/cloudssh/images/open-btn.png
59+
:target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=storage/s3-sdk/list_gcs_buckets.py,storage/s3-sdk/README.rst
60+
61+
62+
63+
64+
To run this sample:
65+
66+
.. code-block:: bash
67+
68+
$ python list_gcs_buckets.py
69+
70+
usage: list_gcs_buckets.py [-h] google_access_key_id google_access_key_secret
71+
72+
positional arguments:
73+
google_access_key_id Your Cloud Storage HMAC Access Key ID.
74+
google_access_key_secret
75+
Your Cloud Storage HMAC Access Key Secret.
76+
77+
optional arguments:
78+
-h, --help show this help message and exit
79+
80+
81+
82+
83+
84+
.. _Google Cloud SDK: https://cloud.google.com/sdk/

‎storage/s3-sdk/README.rst.in

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# This file is used to generate README.rst
2+
3+
product:
4+
name: Google Cloud Storage
5+
short_name: Cloud Storage
6+
url: https://cloud.google.com/storage/docs
7+
description: >
8+
`Google Cloud Storage`_ provides support for S3 API users through the
9+
GCS XML API.
10+
11+
Learn more about migrating data from S3 to GCS at https://cloud.google.com/storage/docs/migrating.
12+
13+
setup:
14+
- install_deps
15+
16+
samples:
17+
- name: List GCS Buckets using S3 SDK
18+
file: list_gcs_buckets.py
19+
show_help: true
20+
21+
cloud_client_library: false
22+
23+
folder: storage/s3-sdk

‎storage/s3-sdk/list_gcs_buckets.py

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
#!/usr/bin/env python
2+
3+
# Copyright 2019 Google, Inc.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
import argparse
18+
# [START storage_s3_sdk_list_buckets]
19+
import boto3
20+
21+
22+
def list_gcs_buckets(google_access_key_id, google_access_key_secret):
23+
"""Lists all GCS buckets using boto3 SDK"""
24+
# Create a new client and do the following:
25+
# 1. Change the endpoint URL to use the
26+
# Google Cloud Storage XML API endpoint.
27+
# 2. Use Cloud Storage HMAC Credentials.
28+
client = boto3.client("s3", region_name="auto",
29+
endpoint_url="https://storage.googleapis.com",
30+
aws_access_key_id=google_access_key_id,
31+
aws_secret_access_key=google_access_key_secret)
32+
33+
# Call GCS to list current buckets
34+
response = client.list_buckets()
35+
36+
# Print bucket names
37+
print("Buckets:")
38+
for bucket in response["Buckets"]:
39+
print(bucket["Name"])
40+
# [END storage_s3_sdk_list_buckets]
41+
42+
43+
if __name__ == "__main__":
44+
parser = argparse.ArgumentParser(
45+
description=__doc__,
46+
formatter_class=argparse.RawDescriptionHelpFormatter)
47+
parser.add_argument("google_access_key_id",
48+
help="Your Cloud Storage HMAC Access Key ID.")
49+
parser.add_argument("google_access_key_secret",
50+
help="Your Cloud Storage HMAC Access Key Secret.")
51+
52+
args = parser.parse_args()
53+
54+
list_gcs_buckets(google_access_key_id=args.google_access_key_id,
55+
google_access_key_secret=args.google_access_key_secret)

‎storage/s3-sdk/list_gcs_buckets_test.py

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Copyright 2019 Google, Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import os
16+
17+
import list_gcs_buckets
18+
19+
BUCKET = os.environ["GOOGLE_CLOUD_PROJECT_S3_SDK"]
20+
KEY_ID = os.environ["STORAGE_HMAC_ACCESS_KEY_ID"]
21+
SECRET_KEY = os.environ["STORAGE_HMAC_ACCESS_SECRET_KEY"]
22+
23+
24+
def test_list_blobs(capsys):
25+
list_gcs_buckets.list_gcs_buckets(google_access_key_id=KEY_ID,
26+
google_access_key_secret=SECRET_KEY)
27+
out, _ = capsys.readouterr()
28+
assert BUCKET in out

‎storage/s3-sdk/requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
boto3==1.9.38

‎testing/secrets.tar.enc

0 Bytes
Binary file not shown.

0 commit comments

Comments
 (0)
0