8000 dataflow: new minimal gpu samples (#6096) · HichemELB/python-docs-samples@7eb1e09 · GitHub

Commit 7eb1e09

authored

dataflow: new minimal gpu samples (GoogleCloudPlatform#6096)

* dataflow: minimal gpu examples * added tests * update dockerignore and cloudignore * remove old config * update header year * removed unused import * add test * make image name unique * fix image name * fix resource name issues * fix lint issues * fix underscore_name * wait for jobs * fix test requirements * fix image prefix * get job status for wait * use correct region * make jobs list not region dependent * use uuid for image tags * update title * fix run command * remove region from job get call * launch jobs from fixtures * fix resource names * use unique images * fix lint issues * pass --no-source when running the job * more logging and changed region * simplified type hints * get all jobs and update wait job status * update header years * tune logging * fix tests * made logging more explicit * use disk_size_gb * use larger disk_size_gb for landsat * list all jobs * wait before querying * include python version on resource names * update beam version * use 30m timeout * more debugging and increase waiting time * update bq query * use shorter flag name alias * made logs clearer * fix typo * add more logging * wait longer for table to be created * remove spurious f-string * add disk size * updated beam version * add comments * add missing fixture * decrease waiting time * increase waiting time * provide staging location * drain before cancel * adjust wait time * adjust timeout * adjust timeout * improve error messages * small refactorings * add missing fixture * updated timeout * containerize gpu tests * remove unused import * reverted streaming-beam changes * updated image versions * restore old sample to avoid breaking docs * update copyright year * fix lint issues * renamed test file * update beam version * renamed test files * adjust to conftest behaving differently with tests in current directory * moved new samples to another directory * simplified instructions * add gcloudignore to reduce test time * reset gpu-workers entirely * test in a single Python version * run subprocess as daemon * update beam version * install g++ to compile google-cloud-profiler * fix build dependencies * fix build dependencies * fix build dependencies * adjust timeout

1 parent 83b915c commit 7eb1e09Copy full SHA for 7eb1e09

32 files changed

+1828

-1

lines changed

dataflow
- flex-templates
  - __init__.py
- gpu-examples
  - conftest.py
  - pytorch-minimal
  - tensorflow-landsat
  - tensorflow-minimal

32 files changed

+1828

-1

lines changed

`‎dataflow/flex-templates/init.py`

Original file line number	Diff line number	Diff line change
`@@ -1 +0,0 @@`
`1`		`-`

Comments

(0)

0