prepare utils for pypi distribution #5848
Conversation
eb3c973 to
b2c68d6
Compare
| @@ -6,7 +6,7 @@ This package is used internally to unit test the validity of all Cloud Composer | |||
|
|
|||
| Add the following to your `requirements-test.txt` file: | |||
|
|
|||
| `git+https://github.com/GoogleCloudPlatform/python-docs-samples.git#egg=dag_test_utils&subdirectory=composer/dag_test_utils` | |||
| `cloud_composer_dag_test_utils` | |||
There was a problem hiding this comment.
should google be in the package name or is this agnostic to our stuffs?
There was a problem hiding this comment.
At the moment, it's agnostic to our stuff, though I could envision it being expanded to be more composer-y in the future. I wasn't sure if there were restrictions on using google in the package name - I'm happy to add it if you think it's appropriate here
There was a problem hiding this comment.
I could go either way on this. I tend to think being upfront that "this is a google package" is an alright thing, but could also think that by not including google in the name it is more likely to be 'hidden' when folks search for things. :)
There was a problem hiding this comment.
Okay! I think I'll keep it as is for now for that 'hidden' reason
Description
I redid the setup.py with a better, more descriptive name, and added a pyproject.toml. I haven't added it to PyPI yet in case we wanted to do something like change the name, but to test locally, you can
composer/dag_test_utilspython3 -m build(you might have topip install build)../workflowspip install ../dag_test_utils/dist/cloud_composer_dag_test_utils-0.0.1.tar.gzimport internal_unit_testing- you may see some deprecation warnings, and those are okay. They're part of underlying Airflow and will be resolved in the next release. If you see anyModuleNotFoundErrorerrors, we should talk because I probably got something wrong.requirements-test.txtand replace the gross URL with../dag_test_utils/dist/cloud_composer_dag_test_utils-0.0.1.tar.gz, then runnox -s py-3.8 -- quickstart_test.py(or any of the tests, or the whole test suite - I just suggest doing one for speed reasons because the pip install of Airflow is slow)I have requested a pywindows cloudtop and will test these distributions on windows and linux as well, and I have properly registered a team account on PyPI for the Cloud Composer DPEs per the OSPO instructions.
Preview of Coming Attractions
Once this is merged, I'll
Note: It's a good idea to open an issue first for discussion.
Checklist
nox -s py-3.6(see Test Environment Setup)nox -s lint(see Test Environment Setup)