Intent of this template

The intent of this repository is to help you get started with implementing new Pydra environments, within the pydra.environments.<yourworkerpackagename> namespace.

To use this template

Click on new repo.
Select this template from the repository template drop down list.
Give your repo a name.
Once the repo is created and cloned, search for CHANGEME (grep -rn CHANGEME . ) and replace with appropriate name.
Rename the namespace package root directory to replace CHANGEME with the name of the package:
- pydra/environments/CHANGEME
Add your new worker class to pydra/environments/<package-name>/__init__.py
You may want to initialize a Sphinx docs directory.
Review the workflow in .github/workflows/pythonpackage.yml. Testing editable installations is probably not useful unless you are reconfiguring namespace packages.
Update this README after creating the new repository.

Features of this template

Tag-based versioning

The setuptools_scm tool allows for versioning based on the most recent git tag. The release process can thus be:

git tag -a 1.0.0
python -m build
twine upload dist/*

Note that uploading to PyPI is done via Continuous integration when a tag is pushed to the repository, so only the first step needs to be donne manually.

Note also that we assume tags will be version numbers and not be prefixed with v or some other string. See setuptools_scm documentation for alternative configurations.

Namespace packages

Namespace packages allow multiple packages to be installed in such a way that they can be imported from a common namespace. In this case, the base namespace is pydra.environments, and each worker package installs a new module, such as pydra.environments.afni.

To do correctly, worker packages must not interfere with files higher in the hierarchy. This includes adding __init__.py files in either pydra or pydra/environments.

Tests

This package comes with a default set of test modules, and we encourage users to use pytest. Tests can be discovered and run using:

pytest --doctest-modules pydra/environments/*

Continuous integration

This template uses GitHub Actions to run tests. To simulate several plausible development or installation environments, we test over all Python versions supported by Pydra, and install Pydra and the current package in both standard and editable modes.

The combination of standard/editable is in particular designed to ensure that namespace packaging does not break. We strongly recommend keeping these tests in place for this reason, as one non-compliant package can potentially affect Pydra or other worker packages.

In addition to verifying installations do not break or conflict, pytest is run on the package, including all tests found in test/ directories and doctests.

Finally, packages are built and uploaded as artifacts for inspection. When a tag is pushed, the packages are uploaded to PyPI if a valid API token is placed in the repository secrets.

Contributing to this template

This template repository is periodically updated as we establish and best practices for worker packages. We welcome contributions from the community to help set a solid baseline for all worker packages.

Sample post-setup README contents

For developers

Install repo in developer mode from the source directory. It is also useful to install pre-commit to take care of styling via black:

pip install -e .[dev]
pre-commit install