Description
Description
Used abbreviation: dependency (dependencies) - dep (deps)
Our policy regarding the deprecation of different upstream and downstream dependencies has been discussed numerous times already in multiple PRs (#4451, #4100, #4259, #4430, #4371, #3820, #3117, #1629, #4356 (comment)). I have tried to put all the considerations into a single place, such that we can review, align, and improve our policies.
As I see it, the reason behind dependency version control is to maintain the balance of:
- Number of supported platforms. In the ideal case, the aim should be to support all possible platforms and all versions of the deps. In reality, the support is as wide as feasible (in the sense of both the development and the maintenance costs).
- Freedom to use the latest upstream features to develop the library.
- Dependency conflict handling. In the Python ecosystem comprehensive integration tests are rare, especially for the old versions of packages. I would say on average the developers go with a reasonable effort rule and test against the fixed/minimal and the latest versions of the deps. Breaking changes in those deps are expected to happen in the major releases, however, they are common also in minor.
Above said, the support cycle comprises the following:
- Minimal version policy. The one we tweak the most.
- Maximal version policy. We allow the latest stable version, no nightly, no pre-releases. Looks good to me and I have no suggestions here.
- Policy on the buggy/conflicting version. We blacklist minor/major versions based on the CI and the issue reports. Most likely, we do not have all the present conflicts covered. No suggestions either.
The deps of skimage can be seen as of the following kinds:
- Hardware platforms
- Operating systems
- Python distributions
- Python
- Mandatory build/install/run deps
- Optional run deps
- Documentation build/run deps
- Testing deps
- Downstream deps (packages that depend on
scikit-image).
Ideally, every item from the above should have a designated policy for the version control. Current policies (based on my knowledge) and my suggestions are below:
- [hardware] Status: Full support for x86-32 and x86-64. Plans for the deprecation of x86, discussion on the support of ARM/32 and ARM/64. Policy: Changes are rare - decide on a case-by-case basis.
- [os] Status: Full support for main desktop OSs (Linux, MacOS, Windows), and more or less all of their major distributions (Ubuntu, Archlinux, CentOS, …, Windows 7, Windows 10). No plans for deprecation. Policy: We do not explicitly condition on the OS versions (i.e. Windows XP/7/8/10, Linux kernel version, MacOS release). Changes are rare - decide on a case-by-case basis.
- [distro] Status: Official community support for
pip,condaon Linux, MacOS, and Windows. Community support for Windows, Debian. No direct support for commercial/third-party distros we have limited control over (e.g. brew on MacOS, launchpad repos on Ubuntu, AUR on Archlinux, Canopy, etc). Policy: Changes are rare - decide on a case-by-case basis. - [python] Status: Full support for CPython 3.6, 3.7, 3.8 - all officially maintained stable versions. Jython, etc are not officially supported. Why: Breaking changes in Python 2to3 transition, new useful language features in Python 3.6, 3.7. Policy: case-by-case. Proposal: Strictly follow NEP 29 adopted by
numpy,scipy,matplotlib(support all minor versions of Python released within the 42 months before the anticipated release date with a minimum of 2 minor versions of Python) https://github.com/numpy/numpy/blob/e7bc0989ea9f99e070c8f54d23dbdd23b9703cda/doc/neps/nep-0029-deprecation_policy.rst. - [build,default] Status: We set the minimal required versions as low as possible such that it does not introduce a significant maintenance burden. Policy: Bumping the versions require strong justification, e.g. incompatibility between the versions of the upstream packages or a discovered major bug upstream. Proposal: For all the deps from these lists strictly follow NEP 29 (support all minor versions released in the prior 24 months from the anticipated release date with a minimum of 3 minor versions) .
- [optional] By definition, these deps are not the core functionality of the library, but they still bring notable functionality increase. Since they are not core, I believe we should spend less attention on the maintenance here. Policy: Same as above, but we require less strong justification. Proposal: Adopting the NEP 29 or its more aggressive modification (e.g. 18 months, 2 minor versions).
- [docs] The deps here can be split into two groups: “functional” (e.g. matplotlib, pandas, seaborn), “infrastructural” (e.g. sphinx, numpydoc). I am assuming that the main use case for the users is to copy and run the code from the website API/examples, and that they mainly do not build the docs locally. Policy: not sure if we have one. Bumping requires some justification. Proposal: for "functional" - same policy as for [optional], for "infrastructural" - use the latest stable or the latest stable major versions in order to have the latest features and bug fixes.
- [test] This mostly concerns
pytest,flake8, andcoverage. The users are likely to run the tests locally, but, to the best of my knowledge, none of the mentioned packages depend on or may conflict with the [build,default] deps. Policy: Also unclear. Probably, similar to [docs]. Proposal: use the latest stable versions or the latest stable major versions. - [downstream] This is just to see how our own deprecation policy is aligned with the upstream. Policy: Breaking change has to be deprecated for the next 2 releases and implemented in the 3rd. It is possible to have a 2-release cycle in special cases, or a breaking change right away in case of a major bug. Proposal: Align the deprecation policy with NEP 29, given that 1 minor release of
skimagetakes approximately 327 days (see the next message).
Speaking of the policies above, we should also keep in mind the frequency of scikit-image releases. For example, our average release cycle is 11 months. Based on this, we can set an anticipated release date and merge the PRs deprecating/removing the support of specific versions, considering that date.
Looking forward to hearing your opinions! I hope we manage to find a consensus here.