SunPy Core Library Versioning and Release Pattern #30
Conversation
|
👍 |
|
Looks fine. Its one hell of a word salad, someone over at semver putting their creative writing to use. |
|
|
||
| The MAJOR, MINOR, and PATCH numbers shall be incremented in the following manner: | ||
|
|
||
| 1. MAJOR version when incompatible API changes are made. |
There was a problem hiding this comment.
can you define API? I assume you mean public-facing API and not developer-facing API (for example how to register a new downloader to FIDO).
There was a problem hiding this comment.
API is defined later in the document.
SEP-0008.md
Outdated
| precise and comprehensive (*See Below*). | ||
|
|
||
| 1. A normal version number MUST take the form X.Y.Z where X, Y, and Z are | ||
| non-negative integers, and MUST NOT contain leading zeroes. X is the major |
There was a problem hiding this comment.
can you be more clear by what you mean by leading zeroes? is 0.1.2 not allowed? but 1.2.0 is? Do you mean leading that X cannot be zero? or can you have 1.02.3?
| introduced within the private code. It MAY include patch level changes. | ||
| Patch version MUST be reset to 0 when minor version is incremented. | ||
|
|
||
| 1. Major version X (X.y.z | X > 0) MUST be incremented if any backwards |
There was a 8000 problem hiding this comment.
I request that the board need to provide some level of input what is mean by changed API. In this scheme it would be possible to release a brand new version by simply changing the name of a single function for example.
There was a problem hiding this comment.
I meant to weigh in on this earlier, sorry! Thanks for the reminder, @ehsteve .
This comment is the fundamental problem that I have with semantic versioning.
It sounds really nice on paper, but becomes a bear to manage in a research code with an inherently unstable API. I've had many projects where we tried to use it and it always comes down to "what do we count as a change."
This is why I strongly favor YY.MM.Patch versioning for all of my research codes that I work on. It is easy to tell when a version was released, temporally, if there is active development, and it also gives an easier time frame for features marked for a certain release.
There was a problem hiding this comment.
So, under YY.MM.Patch scheme we would move to version 17.10.1 ? Do I understand the scheme correctly? I'm curious if there any large public software project that uses this versioning scheme. I think this versioning scheme might be useful for that other large repository of commonly used solar data analysis software.
There was a problem hiding this comment.
Or 17.10.0.
The biggest one I know off the top of my head is Ubuntu. There are others, I'm sure, but I have not looked too hard.
It also makes it easier to know what is a long-term support release (always the April one or whatever) if you want to go that way. Lastly it is helpful for planning features too.
Dev: "This feature will take me 2 months to complete."
Maintainer: "I'll schedule it for 18.01"
I use this at work, and the only catch is that it is hard to tell when APIs are changing (an advantage of semantic versioning). So we compromise by only allowing API-breaking changes every 4 months or every 6 months. New API doesn't count as breaking, and any API that has not hit one of the major releases can be changed until that time without major issue.
There was a problem hiding this comment.
@ehsteve Yes, changing a function name could trigger a requirement for a major version update, that really is the whole objective of this proposal. I believe this document sufficiently defines what the API is and what changes to it require what changes in the version number.
@rhewett I do not like the idea of YY.MM.Patch versioning for SunPy.
I am of the opinion that SunPy is a little different from common research code, we are aiming to be a core library for a lot of different people using Python for solar physics, including for instance the DKIST data center as well as people who are just writing scripts to do science. I do not believe we should have an "inherently unstable API".
I believe this is a good motivation for us to adopt a strict versioning policy where we make it very obvious to our users when their code is going to break and when it isn't. I am aware that this will cause us pain, we will have to manage this and own our mistakes.
In my opinion "what we count as a change" should be strict, i.e. if it breaks code in the wild we release a major version. If this means we end up on version 30 in two years time, so be it.
There was a problem hiding this comment.
@Cadair, OK, that rationale makes sense. I'm trying to bring a counter point because people seem to read semantic versioning and think it makes a ton of sense (I thought this way after I saw it the first time and occasionally still find rationale for using it, too).
However, in practice, semantic versioning has been a royal pain in my ass. One of my jobs is to maintain a few central libraries that are used for many research projects (in many areas: geophysics, numerical computing, high-performance computing, ...) in our department and company. We have the same needs for API stability and that SunPy has, and even some more significant constraints because we have acceptability and validation requirements that must be satisfied on many computational architectures (different compilers supporting different features, different optimizations, different memory usage patterns, different data flows...). We can't afford to have major API breaking changes all the time, but we also need to have them to continue forward progress. I won't say that I know everything about this subject or that I have the best perspective, I'm just trying to share some of the experience I've had with these issues.
I'll leave a few more thoughts. Apologies if they are a little disorganized.
- If you are increasing the version number constantly, you don't have a stable code. Semantic versioning is not getting you stability in the API. This only comes from planning and willpower.
- The only benefit from a stability perspective is that it provides an active disincentive to modify the API. This can also be done with strong organizational willpower.
- A side effect of this is that it can actually hinder the forward progress of the library. If there is too strong a will against changes then things stall.
- Stability, or the perception of stability, can also be achieved by marking certain release versions as long-term support. That is where your core library stability will come from.
- How can you decide a priori which releases will be LTS if you have to increment the major number to patch an API breaking bug? An LTS release needs to be known well in advance, so work can be planned. If you dont know what version it will be because it is 7 months out, people will start to refer to it by the planned release date anyway, and then you have to change how it is referenced when the release happens.
- With YY.MM versioning, it is much easier to set a release schedule and have everyone know when to expect releases. It is impossible to keep track of firefox now that they have switched to incrementing versions so quickly. In fact it has had the opposite effect. I used to know which FF version I was using and now I have no idea. For a browser this is OK, but for a core library like SunPy it needs to be easy to know what version you have, if it is out of date, and by how much.
- This can be achieved with strict major revision counting, too, but semantic versioning in practice doesn't provide anyone (outside of the maintainer) any additional useful information, despite how overly specifically it is prescribed.
- If a change breaks code in the wild, incrementing the major version number does not change that fact. The code in the wild will still have to stay up to date. And there will be no indication of when the next major breaking change will happen.
- How do you know that a change will break code in the wild?
- Semantic versioning doesn't tell you anything about the codes that you can break. An API that has been stable from version X to version X+5 could be 3 years of stability or 3 months. Only institutional knowledge from the maintainer (technical debt) preserves this information. An API that has been stable from 16.08 to 18.04 has been stable for 1 year and 8 months. You know this instantly without having to look at the git history. And that gives you an idea of the risk you take in breaking something in a dependent package.
In summary, if the issue is stability, then no versioning system gets you that. It comes only from planning and following the workflow.
What versioning can give you and the users is extra information about the past and the future. Semantic versioning does not get you this. Date-versioning doesn't give you any information about the volume of changes (neither does semantic versioning, or many other versioning schemes) but it does at least give you (and the users depending on stability) some information about the age of the version you have and how long a particular API has remained stable.
There was a problem hiding this comment.
@rhewett - I really appreciate your list of thoughts. It makes me look from a different perspective. The date versioning seems interesting, though I only think it really works well if there's a scheduled release in place. So, ubuntu has their April and October releases, but besides whether I know how many months/years have passed from one version to another I don't know whether this is bring me a new fancy desktop or just changes that I won't notice. I'm with @Cadair that sunpy shouldn't be treated as a research code, but as a code to use in research - so it should change only in a way that researchers will be aware, it doesn't matter whether you are 10 months or 10 years on the same x version. You know that x+1 may require you to change something.
I completely agree with @rhewett about the concept of stability - sure it won't be given by any system we use to call one or another release, but from planning and willpower. What I believe semantic versioning does is to create a frame with a set of simple rules to think about what is stable. Thinks may go wrong? certainly! but with a numbering and the rules we will put the willpower to keep it going towards the ideal perfect world.
Finally, I would say that once we have a versioning numbering in place (any) then we can plan on what will be LTSs, will be the even mayor releases? will be a particular minor release that have got large adoption or momentum? (eg. python 2.7)
There was a problem hiding this comment.
I agree that the versioning scheme alone is not enough to make sunpy stable and consistent in the way we change things. I do think this proposal is a pretty simple way of advertising these changes to users, assuming we use it correctly.
I also agree that what is an LTS is out of scope for this, but I could imagine the last minor release before I major release being given an extended life as an LTS. This would allow people to keep using that old major version for longer. (I.e. while writing up a thesis)
| Python source, and any attributes on those objects not prefixed with one or more | ||
| underscores**, and are therefore included in the API documentation. | ||
|
|
||
| A change to the public API is anything that affects not just the name, arguments |
There was a problem hiding this comment.
Is a change defined to include new API as well or just changes to existing API?
There was a problem hiding this comment.
A change to the API is adding new functionality, however, it is a backwards compatible change so it would not require a new major version under this scheme.
SEP-0008.md
Outdated
| of matching version numbers, which this specification is compatible with. | ||
|
|
||
|
|
||
| # Future SunPy Releases |
There was a problem hiding this comment.
This section mixes plans for SunPy's development and the versioning description. I recommend that this section be dropped.
SEP-0008.md
Outdated
| a backwards compatible change, and removing it is not backwards compatible. | ||
| Where possible behaviour should be deprecated before it is removed. | ||
|
|
||
| ## SunPy Release Schedule |
There was a problem hiding this comment.
This section mixes plans for SunPy's development and its versioning description. I recommend that this section be dropped.
There was a problem hiding this comment.
Maybe we should have a separate SEP for describing how and when we release. For example, I'd be OK with a new patch release every time a single new patch was implemented. I think when a user finds a bug we should be as responsive as possible, otherwise we could be blocking someone's research.
SEP-0008.md
Outdated
| The MAJOR, MINOR, and PATCH numbers shall be incremented in the following manner: | ||
|
|
||
| 1. MAJOR version when incompatible API changes are made. | ||
| 1. MINOR version when functionality in added in a backwards-compatible manner. |
SEP-0008.md
Outdated
|
|
||
| 1. MAJOR version when incompatible API changes are made. | ||
| 1. MINOR version when functionality in added in a backwards-compatible manner. | ||
| 1. PATCH version when backwards-compatible bug fixes are made. |
There was a problem hiding this comment.
bug fixes to existing functionality are made
| [Semantic Versioning 2.0.0](http://semver.org/spec/v2.0.0.html) which is | ||
| licensed under the terms of the | ||
| [Creative Commons Attribution 3.0](https://creativecommons.org/licenses/by/3.0/) | ||
| licence. PEP 440 is in the public domain. |
There was a problem hiding this comment.
I'd prefer if the phrase "in the public domain" were not used since it's not a well-defined legal phrase. I realize that this is just parroting the language in the PEP – see here for their lack of interest in being more precise – so maybe there's a way to make clear that the phrase is their "fault", not ours.
|
|
||
| A change to the public API is anything that affects not just the name, arguments | ||
| or properties of any object, but any change that alters the behaviour of that | ||
| object (i.e. the values returned from calling it). Deprecating some behaviour is |
There was a problem hiding this comment.
Would changing the value of a physical constant (e.g., the radius of the Sun) be considered API-breaking?
There was a problem hiding this comment.
I wouldn't think so, but reading what's in there it should. A better way to go around constants is by calling them in a way that you can get the old one too, for example via a reference to the source. Then when we change the "default" that would be an API change.
|
Given the productive discussion on this, I want to take a different direction with this, so I am going to close this. See #39 for an issue to track development of a replacement. |
No description provided.