8000 PEP440 compliant versions by warner · Pull Request #67 · python-versioneer/python-versioneer · GitHub
[go: up one dir, main page]

Skip to content

PEP440 compliant versions #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 13, 2015
Merged

PEP440 compliant versions #67

merged 2 commits into from
Feb 13, 2015

Conversation

warner
Copy link
Collaborator
@warner warner commented Feb 12, 2015

This makes get_version() always return a PEP440-compliant string, in one of the following forms:

  • TAG
  • TAG+DISTANCE.gHASH[.dirty]
  • 0+unknown
  • 0+untagged.gHASH[.dirty]
  • 0+unparseable[.dirty]

The "+" separator uses PEP440's "local version" section to represent the non-tag portions, retaining all the information from before, but not triggering "LegacyVersion" warnings. This uses ".dirty" instead of "-dirty" because "-dirty" would trigger a warning and be normalized into ".dirty" anyways.

I don't know if this format will work for everybody. On the down side, it doesn't use the .postNN and .devNN portions, which would impact projects that intentionally distribute not-at-a-tag development versions and depend upon e.g. pip install --pre to distinguish between real releases and these intermediate things. I'm inclined to think that anything deserving of a .post/.dev label also deserves a tag, and that one should never push an untagged release to PyPI. But I'd like to hear more about other folks' workflows before we land this.

On the plus side, this new format contains all the useful information from the old one (distance and git commit-id hash). I was previously frustrated by the apparent tradeoff between 1: making setuptools stop complaining about PEP440-incompatibility, and 2: getting to know the git hash of the tree. This removes the tradeoff. It also seems more readable to me: in 1.2+4.g123abc.dirty, I mentally parse the pieces as:

  • "1.2+4", so that's something tagged "1.2", plus four commits on top
  • "4.g123abc.dirty" is like a three-digit number in a funny non-uniform radix encoding, separated by dots.
  • The first "digit" is 4, which is just an integer, and sorts naturally (as an integer).
  • The second "digit" is g123abc, which doesn't sort in any useful order (PEP440 sorts it alphabetically) but you only ever compare this part for equality, not orderability.
  • And the last "digit" dirty is a boolean, either there or not, and if it's there then the tree is "newer" than a tree without .dirty

So I'm happy with the encoding we get when we're on a tag, or post-tag. The other three cases (0+stuff) are basically a last-ditch effort to produce something PEP440-compatible despite having very little information to go on. I think these are strictly better than a non-compliant unknown, but it does sort of pretend there's an implicit 0 tag on the initial commit, which might seem weird at first.

This also happens to fix #12, "no version until first tag is added". In the future I'd like to change the #12 untagged case to 0+untagged.DISTANCE.gHASH[.dirty], or maybe just 0+DISTANCE.gHASH[.dirty], where DISTANCE is the total number of commits in the repo at that point. But this patch doesn't include that, since it'll involve more code (a call to git log to count the patches when git describe fails to tell us).

PEP440 describes how these "local version identifiers" are supposed to be used. They're mainly for things like Debian downstream patches, so if upstream releases 1.2, downstream might use 1.2+debian1 to indicate they've made one set of local changes. So in one sense, we're abusing this feature to add information that doesn't fit in the other part (which PEP440 calls the "public version identifier"). But in another sense, tagged sources are "upstream", and your development tree (e.g. basically any source that isn't tagged) is a "downstream". This approach gives us public version identifiers (with no "+") for anything that's tagged, which is basically the only thing you should be pushing to PyPI or referencing as a dependency from elsewhere. But if you do build something with untagged commits, you've still got a correctly-sorting traceable identifier (with the "+" and gHASH and .dirty) that should appear in all the right places.

I'm hoping that everyone who's contributed or requested PEP440 compliance changes could take a look at this and tell me if it would work for them. /cc @muggenhor @sebastianneubauer @glennmatthews @marscher @tacaswell @dairiki @matthew-brett

I'm especially interested in folks thoughts about whether TAG[.post0.devDISTANCE] would be better in some cases. When the "template" branch gets done, it should be easy to switch to this other form, but for now if we land this PR, you'll be getting TAG[+DISTANCE.gHASH[.dirty]] -style versions. (fortunately it's a one-line edit to get the other form, and everybody has to manually upgrade their versioneer installation anyways, so nothing will immediately break when we make the next release, and it should be easy for folks who need .post.dev -style to get it).

The short version should now always be PEP440-compliant, in one of the
following forms:

* TAG
* TAG+DISTANCE.gHASH[.dirty]
* 0+unknown
* 0+untagged.gHASH[.dirty]
* 0+unparseable[.dirty]

This uses PEP440's "local version" section to represent the non-tag
portions, retaining all the information from before but not triggering
"LegacyVersion" warnings. ".dirty" is used instead of "-dirty" because
"-dirty" would trigger a warning and be normalized into ".dirty"
anyways.

Thanks to @muggenhor, @sebastianneubauer, and @glennmatthews for the
many attempts to solve this which have been incorporated into this
patch.
@warner
Copy link
Collaborator Author
warner commented Feb 13, 2015

BTW, I looked at pip, and it seems that pip install --pre enables the selection of distributions that have a .pre or .dev in their version strings. Pip mostly ignores .post.

So I don't think my earlier plan of using TAG[.postDISTANCE][.dev0] (where .dev0 means -dirty). Non-dirty between-tags versions become TAG.postNN, and that doesn't qualify for pip install --pre. Dirty versions would qualify (due to the .dev0), but nobody should be publishing or depending upon code that isn't even checked in. (.dev0, like -dirty, is really just a flag to the bug triage team that the submitter has modified their source, and they shouldn't assume it will behave like something they released).

@sebastianneubauer was asking for TAG[.post.devDISTANCE] (with no indication of -dirty). If I'm reading PEP-440 and the pip code correctly:

  • the .post has no number and is only there to make sure it sorts higher than TAG
  • .dev exists to enable pip install --pre
  • the distance then must go on .devNN so post-tag versions sort properly

I guess TAG[.postDISTANCE.dev] would work too, and you could maybe use .dev0 and .dev1 to distinguish -dirty.

I'll see if I can make progress on that template branch to make it easy to get a string of this format.

@glennmatthews
Copy link
Contributor

I like this approach quite a bit, and agree with your thoughts that only explicitly tagged labels should be published to PyPI.

Regarding the dev question, not to muddy the waters too much, but I'd also vaguely considered the idea of auto-incrementing the tag least significant version number to create a proper 'dev' version without needing the post.dev hack - i.e., 1.2.1-30-g5ca8731-dirty could be massaged to 1.2.2.dev30+g5ca8731.dirty, 1.2.1.rc3-3-g23423 to 1.2.1.rc4.dev3+g23423, etc. It's a total hack, but this might also be a better fit for @sebastianneubauer's request for allowing interim builds to be installed with pip install --pre?

@tacaswell
Copy link

My use-cases are mostly for managing inter-tag binary builds (via conda) of packages. I am in the camp that so long as the test are passing any commit on 'master' is suitable to be a release. The big formal releases are important for communication reasons (and give you a big milestone to be able to say 'we are going to break something'). Given my use case the 1.2.6postN+g45679 form is far more useful to me than dev or pre forms.

I am getting server issues from https://www.python.org/dev/peps/pep-0440/ tonight which makes it difficult for me to look at the details.

@warner
Copy link
Collaborator Author
warner commented Feb 13, 2015

Ok, I think I'm going to land this. I'm adding a template for @tacaswell 's format (I'll ask you more questions about it on its own PR). I think this is an overall improvement, and can be made to work with the other workflows folks have described here.

@sebastianneubauer
Copy link

I agree, that it is simple enough to change the exact format individually, maybe it makes sense to give a hint on the documentation for it?
Just some thoughts on the various version schemes that popped up here. My motivation is, that we are in a continuous integration/delivery workflow.
"Dirty" is therefore not interesting for me, we anyhow need to commit in order to trigger the automated tests and subsequent release. Each commit leads to a deployable release which is marked as unstable (because it is not tagged explicitly) in our case 1.2.3.post.devN. Everyone can trivially deploy the latest unstable with 'pip install', or the latest unstable release (last commit passing the tests) with 'pip install --pre'. A tag is mostly a political decision, not a quality measure, because the tests are all passed, therefore in our case it would make sense in principle to upload it to the pypi, but as said, thats mostly a 'political/marketing' decision.
The tags as we use it have semantical meaning, that is,, an increment in the last digit is a compatible release, in the second a breaking change and the first one, once again political. Therefore I would disfavor to auto-increment the least significant digit as the next tag could also be a breaking change, but that is often not clear from the beginning, but technically it would work...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

no version until first tag is added
4 participants
0