8000 Add a build step at the end of .travis.yml that uploads working wheels to a known location · Issue #6493 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

Add a build step at the end of .travis.yml that uploads working wheels to a known location #6493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
njsmith opened this issue Oct 17, 2015 · 13 comments

Comments

@njsmith
Copy link
Member
njsmith commented Oct 17, 2015

There are projects that want to run their tests against the latest numpy master, and it would be a lot easier for them if there was a wheel they could just download. Some of them do pip install git+http://...path/to/numpy/master, but this has two disadvantages: (a) everyone has to rebuild from scratch all the time, (b) sometimes (hopefully rarely, but...) master is broken and this just doesn't work.

The solution is that we should have an after_success: block in .travis.yml so that after each commit has passed its tests, we upload the corresponding wheel somewhere, probably one of these rackspace accounts that everyone seems to have access to.

(Ideally we would only do this if all tests pass, but unfortunately this is impossible with travis currently: travis-ci/travis-ci#929 . In the mean time, uploading each build variant iff its tests pass seems like a reasonable compromise.)

@ogrisel volunteers to do this in a few weeks, so if it's, say, mid-November and it hasn't happened, then someone should nag him ;-)
(#6462 (comment))

@matthew-brett
Copy link
Contributor

I believe the python-Pillow guys have been using travis-after-all to trigger builds for for the OSX wheels for some time:

https://github.com/alrra/travis-after-all
https://github.com/python-pillow/Pillow/blob/master/.travis.yml#L79

I'm not sure how reliable that is.

@ogrisel
Copy link
Contributor
ogrisel commented Oct 17, 2015

I use wheelhouse-uploader for scikit-learn (which I maintain myself): here is an example usage:

https://github.com/scikit-learn/scikit-learn/blob/master/appveyor.yml#L74

Here is the documentation (README.md):

https://github.com/ogrisel/wheelhouse-uploader

I don't have time to do the PR myself this (at PyCON FR right now and sprinting on sklearn next week) but can provide sporadic help if you want to give it a try.

BTW: we already have the following container for stable wheels for the scipy stack:

http://wheels.scipy.org/

we could setup a dedicated container for CI related wheels for the master branch of each project (under Linux assuming they are all built with the same version of Linux on travis) for instance:

http://travis-wheels.scipy.org/

Maybe we can reuse the same rackspace container as http://travis-wheels.scikit-image.org/ and just add a new DNS if the scikit-image people agree (ping @stefanv :). Otherwise I can create a new dedicated container for this workflow.

@mhvk
Copy link
Contributor
mhvk commented Oct 17, 2015

Out of curiosity: how does one ensure this only runs when merging stuff in master? (rather than on every PR that is being tested.)

@ogrisel
Copy link
Contributor
ogrisel commented Oct 17, 2015

Good question, thanks for asking: for security reasons, it does not have the credentials to do the upload in the PR, only after the merge. This makes it skip the upload automatically in PRs but will do the upload once merge in master (or in a release branch), assuming the proper credentials have been configured as a secret variable on the main github repo.

@stefanv
Copy link
Contributor
stefanv commented Oct 20, 2015

I'm happy to add whichever DNS you need.

@ogrisel
Copy link
Contributor
ogrisel commented Dec 15, 2015

@stefanv could you please add a CNAME entry for travis-dev-wheels.scipy.org pointing to?

66ce22ef0bf26a5e2cca-4ffdece11fd3f72855e4665bc61c7445.r49.cf2.rackcdn.com

@ogrisel
Copy link
Contributor
ogrisel commented Dec 24, 2015

@stefanv are you ok to add the DNS entry? It would be great to have a shorter URLs in .travis.yml files of those projects.

@stefanv
Copy link
Contributor
stefanv commented Dec 25, 2015 via email

@ogrisel
Copy link
Contributor
ogrisel commented Dec 26, 2015

For scikit-learn.org yes, but here I would just need to have a new CNAME entry for travis-dev-wheels.scipy.org pointing to:

66ce22ef0bf26a5e2cca-4ffdece11fd3f72855e4665bc61c7445.r49.cf2.rackcdn.com

Hence on the scipy.org domain.

@stefanv
Copy link
Contributor
stefanv commented Dec 26, 2015 via email

@ogrisel
Copy link
Contributor
ogrisel commented Dec 26, 2015

Thanks!

@ogrisel
Copy link
Contributor
ogrisel commented Dec 28, 2015

As #6768 is now merged in master I think this issue can now be closed.

@rgommers
Copy link
Member

Great, thanks Olivier!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants
0