Add config to build aarch64 wheels #47
Conversation
|
hmmm. i think we would have to make sure our encrypted keys get migrated as well? |
|
@hmaarrfk The encrypted keys get migrated too. |
|
Ah very nice. I guess I can get on this. |
|
@hmaarrfk The core team should have admin access on that repo; you don't? |
|
I guess not on travis |
|
Weird, no idea why; but I applied for the beta. |
|
I thought I did this 2y ago when they first announced this. I am completely confused by their settings, admin permissions (I also don't appear to be able to change the settings for this repo on travis-ci.com), how accounts interact between GH and Travis... If someone has specific instructions for what I need to do, please let me know. Even better if I can somehow delegate to you, @hmaarrfk! |
|
I think it is building should be good now. |
|
I think we can merge. The travis org and com thing might just cause the build to get uploaded twice. Not a big deal tough. |
|
Thanks! |
|
WHen you do make that PR, watch out for QT getting installed. Not sure about QT + AARCH64 support on pypi |
|
@jni When would the next release be? |
|
@janaknat Sorry for the delay. I meant to get it done this week but didn't manage. I hope I can do so this coming week! |
|
@jni Any updates on the next release? |
|
@hmaarrfk Any news on when the next release will be? |
|
@janaknat The good news is that the release should happen tomorrow. The release candidate 0.18.0rc0 is already out! The very bad news is that Travis stopped allowing free builds for source so I think we no longer have a way to build aarch wheels!!! 😭😭😭😭😭 |
|
@jni You mean the main repo? Or this wheel builder repo? |
|
@jni have you reached out to the Travis folks for additional credits for OSS builds? we'll reach out to our contacts in parallel to request additional credits. Free builds for OSS are still possible but there was a recent change by TravisCI to eliminate abuse of their good will to provide free builds which has an immediate term impact: https://blog.travis-ci.com/oss-announcement |
|
Hello folks, I have been directed to provide you with credits for this project. The current value is 50,000. Thanks |
|
Great, thank you @muizzk! |
8000
|
@jni Will there be aarch64 wheels with the next rc? |
|
i'm not sure if the plan is going to be to uses these wheels here. Maybe you can help us update to have some aarch builds? Recent changes to travis forced us to look at alternatives. Not too sure what @jni's plan is |
|
@hmaarrfk Github Actions does not support aarch64 build environment. Travis has said that they'll provide more credits to projects. |
|
Got it. Honestly, with conda-forge, we had docker environments that incorporated qemu that would work even when executed on a x84 host. not sure if that is possible. THerefore, we used x86 host, but a docker image that was aarch64. That might be the way to go, at least for the shrot term. The problem, is that qemu took about 6x longer than native. |
I think we just got bumped from 10,000 credits to 50,000 yesterday and already travis-ci.com says we have used 39400 out of 50000, so that will not get us too far. Also, if it still renews each month at 10,000 credits and we have to manually request more each time, that is not a feasible solution. If we remove all jobs except ARM from Travis, we may be okay though. I think that is what scipy-wheels is doing now. IIRC, costs are 10/min on linux, 20/min on Windows and 50/min on OS X.
Yes, I recently used qemu locally to debug some ARM-related failures in scipy.ndimage code and the build took quite a long time. |
|
Agreed @grlee77, let's get everything other than this off of Travis-CI ASAP. |
|
@stefanv Travis-CI has provided more credits. Will the aarch64 wheels make it in the next release? |
|
@janaknat they gave us credits but they ran out in ~24h. I haven't received any build notifications for a while (this repo has a nightly build), so I presume that we are out of credits again. I don't know how to check our credits balance. 50,000 sounds like a lot but at 50 credits a minute on mac, that's only 16h of build time on mac, and a single matrix entry takes close to 1h, so they run out extremely quickly, with no room for experimentation when builds go wrong, which is basically all the time. @grlee77 worked on removing Travis builds from the main repo, but I fear that came too late. I'm working to remove everything but the aarch64 builds from here, but, to reiterate what @stefanv and @grlee77 said, Travis is not a sustainable solution for us. After the build matrix is cleaned up here I think @hmaarrfk's attempt at scikit-image/scikit-image#5098 is the right way forward, but there's still a bit of work to do there, it seems. |
Travis-CI allows for the creation of ARM64 wheels.
Successful build log: https://travis-ci.com/github/janaknat/scikit-image-wheels
Travis-CI also supports using Graviton2 for ARM builds. To do that, the repo would have to migrate to travis-ci.com from travis-ci.org.
Basic steps:
More info at https://docs.travis-ci.com/user/migrate/open-source-repository-migration