10000 Provide x64, arm64, and universal2 arch selectors for darwin · Issue #547 · actions/setup-python · GitHub
[go: up one dir, main page]
Skip to content

Provide x64, arm64, and universal2 arch selectors for darwin #547

New issue

Have a question about this project? Sign up 8000 for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mattip opened this issue Nov 22, 2022 · 18 comments
Closed

Provide x64, arm64, and universal2 arch selectors for darwin #547

mattip opened this issue Nov 22, 2022 · 18 comments
Assignees
@aparnajyothi-y
Labels
feature request New feature or request to improve the current logic needs eyes

Comments

@mattip
Copy link
mattip commented Nov 22, 2022

Description:
Describe your proposal.

As described in this comment by @misl6, it would be nice to add the possibility to choose between arch = universal2, arm64, x64 on darwin (macos).

Justification:
Justification or a use case for your proposal.

Runners for arm64 macos are in the github roadmap, but are not generally available. Once they are, it would be nice to be able to use either "thin" (x86_64 or arm64) or universal2 pythons on that image. Some work toward this was done in PR #114 which was merged then reverted. The design for the arch selector should already be worked out before making the images available:

  • what is the exact syntax of the selector
  • what will the default be

Note that PyPy is going to release both thin arm64 and x86_64 binary packages in its next release, and CPython already has a universal2 package.

I would propose supporting all three alternatives, as the scientific python stack (NumPy, SciPy, and more) and conda-forge have decided to support thin binary packaging only. It would be convenient if this action would make it easy to support those decisions by not forcing a universal2-only solution.

xref issue #108 which discusses the need for arm/arm64 python.

Are you willing to submit a PR?

Yes, but I don't know if it is relevant for an outside contribution.
@mattip mattip added feature request New feature or request to improve the current logic needs triage labels Nov 22, 2022
@MaksimZhukov
Copy link
Contributor

Hello @mattip! Thank you for the suggested idea!
We will consider adding this feature and will let you know as soon as we have any decision.

@jdogmcsteezy
Copy link
jdogmcsteezy commented Nov 28, 2022 •
edited
Loading

@mattip, this states that runners for arm64 macOS are generally available. I do see the open issue you referenced that says "beta". 528 and 507 seem in conflict to me. @MaksimZhukov, what am I missing here?

@mattip
Copy link
Author
mattip commented Nov 28, 2022

The link you posted states that "self-hosted runners are available". That is great, and is probably enough to justify formalizing the syntax for the selectors now. I was relating to the possibility of github-hosted runners, which seems to be not generally available yet.

@jdogmcsteezy
Copy link

@mattip, I should have thought about it a little harder, haha. Thanks for the clarification

@NeilJed
Copy link
NeilJed commented Dec 7, 2022

+1 To this. I'm building an Ubuntu 22.04 runner on ARM64 from the runner-images repo and the lack of ARM64 toolcache packages makes it impossible to use this action. TBH all the "setup/xxx" toolcache packages should be multi-arch.

@mayeut mayeut mentioned this issue Dec 11, 2022
Merged
@mayeut
Copy link
Contributor
mayeut commented Dec 11, 2022

@mattip, PyPy already works out of the box on macOS arm64 self-hosted runners (so should be the same on github-hosted runners once available): https://github.com/mayeut/setup-python/actions/runs/3669884297/jobs/6204031022

I'm not sure if universal2 selector should be supported here at all. At least arm64 & x64 shall (so, that's already the case for PyPy). I opened actions/python-versions#214 to get support for CPython, let's see what maintainers say.

@mayeut mayeut mentioned this issue Dec 11, 2022
Closed
2 tasks
@mattip
Copy link
Author
mattip commented Dec 11, 2022

@mayeut: nice.
FWIW, the run you pointed to above seems to use x86_64 for the CPython run. Is that intentional?

@mayeut
Copy link
Contributor
mayeut commented Dec 11, 2022

Is that intentional?

Yes, since there's no arm64 package in python-versions, I was just checking if reusing the x86_64 python 3.11 one would work as-is (since it's a universal2 pkg underneath, it works).

@alex
Copy link
alex commented Jan 14, 2023

For readability purposes, it'd be great if the action accepted universal2/arm64 selectors -- right now we're doing x64 with a self-hosted arm64 runner and relying on the fact that it happens to fetch a universal2 binary.

@vutting4221 vutting4221 mentioned this issue Jun 22, 2023
Open
@dhermes dhermes mentioned this issue Jun 18, 2024
Merged
@aparnajyothi-y
Copy link
Contributor

Hello @mattip, Thank you for your feature request! We are happy to inform you that Framework builds are available for Python versions 3.11 and above, for both x64 and arm64 architectures. For these versions, the official macOS universal2 Python binaries are downloaded from python.org, archived, and provided along with the installation script for further distribution and installation.

Additionally, Framework builds are available for arm64 packages starting from Python version 3.9 and above. The primary difference for these builds is the adjustment in the tool cache destination.

Please feel free to reach out if you have any further questions or encounter any issues!

@aparnajyothi-y aparnajyothi-y self-assigned this Mar 18, 2025
@Julian
Copy link
Julian commented Mar 23, 2025 •
edited
Loading

For these versions, the official macOS universal2 Python binaries are downloaded from python.org, archived, and provided along with the installation script for further distribution and installation.

Are earlier versions (of Python) and support for installing PyPy on ARM64 still perhaps planned given this is still open? More specifically I'm finding this issue as I seem to get:

Installed versions
  PyPy version 2.7 (x) was not found in the local cache
  Error: PyPy version 2.7 (x) with arch arm64 not found

when attempting to ask for:

    - name: Set up PyPy 2.7
      uses: actions/setup-python@v5
      with:
        python-version: "pypy-2.7"
        architecture: arm64

on macOS (despite there of course being ARM64 builds available).

@aparnajyothi-y
Copy link
Contributor
aparnajyothi-y commented Mar 27, 2025 •
edited
Loading

Hello @Julian, Thank you for your feedback. We appreciate your interest in using PyPy 2.7 with ARM64 architecture. As you mentioned, the PyPy version is available for download and is successfully extracted during the setup process as per the below test run screenshot. However, our current focus is on supporting Python packages, and we do not have immediate plans to include PyPy support.
While we recognize that PyPy offers performance improvements for certain use cases, prioritizing Python packages allows us to maintain broader compatibility and simplify ongoing maintenance. We will continue to assess community needs and may revisit PyPy support in the future.
Additionally, the official macOS universal2 Python binaries are downloaded from python.org, archived, and provided along with the installation script for distribution and installation, as outlined in the documentation.

Thank you for your understanding. Please feel free to reach out if you have any further questions.

Image

@aparnajyothi-y
Copy link
Contributor

Hello @Julian, Please let us know in case of any concerns/clarifications on the above

@Julian
Copy link
Julian commented Apr 3, 2025

Hey @aparnajyothi-y thanks for following up! I'd seen your response but the screenshot confused me because you got different behavior I did! So either I did something wrong then or something else has changed -- specifically in your screenshot I see PyPy ends up getting downloaded and installed, regardless of the first cache miss, whereas in my build it simply failed completely after that! So I made a note to try this again before responding to check whether there was really anything to ask for -- if indeed it installs for ARM64 even if it's not in the local cache that seems totally reasonable behavior to me.

I'll double check but essentially I think you answered my ask!

@aparnajyothi-y
Copy link
Contributor
aparnajyothi-y commented Apr 15, 2025 •
edited
Loading

Hello @Julian, Thanks for the update! Glad the screenshot helped clarify things used the below code snippet. From what I’ve seen, the current behavior does fall back to installing PyPy after a cache miss, which is expected. Please let us know if your re-run shows anything different!

  - name: Set up PyPy 2.7
    uses: actions/setup-python@v5
    with:
      python-version: ""pypy-2.7""
      architecture: arm64

@aparnajyothi-y
Copy link
Contributor

Hello Everyone, Proceeding to close this as this feature is implemented and confirmed.
Please feel free to reach us in case of any further concerns/clarifications are needed

@spott
8000
Copy link
spott commented May 1, 2025

I'd like to ask for x64 thin binaries for python: If you want to build or use x64 on any of the aarch64 runners, then you need to:

a) use a thin binary for python that ensures it always runs x64 (because aarch64 doesn't exist)... or...
b) for every pip line in the entire script use "arch -x86_64" before every python call. This is kinda a pita, as it means you can't use other GitHub Actions that use python (they won't have the required arch prepend).

So I'd like to request that this be reopened to add thin binaries for python for x86-64.

Getting x64 to work on arm runners is going to be difficult without the thin binaries.

@aparnajyothi-y
Copy link
Contributor

Hello @Julian, thank you for the detailed context. The current issue pertains to Framework builds and their availability for various architectures, including arm64 and x64.

Your request for x64 thin binaries on aarch64 runners appears to be a distinct requirement, as it focuses on enabling x64 Python execution without relying on arch -x86_64 prefixes. To help us better track and evaluate this feature, we recommend raising a new issue specifically for x64 thin binary support on ARM runners.

This will ensure proper visibility and allow us to assess it independently. Please feel free to open one, and let us know if you'd like help outlining the request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aparnajyothi-y aparnajyothi-y

Labels
feature request New feature or request to improve the current logic needs eyes
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@alex @spott @Julian @mattip @NeilJed @mayeut @jdogmcsteezy @MaksimZhukov @dusan-trickovic @aparnajyothi-y
0