8000 Use GitHub releases to download python versions (#85) · ffe4/setup-python@e5af64b · GitHub
[go: up one dir, main page]
Skip to content

Commit e5af64b

Browse files
MaksimZhukovMaksimZhukovkonradpabjanbrcrista
authored
Use GitHub releases to download python versions (actions#85)
This pull-request improves `setup-python` action to add ability to download specific version of Python on flight if it is not available by default. **Details:** `setup-python` action will download and install specific Python version from GitHub releases ([actions/python-versions](https://github.com/actions/python-versions/releases)) in case the version is not found in the local cache. All versions of Python available for installation are published in [actions/python-versions](https://github.com/actions/python-versions) repository. All available versions are listed in the [version-manifest.json](https://github.com/actions/python-versions/blob/master/versions-manifest.json) file. **Installation time:** - Ubuntu / macOS: 10-20 seconds - Windows: ~ 1 minute (mostly related to fact that we use MSI installer for Python on Windows) Co-authored-by: MaksimZhukov <v-mazhuk@microsoft.com> Co-authored-by: Konrad Pabjan <konradpabjan@github.com> Co-authored-by: Brian Cristante <33549821+brcrista@users.noreply.github.com>
1 parent 985150d commit e5af64b

File tree

11 files changed

+5427
-4972
lines changed

11 files changed

+5427
-4972
lines changed

‎.github/workflows/test.yml

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: Validate 'setup-python'
2+
on:
3+
push:
4+
pull_request:
5+
schedule:
6+
- cron: 0 0 * * *
7+
8+
jobs:
9+
default-version:
10+
name: Setup default version
11+
runs-on: ${{ matrix.os }}
12+
strategy:
13+
fail-fast: false
14+
matrix:
15+
os: [macos-latest, windows-latest, ubuntu-16.04, ubuntu-18.04]
16+
steps:
17+
- name: Checkout
18+
uses: actions/checkout@v2
19+
20+
- name: setup default python
21+
uses: ./
22+
23+
- name: Validate version
24+
run: python --version
25+
26+
- name: Run simple python code
27+
run: python -c 'import math; print(math.factorial(5))'
28+
29+
setup-versions-from-manifest:
30+
name: Setup ${{ matrix.python }} ${{ matrix.os }}
31+
runs-on: ${{ matrix.os }}
32+
strategy:
33+
fail-fast: false
34+
matrix:
35+
os: [macos-latest, windows-latest, ubuntu-16.04, ubuntu-18.04]
36+
python: [3.5.4, 3.6.7, 3.7.5, 3.8.1]
37+
steps:
38+
- name: Checkout
39+
uses: actions/checkout@v2
40+
41+
- name: setup-python ${{ matrix.python }}
42+
uses: ./
43+
with:
44+
python-version: ${{ matrix.python }}
45+
46+
- name: Validate version
47+
run: |
48+
$pythonVersion = (python --version)
49+
if ("Python ${{ matrix.python }}" -ne "$pythonVersion"){
50+
Write-Host "The current version is $pythonVersion; expected version is ${{ matrix.python }}"
51+
exit 1
52+
}
53+
$pythonVersion
54+
shell: pwsh
55+
56+
- name: Run simple code
57+
run: python -c 'import math; print(math.factorial(5))'

‎README.md

Lines changed: 60 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,9 @@
66

77
This action sets up a Python environment for use in actions by:
88

9-
- optionally installing a version of Python and adding to PATH. Note that this action only uses versions of Python already installed in the cache. The action will fail if no matching versions are found.
9+
- optionally installing and adding to PATH a version of Python that is already installed in the tools cache
10+
- downloading, installing and adding to PATH an available version of Python from GitHub Releases ([actions/python-versions](https://github.com/actions/python-versions/releases)) if a specific version is not available in the tools cache
11+
- failing if a specific version of Python is not preinstalled or available for download
1012
- registering problem matchers for error output
1113

1214
# Usage
@@ -67,13 +69,43 @@ jobs:
6769
run: python -c "import sys; print(sys.version)"
6870
```
6971
72+
Download and set up a version of Python that does not come preinstalled on an image:
73+
```yaml
74+
jobs:
75+
build:
76+
runs-on: ubuntu-latest
77+
strategy:
78+
# in this example, there is a newer version already installed, 3.7.7, so the older version will be downloaded
79+
python-version: [3.5, 3.6, 3.7.4, 3.8]
80+
steps:
81+
- uses: actions/checkout@v2
82+
- uses: actions/setup-python@v1
83+
with:
84+
python-version: ${{ matrix.python }}
85+
- run: python my_script.py
86+
87+
```
88+
7089
# Getting started with Python + Actions
7190

7291
Check out our detailed guide on using [Python with GitHub Actions](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/using-python-with-github-actions).
7392

93+
# Available versions of Python
94+
95+
`setup-python` is able to configure Python from two sources:
96+
97+
- Preinstalled versions of Python in the tools cache on GitHub-hosted runners
98+
- For detailed information regarding the available versions of Python that are installed see [Software installed on GitHub-hosted runners](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/software-installed-on-github-hosted-runners).
99+
- For every minor version of Python, expect only the latest patch to be preinstalled.
100+
- If `3.8.1` is installed for example, and `3.8.2` is released, expect `3.8.1` to be removed and replaced by `3.8.2` in the tools cache.
101+
- If the exact patch version doesn't matter to you, specifying just the major and minor version will get you the latest preinstalled patch version. In the previous example, the version spec `3.8` will use the `3.8.2` Python version found in the cache.
102+
- Downloadable Python versions from GitHub Releases ([actions/python-versions](https://github.com/actions/python-versions/releases))
103+
- All available versions are listed in the [version-manifest.json](https://github.com/actions/python-versions/blob/master/versions-manifest.json) file.
104+
- If there is a specific version of Python that is not available, you can open an issue in the `python-versions` repository.
105+
74106
# Hosted Tool Cache
75107

76-
GitHub hosted runners have a tools cache that comes with Python + PyPy already installed. This tools cache helps speed up runs and tool setup by not requiring any new downloads. There is an environment variable called `RUNNER_TOOL_CACHE` on each runner that describes the location of this tools cache and there is where you will find Python and PyPy installed. `setup-python` works by taking a specific version of Python or PyPy in this tools cache and adding it to PATH.
108+
GitHub hosted runners have a tools cache that comes with a few versions of Python + PyPy already installed. This tools cache helps speed up runs and tool setup by not requiring any new downloads. There is an environment variable called `RUNNER_TOOL_CACHE` on each runner that describes the location of this tools cache and there is where you will find Python and PyPy installed. `setup-python` works by taking a specific version of Python 10000 or PyPy in this tools cache and adding it to PATH.
77109

78110
|| Location |
79111
|------|-------|
@@ -85,35 +117,44 @@ GitHub virtual environments are setup in [actions/virtual-environments](https://
85117
- [Tools cache setup for Ubuntu](https://github.com/actions/virtual-environments/blob/master/images/linux/scripts/installers/hosted-tool-cache.sh)
86118
- [Tools cache setup for Windows](https://github.com/actions/virtual-environments/blob/master/images/win/scripts/Installers/Download-ToolCache.ps1)
87119

120+
# Specifying a Python version
121+
122+
If there is a specific version of Python that you need and you don't want to worry about any potential breaking changes due to patch updates (going from `3.7.5` to `3.7.6` for example), you should specify the exact major, minor, and patch version (such as `3.7.5`)
123+
- The only downside to this is that set up will take a little longer since the exact version will have to be downloaded if the exact version is not already installed on the runner due to more recent versions.
124+
- MSI installers are used on Windows for this, so runs will take a little longer to set up vs Mac and Linux.
125+
126+
You should specify only a major and minor version if you are okay with the most recent patch version being used.
127+
- There will be a single patch version already installed on each runner for every minor version of Python that is supported.
128+
- The patch version that will be preinstalled, will generally be the latest and every time there is a new patch released, the older version that is preinstalled will be replaced.
129+
- Using the most recent patch version will result in a very quick setup since no downloads will be required since a locally installed version Python on the runner will be used.
130+
88131
# Using `setup-python` with a self hosted runner
89132

90-
If you would like to use `setup-python` on a self-hosted runner, you will need to download all versions of Python & PyPy that you would like and setup a similar tools cache locally for your runner.
91-
92-
- Create an global environment variable called `AGENT_TOOLSDIRECTORY` that will point to the root directory of where you want the tools installed. The env variable is preferrably global as it must be set in the shell that will install the tools cache, along with the shell that the runner will be using.
93-
- This env variable is used internally by the runner to set the `RUNNER_TOOL_CACHE` env variable
94-
- Example for Administrator Powershell: `[System.Environment]::SetEnvironmentVariable("AGENT_TOOLSDIRECTORY", "C:\hostedtoolcache\windows", [System.EnvironmentVariableTarget]::Machine)` (restart the shell afterwards)
95-
- Download the appropriate NPM packages from the [GitHub Actions NPM registry](https://github.com/orgs/actions/packages)
96-
- Make sure to have `npm` installed, and then [configure npm for use with GitHub packages](https://help.github.com/en/packages/using-github-packages-with-your-projects-ecosystem/configuring-npm-for-use-with-github-packages#authenticating-to-github-package-registry)
97-
- Create an empty npm project for easier installation (`npm init`) in the tools cache directory. You can delete `package.json`, `package.lock.json` and `node_modules` after all tools get installed
98-
- Before downloading a specific package, create an empty folder for the version of Python/PyPY that is being installed. If downloading Python 3.6.8 for example, create `C:\hostedtoolcache\windows\Python\3.6.8`
99-
- Once configured, download a specific package by calling `npm install`. Note (if downloading a PyPy package on Windows, you will need 7zip installed along with `7z.exe` added to your PATH)
100-
- Each NPM package has multiple versions that determine the version of Python or PyPy that should be installed.
101-
- `npm install @actions/toolcache-python-windows-x64@3.7.61579791175` for example installs Python 3.7.6 while `npm install @actions/toolcache-python-windows-x64@3.6.81579791177` installs Python 3.6.8
102-
- You can browse and find all available versions of a package by searching the GitHub Actions NPM registry
103-
![image](https://user-images.githubusercontent.com/16109154/76194005-87aeb400-61e5-11ea-9b21-ef9111247f84.png)
133+
If you would like to use `setup-python` and a self-hosted runner, there isn't much that you need to do. When `setup-python` is run for the first time with a version of Python that it doesn't have, it will download the appropriate version, and set up the tools cache on your machine. Any subsequent runs will use the Python versions that were previously downloaded.
134+
135+
A few things to look out for when `setup-python` is first setting up the tools cache
136+
- If using Windows, your runner needs to be running as an administrator so that the appropriate directories and files can be setup. On Linux and Mac, you also need to be running with elevated permissions
137+
- On Windows, you need `7zip` installed and added to your `PATH` so that files can be extracted properly during setup
138+
- MSI installers are used when setting up Python on Windows. A word of caution as MSI installers update registry settings
139+
- The 3.8 MSI installer for Windows will not let you install another 3.8 version of Python. If `setup-python` fails for a 3.8 version of Python, make sure any previously installed versions are removed by going to "Apps & Features" in the Settings app.
104140

105141
# Using Python without `setup-python`
106142

107143
`setup-python` helps keep your dependencies explicit and ensures consistent behavior between different runners. If you use `python` in a shell on a GitHub hosted runner without `setup-python` it will default to whatever is in PATH. The default version of Python in PATH vary between runners and can change unexpectedly so we recommend you always use `setup-python`.
108144

109-
# Available versions of Python
145+
# Need to open an issue?
146+
147+
Python versions available for `setup-python` can be found in the [actions/python-versions](https://github.com/actions/python-versions) repository. You can see the build scripts, configurations, and everything that is used. You should open an issue in the `python-versions` repository if:
148+
- something might be compiled incorrectly
149+
- certain modules might be missing
150+
- there is a version of Python that you would like that is currently not available
110151

111-
For detailed information regarding the available versions of Python that are installed see [Software installed on GitHub-hosted runners](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/software-installed-on-github-hosted-runners)
152+
Any remaining issues can be filed in this repository
112153

113154
# License
114155

115156
The scripts and documentation in this project are released under the [MIT License](LICENSE)
116157

117158
# Contributions
118159

119-
Contributions are welcome! See [Contributor's Guide](docs/contributors.md)
160+
Contributions are welcome! See our [Contributor's Guide](docs/contributors.md)

‎__tests__/data/python-release.json

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"version": "1.2.3",
3+
"stable": true,
4+
"release_url": "https://github.com/actions/sometool/releases/tag/1.2.3-20200402.6",
5+
"files": [
6+
{
7+
"filename": "sometool-1.2.3-linux-x64.tar.gz",
8+
"arch": "x64",
9+
"platform": "linux",
10+
"download_url": "https://github.com/actions/sometool/releases/tag/1.2.3-20200402.6/sometool-1.2.3-linux-x64.tar.gz"
11+
}
12+
]
13+
}

‎__tests__/finder.test.ts

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,19 +18,48 @@ const tempDir = path.join(
1818
process.env['RUNNER_TOOL_CACHE'] = toolDir;
1919
process.env['RUNNER_TEMP'] = tempDir;
2020

21+
import * as tc from '@actions/tool-cache';
2122
import * as finder from '../src/find-python';
23+
import * as installer from '../src/install-python';
24+
25+
const pythonRelease = require('./data/python-release.json');
2226

2327
describe('Finder tests', () => {
28+
afterEach(() => {
29+
jest.resetAllMocks();
30+
jest.clearAllMocks();
31+
});
32+
2433
it('Finds Python if it is installed', async () => {
2534
const pythonDir: string = path.join(toolDir, 'Python', '3.0.0', 'x64');
2635
await io.mkdirP(pythonDir);
2736
fs.writeFileSync(`${pythonDir}.complete`, 'hello');
28-
// This will throw if it doesn't find it in the cache (because no such version exists)
37+
// This will throw if it doesn't find it in the cache and in the manifest (because no such version exists)
2938
await finder.findPythonVersion('3.x', 'x64');
3039
});
3140

41+
it('Finds Python if it is not installed, but exists in the manifest', async () => {
42+
const findSpy: jest.SpyInstance = jest.spyOn(
43+
installer,
44+
'findReleaseFromManifest'
45+
);
46+
findSpy.mockImplementation(() => <tc.IToolRelease>pythonRelease);
47+
48+
const installSpy: jest.SpyInstance = jest.spyOn(
49+
installer,
50+
'installCpythonFromRelease'
51+
);
52+
installSpy.mockImplementation(async () => {
53+
const pythonDir: string = path.join(toolDir, 'Python', '1.2.3', 'x64');
54+
await io.mkdirP(pythonDir);
55+
fs.writeFileSync(`${pythonDir}.complete`, 'hello');
56+
});
57+
// This will throw if it doesn't find it in the cache and in the manifest (because no such version exists)
58+
await finder.findPythonVersion('1.2.3', 'x64');
59+
});
60+
3261
it('Errors if Python is not installed', async () => {
33-
// This will throw if it doesn't find it in the cache (because no such version exists)
62+
// This will throw if it doesn't find it in the cache and in the manifest (because no such version exists)
3463
let thrown = false;
3564
try {
3665
await finder.findPythonVersion('3.300000', 'x64');

‎action.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,9 @@ inputs:
88
default: '3.x'
99
architecture:
1010
description: 'The target architecture (x86, x64) of the Python interpreter.'
11-
default: 'x64'
11+
token:
12+
description: Used to pull python distributions from actions/python-versions. Since there's a default, this is typically not supplied by the user.
13+
default: ${{ github.token }}
1214
outputs:
1315
python-version:
1416
description: "The installed python version. Useful when given a version range as input."

0 commit comments

Comments
 (0)
0