8000 Merge pull request #24291 from rgommers/improve-ci-runtime · numpy/numpy@a2ac8fd · GitHub
[go: up one dir, main page]
Skip to content

Commit a2ac8fd

Browse files
mattipmattip
authored
Merge pull request #24291 from rgommers/improve-ci-runtime
CI: improve test suite runtime via pytest parallelism and disabling mypy in most jobs
2 parents 8c1a4c2 + f3d05ea commit a2ac8fd

File tree

5 files changed

+103
-2
lines changed

5 files changed

+103
-2
lines changed

‎.github/workflows/mypy.yml

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
name: Run MyPy
2+
3+
# Mypy is too slow to run as part of regular CI. The purpose of the jobs in
4+
# this file is to cover running Mypy across:
5+
#
6+
# - OSes: Linux, Windows and macOS
7+
# - Python versions: lowest/highest supported versions, and an intermediate one
8+
#
9+
# The build matrix aims for sparse coverage across those two dimensions.
10+
# Use of BLAS/LAPACK and 8000 SIMD is disabled on purpose, because those things
11+
# don't matter for static typing and this speeds up the builds.
12+
#
13+
# This is a separate job file so it's easy to trigger by hand.
14+
15+
on:
16+
pull_request:
17+
branches:
18+
- main
19+
- maintenance/**
20+
paths-ignore:
21+
- 'benchmarks/'
22+
- '.circlecl/'
23+
- 'docs/'
24+
- 'meson_cpu/'
25+
- 'tools/'
26+
workflow_dispatch:
27+
28+
defaults:
29+
run:
30+
shell: bash
31+
32+
concurrency:
33+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
34+
cancel-in-progress: true
35+
36+
permissions:
37+
contents: read # to fetch code (actions/checkout)
38+
39+
jobs:
40+
mypy:
41+
if: "github.repository == 'numpy/numpy'"
42+
name: "MyPy"
43+
runs-on: ${{ matrix.os_python[0] }}
44+
strategy:
45+
matrix:
46+
os_python:
47+
- [ubuntu-latest, '3.10'] # switch to 3.12-dev after mypy is upgraded (see gh-23764)
48+
- [windows-2019, '3.11']
49+
- [macos-12, '3.9']
50+
steps:
51+
- uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
52+
with:
53+
submodules: recursive
54+
fetch-depth: 0
55+
- uses: actions/setup-python@61a6322f88396a6271a6ee3565807d608ecaddd1 # v4.7.0
56+
with:
57+
python-version: ${{ matrix.os_python[1] }}
58+
- name: Install dependencies
59+
run: |
60+
pip install -r build_requirements.txt
61+
pip install -r test_requirements.txt
62+
- name: Build
63+
run: |
64+
spin build -j2 -- -Dallow-noblas=true -Ddisable-optimization=true --vsenv
65+
- name: Run Mypy
66+
run: |
67+
spin mypy

‎.spin/cmds.py

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -925,3 +925,16 @@ def run(ctx, args):
925925
"""
926926
ctx.invoke(build)
927927
ctx.forward(meson_run)
928+
929+
930+
@click.command(context_settings={"ignore_unknown_options": True})
931+
@click.pass_context
932+
def mypy(ctx):
933+
"""Run Mypy tests for NumPy
934+
"""
935+
env = os.environ
936+
env['NPY_RUN_MYPY_IN_TESTSUITE'] = '1'
937+
ctx.params['pytest_args'] = [os.path.join('numpy', 'typing')]
938+
ctx.params['markexpr'] = 'full'
939+
ctx.forward(test)
940+

‎numpy/typing/tests/test_typing.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,21 @@
1818
_C_INTP,
1919
)
2020

21+
22+
# Only trigger a full `mypy` run if this environment variable is set
23+
# Note that these tests tend to take over a minute even on a macOS M1 CPU,
24+
# and more than that in CI.
25+
RUN_MYPY = "NPY_RUN_MYPY_IN_TESTSUITE" in os.environ
26+
if RUN_MYPY and RUN_MYPY not in ('0', '', 'false'):
27+
RUN_MYPY = True
28+
29+
# Skips all functions in this file
30+
pytestmark = pytest.mark.skipif(
31+
not RUN_MYPY,
32+
reason="`NPY_RUN_MYPY_IN_TESTSUITE` not set"
33+
)
34+
35+
2136
try:
2237
from mypy import api
2338
except ImportError:

‎pyproject.toml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -194,7 +194,11 @@ repair-wheel-command = "bash ./tools/wheels/repair_windows.sh {wheel} {dest_dir}
194194
package = 'numpy'
195195

196196
[tool.spin.commands]
197-
"Build" = [".spin/cmds.py:build", ".spin/cmds.py:test"]
197+
"Build" = [
198+
".spin/cmds.py:build",
199+
".spin/cmds.py:test",
200+
".spin/cmds.py:mypy",
201+
]
198202
"Environments" = [
199203
".spin/cmds.py:run", ".spin/cmds.py:ipython",
200204
".spin/cmds.py:python", ".spin/cmds.py:gdb"

‎tools/wheels/cibw_test_command.sh

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,5 +26,7 @@ fi
2626
# Set available memory value to avoid OOM problems on aarch64.
2727
# See gh-22418.
2828
export NPY_AVAILABLE_MEM="4 GB"
29-
python -c "import sys; import numpy; sys.exit(not numpy.test(label='full'))"
29+
# Run full tests with -n=auto. This makes pytest-xdist distribute tests across
30+
# the available N CPU cores: 2 by default for Linux instances and 4 for macOS arm64
31+
python -c "import sys; import numpy; sys.exit(not numpy.test(label='full', extra_argv=['-n=auto']))"
3032
python $PROJECT_DIR/tools/wheels/check_license.py

0 commit comments

Comments
 (0)
0