8000 Merge pull request #24613 from charris/backport-24053 · numpy/numpy@4bf92ea · GitHub
[go: up one dir, main page]
Skip to content

Commit 4bf92ea

Browse files
charrischarris
authored
Merge pull request #24613 from charris/backport-24053
ENH: Adopt new macOS Accelerate BLAS/LAPACK Interfaces, including ILP64
2 parents 4fb4d7a + 784842a commit 4bf92ea

File tree

10 files changed

+307
-17
lines changed

10 files changed

+307
-17
lines changed

‎.cirrus.star

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,4 +48,7 @@ def main(ctx):
4848
if wheel:
4949
return fs.read("tools/ci/cirrus_wheels.yml")
5050

51-
return fs.read("tools/ci/cirrus_macosx_arm64.yml")
51+
if int(pr_number) < 0:
52+
return []
53+
54+
return fs.read("tools/ci/cirrus_arm.yml")

‎.github/workflows/macos.yml

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
name: macOS tests (meson)
2+
3+
on:
4+
pull_request:
5+
branches:
6+
- main
7+
- maintenance/**
8+
9+
permissions:
10+
contents: read # to fetch code (actions/checkout)
11+
12+
env:
13+
CCACHE_DIR: "${{ github.workspace }}/.ccache"
14+
15+
concurrency:
16+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
17+
cancel-in-progress: true
18+
19+
jobs:
20+
x86_conda:
21+
name: macOS x86-64 conda
22+
if: "github.repository == 'numpy/numpy'"
23+
runs-on: macos-latest
24+
strategy:
25+
matrix:
26+
python-version: ["3.11"]
27+
28+
steps:
29+
- uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
30+
with:
31+
submodules: recursive
32+
fetch-depth: 0
33+
34+
- name: Prepare cache dirs and timestamps
35+
id: prep-ccache
36+
shell: bash -l {0}
37+
run: |
38+
mkdir -p "${CCACHE_DIR}"
39+
echo "dir=$CCACHE_DIR" >> $GITHUB_OUTPUT
40+
NOW=$(date -u +"%F-%T")
41+
echo "timestamp=${NOW}" >> $GITHUB_OUTPUT
42+
echo "today=$(/bin/date -u '+%Y%m%d')" >> $GITHUB_OUTPUT
43+
44+
- name: Setup compiler cache
45+
uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3.3.1
46+
id: cache-ccache
47+
with:
48+
path: ${{ steps.prep-ccache.outputs.dir }}
49+
key: ${{ github.workflow }}-${{ matrix.python-version }}-ccache-macos-${{ steps.prep-ccache.outputs.timestamp }}
50+
restore-keys: |
51+
${{ github.workflow }}-${{ matrix.python-version }}-ccache-macos-
52+
53+
- name: Setup Mambaforge
54+
uses: conda-incubator/setup-miniconda@3b0f2504dd76ef23b6d31f291f4913fb60ab5ff3 # v2.2.0
55+
with:
56+
python-version: ${{ matrix.python-version }}
57+
channels: conda-forge
58+
channel-priority: true
59+
activate-environment: numpy-dev
60+
use-only-tar-bz2: false
61+
miniforge-variant: Mambaforge
62+
miniforge-version: latest
63+
use-mamba: true
64+
65+
# Updates if `environment.yml` or the date changes. The latter is needed to
66+
# ensure we re-solve once a day (since we don't lock versions). Could be
67+
# replaced by a conda-lock based approach in the future.
68+
- name: Cache conda environment
69+
uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3.3.1
70+
env:
71+
# Increase this value to reset cache if environment.yml has not changed
72+
CACHE_NUMBER: 1
73+
with:
74+
path: ${{ env.CONDA }}/envs/numpy-dev
75+
key:
76+
${{ runner.os }}--${{ steps.prep-ccache.outputs.today }}-conda-${{ env.CACHE_NUMBER }}-${{ hashFiles('environment.yml') }}
77+
id: envcache
78+
79+
- name: Update Conda Environment
80+
run: mamba env update -n numpy-dev -f environment.yml
81+
if: steps.envcache.outputs.cache-hit != 'true'
82+
83+
- name: Build and Install NumPy
84+
shell: bash -l {0}
85+
run: |
86+
conda activate numpy-dev
87+
CC="ccache $CC" spin build -j2
88+
89+
- name: Run test suite (full)
90+
shell: bash -l {0}
91+
run: |
92+
conda activate numpy-dev
93+
export OMP_NUM_THREADS=2
94+
spin test -j2 -m full
95+
96+
- name: Ccache statistics
97+
shell: bash -l {0}
98+
run: |
99+
conda activate numpy-dev
100+
ccache -s
101+
102+
accelerate:
103+
name: Accelerate ILP64
104+
if: "github.repository == 'numpy/numpy'"
105+
runs-on: macos-13
106+
steps:
107+
- uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
108+
with:
109+
submodules: recursive
110+
fetch-depth: 0
111+
112+
- uses: actions/setup-python@61a6322f88396a6271a6ee3565807d608ecaddd1 # v4.7.0
113+
with:
114+
python-version: '3.10'
115+
116+
- uses: maxim-lobanov/setup-xcode@9a697e2b393340c3cacd97468baa318e4c883d98 # v1.5.1
117+
with:
118+
xcode-version: '14.3'
119+
120+
- name: Install dependencies
121+
run: |
122+
pip install -r build_requirements.txt
123+
pip install pytest pytest-xdist hypothesis
124+
125+
- name: Build NumPy against Accelerate (ILP64)
126+
run: |
127+
spin build -- -Dblas=accelerate -Dlapack=accelerate -Duse-ilp64=true
128+
129+
- name: Show meson-log.txt
130+
if: always()
131+
run: 'cat build/meson-logs/meson-log.txt'
132+
133+
- name: Test
134+
run: |
135+
spin test -j2

‎build_requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@ Cython>=3.0
33
wheel==0.38.1
44
ninja
55
spin==0.5
6+
build

‎doc/release/upcoming_changes/24053.new_feature.rst

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Support for the updated Accelerate BLAS/LAPACK library, including ILP64 (64-bit
2+
integer) support, in macOS 13.3 has been added. This brings arm64 support, and
3+
significant performance improvements of up to 10x for commonly used linear
4+
algebra operations. When Accelerate is selected at build time, the 13.3+
5+
version will automatically be used if available.

‎environment.yml

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,16 +8,17 @@ channels:
88
- conda-forge
99
dependencies:
1010
- python=3.9 #need to pin to avoid issues with builds
11-
- cython>=0.29.30
11+
- cython>=3.0
1212
- compilers
1313
- openblas
1414
- nomkl
1515
- setuptools=59.2.0
16-
- meson >= 0.64.0
1716
- ninja
1817
- pkg-config
1918
- meson-python
20-
- pip # so you can use pip to install spin
19+
- pip
20+
- spin
21+
- ccache
2122
# For testing
2223
- pytest
2324
- pytest-cov

‎numpy/core/src/common/npy_cblas.h

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,21 @@ enum CBLAS_SIDE {CblasLeft=141, CblasRight=142};
2525

2626
#define CBLAS_INDEX size_t /* this may vary between platforms */
2727

28+
#ifdef ACCELERATE_NEW_LAPACK
29+
#if __MAC_OS_X_VERSION_MAX_ALLOWED < 130300
30+
#ifdef HAVE_BLAS_ILP64
31+
#error "Accelerate ILP64 support is only available with macOS 13.3 SDK or later"
32+
#endif
33+
#else
34+
#define NO_APPEND_FORTRAN
35+
#ifdef HAVE_BLAS_ILP64
36+
#define BLAS_SYMBOL_SUFFIX $NEWLAPACK$ILP64
37+
#else
38+
#define BLAS_SYMBOL_SUFFIX $NEWLAPACK
39+
#endif
40+
#endif
41+
#endif
42+
2843
#ifdef NO_APPEND_FORTRAN
2944
#define BLAS_FORTRAN_SUFFIX
3045
#else

‎numpy/distutils/system_info.py

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@
4747
_numpy_info:Numeric
4848
_pkg_config_info:None
4949
accelerate_info:accelerate
50+
accelerate_lapack_info:accelerate
5051
agg2_info:agg2
5152
amd_info:amd
5253
atlas_3_10_blas_info:atlas
@@ -534,6 +535,7 @@ def get_info(name, notfound_action=0):
534535
'lapack_ssl2': lapack_ssl2_info,
535536
'blas_ssl2': blas_ssl2_info,
536537
'accelerate': accelerate_info, # use blas_opt instead
538+
'accelerate_lapack': accelerate_lapack_info,
537539
'openblas64_': openblas64__info,
538540
'openblas64__lapack': openblas64__lapack_info,
539541
'openblas_ilp64': openblas_ilp64_info,
@@ -2015,14 +2017,17 @@ def _check_info(self, info):
20152017

20162018
class lapack_ilp64_opt_info(lapack_opt_info, _ilp64_opt_info_mixin):
20172019
notfounderror = LapackILP64NotFoundError
2018-
lapack_order = ['openblas64_', 'openblas_ilp64']
2020+
lapack_order = ['openblas64_', 'openblas_ilp64', 'accelerate']
20192021
order_env_var_name = 'NPY_LAPACK_ILP64_ORDER'
20202022

20212023
def _calc_info(self, name):
2024+
print('lapack_ilp64_opt_info._calc_info(name=%s)' % (name))
20222025
info = get_info(name + '_lapack')
20232026
if self._check_info(info):
20242027
self.set_info(**info)
20252028
return True
2029+
else:
2030+
print('%s_lapack does not exist' % (name))
20262031
return False
20272032

20282033

@@ -2163,7 +2168,7 @@ def calc_info(self):
21632168

21642169
class blas_ilp64_opt_info(blas_opt_info, _ilp64_opt_info_mixin):
21652170
notfounderror = BlasILP64NotFoundError
2166-
blas_order = ['openblas64_', 'openblas_ilp64']
2171+
blas_order = ['openblas64_', 'openblas_ilp64', 'accelerate']
21672172
order_env_var_name = 'NPY_BLAS_ILP64_ORDER'
21682173

21692174
def _calc_info(self, name):
@@ -2625,13 +2630,27 @@ def calc_info(self):
26252630
link_args.extend(['-Wl,-framework', '-Wl,vecLib'])
26262631

26272632
if args:
2633+
macros = [
2634+
('NO_ATLAS_INFO', 3),
2635+
('HAVE_CBLAS', None),
2636+
('ACCELERATE_NEW_LAPACK', None),
2637+
]
2638+
if(os.getenv('NPY_USE_BLAS_ILP64', None)):
2639+
print('Setting HAVE_BLAS_ILP64')
2640+
macros += [
2641+
('HAVE_BLAS_ILP64', None),
2642+
('ACCELERATE_LAPACK_ILP64', None),
2643+
]
26282644
self.set_info(extra_compile_args=args,
26292645
extra_link_args=link_args,
2630-
define_macros=[('NO_ATLAS_INFO', 3),
2631-
('HAVE_CBLAS', None)])
2646+
define_macros=macros)
26322647

26332648
return
26342649

2650+
class accelerate_lapack_info(accelerate_info):
2651+
def _calc_info(self):
2652+
return super()._calc_info()
2653+
26352654
class blas_src_info(system_info):
26362655
# BLAS_SRC is deprecated, please do not use this!
26372656
# Build or install a BLAS library via your package manager or from

‎numpy/linalg/meson.build

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# Note that `python_xerbla.c` was excluded on Windows in setup.py;
22
# unclear why and it seems needed, so unconditionally used here.
3-
lapack_lite_sources = [
4-
'lapack_lite/python_xerbla.c',
5-
]
3+
python_xerbla_sources = ['lapack_lite/python_xerbla.c']
4+
5+
lapack_lite_sources = []
66
if not have_lapack
7-
lapack_lite_sources += [
7+
lapack_lite_sources = [
88
'lapack_lite/f2c.c',
99
'lapack_lite/f2c_c_lapack.c',
1010
'lapack_lite/f2c_d_lapack.c',
@@ -19,6 +19,7 @@ endif
1919
py.extension_module('lapack_lite',
2020
[
2121
'lapack_litemodule.c',
22+
python_xerbla_sources,
2223
lapack_lite_sources,
2324
],
2425
dependencies: [np_core_dep, blas_dep, lapack_dep],
@@ -29,6 +30,7 @@ py.extension_module('lapack_lite',
2930
py.extension_module('_umath_linalg',
3031
[
3132
'umath_linalg.cpp',
33+
python_xerbla_sources,
3234
lapack_lite_sources,
3335
],
3436
dependencies: [np_core_dep, blas_dep, lapack_dep],

0 commit comments

Comments
 (0)
0