1
1
This file gives an overview of what is necessary to build binary releases for
2
- NumPy. Windows binaries are built here using Wine, they can of course also be
3
- built on Windows itself. Building OS X binaries on another platform is not
4
- possible, but our current OSX binary build procedure uses travis-ci virtual
5
- machines running OSX.
2
+ NumPy.
6
3
7
4
Current build and release info
8
5
==============================
@@ -39,15 +36,15 @@ Release Scripts
39
36
Supported platforms and versions
40
37
================================
41
38
42
- Python 2.6-2. 7 and >=3.2 are the currently supported versions when building
43
- from source. We test numpy against all these versions every time we merge
44
- code to trunk. Binary installers may be available for a subset of these
45
- versions (see below).
39
+ Python 2.7 and >=3.4 are the currently supported versions when building from
40
+ source. We test numpy against all these versions every time we merge code to
41
+ trunk. Binary installers may be available for a subset of these versions (see
42
+ below).
46
43
47
44
OS X
48
45
----
49
46
50
- Python 2.7 and >=3.3 are the versions for which we provide binary installers.
47
+ Python 2.7 and >=3.4 are the versions for which we provide binary installers.
51
48
OS X versions >= 10.6 are supported. We build binary wheels for OSX that are
52
49
compatible with Python.org Python, system Python, homebrew and macports - see
53
50
this `OSX wheel building summary
@@ -56,47 +53,49 @@ this `OSX wheel building summary
56
53
Windows
57
54
-------
58
55
59
- 32-bit Python 2.7, 3.3 , 3.4 are the versions for which we provide binary
60
- installers. Windows XP, Vista and 7 are supported. Our current windows mingw
61
- toolchain is not able to build 64-bit binaries of numpy. We are hoping to
62
- update to a `mingw-w64 toolchain
63
- <https://github.com/numpy/numpy/wiki/Mingw-w64-faq> `_ soon.
56
+ 32-bit Python 2.7, 3.4 , 3.5 are the versions for which we provide binary
57
+ installers. Windows XP, Vista, 7, 8 and 10 are supported. We build numpy
58
+ using the MSVC compilers on Appveyor, but we are hoping to update to a
59
+ `mingw-w64 toolchain < https://github.com/numpy/numpy/wiki/Mingw-w64-faq >`_
60
+ soon.
64
61
65
62
Linux
66
63
-----
67
64
68
- Many distributions include NumPy. Building from source is also relatively
69
- straightforward. Only tarballs are created for Linux, no specific binary
70
- installers are provided (yet) .
65
+ We build and ship ` manylinux1 < https://www.python.org/dev/peps/pep-0513 >`_
66
+ wheels for numpy. Many Linux distributions include their own binary builds
67
+ of NumPy .
71
68
72
69
BSD / Solaris
73
70
-------------
74
- No binaries are provided, but succesful builds on Solaris and BSD have been
71
+
72
+ No binaries are provided, but successful builds on Solaris and BSD have been
75
73
reported.
76
74
77
75
Tool chain
78
76
==========
79
77
78
+ We build all our wheels on cloud infrastructure - so this list of compilers is
79
+ for information and debugging builds locally. See the ``.travis.yml `` and
80
+ ``appveyor.yml `` scripts in the `numpy wheels `_ repo for the definitive source
81
+ of the build recipes.
82
+
80
83
Compilers
81
84
---------
82
85
83
86
The same gcc version is used as the one with which Python itself is built on
84
87
each platform. At the moment this means:
85
88
86
89
* OS X builds on travis currently use `clang `. It appears that binary wheels
87
- for OSX >= 10.6 can be safely built from from OSX 10.9 when building against
88
- the Python from the Python.org installers.
89
- * Windows builds use MinGW 3.4.5. Updating this to a more recent MinGW with
90
- GCC 4.x is desired, but there are still practical difficulties in building
91
- the binary installers .
90
+ for OSX >= 10.6 can be safely built from from the travis-ci OSX 10.9 VMs
91
+ when building against the Python from the Python.org installers;
92
+ * Windows builds use the MSVC version corresponding to the Python being built
93
+ against;
94
+ * Manylinux1 wheels use the gcc provided on the Manylinux docker images .
92
95
93
96
You will need Cython for building the binaries. Cython compiles the ``.pyx ``
94
97
files in the numpy distribution to ``.c `` files.
95
98
96
- Fortran: on OS X gfortran from `this site <http://r.research.att.com/tools/ >`_
97
- is used. On Windows g77 (included in MinGW) is the current default, in the future
98
- this may shift to gfortran as well.
99
-
100
99
Python
101
100
------
102
101
* Python(s) from `python.org <http://python.org >`_
@@ -110,54 +109,6 @@ Building docs
110
109
* Matplotlib
111
110
* Texlive (or MikTeX on Windows)
112
111
113
- Wine
114
- ----
115
- For building Windows binaries on OS X Wine can be used. In Wine the following
116
- needs to be installed:
117
-
118
- * Python 2.6-2.7 and 3.3
119
- * MakeNsis
120
- * CpuId plugin for MakeNsis : this can be found in the NumPy source tree under
121
- tools/win32build/cpucaps and has to be built with MinGW (see SConstruct file in
122
- that dir for details)
123
- * MinGW
124
- * ATLAS, 3x ([No SSE, SSE2, SSE3] for superpack installer) : ATLAS does not
125
- compile under wine or on Windows out of the box. Binaries for ATLAS can be
126
- found in the vendor repository on GitHub (http://github.com/numpy/vendor).
127
-
128
- To install Wine on OS X Snow Leopard the current options are to compile a
129
- current unstable version ,`<http://wiki.winehq.org/MacOSX/Building>`_, or to use
130
- an install script from `here <http://code.google.com/p/osxwinebuilder/ >`_. For
131
- me, the former option did not work (everything compiled, but after installing
132
- Python the command ``import tempfile `` resulted in an exception. The latter
133
- option did work.
134
-
135
- After successful installation and an invocation of the wine executable, a
136
- ~/.wine folder exists - new programs will be installed there in
137
- ~/.wine/drive_c. Installing Windows programs with .exe executables is done by
138
- running
139
-
140
- $ wine yourprog.exe
141
-
142
- and MSI installers can be installed with
143
-
144
- $ msiexec /i yourprog.msi
145
-
146
- For the above to work you probably need to put the wine-1.x.x/bin directory in
147
- your PATH.
148
-
149
- To install MinGW, the easiest option is to use the automated installer on the
150
- MinGW download page. This will give you (at this moment) GCC 3.4.5; GCC 4.x is
151
- still not supported officially by MinGW.
152
-
153
- To be able to use gcc and MakeNsis in Wine, the locations of gcc.exe and
154
- makensis.exe should be added to the Windows environment variable PATH. This can
155
- easily be done by running
156
-
157
- $ wine regedit
158
-
159
- add adding a PATH variable in HKEY_CURRENT_USER/Environment.
160
-
161
112
Virtualenv
162
113
----------
163
114
Virtualenv is a very useful tool to keep several versions of packages around.
@@ -167,18 +118,17 @@ It is also used in the Paver script to build the docs.
167
118
What is released
168
119
================
169
120
170
- Binaries
171
- --------
172
-
173
- Windows binary installers in "superpack" form for Python 2.7/3.3/3.4. A
174
- superpack contains three builds, for SSE2, SSE3 and no SSE.
175
-
176
121
Wheels
177
122
------
178
123
179
- OSX wheels built via travis-ci : see - see `building OSX wheels `_.
124
+ * Windows wheels for Python 2.7, 3.4, 3.5, for 32- and 64-bit, built using
125
+ Appveyor;
126
+ * Dual architecture OSX wheels built via travis-ci;
127
+ * 32- and 64-bit Manylinux1 wheels built via travis-ci.
128
+
129
+ See the `numpy wheels `_ building repository for more detail.
180
130
181
- .. _ build OSX wheels : https://github.com/MacPython/numpy-wheels
131
+ .. _ numpy wheels : https://github.com/MacPython/numpy-wheels
182
132
183
133
Other
184
134
-----
@@ -366,78 +316,36 @@ Increment the release number in setup.py. Release candidates should have "rc1"
366
316
Also create a new version hash in cversions.txt and a corresponding version
367
317
define NPY_x_y_API_VERSION in numpyconfig.h
368
318
369
- Trigger the OSX builds on travis
370
- --------------------------------
319
+ Trigger the wheel builds on travis-ci and Appveyor
320
+ --------------------------------------------------
321
+
322
+ See the `numpy wheels ` repository.
371
323
372
- See ` build OSX wheels `_.
324
+ In that repository edit the files:
373
325
374
- You may need to check the ``.travis.yml `` file of the
375
- https://github.com/MacPython/numpy-wheels repository .
326
+ * ``.travis.yml ``;
327
+ * `` appveyor.yml `` .
376
328
377
- Make sure that the releast tag has been pushed, and that the `` .travis.yml ``
378
- is set thusly::
329
+ In both cases, set the `` BUILD_COMMIT `` variable to the current release tag -
330
+ e.g. `` v1.11.1 ``.
379
331
380
- - NP_COMMIT=latest-tag # comment out to build version in submodule
332
+ Make sure that the release tag has been pushed.
381
333
382
- Trigger a build by doing an empty (or otherwise) commit to the repository::
334
+ Trigger a build by doing a commit of your edits to ``.travis.yml `` and
335
+ ``appveyor.yml `` to the repository::
383
336
384
337
cd /path/to/numpy-wheels
385
- git commit --allow-empty
338
+ # Edit .travis.yml, appveyor.yml
339
+ git commit
386
340
git push
387
341
388
- The wheels, once built, appear in http://wheels.scipy.org
389
-
390
- Trigger Windows builds on Appveyor
391
- ----------------------------------
392
-
393
- See: `build Windows wheels `_
394
-
395
- * Clone / update the https://github.com/numpy/windows-wheel-builder repository;
396
- * Check the ``appveyor.yml `` file in that repository;
397
- * Edit the line starting ``NP_VERSION: `` to give the numpy tag that you want
398
- to build;
399
- * Push up to github to trigger a build.
400
-
401
- The wheels appear in a Rackspace CDN container at:
402
-
403
- * http://58688808cd85529d4031-38dee5dca2544308e91131f21428d924.r12.cf2.rackcdn.com
404
- * https://84c1a9a06db6836f5a98-38dee5dca2544308e91131f21428d924.ssl.cf2.rackcdn.com
405
-
406
- The contents via the HTTPS URL seems to get updated more slowly than via the
407
- HTTP URL, so if you need the binaries quickly, prefer the HTTP URL.
408
-
409
- .. _build Windows wheels : https://github.com/numpy/windows-wheel-builder
342
+ The wheels, once built, appear at a Rackspace container pointed at by:
410
343
411
- Trigger Manylinux builds on travis-ci
412
- -------------------------------------
344
+ * http://wheels.scipy.org
345
+ * https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com
413
346
414
- .. note ::
415
-
416
- Until we move the manylinux build scripts, you'll need to ask
417
- ``@matthew-brett `` to make you a collaborator on the manylinux repos.
418
-
419
- * Clone / update the repository at
420
- https://github.com/matthew-brett/manylinux-builds
421
- * Edit the line in ``.travis.yml `` starting ``NUMPY_VERSIONS= `` to set the
422
- numpy tag to build;
423
- * Push your edits to ``.travis.yml `` up to github to trigger a mass manylinux
424
- build;
425
- * Clone / update the repository at
426
- https://github.com/matthew-brett/manylinux-testing;
427
- * Push an empty commit to the ``manylinux-testing `` repo to trigger a test run
428
- of the newly-built numpy wheels with a range of dependent libraries, as well
429
- as numpy's own unit tests. The tests will take several hours.
430
-
431
- The built wheels will be available from a Rackspace CDN container at:
432
-
433
- * http://ccdd0ebb5a931e58c7c5-aae005c4999d7244ac63632f8b80e089.r77.cf2.rackcdn.com
434
- * https://d9a97980b71d47cde94d-aae005c4999d7244ac63632f8b80e089.ssl.cf2.rackcdn.com
435
-
F438
436
- As for the other Rackspace containers, the HTTP address may update first, and
437
- you should wait 15 minutes after the build finishes before fetching the
438
- binaries. For the manylinux wheels, the time to run ``manylinux-testing `` is
439
- much greater than 15 minutes, so waiting for the tests to pass will be enough
440
- time for the binaries to refresh on Rackspace.
347
+ The HTTP address may update first, and you should wait 15 minutes after the
348
+ build finishes before fetching the binaries.
441
349
442
350
Make the release
443
351
----------------
@@ -471,30 +379,39 @@ downloading all the Windows, Manylinux, OSX wheels and uploading to PyPI.
471
379
::
472
380
473
381
cd ~/wheelhouse # local directory to cache wheel downloads
474
- MANYLINUX_URL=http://ccdd0ebb5a931e58c7c5-aae005c4999d7244ac63632f8b80e089.r77.cf2.rackcdn.com
475
- WINDOWS_URL=http://58688808cd85529d4031-38dee5dca2544308e91131f21428d924.r12.cf2.rackcdn.com
476
- OSX_URL=http://wheels.scipy.org
477
- wheel-uploader -u $MANYLINUX_URL -v -s -t manylinux1 numpy 1.11.1rc1
478
- wheel-uploader -u $WINDOWS_URL -v -s -t win numpy 1.11.1rc1
479
- wheel-uploader -u $OSX_URL -v -s -t macosx numpy 1.11.1rc1
382
+ CDN_URL=https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com
383
+ wheel-uploader -u $CDN_URL -w warehouse -v -s -t win numpy 1.11.1rc1
384
+ wheel-uploader -u $CDN_URL -w warehouse -v -s -t macosx numpy 1.11.1rc1
385
+ wheel-uploader -u $CDN_URL -w warehouse -v -s -t manylinux1 numpy 1.11.1rc1
480
386
481
387
The ``-v `` flag gives verbose feedback, ``-s `` causes the script to sign the
482
- wheels with your GPG key before upload.
483
-
484
- You may well find that these uploads break at some point, with error messages
485
- from the PyPI server. In this case you'll have to continue the uploads by
486
- hand using `twine <https://pypi.python.org/pypi/twine >`_, using something
487
- like::
488
-
489
- twine upload -s numpy-1.11.1rc1-cp34-*.whl
490
-
491
- Do this for the wheel files that ``wheel-uploader `` downloaded, but for which
492
- the upload failed.
493
-
494
- The ``warehouse `` PyPI server seems to be more reliable in receiving automated
495
- wheel uploads. You can set the repository to upload to with the ``-r `` flag
496
- to ``wheel-uploader `` and ``twine ``. The warehouse repository URL for your
497
- ``~/.pypirc `` file is https://upload.pypi.io/legacy/
388
+ wheels with your GPG key before upload. ``-r warehouse `` causes the upload to
389
+ use the Warehouse PyPI server. This is a good idea because the Warehouse
390
+ server seems to be a lot more reliable in receiving automated wheel uploads.
391
+ For this flag to work, you will need a ``warehouse `` section in your
392
+ ``~/.pypirc `` file, of form:
393
+
394
+ [distutils]
395
+ index-servers =
396
+ pypi
397
+ warehouse
398
+
399
+ [pypi]
400
+ username:your_user_name
401
+ password:your_password
402
+
403
+ [warehouse]
404
+ repository: https://upload.pypi.io/legacy/
405
+ username: your_user_name
406
+ password: your_password
407
+
408
+ [server-login]
409
+ username:your_user_name
410
+ password:your_password
411
+
412
+ Don't forget to upload the wheels before the source tarball, so there is no
413
+ period for which people switch from an expected binary install to a source
414
+ install from PyPI.
498
415
499
416
There are two ways to update the source release on PyPI, the first one is::
500
417
@@ -509,9 +426,6 @@ The second way is to upload the PKG_INFO file inside the sdist dir in the
509
426
web interface of PyPI. The source tarball can also be uploaded through this
510
427
interface.
511
428
512
- To push the travis-ci OSX wheels up to PyPI see :
513
- https://github.com/MacPython/numpy-wheels#uploading-the-built-wheels-to-pypi
514
-
515
429
.. _push-tag-and-commit :
516
430
517
431
Push the release tag and commit
0 commit comments