8000 Merge pull request #20182 from HaoZeke/buildf2pydoc · numpy/numpy@f146ec1 · GitHub
[go: up one dir, main page]
Skip to content

Commit f146ec1

Browse files
rgommersrgommers
authored
Merge pull request #20182 from HaoZeke/buildf2pydoc
DOC, MAINT: Update build systems for f2py
2 parents 444a721 + 36041a0 commit f146ec1

15 files changed

+644
-13
lines changed

‎doc/source/f2py/buildtools/cmake.rst

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
.. _f2py-cmake:
2+
3+
===================
4+
Using via ``cmake``
5+
===================
6+
7+
In terms of complexity, ``cmake`` falls between ``make`` and ``meson``. The
8+
learning curve is steeper since CMake syntax is not pythonic and is closer to
9+
``make`` with environment variables.
10+
11+
However, the trade-off is enhanced flexibility and support for most architectures
12+
and compilers. An introduction to the syntax is out of scope for this document,
13+
but this `extensive CMake collection`_ of resources is great.
14+
15+
.. note::
16+
17+
``cmake`` is very popular for mixed-language systems, however support for
18+
``f2py`` is not particularly native or pleasant; and a more natural approach
19+
is to consider :ref:`f2py-skbuild`
20+
21+
Fibonacci Walkthrough (F77)
22+
===========================
23+
24+
Returning to the ``fib`` example from :ref:`f2py-getting-started` section.
25+
26+
.. literalinclude:: ./../code/fib1.f
27+
:language: fortran
28+
29+
We do not need to explicitly generate the ``python -m numpy.f2py fib1.f``
30+
output, which is ``fib1module.c``, which is beneficial. With this; we can now
31+
initialize a ``CMakeLists.txt`` file as follows:
32+
33+
.. literalinclude:: ./../code/CMakeLists.txt
34+
:language: cmake
35+
36+
A key element of the ``CMakeLists.txt`` file defined above is that the
37+
``add_custom_command`` is used to generate the wrapper ``C`` files and then
38+
added as a dependency of the actual shared library target via a
39+
``add_custom_target`` directive which prevents the command from running every
40+
time. Additionally, the method used for obtaining the ``fortranobject.c`` file
41+
can also be used to grab the ``numpy`` headers on older ``cmake`` versions.
42+
43+
This then works in the same manner as the other modules, although the naming
44+
conventions are different and the output library is not automatically prefixed
45+
with the ``cython`` information.
46+
47+
.. code:: bash
48+
49+
ls .
50+
# CMakeLists.txt fib1.f
51+
mkdir build && cd build
52+
cmake ..
53+
make
54+
python -c "import numpy as np; import fibby; a = np.zeros(9); fibby.fib(a); print (a)"
55+
# [ 0. 1. 1. 2. 3. 5. 8. 13. 21.]
56+
57+
This is particularly useful where an existing toolchain already exists and
58+
``scikit-build`` or other additional ``python`` dependencies are discouraged.
59+
60+
.. _extensive CMake collection: https://cliutils.gitlab.io/modern-cmake/

‎doc/source/f2py/distutils.rst renamed to ‎doc/source/f2py/buildtools/distutils.rst

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
.. _f2py-distutils:
2+
13
=============================
24
Using via `numpy.distutils`
35
=============================
@@ -10,23 +12,21 @@ compile Fortran sources, call F2PY to construct extension modules, etc.
1012

1113
.. topic:: Example
1214

13-
Consider the following `setup file`__ for the ``fib`` examples in the previous
14-
section:
15+
Consider the following ``setup_file.py`` for the ``fib`` and ``scalar``
16+
examples from :ref:`f2py-getting-started` section:
1517

16-
.. literalinclude:: ./code/setup_example.py
18+
.. literalinclude:: ./../code/setup_example.py
1719
:language: python
1820

1921
Running
2022

21-
::
23+
.. code-block:: bash
2224
2325
python setup_example.py build
2426
2527
will build two extension modules ``scalar`` and ``fib2`` to the
2628
build directory.
27-
28-
__ setup_example.py
29-
29+
3030
Extensions to ``distutils``
3131
===========================
3232

@@ -57,7 +57,7 @@ Extensions to ``distutils``
5757

5858
Run
5959

60-
::
60+
.. code-block:: bash
6161
6262
python <setup.py file> config_fc build_src build_ext --help
6363
@@ -73,6 +73,6 @@ Extensions to ``distutils``
7373
See ``numpy_distutils/fcompiler.py`` for an up-to-date list of
7474
supported compilers for different platforms, or run
7575

76-
::
76+
.. code-block:: bash
7777
78-
f2py -c --help-fcompiler
78+
python -m numpy.f2py -c --help-fcompiler

‎doc/source/f2py/buildtools/index.rst

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
.. _f2py-bldsys:
2+
3+
=======================
4+
F2PY and Build Systems
5+
=======================
6+
7+
In this section we will cover the various popular build systems and their usage
8+
with ``f2py``.
9+
10+
.. note::
11+
**As of November 2021**
12+
13+
The default build system for ``F2PY`` has traditionally been the through the
14+
enhanced ``numpy.distutils`` module. This module is based on ``distutils`` which
15+
will be removed in ``Python 3.12.0`` in **October 2023**; ``setuptools`` does not
16+
have support for Fortran or ``F2PY`` and it is unclear if it will be supported
17+
in the future. Alternative methods are thus increasingly more important.
18+
19+
20+
Basic Concepts
21+
===============
22+
23+
Building an extension module which includes Python and Fortran consists of:
24+
25+
- Fortran source(s)
26+
- One or more generated files from ``f2py``
27+
28+
+ A ``C`` wrapper file is always created
29+
+ Code with modules require an additional ``.f90`` wrapper
30+
31+
- ``fortranobject.{c,h}``
32+
33+
+ Distributed with ``numpy``
34+
+ Can be queried via ``python -c "import numpy.f2py; print(numpy.f2py.get_include())"``
35+
36+
- NumPy headers
37+
38+
+ Can be queried via ``python -c "import numpy; print(numpy.get_include())"``
39+
40+
- Python libraries and development headers
41+
42+
Broadly speaking there are three cases which arise when considering the outputs of ``f2py``:
43+
44+
Fortran 77 programs
45+
- Input file ``blah.f``
46+
- Generates
47+
48+
+ ``blahmodule.c``
49+
+ ``f2pywrappers.f``
50+
51+
When no ``COMMON`` blocks are present only a ``C`` wrapper file is generated.
52+
Wrappers are also generated to rewrite assumed shape arrays as automatic
53+
arrays.
54+
55+
Fortran 90 programs
56+
- Input file ``blah.f90``
57+
- Generates:
58+
59+
+ ``blahmodule.c``
60+
+ ``blah-f2pywrappers2.f90``
61+
62+
The secondary wrapper is used to handle code which is subdivided into
63+
modules. It rewrites assumed shape arrays as automatic arrays.
64+
65+
Signature files
66+
- Input file ``blah.pyf``
67+
- Generates:
68+
69+
+ ``blahmodule.c``
70+
+ ``blah-f2pywrappers2.f90`` (occasionally)
71+
+ ``f2pywrappers.f`` (occasionally)
72+
73+
Signature files ``.pyf`` do not signal their language standard via the file
74+
extension, they may generate the F90 and F77 specific wrappers depending on
75+
their contents; which shifts the burden of checking for generated files onto
76+
the build system.
77+
78+
.. note::
79+
80+
The signature file output situation is being reconsidered in `issue 20385`_ .
81+
82+
83+
In theory keeping the above requirements in hand, any build system can be
84+
adapted to generate ``f2py`` extension modules. Here we will cover a subset of
85+
the more popular systems.
86+
87+
.. note::
88+
``make`` has no place in a modern multi-language setup, and so is not
89+
discussed further.
90+
91+
Build Systems
92+
==============
93+
94+
.. toctree::
95+
:maxdepth: 2
96+
97+
distutils
98+
meson
99+
cmake
100+
skbuild
101+
102+
.. _`issue 20385`: https://github.com/numpy/numpy/issues/20385

‎doc/source/f2py/buildtools/meson.rst

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
.. _f2py-meson:
2+
3+
===================
4+
Using via ``meson``
5+
===================
6+
7+
The key advantage gained by leveraging ``meson`` over the techniques described
8+
in :ref:`f2py-distutils` is that this feeds into existing systems and larger
9+
projects with ease. ``meson`` has a rather pythonic syntax which makes it more
10+
comfortable and amenable to extension for ``python`` users.
11+
12+
.. note::
13+
14+
Meson needs to be at-least ``0.46.0`` in order to resolve the ``python`` include directories.
15+
16+
17+
Fibonacci Walkthrough (F77)
18+
===========================
19+
20+
21+
We will need the generated ``C`` wrapper before we can use a general purpose
22+
build system like ``meson``. We will acquire this by:
23+
24+
.. code-block:: bash
25+
26+
python -n numpy.f2py fib1.f -m fib2
27+
28+
Now, consider the following ``meson.build`` file for the ``fib`` and ``scalar``
29+
examples from :ref:`f2py-getting-started` section:
30+
31+
.. literalinclude:: ./../code/meson.build
32+
:language: meson
33+
34+
At this point the build will complete, but the import will fail:
35+
36+
.. code-block:: bash
37+
38+
meson setup builddir
39+
meson compile -C builddir
40+
cd builddir
41+
python -c 'import fib2'
42+
Traceback (most recent call last):
43+
File "<string>", line 1, in <module>
44+
ImportError: fib2.cpython-39-x86_64-linux-gnu.so: undefined symbol: FIB_
45+
# Check this isn't a false positive
46+
nm -A fib2.cpython-39-x86_64-linux-gnu.so | grep FIB_
47+
fib2.cpython-39-x86_64-linux-gnu.so: U FIB_
48+
49+
Recall that the original example, as reproduced below, was in SCREAMCASE:
50+
51+
.. literalinclude:: ./../code/fib1.f
52+
:language: fortran
53+
54+
With the standard approach, the subroutine exposed to ``python`` is ``fib`` and
55+
not ``FIB``. This means we have a few options. One approach (where possible) is
56+
to lowercase the original Fortran file with say:
57+
58+
.. code-block:: bash
59+
60+
tr "[:upper:]" "[:lower:]" < fib1.f > fib1.f
61+
python -n numpy.f2py fib1.f -m fib2
62+
meson --wipe builddir
63+
meson compile -C builddir
64+
cd builddir
65+
python -c 'import fib2'
66+
67+
However this requires the ability to modify the source which is not always
68+
possible. The easiest way to solve this is to let ``f2py`` deal with it:
69+
70+
.. code-block:: bash
71+
72+
python -n numpy.f2py fib1.f -m fib2 --lower
73+
meson --wipe builddir
74+
meson compile -C builddir
75+
cd builddir
76+
python -c 'import fib2'
77+
78+
79+
Automating wrapper generation
80+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
81+
82+
A major pain point in the workflow defined above, is the manual tracking of
83+
inputs. Although it would require more effort to figure out the actual outputs
84+
for reasons discussed in :ref:`f2py-bldsys`.
85+
86+
However, we can augment our workflow in a straightforward to take into account
87+
files for which the outputs are known when the build system is set up.
88+
89+
.. literalinclude:: ./../code/meson_upd.build
90+
:language: meson
91+
92+
This can be compiled and run as before.
93+
94+
.. code-block:: bash
95+
96+
rm -rf builddir
97+
meson setup builddir
98+
meson compile -C builddir
99+
cd builddir
100+
python -c "import numpy as np; import fibby; a = np.zeros(9); fibby.fib(a); print (a)"
101+
# [ 0. 1. 1. 2. 3. 5. 8. 13. 21.]
102+
103+
Salient points
104+
===============
105+
106+
It is worth keeping in mind the following:
107+
108+
* ``meson`` will default to passing ``-fimplicit-none`` under ``gfortran`` by
109+
default, which differs from that of the standard ``np.distutils`` behaviour
110+
111+
* It is not possible to use SCREAMCASE in this context, so either the contents
112+
of the ``.f`` file or the generated wrapper ``.c`` needs to be lowered to
113+
regular letters; which can be facilitated by the ``--lower`` option of
114+
``F2PY``

0 commit comments

Comments
 (0)
0