BUG: f2py cannot find .mod files automatically for the new meson backend #27018
Comments
|
Hi, @HaoZeke , would you please add the tag |
|
Thanks for the issue and ping. I'm thinking that there is probably no very good default solution here, since the To make this automatic, something which we could support but really probably shouldn't is to take a list of fortran files to be compiled and linked but not otherwise processed by F2PY. In general though, for all but trivial cases this will not be enough so it doesn't make a lot of sense as a feature. I think, with the new backend projects are slightly encouraged to transition the entire build system to Another option is to try to pass the |
|
Oh, thanks for the quick and detailed response! My personal viewpoint is: if meson backend cannot auto-detect .mod files, it would be better to provide a command line argument, say Thanks again for your kind help. |
|
The issue here is that the module cannot be found if there is no executable code inside. I had the same problem and posted a question to Stackexchange: https://stackoverflow.com/questions/78839287/f2py-in-numpy-2-0-1-does-not-expose-variables-the-way-numpy-1-26-did-how-can-i/78851726#78851726 An ugly workaround is to simply add some executable code inside the module, so it contains not only assignments. This got me unstuck. Just follow the answer on Stackexchange. And I still think this is a bug, but it can be worked around. |
No that's not really the issue discussed here. Adding executable code makes F2PY bind it, which has the same problem as discussed in the original code, i.e. there will be bindings which shouldn't be exposed to the user. Arguably the right fix is not the stackexchange answer (might answer there later) but as discussed to make the project skeleton, and modify the meson.build to build the fortran module, before linking it to the target generated by f2py. EDIT: The stackexchange question is actually separate and would be a bug, if the user passes a variable only fortran file, F2PY should wrap it, could you open another issue for that @P-Kaempf? |
|
The Stackexchange answer is for a specific question. Maybe it is not exactly your question, but the answer still holds true for the original question. Did you really downvote it? That helps nobody. |
That was hasty, and reverted (though it is worth noting that it was irrelevant to this issue). Please open another issue related to the question. |
|
Dear all, Unfortunately, my Everything was working fine with distutils, but I have not found a solution with meson. Do you suggest that I wait before migrating? Thank you for your help, |
Could you please provide a minimum working example showcasing the bug so it can be reproduced? In theory if the |
Hi @HaoZeke , is there any plan for making current f2py support the "automatically find .mod" condition or adding documentations to show how to manually achieve this (by transition build system to meson)? We are looking forward to a final solution to this issue since it has prevent us from using new versions of numpy-f2py for a while. |
|
Part of a workaround: However, the object code still cannot be found when loading the python module, maybe it is not linked. So still ping. The ability to add precompiled objects of modules is essential for f90wrap to work properly, see jameskermode/f90wrap#226 and in CMake build process. Many people will want to continue using CMake and scikit-build-core and not switch to Meson. |
|
This is a bit of a red herring. The compiler and Specifically for this example: ➜ python -m numpy.f2py -c a.o b.f90 -m test --build-dir bdir -I$(pwd) --backend meson
Using meson backend
Will pass --lower to f2py
See https://numpy.org/doc/stable/f2py/buildtools/meson.html
Reading fortran codes...
Reading file 'b.f90' (format:free)
Post-processing...
Block: test
Block: prt_n
In: :test:b.f90:prt_n
get_useparameters: no module para info used by prt_n
Applying post-processing hooks...
character_backward_compatibility_hook
Post-processing (stage 2)...
Building modules...
Building module "test"...
Generating possibly empty wrappers"
Maybe empty "test-f2pywrappers.f"
Constructing wrapper function "prt_n"...
prt_n()
Wrote C/API module "test" to file "./testmodule.c"
The Meson build system
Version: 1.6.0
Source dir: /home/rgoswami/Git/Github/Quansight/numscipy_playground/numpy_bugs/gh-27018/bdir
Build dir: /home/rgoswami/Git/Github/Quansight/numscipy_playground/numpy_bugs/gh-27018/bdir/bbdir
Build type: native build
Project name: test
Project version: 0.1
Fortran compiler for the host machine: /home/rgoswami/micromamba/envs/numpy-dev/bin/x86_64-conda-linux-gnu-gfortran (gcc 13.3.0 "GNU Fortran (conda-forge gcc 13.3.0-1) 13.3.0")
Fortran linker for the host machine: /home/rgoswami/micromamba/envs/numpy-dev/bin/x86_64-conda-linux-gnu-gfortran ld.bfd 2.43
C compiler for the host machine: /home/rgoswami/micromamba/envs/numpy-dev/bin/x86_64-conda-linux-gnu-cc (gcc 13.3.0 "x86_64-conda-linux-gnu-cc (conda-forge gcc 13.3.0-1) 13.3.0")
C linker for the host machine: /home/rgoswami/micromamba/envs/numpy-dev/bin/x86_64-conda-linux-gnu-cc ld.bfd 2.43
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program /home/rgoswami/micromamba/envs/numpy-dev/bin/python found: YES (/home/rgoswami/micromamba/envs/numpy-dev/bin/python)
Found pkg-config: YES (/home/rgoswami/micromamba/envs/numpy-dev/bin/pkg-config) 0.29.2
Run-time dependency python found: YES 3.12
Library quadmath found: YES
Build targets in project: 1
Found ninja-1.12.1 at /home/rgoswami/micromamba/envs/numpy-dev/bin/ninja
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /home/rgoswami/micromamba/envs/numpy-dev/bin/ninja -C /home/rgoswami/Git/Github/Quansight/numscipy_playground/numpy_bugs/gh-27018/bdir/bbdir
ninja: Entering directory `/home/rgoswami/Git/Github/Quansight/numscipy_playground/numpy_bugs/gh-27018/bdir/bbdir'
[6/6] Linking target test.cpython-312-x86_64-linux-gnu.soThis then works as expected: >>> import test
>>> test.prt_n()
n= 5@krystophny the reason why it isn't enough to set the environment variable is because internally when py.extension_module('test',
[
'''b.f90''',
'''testmodule.c''',
'''test-f2pywrappers.f''',
fortranobject_c
],
include_directories: [
inc_np,
'''/home/rgoswami/Git/Github/numscipy_playground/numpy_bugs/gh-27018''',
],
dependencies : [
py_dep,
quadmath_dep,
],
install : true)Closing for now, but perhaps a documentation update would be worthwhile. |
|
Thanks a lot, @HaoZeke . You have shown a great solution for the original problem I proposed in this issue, which is adding and b.f90 Compiling commands are The file Do you have any suggestion? Best wishes, |
Thanks for the report (and the new issue). I've responded there, please feel free to continue the discussion there. |
Uh oh!
There was an error while loading. Please reload this page.
Describe the issue:
I'm trying to compile my Fortran code using
f2py. For python>=3.12, the meson backend is now the default. But meson/f2py cannot automatically find the generated .mod files in the current directory. These .mod files could be automatically found for older versions of python and distutils backend. By the way, gcc/gfortran in this machine is 4.8.5.Thanks for any kind help and suggestion!
Best,
Jingxiang
Reproduce the code example:
A simple example is shown using two Fortran files and one Python script:
here is a.f90
b.f90
Compiling commands are
The first command would generate
a.oandpara.modfiles in the current directory.para.modcan be automatically detected for the distutils backend, but not for the meson backend.test.py
Assuming that the test.xxx.so is generated (which not True for the meson backend), the result would be
n= 5.Error message:
After running
f2py -c a.o b.f90 -m test, the error message isPython and NumPy Versions:
python>=3.12 and numpy>=1.26
Runtime Environment:
Context for the issue:
I'm a developer of the open source package MOKIT. By using
f2py, we offer lots of convenient and efficient Python APIs to users in the computational chemistry field. We are grateful for your numpy f2py functionalities. MOKIT source code are mainly written using Fortran, so we heavily reply on thef2pyto obtain corresponding .so files.Currently my workaround is to change all
.ofilenames into.f90ones in f2py commands, e.g.This is a tiny change for this simple example. But a somewhat big work to do in MOKIT. And this change would make
test.socontain more APIs than previous. For example, if there are many subroutines ina.f90, it will be all embraced intotest.so. This problem does not exist when the distutils backend is used previously. I do not know whether this is a feature or bug for the new meson backend. I'm wondering if there is any elegant solution.The text was updated successfully, but these errors were encountered: