8000 [f2py] BUG: missing Fortran modules in version 1.26.3 · Issue #25867 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

[f2py] BUG: missing Fortran modules in version 1.26.3 #25867

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy sta 8000 tement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mancellin opened this issue Feb 21, 2024 · 10 comments
Closed

[f2py] BUG: missing Fortran modules in version 1.26.3 #25867

mancellin opened this issue Feb 21, 2024 · 10 comments
Labels
00 - Bug component: numpy.f2py
Milestone
2.0.0 release

Comments

@mancellin
Copy link

Describe the issue:

I have a Python+Fortran library built with meson-python and f2py.

Something in version 1.26.3 is causing pip install --no-build-isolation . to compile incomplete binary extension where some Fortran modules are missing. (The ones with a filename starting with a capital letter, coincidentally?)

I browsed the list of commits but could not guess where the issue might come from. Do you have any clue?

The meson file: https://github.com/capytaine/capytaine/blob/df41fb23323f5b55ba4aed0c3f4667f86c25af11/meson.build

Reproduce the code example:

pip install --no-build-isolation .  # works on 1.26.2, not on 1.26.3
pip install .  # works on all versions

Error message:

No response

Python and NumPy Versions:

Python 3.11, Numpy 1.26.2 or 1.26.3

Runtime Environment:

No response

Context for the issue:

No response
@charris
Copy link
Member
charris commented Feb 21, 2024

@HaoZeke Thoughts?

@rgommers
Copy link
Member

Thanks for the report @mancellin.

The --no-build-isolation is a bit of a red herring, there is no direct interaction between that flag and f2py. Without it, you're building against numpy 1.26.2 (package uses oldest-supported-numpy, which selects 1.26.2 for Python 3.12). So the bug report here is "things work with 1.26.2, and don't work with 1.26.3".

Can you please add build logs and/or tracebacks that allow us to diagnose the regression?

@mancellin
Copy link
Author
mancellin commented Feb 22, 2024 •
edited
Loading

Good point @rgommers, thank you.
So I guess I can forget about pip and just check what happens when I call meson compile.

I guess the main divergence is when the following in 1.26.2:

Building modules...
    Building module "Delhommeau_float64"...
		Constructing F90 module support for "floating_point_precision"...
		  Variables: pre
		Constructing F90 module support for "constants"...
		  Variables: zero one pi ii
		Constructing F90 module support for "delhommeau_integrals"...
		Creating wrapper for Fortran function "numerical_integration"("numerical_integration")...
            Constructing wrapper function "delhommeau_integrals.numerical_integration"...
              integrals = numerical_integration(r,z,[nb_integration_points])
		Creating wrapper for Fortran function "asymptotic_approximations"("asymptotic_approximations")...
            Constructing wrapper function "delhommeau_integrals.asymptotic_approximations"...
              integrals = asymptotic_approximations(r,z)
		Creating wrapper for Fortran function "construct_tabulation"("construct_tabulation")...
            Constructing wrapper function "delhommeau_integrals.construct_tabulation"...
              tabulation = construct_tabulation(r_range,z_range,nb_integration_points)
		Creating wrapper for Fortran function "default_r_spacing"("default_r_spacing")...
            Constructing wrapper function "delhommeau_integrals.default_r_spacing"...
              default_r_spacing = default_r_spacing(nr)
		Creating wrapper for Fortran function "default_z_spacing"("default_z_spacing")...
            Constructing wrapper function "delhommeau_integrals.default_z_spacing"...
              default_z_spacing = default_z_spacing(nz)
		Creating wrapper for Fortran function "pick_in_default_tabulation"("pick_in_default_tabulation")...
            Constructing wrapper function "delhommeau_integrals.pick_in_default_tabulation"...
              integrals = pick_in_default_tabulation(r,z,r_range,z_range,tabulation)

[...]

becomes in 1.26.3:

Building modules...
    Building module "Delhommeau_float64"...
		Constructing F90 module support for "floating_point_precision"...
		  Variables: pre
		Constructing F90 module support for "constants"...
		  Variables: zero one pi ii
		Constructing F90 module support for "delhommeau_integrals"...
			Skipping delhommeau_integrals since it is in 'use'...
[...]

I ran the following

set NP_VERSION "1.26.2"
python -m venv /tmp/envs/capy_np$NP_VERSION
source /tmp/envs/capy_np$NP_VERSION/bin/activate.fish
python -m pip install numpy==$NP_VERSION charset_normalizer
mkdir -p /tmp/logs
meson setup --wipe /tmp/meson_build/capy_np$NP_VERSION/ > /tmp/logs/log_np$NP_VERSION 2>&1
meson compile -C /tmp/meson_build/capy_np$NP_VERSION >> /tmp/logs/log_np$NP_VERSION 2>>&1

and the full logs are
log_np1.26.2.gz
log_np1.26.3.gz

@mancellin
Copy link
Author
mancellin commented Feb 22, 2024 •
edited
Loading

Searching the log message on Github, I found

numpy/numpy/f2py/f90mod_rules.py

Lines 115 to 117 in e920ca4

if m['name'] in usenames and not onlyvars:
outmess(f"\t\t\tSkipping {m['name']} since it is in 'use'...\n")
continue

and indeed the missing modules are the ones that are used by other modules and not containing only variables.

I guess this is related to #25337

@mancellin mancellin changed the title [f2py] BUG: broken installation with --no-build-isolation in version 1.26.3 [f2py] BUG: missing Fortran modules in version 1.26.3 Feb 22, 2024
@HaoZeke
Copy link
Member
HaoZeke commented Feb 22, 2024

Thanks for the detailed report @mancellin, there is a fix for this on main, as part of #25337, and I know some of the patches have been left for 2.x
Do you know if it also errors out on the current main? (am traveling, will check over the weekend as well)

@mancellin
Copy link
Author

Do you know if it also errors out on the current main?

Are there some kind of nightly builds somewhere or do I need to compile from source?

@rgommers
Copy link
Member

There are nightlies at https://anaconda.org/scientific-python-nightly-wheels/numpy. Note that they're not ABI-compatible with 1.26, so if you need scipy/pandas/etc., you should also get those from https://anaconda.org/scientific-python-nightly-wheels.

@mancellin mancellin mentioned this issue Feb 22, 2024
Closed
@mancellin
Copy link
Author

It seems that the behavior of the nightly Numpy build is the same as 1.26.3 and the 'used' Fortran modules are missing.

DATA
    __f2py_numpy_version__ = '1.26.2'
    constants = <fortran object>
    delhommeau_integrals = <fortran object>
    floating_point_precision = <fortran object>
    green_rankine = <fortran object>
    green_wave = <fortran object>
    matrices = <fortran object>
    old_prony_decomposition = <fortran object>

DATA
    __f2py_numpy_version__ = '2.0.0.dev0+git20240220.6aab039'
    constants = <fortran object>
    floating_point_precision = <fortran object>
    matrices = <fortran object>
    old_prony_decomposition = <fortran object>

@charris charris added this to the 2.0.0 release milestone Feb 22, 2024
@mancellin
Copy link
Author

I'm confused by #25361, because the comment on the PR does not seem to match the code:

if a module has functions (is not an only variable module) it shouldn't be skipped.

if m['name'] in usenames and not onlyvars:  # Used module with functions
    outmess(f"\t\t\tSkipping {m['name']} since it is in 'use'...\n")
    continue  # Skip

@HaoZeke
Copy link
Member
HaoZeke commented Feb 23, 2024

I'm confused by #25361, because the comment on the PR does not seem to match the code:

if a module has functions (is not an only variable module) it shouldn't be skipped.

if m['name'] in usenames and not onlyvars:  # Used module with functions
    outmess(f"\t\t\tSkipping {m['name']} since it is in 'use'...\n")
    continue  # Skip

I think that's the right place for a fix, though a better approach would be to only generate interfaces to symbols / routines marked as public (which your code does correctly and should be handled better in F2PY).

HaoZeke added a commit to HaoZeke/numpy that referenced this issue Feb 25, 2024
@HaoZeke
TST: Add a test for numpygh-25867
7a6036b
charris added a commit that referenced this issue Feb 25, 2024
@charris
Merge pull request #25881 from HaoZeke/gh25867_fix_use_module_func_sub
90f8eeb 
BUG: Fix gh-25867 for used functions and subroutines
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug component: numpy.f2py
Projects
None yet
Milestone
2.0.0 release
Development

No branches or pull requests
4 participants
@charris @rgommers @HaoZeke @mancellin
0