BUG: `f2py -c` does not work on Windows with meson (Python 3.12) #25000

jmrohwer · 2023-10-25T13:58:55Z

Describe the issue:

I use f2py to compile various fortran modules. Testing this on Windows with Python 3.12 and the new meson-based build system, the *.pyd and *.dll binary files are not produced, instead the directory remains unchanged, no new files added. However, the output from the compile process (pasted below) does not show an obvious error.

When doing this on Python 3.11 using distutils and the same command, the following files are produced:

hi.cp311-win_amd64.pyd
hi\.libs\libhi.LM6TMX7H4VE3WXT3WJV3AXCNAQQAXT2U.gfortran-win_amd64.dll

After adding hi\.libs to the DLL path with os.add_dll_directory(os.path.join(os.getcwd(), 'hi/.libs')) I can then run

>>> import hi
>>> hi
<module 'hi' from 'C:\\Users\\jr\\Documents\\Python\\f2py_test\\hi.cp311-win_amd64.pyd'>
>>> hi.hello()
 Hello from fortran

Under Python 3.12 if I specify the build directory (this command also generates the output in "Error message" below),

python3.12 -m numpy.f2py -c hi.pyf hi.f --build-dir build-windows

then .\build-windows\bbdir\ contains the following files:

hi.cp312-win_amd64.pyd
hi.cp312-win_amd64.dll.a
Note the library ends in *.dll.a not *.dll.

Also .\build-windows\bbdir\hi.cp312-win_amd64.pyd.p\ contains a bunch of *.obj files from the compile. However, the module can't be imported.

>>> os.chdir('build-windows/bbdir/')
>>> os.listdir(os.getcwd())
['.gitignore', '.hgignore', '.ninja_deps', '.ninja_log', 'build.ninja', 'compile_commands.json', 'hi.cp312-win_amd64.dll.a', 'hi.cp312-win_amd64.pyd', 'hi.cp312-win_amd64.pyd.p', 'meson-info', 'meson-logs', 'meson-private']
>>> import hi
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: DLL load failed while importing hi: The specified module could not be found.

Reproduce the code example:

**hi.f**

      SUBROUTINE HELLO()
      PRINT*,"Hello from fortran"
      END

**hi.pyf**

!    -*- f90 -*-
! Note: the context of this file is case sensitive.

python module hi ! in 
    interface  ! in :hi
        subroutine hello ! in :hi:hi.f
        end subroutine hello
    end interface 
end python module hi

! This file was auto-generated with f2py (version:1.23.2).
! See:
! https://web.archive.org/web/20140822061353/http://cens.ioc.ee/projects/f2py2e

Error message:

PS C:\Users\jr\Documents\Python\f2py_test> python3.12 -m numpy.f2py -c hi.pyf hi.f --build-dir build-windows
Cannot use distutils backend with Python 3.12, using meson backend instead.Using meson backend
Will pass --lower to f2py
See https://numpy.org/doc/stable/f2py/buildtools/meson.htmlReading fortran codes...
        Reading file 'hi.pyf' (format:free)
Post-processing...
        Block: hi
                        Block: hello
Applying post-processing hooks...
  character_backward_compatibility_hook
Post-processing (stage 2)...
Building modules...
    Building module "hi"...
    Generating possibly empty wrappers"
    Maybe empty "hi-f2pywrappers.f"
        Constructing wrapper function "hello"...
          hello()
    Wrote C/API module "hi" to file ".\himodule.c"
The Meson build system
Version: 1.2.3
Source dir: C:\Users\jr\Documents\Python\f2py_test\build-windows
Build dir: C:\Users\jr\Documents\Python\f2py_test\build-windows\bbdir
Build type: native build
Project name: hi
Project version: 0.1
Fortran compiler for the host machine: gfortran (gcc 12.2.0 "GNU Fortran (x86_64-posix-seh-rev2, Built by MinGW-W64 project) 12.2.0")
Fortran linker for the host machine: gfortran ld.bfd 2.39
C compiler for the host machine: gcc (gcc 12.2.0 "gcc.exe (x86_64-posix-seh-rev2, Built by MinGW-W64 project) 12.2.0")
C linker for the host machine: gcc ld.bfd 2.39
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (C:\Python312\python.exe)
Run-time dependency python found: YES 3.12
Build targets in project: 1

Found ninja-1.11.1 at C:\ProgramData\chocolatey\bin\ninja.EXE
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: C:\ProgramData\chocolatey\bin\ninja.EXE -C C:/Users/jr/Documents/Python/f2py_test/build-windows/bbdir
ninja: Entering directory `C:/Users/jr/Documents/Python/f2py_test/build-windows/bbdir'
[3/6] Compiling C object hi.cp312-win_amd64.pyd.p/186a87aefc5816048ccf8f910da29dbb8286fc40_.._.._f2py_src_fortranobject.c.obj
C:/Python312/Lib/site-packages/numpy/f2py/src/fortranobject.c: In function 'fortran_doc':
C:/Python312/Lib/site-packages/numpy/f2py/src/fortranobject.c:343:52: warning: unknown conversion type character 'z' in format [-Wformat=]
  343 |             "fortranobject.c: fortran_doc: len(p)=%zd>%zd=size:"
      |                                                    ^
C:/Python312/Lib/site-packages/numpy/f2py/src/fortranobject.c:343:56: warning: unknown conversion type character 'z' in format [-Wformat=]
  343 |             "fortranobject.c: fortran_doc: len(p)=%zd>%zd=size:"
      |                                                        ^
C:/Python312/Lib/site-packages/numpy/f2py/src/fortranobject.c:343:13: warning: too many arguments for format [-Wformat-extra-args]
  343 |             "fortranobject.c: fortran_doc: len(p)=%zd>%zd=size:"
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[6/6] Linking target hi.cp312-win_amd64.pyd

Runtime information:

>>> print(numpy.__version__)
1.26.1
>>> print(sys.version)
3.12.0 (tags/v3.12.0:0fb18b0, Oct  2 2023, 13:03:39) [MSC v.1935 64 bit (AMD64)]
>>> print(numpy.show_runtime())
[{'numpy_version': '1.26.1',
  'python': '3.12.0 (tags/v3.12.0:0fb18b0, Oct  2 2023, 13:03:39) [MSC v.1935 '
            '64 bit (AMD64)]',
  'uname': uname_result(system='Windows', node='DESKTOP-Q4HRMEM', release='10', version='10.0.19045', machine='AMD64')},
 {'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
                      'found': ['SSSE3',
                                'SSE41',
                                'POPCNT',
                                'SSE42',
                                'AVX',
                                'AVX2'],
                      'not_found': ['F16C',
                                    'FMA3',
                                    'AVX512F',
                                    'AVX512CD',
                                    'AVX512_SKX',
                                    'AVX512_CLX',
                                    'AVX512_CNL',
                                    'AVX512_ICL']}},
 {'architecture': 'Haswell',
  'filepath': 'C:\\Python312\\Lib\\site-packages\\numpy.libs\\libopenblas64__v0.3.23-293-gc2f4bdbb-gcc_10_3_0-2bde3a66a51006b2b53eb373ff767a3f.dll',
  'internal_api': 'openblas',
  'num_threads': 2,
  'prefix': 'libopenblas',
  'threading_layer': 'pthreads',
  'user_api': 'blas',
  'version': '0.3.23.dev'}]

PS C:\Users\jr\Documents\Python\f2py_test> gfortran --version
GNU Fortran (x86_64-posix-seh-rev2, Built by MinGW-W64 project) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

PS C:\Users\jr\Documents\Python\f2py_test> meson --version
1.2.3
PS C:\Users\jr\Documents\Python\f2py_test> ninja --version
1.11.1

VS Build tools 2022 (version 17.3.1) but from the output above it seems gcc is detected as C compiler. Not sure if this has to do with the issue.

Context for the issue:

I need to compile Fortran modules across platforms, on linux and macOS the same command (f2py -c hi.pyf hi.f) works flawlessly:

Python 3.12.0 (main, Oct 21 2023, 12:40:24) [GCC 13.2.1 20230801] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hi
>>> hi
<module 'hi' from '/home/jr/Documents/Python/f2py_testing/hi/hi.cpython-312-x86_64-linux-gnu.so'>
>>> hi.hello()
 Hello from fortran

Further context in #22211 and my comments in #22225.

The text was updated successfully, but these errors were encountered:

HaoZeke · 2023-10-28T13:26:55Z

Thanks for the report. The build process seems to have been completed correctly (no errors). I haven't tested meson on windows much but does it work if you rename the dll? Could you also add the meson.build file being used?

Edit: on closer inspection this is because the wrong compiler stack is detected, could you try with the environment variables set for the right compilers?

jmrohwer · 2023-10-30T19:32:01Z

Edit: on closer inspection this is because the wrong compiler stack is detected, could you try with the environment variables set for the right compilers?

Any pointers on how to do this? Windows is not my main OS and I just need to be able to compile this to distribute my software across multiple platforms. As mentioned I have VS build tools installed but it does not seem to be used. The distutils based compile for Python 3.11 and earlier just "did the right thing", selecting gfortran for the Fortran but the MS compiler for the C. Is this new build process with meson documented somewhere for Windows? I have always found compiling on Window 8000 s very challenging...

jmrohwer · 2023-10-30T19:34:33Z

I'm attaching a ZIP of the build directory, this has the meson build file, all the logs, generated C code, etc.
build-windows.zip

HaoZeke · 2023-10-31T04:58:40Z

I likely won't be able to boot into a windows machine until the weekend but from the meson documentation I think passing --vsenv should work, this patch:

diff --git i/numpy/f2py/_backends/_meson.py w/numpy/f2py/_backends/_meson.py
index 3176a5e08..13eaf0569 100644
--- i/numpy/f2py/_backends/_meson.py
+++ w/numpy/f2py/_backends/_meson.py
@@ -92,6 +92,7 @@ def _get_build_command(self):
             "meson",
             "setup",
             self.meson_build_dir,
+            "--vsenv", # gh-25000
         ]
 
     def write_meson_build(self, build_dir: Path) -> None:

Ought to work.

jmrohwer · 2023-10-31T11:18:15Z

There is progress, this now finds the VS compiler but it fails during the linking stage:
LINK : fatal error LNK1181: cannot open input file 'gfortran.lib'

The installed gfortran (from mingw) has gfortran.a and gfortran.dll.a but no gfortran.lib:

    Directory: C:\ProgramData\chocolatey\lib\mingw\tools\install\mingw64\lib\gcc\x86_64-w64-mingw32\12.2.0


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----        09/12/2022     01:37                finclude
d-----        09/12/2022     01:38                include
d-----        09/12/2022     01:15                include-fixed
d-----        09/12/2022     01:36                install-tools
-a----        09/12/2022     01:37            690 crtbegin.o
-a----        09/12/2022     01:37            891 crtend.o
-a----        09/12/2022     01:37           2879 crtfastmath.o
-a----        09/12/2022     01:38          61636 libatomic.a
-a----        09/12/2022     01:38         128200 libatomic.dll.a
-a----        09/12/2022     01:37          61120 libcaf_single.a
-a----        09/12/2022     01:37        6074900 libgcc.a
-a----        09/12/2022     01:37          56432 libgcc_eh.a
-a----        09/12/2022     01:37         257802 libgcov.a
-a----        09/12/2022     01:37        4667296 libgfortran.a
-a----        09/12/2022     01:37        1332060 libgfortran.dll.a
-a----        09/12/2022     01:37            269 libgfortran.spec
-a----        09/12/2022     01:37         384208 libgomp.a
-a----        09/12/2022     01:37         472688 libgomp.dll.a
-a----        09/12/2022     01:37            164 libgomp.spec
-a----        09/12/2022     01:37         503102 libquadmath.a
-a----        09/12/2022     01:37          80196 libquadmath.dll.a
-a----        09/12/2022     01:37          13854 libssp.a
-a----        09/12/2022     01:37          12372 libssp.dll.a
-a----        09/12/2022     01:37            824 libssp_nonshared.a
-a----        09/12/2022     01:37        7032740 libstdc++.a
-a----        09/12/2022     01:37        5578672 libstdc++.dll.a
-a----        09/12/2022     01:37           2415 libstdc++.dll.a-gdb.py
-a----        09/12/2022     01:37         620494 libstdc++fs.a
-a----        09/12/2022     01:37         342108 libsupc++.a

The complete traceback:

PS C:\Users\jr\Documents\Python\f2py_test> python3.12 -m numpy.f2py -c hi.pyf hi.f --build-dir build-windows
Cannot use distutils backend with Python 3.12, using meson backend instead.Using meson backend
Will pass --lower to f2py
See https://numpy.org/doc/stable/f2py/buildtools/meson.htmlReading fortran codes...
        Reading file 'hi.pyf' (format:free)
Post-processing...
        Block: hi
                        Block: hello
Applying post-processing hooks...
  character_backward_compatibility_hook
Post-processing (stage 2)...
Building modules...
    Building module "hi"...
    Generating possibly empty wrappers"
    Maybe empty "hi-f2pywrappers.f"
        Constructing wrapper function "hello"...
          hello()
    Wrote C/API module "hi" to file ".\himodule.c"
The Meson build system
Version: 1.2.3
Source dir: C:\Users\jr\Documents\Python\f2py_test\build-windows
Build dir: C:\Users\jr\Documents\Python\f2py_test\build-windows\bbdir
Build type: native build
Project name: hi
Project version: 0.1
Activating VS 17.3.1
Fortran compiler for the host machine: gfortran (gcc 12.2.0 "GNU Fortran (x86_64-posix-seh-rev2, Built by MinGW-W64 project) 12.2.0")
Fortran linker for the host machine: gfortran ld.bfd 2.39
C compiler for the host machine: cl (msvc 19.33.31629 "Microsoft (R) C/C++ Optimizing Compiler Version 19.33.31629 for x64")
C linker for the host machine: link link 14.33.31629.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (C:\Python312\python.exe)
Run-time dependency python found: YES 3.12
Build targets in project: 1

hi 0.1

  User defined options
    vsenv: True

Found ninja-1.10.2 at "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja\ninja.EXE"

Visual Studio environment is needed to run Ninja. It is recommended to use Meson wrapper:
C:\Python312\Scripts\meson compile -C bbdir
Activating VS 17.3.1
INFO: automatically activated MSVC compiler environment
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja\ninja.EXE" -C C:/Users/jr/Documents/Python/f2py_test/build-windows/bbdir
ninja: Entering directory `C:/Users/jr/Documents/Python/f2py_test/build-windows/bbdir'
[6/6] Linking target hi.cp312-win_amd64.pyd
FAILED: hi.cp312-win_amd64.pyd
"link"  /MACHINE:x64 /OUT:hi.cp312-win_amd64.pyd hi.cp312-win_amd64.pyd.p/hi.f.obj hi.cp312-win_amd64.pyd.p/himodule.c.obj hi.cp312-win_amd64.pyd.p/hi-f2pywrappers.f.obj hi.cp312-win_amd64.pyd.p/186a87aefc5816048ccf8f910da29dbb8286fc40_.._.._f2py_src_fortranobject.c.obj "/LIBPATH:C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0" "/LIBPATH:C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/lib/gcc/x86_64-w64-mingw32/12.2.0" "/LIBPATH:C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin/../lib/gcc" "/LIBPATH:C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/lib/gcc" "/LIBPATH:C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/lib/../lib" "/LIBPATH:C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/x86_64-w64-mingw32/lib" "/LIBPATH:C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../lib" "/LIBPATH:C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/lib" "/LIBPATH:C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/lib" "/LIBPATH:C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../.." "/nologo" "/release" "/nologo" "/OPT:REF" "/DLL" "/IMPLIB:hi.cp312-win_amd64.lib" "C:\Python312\libs\python312.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib" "gfortran.lib"
LINK : fatal error LNK1181: cannot open input file 'gfortran.lib'
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "c:\python312\Lib\site-packages\numpy\f2py\__main__.py", line 5, in <module>
    main()
  File "c:\python312\Lib\site-packages\numpy\f2py\f2py2e.py", line 732, in main
    run_compile()
  File "c:\python312\Lib\site-packages\numpy\f2py\f2py2e.py", line 705, in run_compile
    builder.compile()
  File "c:\python312\Lib\site-packages\numpy\f2py\_backends\_meson.py", line 132, in compile
    self.run_meson(self.build_dir)
  File "c:\python312\Lib\site-packages\numpy\f2py\_backends\_meson.py", line 125, in run_meson
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['meson', 'compile', '-C', 'bbdir']' returned non-zero exit status 1.

HaoZeke · 2023-10-31T14:17:49Z

@jmrohwer this is interesting enough that I've raised an issue over at meson (mesonbuild/meson#12448).

HaoZeke · 2023-11-05T20:26:08Z

TL;DR

Document the compatible toolchains and their installation better, @rgommers has an overview here

Also more specifically:

Depend on quadmath where present

Documentation can be tracked separately; this issue can be closed as soon as the quadmath patch lands (and if it works for @jmrohwer).

Process

I started out with a fresh windows machine and will document the steps I took.

First steps

MinGW via the SourceForge installer
- Basic setup with g++, fortran and developer tools
- Setup environment variables for ease of use (as demonstrated here)
Git via the Windows Git installer
- Bundled openssh
- With MinTTY but also the experimental support for launching interactive Python)
Python 3.12 from the official download page
- with admin privileges and adding python.exe to the path
- Disabled path length limit
Windows Powershell was used as the terminal
pip was used to get meson

At this point, python is not found by meson as a run-time dependency via sysconfig, since my machine is 64-bit..

Program python3 found: YES (C:\Users\blah\AppData\Local\Programs\Python\Python312\python.exe)
Need python for x86, but found x86_64
Run-time dependency python found: NO (tried sysconfig)

Back to the drawing board...

Second attempt

Uninstall mingw via the MinGW installation manager
- Remove the environment variable
Grab the 64-bit mingw
- Cygwin is kinda overkill, MSYS2 is a reasonable "lock-in" option..
Personally went with the portable mingw-builds, which has a nice installer.
- 64-bit architecture
- win32 thread model
- msvcrt C runtime
- Put it in %HOME%/mingw64
- Add C:/Users/blah/mingw64/mingw64/bin to path
numpy and ninja also via pip

At this point..

PS C:\Users\blah\build-windows\build-windows> meson setup bdir
The Meson build system
Version: 1.2.3
Source dir: C:\Users\blah\build-windows\build-windows
Build dir: C:\Users\blah\build-windows\build-windows\bdir
Build type: native build
Project name: hi
Project version: 0.1
Fortran compiler for the host machine: gfortran (gcc 13.2.0 "GNU Fortran (x86_64-win32-seh-rev1, Built by MinGW-Builds project) 13.2.0")
Fortran linker for the host machine: gfortran ld.bfd 2.39
C compiler for the host machine: gcc (gcc 13.2.0 "gcc (x86_64-win32-seh-rev1, Built by MinGW-Builds project) 13.2.0")
C linker for the host machine: gcc ld.bfd 2.39
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (C:\Users\blah\AppData\Local\Programs\Python\Python312\python.exe)
Run-time dependency python found: YES 3.12
Build targets in project: 1

Found ninja-1.11.1.git.kitware.jobserver-1 at "C:\Users\Rohit Goswami\AppData\Local\Programs\Python\Python312\Scripts\ninja.EXE"
PS C:\Users\blah\build-windows\build-windows> meson compile -C bdir
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: "C:\Users\blah\AppData\Local\Programs\Python\Python312\Scripts\ninja.EXE" -C "C:/Users/blah/build-windows/build-windows/bdir"
ninja: Entering directory `C:/Users/blah/build-windows/build-windows/bdir'
[6/6] Linking target hi.cp312-win_amd64.pyd
PS C:\Users\blah\build-windows\build-windows> cd bdir
PS C:\Users\blah\build-windows\build-windows\bdir> python
Python 3.12.0 (tags/v3.12.0:0fb18b0, Oct  2 2023, 13:03:39) [MSC v.1935 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import hi
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: DLL load failed while importing hi: The specified module could not be found.

The files at this point are:

1646 hi.cp312-win_amd64.dll.a
100428 hi.cp312-win_amd64.pyd

Reproducing what is noted in the issue :)

RTools approach (SciPy)

Used the SciPy documentation for setting up the 64-bit compilers, after removing the environment variable.

Grabbed chocolatey
As of this issue choco install rtools -y --no-progress --force --version=4.0.0.20220206.
- Added C:\rtools40\mingw64\bin to my PATH

This doesn't go very far at all:

Fortran compiler for the host machine: gfortran (gcc 8.3.0 "GNU Fortran (Built by Jeroen for the R-project) 8.3.0")
Fortran linker for the host machine: gfortran ld.bfd 2.33.1
C compiler for the host machine: cc (gcc 8.3.0 "cc (Built by Jeroen for the R-project) 8.3.0")
C linker for the host machine: cc ld.bfd 2.33.1
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (C:\Users\blah\AppData\Local\Programs\Python\Python312\python.exe)
Run-time dependency python found: YES 3.12
...
[6/6] Linking target hi.cp312-win_amd64.pyd
FAILED: hi.cp312-win_amd64.pyd
"cc"  -o hi.cp312-win_amd64.pyd hi.cp312-win_amd64.pyd.p/hi.f.obj hi.cp312-win_amd64.pyd.p/himodule.c.obj hi.cp312-win_amd64.pyd.p/hi-f2pywrappers.f.obj hi.cp312-win_amd64.pyd.p/7359b626462c6d5eeac3d0dfde0a8b196a4faf70_.._.._f2py_src_fortranobject.c.obj "-LC:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0" "-LC:/rtools40/mingw64/lib/gcc/x86_64-w64-mingw32/8.3.0" "-LC:/rtools40/mingw64/bin/../lib/gcc" "-LC:/rtools40/mingw64/lib/gcc" "-LC:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/lib/../lib" "-LC:/rtools40/mingw64/x86_64-w64-mingw32/lib" "-LC:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib" "-LC:/rtools40/mingw64/lib" "-LC:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/lib" "-LC:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../.." "-Wl,--allow-shlib-undefined" "-Wl,-O1" "-shared" "-Wl,--start-group" "-Wl,--out-implib=hi.cp312-win_amd64.dll.a" "C:\Users\Rohit Goswami\AppData\Local\Programs\Python\Python312\python312.dll" "-lkernel32" "-luser32" "-lgdi32" "-lwinspool" "-lshell32" "-lole32" "-loleaut32" "-luuid" "-lcomdlg32" "-ladvapi32" "-lgfortran" "-lm" "-Wl,--end-group"
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/libgfortran.a(write.o):(.text$get_precision+0x156): undefined reference to `quadmath_snprintf'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/libgfortran.a(write.o):(.text$get_float_string+0x170): undefined reference to `quadmath_snprintf'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/libgfortran.a(write.o):(.text$get_float_string+0xa62): undefined reference to `quadmath_snprintf'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/libgfortran.a(write.o):(.text$get_float_string+0x165c): undefined reference to `quadmath_snprintf'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/libgfortran.a(write.o):(.text$get_float_string+0x173a): undefined reference to `quadmath_snprintf'
collect2.exe: error: ld returned 1 exit status

Before fixing this, we can digress and obtain python via choco for more consistency.

With `choco python`

Removed existing python-for-windows via Add or Remove Programs.
Grabbed python via choco install python312
- Also pip install numpy meson ninja --user (path problems without)
- Added the pip directory to path C:\Users\blah\AppData\Roaming\Python\Python312\Scripts

Note that installing meson with choco doesn't work since it will not pick up the pip packages for some reason. However, this leads to the same error as before.

Fortran compiler for the host machine: gfortran (gcc 8.3.0 "GNU Fortran (Built by Jeroen for the R-project) 8.3.0")
Fortran linker for the host machine: gfortran ld.bfd 2.33.1
C compiler for the host machine: cc (gcc 8.3.0 "cc (Built by Jeroen for the R-project) 8.3.0")
C linker for the host machine: cc ld.bfd 2.33.1
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (C:\Python312\python.exe)
Run-time dependency python found: YES 3.12
Build targets in project: 1
...
"cc"  -o hi.cp312-win_amd64.pyd hi.cp312-win_amd64.pyd.p/hi.f.obj hi.cp312-win_amd64.pyd.p/himodule.c.obj hi.cp312-win_amd64.pyd.p/hi-f2pywrappers.f.obj hi.cp312-win_amd64.pyd.p/ea32e2bb3c05117e09f38f1bbb477aa287846ddd_.._.._f2py_src_fortranobject.c.obj "-LC:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0" "-LC:/rtools40/mingw64/lib/gcc/x86_64-w64-mingw32/8.3.0" "-LC:/rtools40/mingw64/bin/../lib/gcc" "-LC:/rtools40/mingw64/lib/gcc" "-LC:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/lib/../lib" "-LC:/rtools40/mingw64/x86_64-w64-mingw32/lib" "-LC:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib" "-LC:/rtools40/mingw64/lib" "-LC:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/lib" "-LC:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../.." "-Wl,--allow-shlib-undefined" "-Wl,-O1" "-shared" "-Wl,--start-group" "-Wl,--out-implib=hi.cp312-win_amd64.dll.a" "C:\Python312\python312.dll" "-lkernel32" "-luser32" "-lgdi32" "-lwinspool" "-lshell32" "-lole32" "-loleaut32" "-luuid" "-lcomdlg32" "-ladvapi32" "-lgfortran" "-lm" "-Wl,--end-group"
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/libgfortran.a(write.o):(.text$get_precision+0x156): undefined reference to `quadmath_snprintf'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/libgfortran.a(write.o):(.text$get_float_string+0x170): undefined reference to `quadmath_snprintf'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/libgfortran.a(write.o):(.text$get_float_string+0xa62): undefined reference to `quadmath_snprintf'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/libgfortran.a(write.o):(.text$get_float_string+0x165c): undefined reference to `quadmath_snprintf'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/libgfortran.a(write.o):(.text$get_float_string+0x173a): undefined reference to `quadmath_snprintf'
collect2.exe: error: ld returned 1 exit status

`quadmath` additions

At which point, for the rtools toolchain, the fix is simple enough (with or without choco python):

quadmath_dep = meson.get_compiler('fortran').find_library('quadmath')
...
                     py_dep,
                     quadmath_dep,

Which will now compile, and work:

Library quadmath found: YES
Build targets in project: 1

Found ninja-1.11.1.git.kitware.jobserver-1 at "C:\Users\blah\AppData\Roaming\Python\Python312\Scripts\ninja.EXE"
Cleaning... 0 files.

[4/4] Linking target hi.cp312-win_amd64.pyd
cd bdir
 python -c "import hi; print(hi.hello())"
 Hello from fortran

jmrohwer · 2023-11-05T20:45:40Z

Thanks, I had figured out something similar but had not yet gotten around to documenting it here. I have also gotten the compilation to work in a similar way.

Closes numpygh-25000

jmrohwer · 2023-11-06T07:41:03Z

Incidentally, in my own project (https://github.com/PySCeS/pysces) I use scikit-build and cmake in combination with f2py to build the extension module for the Fortran sources according to https://numpy.org/doc/stable/f2py/buildtools/skbuild.html. The problem is that under Windows, cmake invokes the MSVC compiler by default, and I have not been able to override this even by putting the MinGW compilers in the beginning of the PATH. This causes the build to fail. Of course I could uninstall MSVC 😄 but the CI platform on Github actions has MSVC preinstalled.

Do you know if it is possible to force cmake to use the MinGW compilers even when MSVC is installed?

rgommers · 2023-11-06T09:28:57Z

For compiler selection, you typically want to use an environment variable (e.g., CMAKE_C_COMPILER, CMAKE_CXX_COMPILER) or use a toolchain file. See for example https://stackoverflow.com/questions/56229493/how-do-you-specify-a-specific-compiler-in-for-cmake-builds-in-windows.

Closes numpygh-25000

HaoZeke · 2024-08-18T21:18:09Z

One issue relating to quadmath is that it shouldn't be explicitly linked in, see discussions relating to #15049 and MacPython/openblas-libs#82 (comment)

mattip · 2024-08-18T22:02:22Z

It is fine to link to quadmath under certain conditions. Rather than dive into the discussion of the conditions, we found it easier not to link to it. But other users may decide differently, they should check the LGPL license and make their own decisions.

HaoZeke · 2024-08-18T23:42:10Z

It is fine to link to quadmath under certain conditions. Rather than dive into the discussion of the conditions, we found it easier not to link to it. But other users may decide differently, they should check the LGPL license and make their own decisions.

Makes sense, generally I'd think for generated code especially (like F2PY), it is a rather surprising default; atleast it should signal more than it currently does :)

jmrohwer added the 00 - Bug label Oct 25, 2023

charris added the component: numpy.f2py label Oct 25, 2023

HaoZeke self-assigned this Oct 31, 2023

jmrohwer mentioned this issue Oct 31, 2023

Handle more Biomodels models biosimulators/Biosimulators_PySCeS#7

Open

HaoZeke mentioned this issue Oct 31, 2023

BUG: Trouble with gfortran and vscode mesonbuild/meson#12448

Closed

HaoZeke added a commit to HaoZeke/numpy that referenced this issue Nov 5, 2023

BLD,BUG: quadmath required where available [f2py]

06d9e43

Closes numpygh-25000

HaoZeke mentioned this issue Nov 5, 2023

BLD,BUG: quadmath required where available [f2py] #25073

Merged

HaoZeke added a commit to HaoZeke/numpy that referenced this issue Nov 12, 2023

BLD,BUG: quadmath required where available [f2py]

a68ada7

Closes numpygh-25000

HaoZeke mentioned this issue Nov 12, 2023

TST: Use meson for testing f2py #25111

Merged

7 tasks

mattip closed this as completed in #25073 Nov 13, 2023

This was referenced Nov 16, 2023

BUG: Fix moving compiled executable to root with f2py -c on Windows #25160

Merged

Proposed release 1.1.2 PySCeS/pysces#93

Merged

charris pushed a commit to charris/numpy that referenced this issue Nov 19, 2023

BLD,BUG: quadmath required where available [f2py]

b424cfe

Closes numpygh-25000

charris mentioned this issue Nov 19, 2023

BLD,BUG: quadmath required where available [f2py] #25189

Merged

charris pushed a commit to charris/numpy that referenced this issue Nov 19, 2023

BLD,BUG: quadmath required where available [f2py]

b2550d1

Closes numpygh-25000

charris mentioned this issue Nov 21, 2023

BUG: Fix moving compiled executable to root with f2py -c on Windows #25218

Merged

jmrohwer mentioned this issue Dec 15, 2023

Implementing the meson-python build backend PySCeS/pysces#94

Merged

This was referenced Jun 16, 2024

BUG: Handle --f77flags and --f90flags for meson #26703

Merged

TST: Skip an f2py module test on Windows #26706

Merged

nbelakovski mentioned this issue Jun 16, 2024

ENH: optimize: integrate the PRIMA library for COBYLA scipy/scipy#20964

Closed

HaoZeke mentioned this issue Aug 21, 2024

BUG: f2py in numpy 2.0.1 cannot compile code with explicit openmp procedure calls [Intel compilers] #27163

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: `f2py -c` does not work on Windows with meson (Python 3.12) #25000

BUG: `f2py -c` does not work on Windows with meson (Python 3.12) #25000

BUG: f2py -c does not work on Windows with meson (Python 3.12) #25000

BUG: f2py -c does not work on Windows with meson (Python 3.12) #25000

Comments

Describe the issue:

Reproduce the code example:

Error message:

Runtime information:

Context for the issue:

TL;DR

Process

First steps

Second attempt

RTools approach (SciPy)

With choco python

quadmath additions

BUG: `f2py -c` does not work on Windows with meson (Python 3.12) #25000

BUG: `f2py -c` does not work on Windows with meson (Python 3.12) #25000

With `choco python`

`quadmath` additions