Closed
Description
Describe the issue:
Hello, this is my first numpy ticket, so hopefully, I can follow the basic guidelines. I searched through the existing issues, and while there are others similar, I think mine has not yet been raised.
I'm trying to wrap some Fortran using F2PY via numpy.distutils, and the generated package and all of its modules are working great. Still, I'm making every effort to reduce the number of tasks invoked by the build_ext command and clean up the console output. But something is incorrectly appending executables and source files that I did not specify. I believe it is clear that the problem is that somewhere the code is splitting the executable path at whitespaces, even after it correctly interprets the same path beforehand.
I'm on Windows 10 and have the ifort compiler provided by Intel Parallel Studio XE 2019 installed at C:\Program Files (x86)\IntelSWTools. There are a few lines in the output from my setup.py script that indicate the problem:
running build_ext
running build_src
build_src
building extension "..py_alchemyCore" sources
f2py options: []
adding 'build\src.win-amd64-3.9\build\src.win-amd64-3.9\fortranobject.c' to sources.
adding 'build\src.win-amd64-3.9\build\src.win-amd64-3.9\' to include_dirs.
adding 'build\src.win-amd64-3.9\py_alchemyCore-f2pywrappers2.f90' to sources.
build_src: building npy-pkg config files
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
customize MSVCCompiler using build_ext
customize IntelEM64VisualFCompiler
Found executable C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe
Could not locate executable C:\Program
customize IntelEM64VisualFCompiler using build_ext
building '..py_alchemyCore' extension
compiling C sources
C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DF2PY_REPORT_ON_ARRAY_COPY=1 -Ibuild\src.win-amd64-3.9\build\src.win-amd64-3.9\ -IC:\Users\cwong\.venv\alchemy-env\lib\site-packages\numpy\core\include -IC:\Users\cwong\.venv\alchemy-env\include -IC:\Python39\include -IC:\Python39\include -IC:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt /Tcbuild\src.win-amd64-3.9\py_alchemyCoremodule.c /Fobuild\temp.win-amd64-3.9\Release\build\src.win-amd64-3.9\py_alchemyCoremodule.obj
C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DF2PY_REPORT_ON_ARRAY_COPY=1 -Ibuild\src.win-amd64-3.9\build\src.win-amd64-3.9\ -IC:\Users\cwong\.venv\alchemy-env\lib\site-packages\numpy\core\include -IC:\Users\cwong\.venv\alchemy-env\include -IC:\Python39\include -IC:\Python39\include -IC:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt /Tcbuild\src.win-amd64-3.9\build\src.win-amd64-3.9\fortranobject.c /Fobuild\temp.win-amd64-3.9\Release\build\src.win-amd64-3.9\build\src.win-amd64-3.9\fortranobject.obj
compiling Fortran sources
Fortran f77 compiler: C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe /nologo /MD /nbs /names:lowercase /assume:underscore /recursive /O1 /QaxSSE3
Fortran f90 compiler: C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe /nologo /MD /nbs /names:lowercase /assume:underscore /recursive /O1 /QaxSSE3
Fortran fix compiler: C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe /nologo /MD /nbs /names:lowercase /assume:underscore /recursive /O1 /QaxSSE3
compile options: '-DF2PY_REPORT_ON_ARRAY_COPY=1 -Ibuild\src.win-amd64-3.9\build\src.win-amd64-3.9\ -IC:\Users\cwong\.venv\alchemy-env\lib\site-packages\numpy\core\include -IC:\Users\cwong\.venv\alchemy-env\include -IC:\Python39\include -IC:\Python39\include -c'
extra f90 options: '/free'
ifort.exe:f90: ..\current\alchemy_core.for
ifort: command line warning #10161: unrecognized source type 'Files'; object file assumed
ifort: command line warning #10161: unrecognized source type '(x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe'; object file assumed
ifort: command line warning #10006: ignoring unknown option '/c '
ifort: warning #10145: no action performed for file 'Files'
ifort: warning #10145: no action performed for file '(x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe'
extra f90 options: '/free'
ifort.exe:f90: dummy_code.f90
ifort: command line warning #10161: unrecognized source type 'Files'; object file assumed
ifort: command line warning #10161: unrecognized source type '(x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe'; object file assumed
ifort: command line warning #10006: ignoring unknown option '/c '
ifort: warning #10145: no action performed for file 'Files'
ifort: warning #10145: no action performed for file '(x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe'
extra f90 options: '/free'
ifort.exe:f90: build\src.win-amd64-3.9\py_alchemyCore-f2pywrappers2.f90
ifort: command line warning #10161: unrecognized source type 'Files'; object file assumed
ifort: command line warning #10161: unrecognized source type '(x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe'; object file assumed
ifort: command line warning #10006: ignoring unknown option '/c '
ifort: warning #10145: no action performed for file 'Files'
ifort: warning #10145: no action performed for file '(x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe'
C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\compiler\lib\intel64 /LIBPATH:C:\Users\cwong\.venv\alchemy-env\libs /LIBPATH:C:\Python39\libs /LIBPATH:C:\Python39 /LIBPATH:C:\Users\cwong\.venv\alchemy-env\PCbuild\amd64 /LIBPATH:C:\Users\cwong\.venv\alchemy-env\libs /LIBPATH:C:\Python39\libs /LIBPATH:C:\Python39 /LIBPATH:C:\Users\cwong\.venv\alchemy-env\PCbuild\amd64 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\ATLMFC\lib\x64 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\lib\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\ucrt\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64 /EXPORT:PyInit_py_alchemyCore build\temp.win-amd64-3.9\Release\build\src.win-amd64-3.9\py_alchemyCoremodule.obj build\temp.win-amd64-3.9\Release\build\src.win-amd64-3.9\build\src.win-amd64-3.9\fortranobject.obj build\temp.win-amd64-3.9\Release\alchemy_core\current\alchemy_core.o build\temp.win-amd64-3.9\Release\dummy_code.o build\temp.win-amd64-3.9\Release\build\src.win-amd64-3.9\py_alchemyCore-f2pywrappers2.o /OUT:py_alchemyCore.cp39-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.9\Release\build\src.win-amd64-3.9\py_alchemyCore.cp39-win_amd64.lib
First, after it correctly locates my compiler executable, it mysteriously tries to find another one C:\Program:
Found executable C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe
Could not locate executable C:\Program
Second, it tries to run ifort with two files named Files and (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe:
Fortran f90 compiler: C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe /nologo /MD /nbs /names:lowercase /assume:underscore /recursive /O1 /QaxSSE3
And finally, the compiler complains that it doesn't recognize these files and ignores them, thankfully. Somehow, the executable path is split like C:\Program, Files, (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64\ifort.exe, and then passed around all willy-nilly. I'm not sure if this is a numpy issue, but it seems somewhat known. The commonly accepted solution appears to be to use a different C compiler like mingw or intelemw, albeit I prefer to stick with the default msvc one.
Reproduce the code example:
import os, sys
from numpy.distutils.core import setup
from numpy.distutils.misc_util import Configuration
def configuration(parent_package='', top_path=None):
arch = 'intel64'
intel_dir = os.path.join('C:', os.sep, 'Program Files (x86)', 'IntelSWTools',
'compilers_and_libraries', 'windows')
os.environ['Path'] += ';' + os.path.join(intel_dir, 'bin', arch)
config = Configuration('.', parent_package, top_path)
config.add_extension(
name='py_alchemyCore',
sources=[os.path.join('..', 'current', 'alchemy_core.for'), 'dummy_code.f90'],
library_dirs=[os.path.join(intel_dir, 'compiler', 'lib', arch)],
define_macros=[('F2PY_REPORT_ON_ARRAY_COPY', 1)],
extra_f90_compile_args=['/free']
)
return config
if __name__ == '__main__':
default_args = ['build_ext', '--inplace', '--force', '--fcompiler=intelvem']
setup(script_args=default_args if len(sys.argv) == 1 else None, configuration=configuration)Error message:
No response
NumPy/Python version information:
1.19.5 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit (AMD64)]