10000 BLD: fix build issues with MSVC10 on Windows. Closes gh-4245. by rgommers · Pull Request #4892 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

BLD: fix build issues with MSVC10 on Windows. Closes gh-4245. #4892

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 20, 2014

Conversation

rgommers
Copy link
Member

Note that there are a few similar patches posted to gh-4101 and gh-4245. Those were all patches to msvc9compiler in Python distutils. Monkeypatching MSVCCompiler.link is less easy than this change to config._check_compiler; effect should be the same.

I don't have MSVC and TravisCI tests don't help here, so needs some manual testing. I'll ping some people who have had issues with this.

@juliantaylor
Copy link
Contributor

I get:

MSVCCompiler has no attribute config_version

at config.py line 80

@juliantaylor
Copy link
Contributor

right you can't access double underscore names in python, they are mangled

@juliantaylor
Copy link
Contributor

distutils.msvccompiler.get_build_version() might work

@rgommers
Copy link
Member Author

Right, forgot about that one. Why mangle a version number? I'll update to get_build_version.

@rgommers
Copy link
Member Author

updated. skipping Travis tests, changes not covered by any tests anyway.

@juliantaylor
Copy link
Contributor

seems to get past the point that caused problems before, now failing due to ldexp/frexp stuff, probably related to the generation of these ufuncs not this PR

@rgommers
Copy link
Member Author

For reference, ldexp/frexp pr is gh-4852.

@mcleu
Copy link
mcleu commented Jul 20, 2014

So when I got the bug last time I was using pip install numpy, this time I dowloaded your fork rgommers/msvc10-fix, (build 1dd9f0d37e0f7c1d60740a64c84be7840f23f78d) and then ran it as python setup.py install.
I was missing cython, so I had to pip install that first. I also reverted the change I had done following #4101 back to mfinfo = self.manifest_get_embed_info(target_desc, ld_args)
It does not build successfully, failed with exit status 1120

Here is the last few lines, though I'm not sure it's very helpful.

npymath.lib(npy_math.obj) : error LNK2019: unresolved external symbol _ldexpl re
ferenced in function _npy_ldexpl
npymath.lib(npy_math.obj) : error LNK2019: unresolved external symbol _frexpl re
ferenced in function _npy_frexpl
build\lib.win32-3.4\numpy\core\multiarray.pyd : fatal error LNK1120: 2 unresolve
d externals
error: Command "C:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\link.exe /D
LL /nologo /INCREMENTAL:NO /LIBPATH:C:\Python34\libs /LIBPATH:C:\Python34\PCbuil
d /LIBPATH:build\temp.win32-3.4 npymath.lib npysort.lib /EXPORT:PyInit_multiarra
y build\temp.win32-3.4\Release\numpy\core\src\multiarray\alloc.obj build\temp.wi
n32-3.4\Release\numpy\core\src\multiarray\arrayobject.obj build\temp.win32-3.4\R
elease\build\src.win32-3.4\numpy\core\src\multiarray\arraytypes.obj build\temp.w
in32-3.4\Release\numpy\core\src\multiarray\array_assign.obj build\temp.win32-3.4
\Release\numpy\core\src\multiarray\array_assign_scalar.obj build\temp.win32-3.4\
Release\numpy\core\src\multiarray\array_assign_array.obj build\temp.win32-3.4\Re
lease\numpy\core\src\multiarray\buffer.obj build\temp.win32-3.4\Release\numpy\co
re\src\multiarray\calculation.obj build\temp.win32-3.4\Release\numpy\core\src\mu
ltiarray\common.obj build\temp.win32-3.4\Release\numpy\core\src\multiarray\conve
rt.obj build\temp.win32-3.4\Release\numpy\core\src\multiarray\convert_datatype.o
bj build\temp.win32-3.4\Release\numpy\core\src\multiarray\conversion_utils.obj b
uild\temp.win32-3.4\Release\numpy\core\src\multiarray\ctors.obj build\temp.win32
-3.4\Release\numpy\core\src\multiarray\datetime.obj build\temp.win32-3.4\Release
\numpy\core\src\multiarray\datetime_strings.obj build\temp.win32-3.4\Release\num
py\core\src\multiarray\datetime_busday.obj build\temp.win32-3.4\Release\numpy\co
re\src\multiarray\datetime_busdaycal.obj build\temp.win32-3.4\Release\numpy\core
\src\multiarray\descriptor.obj build\temp.win32-3.4\Release\numpy\core\src\multi
array\dtype_transfer.obj build\temp.win32-3.4\Release\build\src.win32-3.4\numpy\
core\src\multiarray\einsum.obj build\temp.win32-3.4\Release\numpy\core\src\multi
array\flagsobject.obj build\temp.win32-3.4\Release\numpy\core\src\multiarray\get
set.obj build\temp.win32-3.4\Release\numpy\core\src\multiarray\hashdescr.obj bui
ld\temp.win32-3.4\Release\numpy\core\src\multiarray\item_selection.obj build\tem
p.win32-3.4\Release\numpy\core\src\multiarray\iterators.obj build\temp.win32-3.4
\Release\build\src.win32-3.4\numpy\core\src\multiarray\lowlevel_strided_loops.ob
j build\temp.win32-3.4\Release\numpy\core\src\multiarray\mapping.obj build\temp.
win32-3.4\Release\numpy\core\src\multiarray\methods.obj build\temp.win32-3.4\Rel
ease\numpy\core\src\multiarray\multiarraymodule.obj build\temp.win32-3.4\Release
\build\src.win32-3.4\numpy\core\src\multiarray\nditer_templ.obj build\temp.win32
-3.4\Release\numpy\core\src\multiarray\nditer_api.obj build\temp.win32-3.4\Relea
se\numpy\core\src\multiarray\nditer_constr.obj build\temp.win32-3.4\Release\nump
y\core\src\multiarray\nditer_pywrap.obj build\temp.win32-3.4\Release\numpy\core\
src\multiarray\number.obj build\temp.win32-3.4\Release\numpy\core\src\multiarray
\numpymemoryview.obj build\temp.win32-3.4\Release\numpy\core\src\multiarray\nump
yos.obj build\temp.win32-3.4\Release\numpy\core\src\multiarray\refcount.obj buil
d\temp.win32-3.4\Release\numpy\core\src\multiarray\sequence.obj build\temp.win32
-3.4\Release\numpy\core\src\multiarray\shape.obj build\temp.win32-3.4\Release\nu
mpy\core\src\multiarray\scalarapi.obj build\temp.win32-3.4\Release\build\src.win
32-3.4\numpy\core\src\multiarray\scalartypes.obj build\temp.win32-3.4\Release\nu
mpy\core\src\multiarray\usertypes.obj build\temp.win32-3.4\Release\numpy\core\sr
c\multiarray\ucsnarrow.obj /OUT:build\lib.win32-3.4\numpy\core\multiarray.pyd /I
MPLIB:build\temp.win32-3.4\Release\numpy\core\src\multiarray\multiarray.lib /MAN
IFESTFILE:build\temp.win32-3.4\Release\numpy\core\src\multiarray\multiarray.pyd.
manifest" failed with exit status 1120

@rgommers
Copy link
Member Author

@mcleung yes, @juliantaylor also just noticed that. That should be a recently introduced regression. I'll create a second branch based on a commit just before that regression for you to test now.

@rgommers
Copy link
Member Author

@mcleung could you try this branch: https://github.com/rgommers/numpy/tree/msvc10-fix-test?

Regressions with MSVC happen once in a while unfortunately because we don't have CI set up for it.

@juliantaylor
Copy link
Contributor

I'm just testing based on the 1.9 branch with 2.7, 3.3 and 3.4, seems to be working fine and does the same thing I always patched my local python installations too anyway so it should be safe.

@rgommers can you rebase it onto the branch point, I think this should go into 1.9
run this command on the branch (assuming origin is the main numpy repo and it was branched from master):

git rebase --onto $(git merge-base origin/master origin/maintenance/1.9.x) $(git merge-base origin/master HEAD)

possibly the commits could also be squashed.

Note that there are a few similar patches posted to numpygh-4101 and numpygh-4245.
Those were all patches to msvc9compiler in Python distutils.
Monkeypatching ``MSVCCompiler.link`` is less easy than this change
to ``config._check_compiler``; effect should be the same.

Also updates the error message shown when initializing MSVC fails.

[ci skip]
@rgommers
Copy link
Member Author

sure, done.

# flags. See issues gh-4245 and gh-4101 for details. Also
# relevant are issues 4431 and 16296 on the Python bug tracker.
from distutils import msvc9compiler
if msvc9compiler.get_build_version() >= 10:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8000

I think it would be better to put this block into a try..except
newer python versions will use newer compilers and could fix the distutils issue we are working around
when this happens this could break

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added one at first, but then removed it because a try-except block isn't going to help in that case (the code below can't really fail, and get_build_version shouldn't either). Doesn't hurt either though.

@mcleu
Copy link
mcleu commented Jul 20, 2014

It installed correctly, and then when I opened python and imported numpy I got errors. A quick reboot seemed to have fixed it.

>>> numpy.version.version
'1.10.0.dev-Unknown'

A quick test shows that all seems to be working fine.

@juliantaylor
Copy link
Contributor

k in it goes, thanks

juliantaylor added a commit that referenced this pull request Jul 20, 2014
BLD: fix build issues with MSVC10 on Windows.  Closes gh-4245.
@juliantaylor juliantaylor merged commit a28bfa5 into numpy:master Jul 20, 2014
juliantaylor added a commit that referenced this pull request Jul 20, 2014
BLD: fix build issues with MSVC10 on Windows.  Closes gh-4245.
@mcleu
Copy link
mcleu commented Jul 20, 2014

You may have merged the wrong one. I'm not too sure what the rebasing did.
Anyhow, I just uninstalled numpy and downloaded a fresh copy from the master branch (as I don't want to have a 1.10 dev version) and I am getting a similar error to the one I pointed out earlier /pull/4892#issuecomment-49559522 failed with exit status 1120.
I believe the one you want to merge is msvc10-fix-test

@juliantaylor
Copy link
Contributor

the master branch is 1.10 and currently does not compile with msvc, please try the maintenance/1.9.x branch which should work fine

@rgommers rgommers deleted the msvc10-fix branch July 20, 2014 22:10
@rgommers
Copy link
Member Author

@mcleung thanks for testing. Julian did merge the right branch. The -test branch contains the exact same code, just based on an older commit.

@mcleu
Copy link
mcleu commented Jul 20, 2014

Thanks to you guys for fixing it so quickly. I'm just doing my part reporting bugs when I find them. I'm a regular user and want to say thanks. Upgrading to 3.4 from 2.X is causing more trouble then I anticipated. Oh well, at least it's fixed now when others adopt 3.X!

@rgommers
Copy link
Member Author

@mcleung thank you for the help. And it wasn't that quick, first issue opened for this was in December...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0