8000 Compiling numpy 1.7.1/1.8.0 for python 3.3 on Windows with VS2010/SDK fails with "broken toolchain" due to missing manifest · Issue #4101 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

Compiling numpy 1.7.1/1.8.0 for python 3.3 on Windows with VS2010/SDK fails with "broken toolchain" due to missing manifest #4101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ghost opened this issue Dec 5, 2013 · 8 comments
Closed

Compiling numpy 1.7.1/1.8.0 for python 3.3 on Windows with VS2010/SDK fails with "broken toolchain" due to missing manifest #4101

ghost opened this issue Dec 5, 2013 · 8 comments
Labels
component: numpy.distutils

Comments

@ghost
Copy link
ghost commented Dec 5, 2013

I ran into this, and the solution found on numpy-discussion worked for me, but does
turn up on a search of the issues.

Summary:

  • Python <3.2 are compiled with VS2008 (SDK v7.0,"sdk for win7 + NET 3.5.1")
  • Python 3.3 (+?) is compiled with VS2010 (SDK v7.1, "sdk for win7 + NET 4")
  • It's often recommended that c extensions (such as numpy) be compiled using the same compiler
    used to compile the python binaries, to ensure CRT compatability, so using VS2010/SDK 7.1 to compile
    numpy for python 3.3 is implied.
  • VS2010 (msvc10) behaves differently with regards to manifest files then VS2008 (msvc9) but the cpython distutils, which the numpy build process relies on,
    have not been altered to allow for this (as of python 3.3.3).
  • a cpython issue was filed in dec/2012, but has stalled.
  • The issue manifests as a "broken toolchain" exception when attempting to build
    numpy (both 1.7.1 and 1.8.0), which results from the following compiler error message.
Found executable C:\Program Files\Microsoft SDKs\Windows\v7.0A\bin\mt.exe

_configtest.exe.manifest : general error c1010070: Failed to load and
parse the manifest. The system cannot find the file specified.

The fix suggested by Christoph Gohlke on the ml is to edit the file "Lib/distutils/msvc9compiler.py:648", Located under your python 3.3 install directory, and modify the line from:

mfinfo = self.manifest_get_embed_info(target_desc, ld_args)

to

mfinfo = None

I've tried, and it has worked for me. Thanks goes to @cgohlke, the intrepid crusader for windows pydata binaries.

There, that should give pagerank something to chew on.
@rgommers
Copy link
Member

Thanks, useful to have this summarized.

http://mail.scipy.org/pipermail/numpy-discussion/2012-November/064524.html does mention that this works now with numpy master (at least under some circumstances). So do you think it's necessary to do anything else in numpy.distutils, or can we close this issue?

@ghost
Copy link
Author
ghost commented Dec 12, 2013

It certainly didn't work for me very recently, perhaps it's mingw compilation that was fixed?
The fix above is to modify cpython's distutils, I don't think the problem is specific to numpy.
It's just common to come across it when compiling numpy (though the error message you
get isn't very revealing).

I mostly just wanted the search engines to pick this up and since it's now the first
result on google for the search queries I tried: mission accomplished. Feel free to close.

@rgommers
Copy link
Member

It would still be possible to fix this bug in numpy.distutils then, but I don't think that's the right thing to do. So closing.

EDIT (Feb 2014): I think I've changed my mind on this one).

@juliantaylor
Copy link
Contributor

for reference I got numpy to build with vc10 with this patch and using

SET VS90COMNTOOLS=%VS100COMNTOOLS%

--- a/msvc9compiler.py
+++ b/msvc9compiler.py
@@ -623,6 +623,7 @@ class MSVCCompiler(CCompiler) :
             temp_manifest = os.path.join(
                     build_temp,
                     os.path.basename(output_filename) + ".manifest")
+            ld_args.append('/MANIFEST')
             ld_args.append('/MANIFESTFILE:' + temp_manifest)

             if extra_preargs:

found it in http://bugs.python.org/issue4431

@ghost
Copy link
Author
ghost commented Dec 12, 2013

changing VS90COMNTOOLS means you're using a different version of the compiler
then the one the python binary was compiled with. It does compile and run but the libc (Well, crt)
is different between the versions and that may cause mysterious unspecified errors
down the line. I haven't come across any concrete cases, it's just lore and common sense.
Compiling with the proper version eliminates the window for compat problems.

@juliantaylor juliantaylor mentioned this issue Jan 30, 2014
Closed
@ghost
Copy link
Author
ghost commented Feb 4, 2014

Compiling still broken by default as of python 3.4b3

@rgommers
Copy link
Member
rgommers commented Feb 4, 2014

See EDIT above. Since gh-4245 is a duplicate of this issue that is still open, so not reopening this one.

@rgommers
Copy link
Member
rgommers commented Feb 4, 2014

@juliantaylor why did you need SET VS90COMNTOOLS=%VS100COMNTOOLS%? It should be possible to fix this with a distutils-only patch and plain python setup.py install.

@rgommers rgommers mentioned this issue Jul 20, 2014
Merged
rgommers pushed a commit to rgommers/numpy that referenced this issue Jul 20, 2014
BLD: fix build issues with MSVC10 on Windows. Closes numpygh-4245.
7f382e0 
Note that there are a few similar patches posted to numpygh-4101 and numpygh-4245.
Those were all patches to msvc9compiler in Python distutils.
Monkeypatching ``MSVCCompiler.link`` is less easy than this change
to ``config._check_compiler``; effect should be the same.

(cherry picked from commit 2aa755eedf098b2357dcb6d11fa16719ada25ccc)
rgommers pushed a commit to rgommers/numpy that referenced this issue Jul 20, 2014
BLD: fix build issues with MSVC10 on Windows. Closes numpygh-4245.
fa0ec11 
Note that there are a few similar patches posted to numpygh-4101 and numpygh-4245.
Those were all patches to msvc9compiler in Python distutils.
Monkeypatching ``MSVCCompiler.link`` is less easy than this change
to ``config._check_compiler``; effect should be the same.

Also updates the error message shown when initializing MSVC fails.

[ci skip]
tadeu pushed a commit to ESSS/numpy that referenced this issue Apr 5, 2016
@tadeu
BLD: fix build issues with MSVC10 on Windows. Closes numpygh-4245.
ed05e26 
Note that there are a few similar patches posted to numpygh-4101 and numpygh-4245.
Those were all patches to msvc9compiler in Python distutils.
Monkeypatching ``MSVCCompiler.link`` is less easy than this change
to ``config._check_compiler``; effect should be the same.

Also updates the error message shown when initializing MSVC fails.

[ci skip]

(cherry picked from commit fa0ec11)
@ghost ghost mentioned this issue Jul 23, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component: numpy.distutils
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rgommers @juliantaylor
0