8000 MSVCCompiler: Fix for feedback loop in os.environ[] by mingwandroid · Pull Request #7926 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

MSVCCompiler: Fix for feedback loop in os.environ[] #7926

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

mingwandroid
Copy link
Contributor

Each time an MSVCCompiler is instantiated, os.environ['lib'] and
os.environ['include'] doubles. This leads to hitting the 32,768
character limit.

Each time an MSVCCompiler is instantiated, os.environ['lib'] and
os.environ['include'] doubles.  This leads to hitting the 32,768
character limit.
@juliantaylor
Copy link
Contributor

you could just do if environ_lib is not None and environ_lib not in os.environ['lib']

also the joining should probably be ';'.join(a, b) instead of just a +. Is this code even functional to begin with?

@charris charris added this to the 1.11.2 release milestone Aug 15, 2016
@charris
Copy link
Member
charris commented Aug 15, 2016

Be good to get this finished up.

@mingwandroid
Copy link
Contributor Author

Sure thing, I'm just a little busy at present. Feel free to reject mine in favour of a better version from @juliantaylor, if Julian has more bandwidth.

@charris
Copy link
Member
charris commented Aug 16, 2016

@mingwandroid Out of curiosity, how did you manage to get to the 32,768 limit? I think os.environ is reinitialized when python is started up. Is there a putenv somewhere?

EDIT: Ah, I see that putenv is called when os.environ is modified. So the enviroment variable string doubles in size each time the compiler is instantiated.

@charris
Copy link
Member
charris commented Aug 16, 2016

I don't see why the environment should be updated in any case, only MSVCCompiler does that. BTW, note that MSVCCompiler is defined in both msvccompiler.py and msvc9compiler.py.

@juliantaylor @cgohlke Thoughts?

@charris
Copy link
Member
charris commented Aug 16, 2016

@dzagorny These classes are yours. What is the reason for the environ updates?

@mingwandroid
Copy link
Contributor Author
mingwandroid commented Aug 17, 2016

Compiling numpy itself with an initially quite large os.environ['include'] hits it for me. It seemed to me that this instances of this class were initialized at least 6 times.

EDIT: Oops, I just read your EDIT! Yeah, that.

@charris
Copy link
Member
charris commented Aug 17, 2016

@mingwandroid What happens if you remove the environment updates altogether?

@charris
Copy link
Member
charris commented Aug 22, 2016

Ah, I see what the original is trying to do. The base initialize overwrites the current environment variables, possibly with the same values, but possibly not, so a attempt is made to preserve the original and merge it with the new after calling the call to base initialize. The best fix is probably to do a proper merge using a set.

@charris
Copy link
Member
charris commented Aug 23, 2016

I put up an alternative fix in #7963.

@charris
Copy link
Member
charris commented Aug 23, 2016

It still isn't clear to me when not merging the original environment variables became a problem, the base class certainly doesn't do it. Perhaps that was due to using non-microsoft compilers.

@charris charris removed this from the 1.11.2 release milestone Aug 25, 2016
@charris
Copy link
Member
charris commented Aug 25, 2016

OK, I went with the alternate. Thanks @mingwandroid for initiating a fix for this problem.

@charris charris closed this Aug 25, 2016
@homu
Copy link
Contributor
homu commented Aug 25, 2016

☔ The latest upstream changes (presumably #7963) made this pull request unmergeable. Please resolve the merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0