-
-
Notifications
You must be signed in to change notification settings - Fork 32.1k
Parallel build race condition on AIX since python-2.7 #63720
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Since python-3.2, there is a race condition building in parallel on AIX: Consider these Makefile(.pre.in) rules: $(BUILDPYTHON): ... Modules/_testembed: ... Modules/_freeze_importlib: ... On AIX, the variables get these values: LINKCC = $(srcdir)/Modules/makexp_aix Modules/python.exp ... Now $(BUILDPYTHON) and Modules/_testembed may run in parallel, causing Modules/python.exp to be created by two instances of makexp_aix eventually running at the same time. Attached patch fixes this problem for cpython tip (doubt supporting AIX 4.1 and earlier still is necessary). Thank you! |
Wouldn't it be better if linking _testembed generated _testembed.exp instead of generating python.exp? I hope using $@.exp somehow could help. Hard-coding the name of the export file sounds like a flaw in the first place. |
I'm unsure about the real purpose of _testembed, but given the name it does make sense to me to export the same symbols as $(BUILDPYTHON), thus reusing python.exp. |
+1 |
Patch including configure update now. |
I also have found this goes back since Python 2.7. I have refreshed the patched for the tip of CPython. What can I do to help push this forward? |
I may be able to simplify the build on AIX by removing ld_so_aix and python.exp entirely. Would this be a preferred solution if I am able to get something working? If so, should I create a separate issue to track the change? |
The original patch is a bit dated, do we still need the export symbol generation? "Use the -G option to create a shared library from the generated object files, and to enable runtime linking with applications that support it." "If you do not specify a -bE option, all the global symbols are exported except for those symbols that have the hidden or internal visibility attribute." |
Yes, export file generation still is required. Python does not need to utilize runtime linking. Using -G is a very bad choice and severely discouraged with severe performance and other penalties. |
Okay, thanks. The -G option is of course attractive for Linux projects I've added support for libmpdec/libmpdec++ (next release) for AIX-style I can confirm that -G is substantially slower, so I'm going to commit this |
I can't find the reason for:
Modules/python.exp is hardcoded elsewhere, and I'd rather set |
This is in master and 3.9.1. I'll not backport to 3.8 because a release candidate is imminent. |
I have been asked to backport this to 3.8. There's a very small window 3.8.6: Monday, 2020-09-21 Backporting procedures since 3.8 are unclear and a source of There's a simple solution for 3.8: Do not use the parallel build, For the buildbots you can ask the operator for a custom command |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: