-
Notifications
You must be signed in to change notification settings - Fork 4
Building 3rd party Depende 8000 ncies
A number of python packages require 3rd party libraries that are not supplied by Apple. In order to build and use these libs, you need to build them in a way that is compatible with the Python you are running.
If you simply need to get a package working on your own machine (or ones like it), then it is generally easiest to use HomeBrew or MacPorts to build and install the libs for you. And many Python packages are configured to look for those libs at build time.
But if you want to re-distribute your builds to folks that may have older systems than yours, you'll need to build them carefully.
There is also gitHub project aimed at consolidating the effort to build 3rd party libs, but it is very incomplete and poorly tested. So you are pretty much on your own. However, if you do get a build script going, consider contributing it here:
[https://github.com/MacPython/mac-builds]
For the most part, if you want to re-distribute your builds (as binary wheels, hopefully), you'll want to build for the python.org - distributed binaries. If you build for those, the binary will run on newer systems and newer python builds as well (Apple's, Homebrew, etc), so it's a good target.
There are two versions of binaries on python.org: PPC+32bitIntel and 32+64bitIntel. PPC is pretty much gone, and it's really hard to build for that now, so you probably want to support the 32+64bit Intel builds. These are built on/for OS-X 10.6 and above.
Apple has done a pretty good job of providing SDKs (Softare Deveopment kits) for each version of their system. If you build against the 10.6 SDK, it shoudl run on 10.6 and newer systems. In theory, this is as simple as setting:
MACOSX_DEPLOYMENT_TARGET=10.6
Before you build. IN practice, that doesn't always work, see below for more deatail
The Apple builds are also built for both 32bit and 64 bit systems. 32 bit systema are getting pretty rare, bu they are still out there, and there are some issues wtill with things like 64bit Tk, so it's nce to support both. You do that by passing both the flags to the compiler:
-arch i386 and -arch x86_64
This will usually work but not always (OpenSSL is an example of one where it doesn't, as well as HDF5). If the package build makes arch dependent decisions at configure time (or the equivalent) or arch-dependent values are in generated include files or something similar, the safest approach is to build each arch separately and then use lipo to combine the separate archs into universal file(s) to be installed. Fortunately, most popular libs either now natively support universal builds or someone has figured out how to do it. The build recipes in MacPorts or homebrew can be of help here.
(this from a post to the macpython list my Ned Deily -- thanks Ned!)
It all depends on what you are building. I assume you're talking about non-Python packages here. Many C modules won't be a problem but some will if they use OS-supplied APIs that have changed or been added between the deployment target version of OS X and the build systems's target. If you need to support the full range of systems, your safest best remains to build on deployment target system. Otherwise, until proven otherwise for the particular package, you should be using the SDK for the deployment target version and hope that the build system (Makefile, whatever) for the package builds correctly with an SDK. In many cases it will because Apple cleverly hid much of the SDK and universal arch complexity under the covers in the compiler driver wrappers for cc/gcc/clang. Basically when you use an SDK, the Apple compiler driver automatically prepends the SDK path to all "system" files, those beginning with /System/Library, /Library, and /usr, so that they are picked up from the SDK directory rather than from the running system. But, if the package's build process does something clever, like making its own decisions about where to look for libs or include files that does not take the SDK path into account, you can end up with subtle errors, where the package build is basing decisions on the build system's installed include files (i.e. the 10.9 version) whereas the compiler is actually building with the SDK version (i.e. 10.6) of the file. (Builds of Python itself have run into this problem in the past.)
If you want to fully support universal architectures (Intel-32 and Intel-64 for the python.org 64-bit/32-bit installers), you also need to build universal libraries. Adding -arch i386 and -arch x86_64 will usually work but not always (OpenSSL is an example of one where it doesn't). If the package build makes arch dependent decisions at configure time (or the equivalent) or arch-dependent values are in generated include files or something similar, the safest approach is to build each arch separately and then use lipo to combine the separate archs into universal file(s) to be installed. Fortunately, most popular libs either now natively support universal builds or someone has figured out how to do it. The build recipes in MacPorts or homebrew can be of help here.
On the most recent versions of OS X, particularly on 10.9, there is better support from the command line tools for building with SDKs by setting the SDKROOT and DEVELOPER_DIR environment variables. See the man page for xcrun (man 1 xcrun) for more details.
Also, note that when building C extension modules via Distutils in current Pythons from python.org, the SDK used to build Python is looked for in its original path (for example, /Developer/SDKs/MacOSX10.6.sdk) and, if not found there, the ext module build proceeds without an SDK, using the headers and libs of the installed command line tools. Most of the time that is fine if you are building something to be used solely on the build system's OS level and probably for later systems.