8000 trunk fails to build in AIX · Issue #17829 · matplotlib/matplotlib · GitHub
[go: up one dir, main page]
Skip to content

trunk fails to build in AIX #17829

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ayappanec opened this issue Jul 3, 2020 · 5 comments · Fixed by #17831
Closed

trunk fails to build in AIX #17829

ayappanec opened this issue Jul 3, 2020 · 5 comments · Fixed by #17831
Labels
Release critical For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.
Milestone
v3.3.0

Comments

@ayappanec
Copy link
Contributor

Building the trunk fails in AIX with the following errors

builds/unix/ftsystem.c: In function 'ft_close_stream_by_munmap':
builds/unix/ftsystem.c:197:5: warning: implicit declaration of function 'munmap' [-Wimplicit-function-declaration]
munmap( (MUNMAP_ARG_CAST)stream->descriptor.pointer, stream->size );
^~~~~~
builds/unix/ftsystem.c: In function 'FT_Stream_Open':
builds/unix/ftsystem.c:234:18: error: storage size of 'stat_buf' isn't known
struct stat stat_buf;
^~~~~~~~
builds/unix/ftsystem.c:241:12: warning: implicit declaration of function 'open'; did you mean 'fopen'? [-Wimplicit-function-declaration]
file = open( filepathname, O_RDONLY );
^~~~
fopen
builds/unix/ftsystem.c:241:32: error: 'O_RDONLY' undeclared (first use in this function)
file = open( filepathname, O_RDONLY );

This happens during the freetype build.
I am able to overcome this issue by making use of system freetype setting via setup.cfg

AIX build mechanism, unlike linux or other well tested opensource platforms, is very unique and needs special compiler flags.
So the technique of downloading freetype source code and building it just using the configure is not going to work in AIX.

The point here is once there is a new matplotlib release and when users start using "pip install matplotlib" , AIX users will face this issue. So i would probably suggest for AIX platform, matplotlib has to rely on system softwares ( in this case it's freetype)
@tacaswell tacaswell added this to the v3.3.0 milestone Jul 3, 2020
@tacaswell tacaswell added the Release critical For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions. label Jul 3, 2020
@tacaswell
Copy link
Member

Is it a matter of flags or do we need to patch the freetype as well? If it is flags / proper calls to the compilers we can add the details to

matplotlib/setupext.py

Lines 518 to 643 in b2a11c7

class FreeType(SetupPackage):
name = "freetype"
def add_flags(self, ext):
ext.sources.insert(0, 'src/checkdep_freetype2.c')
if options.get('system_freetype'):
pkg_config_setup_extension(
# FreeType 2.3 has libtool version 9.11.3 as can be checked
# from the tarball. For FreeType>=2.4, there is a conversion
# table in docs/VERSIONS.txt in the FreeType source tree.
ext, 'freetype2',
atleast_version='9.11.3',
alt_exec=['freetype-config'],
default_libraries=['freetype'])
ext.define_macros.append(('FREETYPE_BUILD_TYPE', 'system'))
else:
src_path = Path('build', f'freetype-{LOCAL_FREETYPE_VERSION}')
# Statically link to the locally-built freetype.
# This is certainly broken on Windows.
ext.include_dirs.insert(0, str(src_path / 'include'))
if sys.platform == 'win32':
libfreetype = 'libfreetype.lib'
else:
libfreetype = 'libfreetype.a'
ext.extra_objects.insert(
0, str(src_path / 'objs' / '.libs' / libfreetype))
ext.define_macros.append(('FREETYPE_BUILD_TYPE', 'local'))
def do_custom_build(self, env):
# We're using a system freetype
if options.get('system_freetype'):
return
src_path = Path('build', f'freetype-{LOCAL_FREETYPE_VERSION}')
# We've already built freetype
if sys.platform == 'win32':
libfreetype = 'libfreetype.lib'
else:
libfreetype = 'libfreetype.a'
# bailing because it is already built
if (src_path / 'objs' / '.libs' / libfreetype).is_file():
return
# do we need to download / load the source from cache?
if not src_path.exists():
os.makedirs('build', exist_ok=True)
tarball = f'freetype-{LOCAL_FREETYPE_VERSION}.tar.gz'
target_urls = [
(f'https://downloads.sourceforge.net/project/freetype'
f'/freetype2/{LOCAL_FREETYPE_VERSION}/{tarball}'),
(f'https://download.savannah.gnu.org/releases/freetype'
f'/{tarball}')
]
for tarball_url in target_urls:
try:
tar_contents = download_or_cache(tarball_url,
LOCAL_FREETYPE_HASH)
break
except Exception:
pass
else:
raise IOError(
f"Failed to download FreeType. Please download one of "
f"{target_urls} and extract it into {src_path} at the "
f"top-level of the source repository.")
print(f"Extracting {tarball}")
with tarfile.open(fileobj=tar_contents, mode="r:gz") as tgz:
tgz.extractall("build")
print(f"Building freetype in {src_path}")
if sys.platform != 'win32': # compilation on non-windows
env = {**env, "CFLAGS": "{} -fPIC".format(env.get("CFLAGS", ""))}
subprocess.check_call(
["./configure", "--with-zlib=no", "--with-bzip2=no",
"--with-png=no", "--with-harfbuzz=no", "--enable-static",
"--disable-shared"],
env=env, cwd=src_path)
subprocess.check_call(["make"], env=env, cwd=src_path)
else: # compilation on windows
shutil.rmtree(src_path / "objs", ignore_errors=True)
msbuild_platform = (
'x64' if platform.architecture()[0] == '64bit' else 'Win32')
base_path = Path("build/freetype-2.6.1/builds/windows")
vc = 'vc2010'
sln_path = (
base_path / vc / "freetype.sln"
)
# https://developercommunity.visualstudio.com/comments/190992/view.html
(sln_path.parent / "Directory.Build.props").write_text("""
<Project>
<PropertyGroup>
<!-- The following line *cannot* be split over multiple lines. -->
<WindowsTargetPlatformVersion>$([Microsoft.Build.Utilities.ToolLocationHelper]::GetLatestSDKTargetPlatformVersion('Windows', '10.0'))</WindowsTargetPlatformVersion>
</PropertyGroup>
</Project>
""")
# It is not a trivial task to determine PlatformToolset to plug it
# into msbuild command, and Directory.Build.props will not override
# the value in the project file.
# The DefaultPlatformToolset is from Microsoft.Cpp.Default.props
with open(base_path / vc / "freetype.vcxproj", 'r+b') as f:
toolset_repl = b'PlatformToolset>$(DefaultPlatformToolset)<'
vcxproj = f.read().replace(b'PlatformToolset>v100<',
toolset_repl)
assert toolset_repl in vcxproj, (
'Upgrading Freetype might break this')
f.seek(0)
f.truncate()
f.write(vcxproj)
cc = ccompiler.new_compiler()
cc.initialize() # Get msbuild in the %PATH% of cc.spawn.
cc.spawn(["msbuild", str(sln_path),
"/t:Clean;Build",
f"/p:Configuration=Release;Platform={msbuild_platform}"])
# Move to the corresponding Unix build path.
(src_path / "objs" / ".libs").mkdir()
# Be robust against change of FreeType version.
lib_path, = (src_path / "objs" / vc / msbuild_platform).glob(
"freetype*.lib")
shutil.copy2(lib_path, src_path / "objs/.libs/libfreetype.lib")
.

How can we reliably tell if we are on AIX? Probably the simplest change would be change

matplotlib/setupext.py

Lines 138 to 144 in b2a11c7

options = {
'backend': config.get('rc_options', 'backend', fallback=None),
'system_freetype': config.getboolean('libs', 'system_freetype',
fallback=False),
'system_qhull': config.getboolean('libs', 'system_qhull',
fallback=False),
}

to

--- a/setupext.py
+++ b/setupext.py
@@ -138,7 +138,7 @@ if os.path.exists(setup_cfg):
 options = {
     'backend': config.get('rc_options', 'backend', fallback=None),
     'system_freetype': config.getboolean('libs', 'system_freetype',
-                                         fallback=False),
+                                         fallback=am_i_on_aix()),
     'system_qhull': config.getboolean('libs', 'system_qhull',
                                       fallback=False),
 }

We should document more prominently the MPLSETUPCFG env which will let you point at a custom cfg file with out having to checkout the source.

@tacaswell
Copy link
Member

Ah, found it, was not looking in the right part of the docs

@tacaswell
Copy link
Member

@ayappanec Do you have the same issue with qhull?

tacaswell added a commit to tacaswell/matplotlib that referenced this issue Jul 3, 2020
@tacaswell
BLD: default to system freetype on AIX
6b808d1 
Our internal freetype build fails due to incorrect compiler flags.
The easy path to make sure AIX users will be able to pip install is to
default to system freetype when on AIX.

closes matplotlib#17829
@tacaswell tacaswell mentioned this issue Jul 3, 2020
Merged
@tacaswell
Copy link
Member

Can you please confirm that this works for you again @ayappanec ?

Thank you for reporting this!

@ayappanec
Copy link
Contributor Author

@tacaswell Great. It works now. Thank you for the quick response and support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Release critical For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.
Projects
None yet
Milestone
v3.3.0
Development

Successfully merging a pull request may close this issue.

BLD: default to system freetype on AIX tacaswell/matplotlib
2 participants
@tacaswell @ayappanec
0