8000 BLD,API: (distutils) Force strict floating point error model on clang by seberg · Pull Request #19049 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

BLD,API: (distutils) Force strict floating point error model on clang #19049

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 20, 2021
Merged

BLD,API: (distutils) Force strict floating point error model on clang #19049

merged 2 commits into from
May 20, 2021

Conversation

seberg
Copy link
Member
@seberg seberg commented May 19, 2021 •
edited by charris
Loading

I am not quite sure this is the right way to do it, it feels hackish... but distutils tends to I guess.

I tried removing some of the hacks that were probably in place just because of the missing compile time flag, so hopefully this will work out. Do we test clang properly in CI? Otherwise we will have to double check before merging (I would have to set up clang first).

Closes gh-18005

@seberg
API,BLD: Always force -ffp-exception-behavior=strict on clang
817684a 
This also affects all users of numpy distutils
@seberg seberg added 00 - Bug 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes 36 - Build Build related PR labels May 19, 2021
@charris
Copy link
Member
charris commented May 19, 2021

I'm going to ping @rgommers here.

@charris charris added the 09 - Backport-Candidate PRs tagged should be backported label May 19, 2021
@seberg seberg force-pushed the clang-fperr-bug branch from 258c446 to b6c550b Compare May 19, 2021 23:31
@rgommers
Copy link
Member

Do we test clang properly in CI?

We'll only be testing it on macOS I believe. So it depends, if it's a simple flag that may be enough.

rgommers
rgommers approved these changes May 20, 2021
View reviewed changes
Copy link
Member
@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This way of inserting the flag seems fine to me. I agree it feels hacky, but there's also no "one obvious place" where all the compiler flag modifications go - doing it in CCompiler_customize should be fairly robust.

@charris
Copy link
Member
charris commented May 20, 2021

Perhaps @r-devulap can check. I'd be more comfortable if the other workarounds hadn't been removed, we want to be sure this works first.

@r-devulap
Copy link
Member

I was going through the details of #13623 and looks like that was bug on a specific version of clang and might have nothing to do with enforcing strict floating point compliance. I think I agree with @charris, I would keep the workarounds. They are pretty harmless anyway.

@r-devulap
Copy link
Member

Adding the ffp-exception-behavior=strict flag should prevent such problems in the future. +1 to that.

@seberg seberg force-pushed the clang-fperr-bug branch from b6c550b to a303b33 Compare May 20, 2021 16:45
@seberg
Copy link
Member Author
seberg commented May 20, 2021

Yeah, reverting those changes was too optimistic. Removed it.

@charris charris merged commit 4732c24 into numpy:main May 20, 2021
@charris
Copy link
Member
charris commented May 20, 2021

Thanks Sebastian.

@seberg seberg deleted the clang-fperr-bug branch May 20, 2021 17:31
@asmeurer asmeurer mentioned this pull request May 20, 2021
Open
@ilayn
Copy link
Contributor
ilayn commented May 22, 2021

This seems to break scipy build on windows ☹️ and breaks the build surprisingly and exactly at the same place that #18295 fixed. I don't have the beginning of the log but it seems like it is early enough for the meat of it. I'm using OpenBLAS 0.3.15, Python 3.9.2 and latest Numpy on master. Dissected on the distutils commits and it is indeed this one starts coughing.

Pinging @mckib2 as maybe he can recognize something relevant

build.log

@charris
Copy link
Member
charris commented May 22, 2021

This seems to break scipy build on windows

Strange, the new flag addition should not even take place. The problem seems to be undefined reference to 'xerror_'. Can you put some print statements in the change to see what might be going on? Is it still an error if the if is explicitly False?

@ilayn
Copy link
Contributor
ilayn commented May 22, 2021

My bad, everything was scrambled because I still worked with master while the fixes went into main, so in my script then hell broke loose. Because dissect works with commit numbers regardless of the branch name.

Sorry for the noise.

@mckib2
Copy link
Contributor
mckib2 commented May 22, 2021

@ilayn Is this still a problem or did switching to main resolve the build problems?

@ilayn
Copy link
Contributor
ilayn commented May 22, 2021

All good after fixing the branches and commits

@charris charris added 09 - Backport-Candidate PRs tagged should be backported and removed 09 - Backport-Candidate PRs tagged should be backported 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes labels Jun 6, 2021
@charris charris added this to the 1.20.4 release milestone Jun 6, 2021
@charris charris mentioned this pull request Jun 13, 2021
Merged
@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Jun 13, 2021
@charris charris removed this from the 1.20.4 release milestone Jun 13, 2021
@seberg
Copy link
Member Author
seberg commented Jul 7, 2021

@rgommers, I just noticed on another PR this build log: https://dev.azure.com/numpy/27346c6a-2774-4eac-bf85-e068127c0ccc/_apis/build/builds/18280/logs/57

Which does not include exception-behavior anywhere, but it probably has to? So that this code doesn't actually work/do anything?

@rgommers
Copy link
Member
rgommers commented Jul 7, 2021

Argh, this is why I'm working so hard on a new build system - so we can stop trying to debug the current one.

Guess I didn't read the PR description - review made sense, but no one actually tested it.

@rgommers
Copy link
Member
rgommers commented Jul 7, 2021

Maybe I did test, can't remember. The code path does get hit, but only once in the "test compile" stage and once when compiling numpy extensions (in the customize UnixCCompiler stage):

creating build/temp.macosx-10.9-x86_64-3.9/numpy/random
creating build/temp.macosx-10.9-x86_64-3.9/numpy/random/src
creating build/temp.macosx-10.9-x86_64-3.9/numpy/random/src/distributions
compile options: '-Inumpy/core/include -Inumpy/core/include/numpy -Ibuild/src.macosx-10.9-x86_64-3.9/numpy/distutils/include -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/Users/rgommers/mambaforge/envs/scipy-dev/include/python3.9 -Inumpy/core/src/common -Inumpy/core/src/npymath -c'
extra options: '-msse -msse2 -msse3'
x86_64-apple-darwin13.4.0-clang: numpy/random/src/distributions/logfactorial.c
x86_64-apple-darwin13.4.0-clang: numpy/random/src/distributions/distributions.c
x86_64-apple-darwin13.4.0-clang: numpy/random/src/distributions/random_mvhg_marginals.c
x86_64-apple-darwin13.4.0-clang: numpy/random/src/distributions/random_mvhg_count.c
x86_64-apple-darwin13.4.0-clang: numpy/random/src/distributions/random_hypergeometric.c
x86_64-apple-darwin13.4.0-ar: adding 5 object files to build/temp.macosx-10.9-x86_64-3.9/libnpyrandom.a
x86_64-apple-darwin13.4.0-ranlib:@ build/temp.macosx-10.9-x86_64-3.9/libnpyrandom.a
creating numpy/random/lib
customize UnixCCompiler
customize UnixCCompiler using new_build_ext
Python 3.9.5 | packaged by conda-forge | (default, Jun 19 2021, 00:27:35) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.24.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: exit

CCompilerOpt.__init__[781] : hit the memory cache
CCompilerOpt.generate_dispatch_header[2267] : generate CPU dispatch header: (build/src.macosx-10.9-x86_64-3.9/numpy/distutils/include/npy_cpu_dispatch_config.h)
building 'numpy.core._multiarray_tests' extension
compiling C sources
C compiler: x86_64-apple-darwin13.4.0-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/rgommers/mambaforge/envs/scipy-dev/include -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/rgommers/mambaforge/envs/scipy-dev/include -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/rgommers/mambaforge/envs/scipy-dev/include -D_FORTIFY_SOURCE=2 -isystem /Users/rgommers/mambaforge/envs/scipy-dev/include

Maybe we end up not winning the monkey patching race or something. This seems like a good moment to share this picture:

image

@rgommers
Copy link
Member
rgommers commented Jul 7, 2021

I don't have time to look at it more right now. There's no real design, it's just trial and error. We should find a place where the change doesn't get overwritten by something else.

@seberg
Copy link
Member Author
seberg commented Jul 7, 2021

Sure, I only noticed this because fmod(1, 0) does not set the correct floating point error flags on Mac. But I am not sure that is a problem with clang at all, it may also be the fmod in libc (can't reproduce it on linux anyway).

This was referenced Jul 7, 2021
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rgommers rgommers rgommers approved these changes

Assignees
No one assigned
Labels
00 - Bug 36 - Build Build related PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

BLD: Do not use clang default to ignore floating point exceptions
6 participants
@seberg @charris @rgommers @r-devulap @ilayn @mckib2
0