10000 MAINT: default to C11 rather than C99, fix most build warnings with Clang 14 by rgommers · Pull Request #25072 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

MAINT: default to C11 rather than C99, fix most build warnings with Clang 14 #25072

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Dec 13, 2023

Conversation

rgommers
Copy link
Member
@rgommers rgommers commented Nov 5, 2023

The large number of build warnings on macOS was getting annoying, so I dug in and solved all but a handful of them (it was 51, now there are 5 left). Those last ones were tricky to address, and silencing -Wunused-function didn't seem quite right.

One main cause of warnings was the use of C11 features. Rather than trying to change that code, it made sense to start requiring C11 explicitly. The biggest change in C11 was to make previously-mandatory C99 features like complex.h and complex types optional - and we were already allowing that of course, because MSVC doesn't support complex types. Since all compilers we care about support C11, and the typedef redefinitions that we use in multiple places are a C11-only feature, documenting that C11 is what we require see and compiling with -std=c11 seems fine.

This PR also cleans up uses of #pragma GCC diagnostic. Those should be avoided, especially for -Wmaybe-uninitialized, because Clang also looks at these pragmas and emits warnings when it doesn't understand them (and -Wmaybe-uninitialized doesn't exist there). There are usually better ways to deal with the problem, like changing the code that emits the warnings.

Finally worth pointing out is one odd bug in loops_minmax.dispatch.c.src which disabled rather than enabled some code that was meant to run on Arm only. This was clearly a typo; tests all seem to pass; I didn't investigate performance.

@rgommers rgommers added this to the 2.0.0 release milestone Nov 5, 2023
seberg
seberg previously requested changes Nov 6, 2023
Copy link
Member
@seberg seberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but that arm typo does not look like a typo, I can see it leading to uninitialized unused variables though.

I assume nobody will care about C11 vs. C99. If anyone does (and actually has problems with those redefinitions), I suspect they can untangle the imports to fix it with some time.

/* At least GCC 9.2 issues spurious warnings for arg2 below. */
#pragma GCC diagnostic push /* matching pop after function and repeat */
#pragma GCC diagnostic ignored "-Wmaybe-uninitialized"
#endif
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If CI doesn't annoy us anymore, great that we don't need to add the unnecessary init.

return npyv_mul_@sfx@(hypot, larger);
}
#endif // VECTOR
/**end repeat**/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For those not reading the commit message carefully: A copy of this is in loops_unary_complex.dispatch.c.src where it is actually used.

@rgommers
Copy link
Member Author
rgommers commented Nov 6, 2023

One CI failure too, with Clang-cl on Windows:

numpy/random/_common.cp311-win_amd64.pyd.p/numpy/random/_common.pyx.c(25481,43): error: operand of type '_Fcomplex' (aka 'struct _C_float_complex') where arithmetic or pointer type is required
      return x + y*(__pyx_t_float_complex)_Complex_I;

Probably due to -std=c11, will check.

@seberg
Copy link
Member
seberg commented Nov 6, 2023

Right, I just rebased gh-25044 I suspect that might help.

EDIT: Well, actually, I am not sure how clang-cl behaves w.r.t. to the complex definitions, TBH...

@lysnikolaou
Copy link
Member
lysnikolaou commented Nov 6, 2023

The clang-cl failure probably is a Cython bug. It's likely related to this line, where Cython tries to detect c11 and it assumes that complex support is okay for c11 (which it is, but Cython's logic later on is to just fall back to using a struct for all MSVC-like environments).

Should we submit a bug report there?

@rgommers
Copy link
Member Author
rgommers commented Nov 6, 2023

Good catch @lysnikolaou! Yes please! If you could take that one on, that'd be helpful.

@lysnikolaou
Copy link
Member

The Cython bug has been fixed in cython/cython#5809. Should we try Windows CI out with cython master?

@rgommers
Copy link
Member Author

The Cython bug has been fixed in cython/cython#5809. Should we try Windows CI out with cython master?

I tried this on my fork, CI passes now with both Clang-cl and MSVC with the Cython 3.0.x branch. Once 3.0.6 is out we can move this forward.

@charris charris added the 09 - Backport-Candidate PRs tagged should be backported label Nov 14, 2023
@charris
Copy link
Member
charris commented Nov 14, 2023

I put a backport label on this, seems pretty straight forward.

@rgommers rgommers force-pushed the fix-build-warnings branch 2 times, most recently from 13ac7be to 1960458 Compare December 11, 2023 19:28
@rgommers rgommers marked this pull request as ready for review December 11, 2023 20:34
@rgommers
Copy link
Member Author

This is happy now that the Cython issue for C11 was fixed and released in 3.0.6

I put a backport label on this, seems pretty straight forward.

I'd be fine either way with backporting the MAINT commits yes or no. I'd prefer not to backport the BLD ones though.

…t warnings

Also a simple fix for a `-Wmaybe-uninitialized` warning in
`special_integer_comparisons.cpp` (GCC isn't smart enough to
understand the `default` part of the switch statement without
the added line).
…arning

That warning was, with clang on macOS:
```
../numpy/_core/src/npysort/selection.cpp:529:15: warning: result of comparison of constant 1 with expression of type 'NPY_SELECTKIND' is always false [-Wtautological-constant-out-of-range-compare]
    if (which >= NPY_NSELECTS) {
```
The warning was:
```
[2/3] Compiling C object numpy/_core/_multiarray_umath.cpython-39-darwin.so.p/src_multiarray_item_selection.c.o
../numpy/_core/src/multiarray/item_selection.c:2371:1: warning: unused function 'count_nonzero_bytes_384' [-Wunused-function]
count_nonzero_bytes_384(const npy_uint64 * w)
```
In effect we were already using C11 for a long time, because that's
where `complex.h` and complex types became optional rather than
mandatory. Also, there were several places where we used typedef
redefinitions, which is a C11 feature, as indicated by build warnings
like these:

```
In file included from ../numpy/_core/src/multiarray/common.h:6:
../numpy/_core/include/numpy/ndarraytypes.h:308:31: warning: redefinition of typedef 'NpyAuxData' is a C11 feature [-Wtypedef-redefinition]
typedef struct NpyAuxData_tag NpyAuxData;
                              ^
../numpy/_core/src/umath/loops.h.src:37:31: note: previous definition is here
typedef struct NpyAuxData_tag NpyAuxData;
                              ^

In file included from ../numpy/_core/src/multiarray/array_method.h:14:
../numpy/_core/include/numpy/_dtype_api.h:108:3: warning: redefinition of typedef 'PyArrayMethod_Context' is a C11 feature [-Wtypedef-redefinition]
} PyArrayMethod_Context;
  ^
../numpy/_core/src/umath/loops.h.src:36:42: note: previous definition is here
typedef struct PyArrayMethod_Context_tag PyArrayMethod_Context;

```
The warning with Clang 14 is:

```
[3/4] Compiling C object numpy/_core/_multiarray_umath.cpython-39-darwin.so.p/meson-generated_einsum.c.o
../numpy/_core/src/multiarray/einsum.c.src:408:32: warning: unknown warning group '-Wmaybe-uninitialized', ignored [-Wunknown-warning-option]
                               ^

[173/301] Compiling C object numpy/_core/_multiarray_umath.cpython-39-darwin.so.p/meson-generated_scalarmath.c.o
../numpy/_core/src/umath/scalarmath.c.src:1810:36: warning: unknown warning group '-Wmaybe-uninitialized', ignored [-Wunknown-warning-option]
    #pragma GCC diagnostic ignored "-Wmaybe-uninitialized"
                                   ^
```

These pragmas are outdated and shouldn't be used. Instead, use
compiler flags (if really needed) that are feature-tested for actually
existing for the compiler that is used in the build.
These were yielding unused function warnings with Clang 14, and
turned out to indeed be unused. The code was an exact copy of
the same function in `loops_unary_complex_dispatch.c.src`, where
it was used.
Needed to include the bug fix for using complex types with C11
(Cython PR 5809).
@rgommers rgommers dismissed seberg’s stale review December 13, 2023 09:20

Action to drop the loops_minmax.dispatch.c.src change was taken

@seberg
Copy link
Member
seberg commented Dec 13, 2023

Thanks Ralf, looks all good now, let's get this in.

@seberg seberg merged commit a7b7e55 into numpy:main Dec 13, 2023
@rgommers rgommers deleted the fix-build-warnings branch December 13, 2023 15:46
@charris
Copy link
Member
charris commented Dec 13, 2023

I'd be fine either way with backporting the MAINT commits yes or no.

I'm going to skip this. There are enough differences between 1.26.x and main that it feels risky. I plan on maybe two more 1.26 releases, and if we can get through those I'll be happy.

@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Dec 13, 2023
@ngoldbaum
Copy link
Member

Thanks for this, the warnings were indeed annoying!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0