8000 ndarray.fill crashes in master · Issue #12503 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

ndarray.fill crashes in master #12503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
oleksandr-pavlyk opened this issue Dec 7, 2018 · 15 comments
Closed

ndarray.fill crashes in master #12503

oleksandr-pavlyk opened this issue Dec 7, 2018 · 15 comments
Milestone
1.16.0 release

Comments

@oleksandr-pavlyk
Copy link
Contributor
oleksandr-pavlyk commented Dec 7, 2018 •
edited
Loading

Reproducing code example:

A test scipy._lib.tests.test__utils::test__aligned_zeros from 1.2.0rc2, with NumPy built from sources sporadically crashes.

(tscp) [13:28:40 fxsatlin03 test_tmp]$ pytest -v --pyargs scipy._lib.tests.test__util::test__aligned_zeros
================================================= test session starts ==================================================
platform linux -- Python 3.6.5, pytest-3.8.1, py-1.6.0, pluggy-0.7.1 -- ~/miniconda3_cb3/envs/tscp/bin/python
cachedir: .pytest_cache
rootdir: ~/test_tmp, inifile:
collected 1 item

test__util.py::test__aligned_zeros <- ~/miniconda3_cb3/envs/tscp/lib/python3.6/site-packages/scipy/_lib/tests/test__util.py Aborted (core dumped)

The best reproducer I could build is

import numpy as np

buf = np.empty(4, np.uint8)
buf = buf[2:3][:-1]
data = np.ndarray((0,), np.double, buf, order='C')
data.fill(0) # crash

Error message:

(tscp) [13:26:20 vmlin03 test_tmp]$ python scipy_utils.py
python: numpy/core/src/multiarray/lowlevel_strided_loops.c.src:816: _aligned_cast_longlong_to_double: Assertion `npy_is_aligned(dst, __builtin_offsetof(struct {char c; npy_double v;}, v))' failed.
Aborted (core dumped)

Numpy/Python version information:

In [1]: import numpy as np

In [2]: np.__version__
Out[2]: '1.16.0.dev0+22d6157'
@tylerjereddy
Copy link
Contributor

I can't reproduce the NumPy-only test case crash above locally on Ubuntu at the same NumPy commit hash.

I haven't seen that SciPy test failure in dev CI or wheel builds either. Anything different about your configuration vs. what we use for i.e., CI?

@charris
Copy link
Member
charris commented Dec 7, 2018

I can't reproduce either, using latest master and both Python2 and Python3 on Fedora.

What compiler/optimization are you using?

@charris
Copy link
Member
charris commented Dec 7, 2018

How sporadic is "sporadic"?

@oleksandr-pavlyk
Copy link
Contributor Author

Thank you all for checking. I am compiling using Intel compiler, and compiling with -O3.

pytest calls crashes 3 out of 4 times, but my stand alone example crashes every time I tried.

I will report my findings here.

@tylerjereddy
Copy link
Contributor

Intel compiler

18.x series? If the compiler is the difference vs. our test infrastructure I'm not sure we have an easy way to CI test with Intel compilers because of the license stuff, if I'm not mistaken.

@oleksandr-pavlyk
Copy link
Contributor Author

As I said, I am using Intel C Compiler 2016 update 3, but I can reproduce the problem when compiling the entire NumPy with -g -O0:

(gdb) f 4
#4  0x00002aaaaf6a9695 in _aligned_cast_longlong_to_double (dst=0x555555ac1142 "", dst_stride=8,
    src=0x7fffffffc5a0 "", src_stride=0, N=0, __NPY_UNUSED_TAGGEDsrc_itemsize=8, __NPY_UNUSED_TAGGEDdata=0x0)
    at numpy/core/src/multiarray/lowlevel_strided_loops.c.src:816
816         assert(npy_is_aligned(dst, _ALIGN(_TYPE2)));

when running the above snippet.

I will run more experiments.

@oleksandr-pavlyk
Copy link
Contributor Author

Indeed, using the current master, f07a38d, the problem is not reproducible if compiling with gcc (I was using 4.8.4), but I get a reliable crash with both icc 2016, and with icc 2018.

@oleksandr-pavlyk
Copy link
Contributor Author

Ok, I can reproduce the problem with gcc as well. The reason the problem was hidden when compiling with gcc, was that -DNDEBUG was used.

If I undo it, by compiling master as

CFLAGS="-g -O0 -UNDEBUG -D_DEBUG -I$CONDA_PREFIX/include $CFLAGS" LDFLAGS="-L$CONDA_PREFIX/lib -Wl,-rpath,$CONDA_PREFIX/lib $LDFLAGS" python setup.py build_ext --force --inplace

I get the assert causing the crash:

Python 3.6.5 |Intel Corporation| (default, Jul 11 2018, 20:01:00)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import numpy as np
>>> np.__version__
'1.16.0.dev0+f07a38d'
>>> buf = np.empty(4, np.uint8)
>>> buf = buf[2:3][:-1]
>>> data = np.ndarray((0,), np.double, buf, order='C')
>>> data.__array_interface__['data'][0] % 64
50
>>> buf.__array_interface__['data'][0] % 64
50
>>> data.fill(0)
python: numpy/core/src/multiarray/lowlevel_strided_loops.c.src:815: _aligned_cast_longlong_to_double: Assertion `npy_is_aligned(dst, __builtin_offsetof (struct {char c; npy_double v;}, v))' failed.
Aborted (core dumped)

@oleksandr-pavlyk
Copy link
Contributor Author

The SHA were this problem was reproduced is the root of maintenance/1.16.x branch.

Unless it is a matter of the assert needing to be fixed, perhaps this issue should be fixed in 1.16

@tylerjereddy
Copy link
Contributor

Does this mean we are missing / need a CI entry that runs tests with C-level assert macros active?

@charris charris added this to the 1.16.0 release milestone Dec 7, 2018
@mattip
Copy link
Member
mattip commented Dec 8, 2018

The DEBUG travis build should not have the -DNDEBUG flag, I have seen it assert before.

@mattip
Copy link
Member
mattip commented Dec 8, 2018

Can reproduce when CFLAGS="-g -O0 -UNDEBUG -D_DEBUG" python runtests.py -i

@charris
Copy link
Member
charris commented Dec 8, 2018

The problem PR is #6377.

@charris
Copy link
Member
charris commented Dec 8, 2018

@ahaldane Want to take a look?

@ahaldane
Copy link
Member
ahaldane commented Dec 8, 2018 •
edited
Loading

OK, I see the problem.

It is actually related to the NPY_RELAXED_STRIDES_CHECKING bugfix from this commit, and is not due to the complex alignment change that PR was focused on.

In that commit I tried to add RELAXED_STRIDE checking which was missing from raw_array_is_aligned. That code says that size-0 arrays are "always aligned" (see comment there), which means that the copy code goes down the "aligned" copy-code path. But that code-path has an assertion which does not account for relaxed_stride definition of alignment.

This didn't happen in 1.15 because raw_array_is_aligned didn't do relaxed_stride checking then (which I think was a bug).

@ahaldane ahaldane mentioned this issue Dec 8, 2018
Merged
ahaldane added a commit to ahaldane/numpy that referenced this issue Dec 10, 2018
@ahaldane
MAINT: don't check alignment size=0 arrays (RELAXED_STRIDES)
d7ab72d 
Don't check alignment of size-0 arrays in copy-loops, because of
RELAXED_STRIDES.

Fixes numpy#12503
charris pushed a commit to charris/numpy that referenced this issue Dec 16, 2018
@ahaldane @charris
MAINT: don't check alignment size=0 arrays (RELAXED_STRIDES)
5b4cfeb 
Don't check alignment of size-0 arrays in copy-loops, because of
RELAXED_STRIDES.

Fixes numpy#12503
@charris charris mentioned this issue Dec 16, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
1.16.0 release
Development

No branches or pull requests
5 participants
@charris @mattip @tylerjereddy @ahaldane @oleksandr-pavlyk
0