8000 MAINT: Remove the RELAXED_STRIDES_CHECKING env variable by seberg · Pull Request #21039 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

MAINT: Remove the RELAXED_STRIDES_CHECKING env variable #21039

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 13, 2022
Merged

MAINT: Remove the RELAXED_STRIDES_CHECKING env variable #21039

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0457cc7
MAINT: Remove the RELAXED_STRIDES_CHECKING env variable
seberg Feb 11, 2022
c06d45f
DOC: Fix striding related comments based on review
seberg Feb 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/actions/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ runs:
shell: bash
run: |
echo NPY_RELAXED_STRIDES_DEBUG $NPY_RELAXED_STRIDES_DEBUG
echo NPY_RELAXED_STRIDES_CHECKING $NPY_RELAXED_STRIDES_CHECKING
echo CHECK_BLAS $CHECK_BLAS
echo DOWNLOAD_OPENBLAS $DOWNLOAD_OPENBLAS
echo USE_DEBUG $USE_DEBUG
Expand Down
5 changes: 2 additions & 3 deletions .github/workflows/build_test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,13 +136,13 @@ jobs:
python-version: ${{ env.PYTHON_VERSION }}
- uses: ./.github/actions

no_relaxed_strides:
relaxed_strides_debug:
needs: [smoke_test]
runs-on: ubuntu-latest
env:
NPY_RELAXED_STRIDES_CHECKING: 0
CHECK_BLAS: 1
NPY_USE_BLAS_ILP64: 1
NPY_RELAXED_STRIDES_DEBUG: 1
steps:
- uses: actions/checkout@v2
with:
Expand All @@ -158,7 +158,6 @@ jobs:
runs-on: ubuntu-latest
env:
USE_WHEEL: 1
NPY_RELAXED_STRIDES_DEBUG: 1
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decided to move this to the other one (so use_wheel is just that and nothing else) and rename that one.

steps:
- uses: actions/checkout@v2
with:
Expand Down
5 changes: 5 additions & 0 deletions doc/release/upcoming_changes/20220.compatibility.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
``NPY_RELAXED_STRIDES_CHECKING`` has been removed
-------------------------------------------------
NumPy cannot be compiled with ``NPY_RELAXED_STRIDES_CHECKING=0``
anymore. Relaxed strides have been the default for many years and
the option was initially introduced to allow a smoother transition.
17 changes: 3 additions & 14 deletions doc/source/reference/arrays.ndarray.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,26 +161,15 @@ An array is considered aligned if the memory offsets for all elements and the
base offset itself is a multiple of `self.itemsize`. Understanding
`memory-alignment` leads to better performance on most hardware.

.. note::

Points (1) and (2) can currently be disabled by the compile time
environmental variable ``NPY_RELAXED_STRIDES_CHECKING=0``,
which was the default before NumPy 1.10.
No users should have to do this. ``NPY_RELAXED_STRIDES_DEBUG=1``
can be used to help find errors when incorrectly relying on the strides
in C-extension code (see below warning).

You can check whether this option was enabled when your NumPy was
built by looking at the value of ``np.ones((10,1),
order='C').flags.f_contiguous``. If this is ``True``, then your
NumPy has relaxed strides checking enabled.

.. warning::

It does *not* generally hold that ``self.strides[-1] == self.itemsize``
for C-style contiguous arrays or ``self.strides[0] == self.itemsize`` for
Fortran-style contiguous arrays is true.

``NPY_RELAXED_STRIDES_DEBUG=1`` can be used to help find errors when
incorrectly relying on the strides in C-extension code (see below warning).

Data in new :class:`ndarrays <ndarray>` is in the :term:`row-major`
(C) order, unless otherwise specified, but, for example, :ref:`basic
array slicing <arrays.indexing>` often produces :term:`views <view>`
Expand Down
18 changes: 7 additions & 11 deletions doc/source/reference/global_state.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,19 +70,15 @@ Debugging-Related Options
Relaxed Strides Checking
------------------------

The *compile-time* environment variables::
The *compile-time* environment variable::

NPY_RELAXED_STRIDES_DEBUG=0
NPY_RELAXED_STRIDES_CHECKING=1

control how NumPy reports contiguity for arrays.
The default that it is enabled and the debug mode is disabled.
This setting should always be enabled. Setting the
debug option can be interesting for testing code written
in C which iterates through arrays that may or may not be
contiguous in memory.
Most users will have no reason to change these; for details
see the :ref:`memory layout <memory-layout>` documentation.

can be set to help debug code written in C which iteraters through arrays
manually. When an array is contiguous and iterated in a contiguous manner,
its ``strides`` should not be queried. This option can help find errors where
the ``strides`` are incorrectly used.
For details see the :ref:`memory layout <memory-layout>` documentation.


Warn if no memory allocation policy when deallocating data
Expand Down
8 changes: 3 additions & 5 deletions numpy/core/include/numpy/ndarraytypes.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -834,11 +834,9 @@ typedef int (PyArray_FinalizeFunc)(PyArrayObject *, PyObject *);
* 1-d array is C_CONTIGUOUS it is also F_CONTIGUOUS. Arrays with
* more then one dimension can be C_CONTIGUOUS and F_CONTIGUOUS
* at the same time if they have either zero or one element.
* If NPY_RELAXED_STRIDES_CHECKING is set, a higher dimensional
* array is always C_CONTIGUOUS and F_CONTIGUOUS if it has zero elements
* and the array is contiguous if ndarray.squeeze() is contiguous.
* I.e. dimensions for which `ndarray.shape[dimension] == 1` are
* ignored.
* A higher dimensional array is always C_CONTIGUOUS and F_CONTIGUOUS if it
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe "or" instead of "and"? This make is sound like the array is both, which is probably true for empty arrays, but squeezed arrays might be one of the other. Maybe rewrite?

* has zero elements or when `array.squeeze()` is contiguous.
* I.e. dimensions for which `array.shape[dimension] == 1` are ignored.
*/

/*
Expand Down
17 changes: 7 additions & 10 deletions numpy/core/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@
# Set to True to enable relaxed strides checking. This (mostly) means
# that `strides[dim]` is ignored if `shape[dim] == 1` when setting flags.
NPY_RELAXED_STRIDES_CHECKING = (os.environ.get('NPY_RELAXED_STRIDES_CHECKING', "1") != "0")
if not NPY_RELAXED_STRIDES_CHECKING:
raise SystemError(
"Support for NPY_RELAXED_STRIDES_CHECKING=0 has been remove as of "
"NumPy 1.23. This error will eventually be removed entirely.")

# Put NPY_RELAXED_STRIDES_DEBUG=1 in the environment if you want numpy to use a
# bogus value for affected strides in order to help smoke out bad stride usage
Expand Down Expand Up @@ -481,13 +485,9 @@ def generate_config_h(ext, build_dir):
if can_link_svml():
moredefs.append(('NPY_CAN_LINK_SVML', 1))

# Use relaxed stride checking
if NPY_RELAXED_STRIDES_CHECKING:
moredefs.append(('NPY_RELAXED_STRIDES_CHECKING', 1))
else:
moredefs.append(('NPY_RELAXED_STRIDES_CHECKING', 0))

# Use bogus stride debug aid when relaxed strides are enabled
# Use bogus stride debug aid to flush out bugs where users use
# strides of dimensions with length 1 to index a full contiguous
# array.
if NPY_RELAXED_STRIDES_DEBUG:
moredefs.append(('NPY_RELAXED_STRIDES_DEBUG', 1))
else:
Expand Down Expand Up @@ -583,9 +583,6 @@ def generate_numpyconfig_h(ext, build_dir):
moredefs.extend(cocache.check_ieee_macros(config_cmd)[1])
moredefs.extend(cocache.check_complex(config_cmd, mathlibs)[1])

if NPY_RELAXED_STRIDES_CHECKING:
moredefs.append(('NPY_RELAXED_STRIDES_CHECKING', 1))

if NPY_RELAXED_STRIDES_DEBUG:
moredefs.append(('NPY_RELAXED_STRIDES_DEBUG', 1))

Expand Down
4 changes: 0 additions & 4 deletions numpy/core/src/common/array_assign.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,6 @@ raw_array_is_aligned(int ndim, npy_intp const *shape,
int i;

for (i = 0; i < ndim; i++) {
#if NPY_RELAXED_STRIDES_CHECKING
/* skip dim == 1 as it is not required to have stride 0 */
if (shape[i] > 1) {
/* if shape[i] == 1, the stride is never used */
Expand All @@ -120,9 +119,6 @@ raw_array_is_aligned(int ndim, npy_intp const *shape,
/* an array with zero elements is always aligned */
return 1;
}
#else /* not NPY_RELAXED_STRIDES_CHECKING */
align_check |= (npy_uintp)strides[i];
#endif /* not NPY_RELAXED_STRIDES_CHECKING */
}

return npy_is_aligned((void *)align_check, alignment);
Expand Down
20 changes: 6 additions & 14 deletions numpy/core/src/multiarray/buffer.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -498,14 +498,11 @@ _buffer_info_new(PyObject *obj, int flags)
assert((size_t)info->shape % sizeof(npy_intp) == 0);
info->strides = info->shape + PyArray_NDIM(arr);

#if NPY_RELAXED_STRIDES_CHECKING
/*
* When NPY_RELAXED_STRIDES_CHECKING is used, some buffer users
* may expect a contiguous buffer to have well formatted strides
* also when a dimension is 1, but we do not guarantee this
* internally. Thus, recalculate strides for contiguous arrays.
* (This is unnecessary, but has no effect in the case where
* NPY_RELAXED_STRIDES CHECKING is disabled.)
* Some buffer users may expect a contiguous buffer to have well
* formatted strides also when a dimension is 1, but we do not
* guarantee this internally. Thus, recalculate strides for
* contiguous arrays.
*/
int f_contiguous = (flags & PyBUF_F_CONTIGUOUS) == PyBUF_F_CONTIGUOUS;
if (PyArray_IS_C_CONTIGUOUS(arr) && !(
Expand All @@ -526,11 +523,6 @@ _buffer_info_new(PyObject *obj, int flags)
}
}
else {
#else /* NPY_RELAXED_STRIDES_CHECKING */
/* We can always use the arrays strides directly */
{
#endif

for (k = 0; k < PyArray_NDIM(arr); ++k) {
info->shape[k] = PyArray_DIMS(arr)[k];
info->strides[k] = PyArray_STRIDES(arr)[k];
Expand Down Expand Up @@ -708,8 +700,8 @@ _buffer_get_info(void **buffer_info_cache_ptr, PyObject *obj, int flags)
if (info->ndim > 1 && next_info != NULL) {
/*
* Some arrays are C- and F-contiguous and if they have more
* than one dimension, the buffer-info may differ between
* the two due to RELAXED_STRIDES_CHECKING.
* than one dimension, the buffer-info may differ between the
* two because strides for length 1 dimension may be adjusted.
* If we export both buffers, the first stored one may be
* the one for the other contiguity, so check both.
* This is generally very unlikely in all other cases, since
Expand Down
14 changes: 0 additions & 14 deletions numpy/core/src/multiarray/ctors.c
View file
Open in desktop
< F438 tr data-hunk="00f372f2c53d24ecd62514fe152c06b9e4ae26875ea44bbb1657b9e50a40d9db" class="show-top-border">
Original file line number Diff line number Diff line change
Expand Up @@ -4020,7 +4020,6 @@ _array_fill_strides(npy_intp *strides, npy_intp const *dims, int nd, size_t item
int inflag, int *objflags)
{
int i;
#if NPY_RELAXED_STRIDES_CHECKING
npy_bool not_cf_contig = 0;
npy_bool nod = 0; /* A dim != 1 was found */

Expand All @@ -4034,7 +4033,6 @@ _array_fill_strides(npy_intp *strides, npy_intp const *dims, int nd, size_t item
nod = 1;
}
}
#endif /* NPY_RELAXED_STRIDES_CHECKING */

/* Only make Fortran strides if not contiguous as well */
if ((inflag & (NPY_ARRAY_F_CONTIGUOUS|NPY_ARRAY_C_CONTIGUOUS)) ==
Expand All @@ -4044,7 +4042,6 @@ _array_fill_strides(npy_intp *strides, npy_intp const *dims, int nd, size_t item
if (dims[i]) {
itemsize *= dims[i];
}
#if NPY_RELAXED_STRIDES_CHECKING
else {
not_cf_contig = 0;
}
Expand All @@ -4054,13 +4051,8 @@ _array_fill_strides(npy_intp *strides, npy_intp const *dims, int nd, size_t item
strides[i] = NPY_MAX_INTP;
}
#endif /* NPY_RELAXED_STRIDES_DEBUG */
#endif /* NPY_RELAXED_STRIDES_CHECKING */
}
#if NPY_RELAXED_STRIDES_CHECKING
if (not_cf_contig) {
#else /* not NPY_RELAXED_STRIDES_CHECKING */
if ((nd > 1) && ((strides[0] != strides[nd-1]) || (dims[nd-1] > 1))) {
#endif /* not NPY_RELAXED_STRIDES_CHECKING */
*objflags = ((*objflags)|NPY_ARRAY_F_CONTIGUOUS) &
~NPY_ARRAY_C_CONTIGUOUS;
}
Expand All @@ -4074,7 +4066,6 @@ _array_fill_strides(npy_intp *strides, npy_intp const *dims, int nd, size_t item
if (dims[i]) {
itemsize *= dims[i];
}
#if NPY_RELAXED_STRIDES_CHECKING
else {
not_cf_contig = 0;
}
Expand All @@ -4084,13 +4075,8 @@ _array_fill_strides(npy_intp *strides, npy_intp const *dims, int nd, size_t item
strides[i] = NPY_MAX_INTP;
}
#endif /* NPY_RELAXED_STRIDES_DEBUG */
#endif /* NPY_RELAXED_STRIDES_CHECKING */
}
#if NPY_RELAXED_STRIDES_CHECKING
if (not_cf_contig) {
#else /* not NPY_RELAXED_STRIDES_CHECKING */
if ((nd > 1) && ((strides[0] != strides[nd-1]) || (dims[0] > 1))) {
#endif /* not NPY_RELAXED_STRIDES_CHECKING */
*objflags = ((*objflags)|NPY_ARRAY_C_CONTIGUOUS) &
~NPY_ARRAY_F_CONTIGUOUS;
}
Expand Down
42 changes: 5 additions & 37 deletions numpy/core/src/multiarray/flagsobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,20 +105,11 @@ PyArray_UpdateFlags(PyArrayObject *ret, int flagmask)
*
* According to these rules, a 0- or 1-dimensional array is either both
* C- and F-contiguous, or neither; and an array with 2+ dimensions
* can be C- or F- contiguous, or neither, but not both. Though there
* there are exceptions for arrays with zero or one item, in the first
* case the check is relaxed up to and including the first dimension
* with shape[i] == 0. In the second case `strides == itemsize` will
* can be true for all dimensions and both flags are set.
*
* When NPY_RELAXED_STRIDES_CHECKING is set, we use a more accurate
* definition of C- and F-contiguity, in which all 0-sized arrays are
* contiguous (regardless of dimensionality), and if shape[i] == 1
* then we ignore strides[i] (since it has no affect on memory layout).
* With these new rules, it is possible for e.g. a 10x1 array to be both
* C- and F-contiguous -- but, they break downstream code which assumes
* that for contiguous arrays strides[-1] (resp. strides[0]) always
* contains the itemsize.
* can be C- or F- contiguous, or neither, but not both (unless it has only
* a single element).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not true, Maybe "one dimensional and contiguous"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is true based on the simplified (old) rules that are given above?

* We correct this, however. When a dimension has length 1, its stride is
* never used and thus has no effect on the memory layout.
* The above rules thus only apply when ignorning all size 1 dimenions.
*/
static void
_UpdateContiguousFlags(PyArrayObject *ap)
Expand All @@ -131,7 +122,6 @@ _UpdateContiguousFlags(PyArrayObject *ap)
sd = PyArray_ITEMSIZE(ap);
for (i = PyArray_NDIM(ap) - 1; i >= 0; --i) {
dim = PyArray_DIMS(ap)[i];
#if NPY_RELAXED_STRIDES_CHECKING
/* contiguous by definition */
if (dim == 0) {
PyArray_ENABLEFLAGS(ap, NPY_ARRAY_C_CONTIGUOUS);
Expand All @@ -144,17 +134,6 @@ _UpdateContiguousFlags(PyArrayObject *ap)
}
sd *= dim;
}
#else /* not NPY_RELAXED_STRIDES_CHECKING */
if (PyArray_STRIDES(ap)[i] != sd) {
is_c_contig = 0;
break;
}
/* contiguous, if it got this far */
if (dim == 0) {
break;
}
sd *= dim;
#endif /* not NPY_RELAXED_STRIDES_CHECKING */
}
if (is_c_contig) {
PyArray_ENABLEFLAGS(ap, NPY_ARRAY_C_CONTIGUOUS);
Expand All @@ -167,24 +146,13 @@ _UpdateContiguousFlags(PyArrayObject *ap)
sd = PyArray_ITEMSIZE(ap);
for (i = 0; i < PyArray_NDIM(ap); ++i) {
dim = PyArray_DIMS(ap)[i];
#if NPY_RELAXED_STRIDES_CHECKING
if (dim != 1) {
if (PyArray_STRIDES(ap)[i] != sd) {
PyArray_CLEARFLAGS(ap, NPY_ARRAY_F_CONTIGUOUS);
return;
}
sd *= dim;
}
#else /* not NPY_RELAXED_STRIDES_CHECKING */
if (PyArray_STRIDES(ap)[i] != sd) {
PyArray_CLEARFLAGS(ap, NPY_ARRAY_F_CONTIGUOUS);
return;
}
if (dim == 0) {
break;
}
sd *= dim;
#endif /* not NPY_RELAXED_STRIDES_CHECKING */
}
PyArray_ENABLEFLAGS(ap, NPY_ARRAY_F_CONTIGUOUS);
return;
Expand Down
5 changes: 3 additions & 2 deletions numpy/core/src/multiarray/multiarraymodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1533,8 +1533,9 @@ PyArray_EquivTypenums(int typenum1, int typenum2)

/*** END C-API FUNCTIONS **/
/*
* NPY_RELAXED_STRIDES_CHECKING: If the strides logic is changed, the
* order specific stride setting is not necessary.
* NOTE: The order specific stride setting is not necessary to preserve
* contiguity and could be removed. However, this way the resulting
* strides do currently look clearer for fortran order inputs.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe "strides look better for . . ."?

*/
static NPY_STEALS_REF_TO_ARG(1) PyObject *
_prepend_ones(PyArrayObject *arr, int nd, int ndmin, NPY_ORDER order)
Expand Down
7 changes: 2 additions & 5 deletions numpy/core/src/multiarray/shape.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,11 +244,9 @@ PyArray_Newshape(PyArrayObject *self, PyArray_Dims *newdims,
* in order to get the right orientation and
* because we can't just re-use the buffer with the
* data in the order it is in.
* NPY_RELAXED_STRIDES_CHECKING: size check is unnecessary when set.
*/
Py_INCREF(self);
if ((PyArray_SIZE(self) > 1) &&
((order == NPY_CORDER && !PyArray_IS_C_CONTIGUOUS(self)) ||
if (((order == NPY_CORDER && !PyArray_IS_C_CONTIGUOUS(self)) ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just check if this is correct.

(order == NPY_FORTRANORDER && !PyArray_IS_F_CONTIGUOUS(self)))) {
int success = 0;
success = _attempt_nocopy_reshape(self, ndim, dimensions,
Expand Down Expand Up @@ -1000,7 +998,6 @@ PyArray_Flatten(PyArrayObject *a, NPY_ORDER order)
* If an axis flagged for removal has a shape larger than one,
* the aligned flag (and in the future the contiguous flags),
* may need explicit update.
* (check also NPY_RELAXED_STRIDES_CHECKING)
*
* For example, this can be used to remove the reduction axes
* from a reduction result once its computation is complete.
Expand All @@ -1024,6 +1021,6 @@ PyArray_RemoveAxesInPlace(PyArrayObject *arr, const npy_bool *flags)
/* The final number of dimensions */
fa->nd = idim_out;

/* May not be necessary for NPY_RELAXED_STRIDES_CHECKING (see comment) */
/* NOTE: This is only necessary if a dimension with size != 1 was removed */
PyArray_UpdateFlags(arr, NPY_ARRAY_C_CONTIGUOUS | NPY_ARRAY_F_CONTIGUOUS);
}
Loading
0