-
-
Notifications
You must be signed in to change notification settings - Fork 10.9k
ENH: Add shape to *_like() array creation #13046
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
eric-wieser
merged 24 commits into
numpy:master
from
pentschev:add-shape-to-like-array-creation
Apr 25, 2019
Merged
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
f885c08
ENH: Added shape argument to *_like() array creation functions
pentschev 5ad61b6
ENH: C backend adjustments for shape argument on *_like()
pentschev fd3e270
TST: Added test for shape argument in *_like() functions
pentschev b7202a7
ENH: Added PyArray_NewLikeArrayWithShape()
pentschev 013cbce
BUG: Fix for PyArray_NewLikeArrayWithShape strides and ndim == 0
pentschev 807f512
REL: Updates for C-API, version 1.17.x
pentschev 95bbfd0
Revert "REL: Updates for C-API, version 1.17.x"
pentschev 43b9828
Revert exposing PyArray_NewLikeArrayWithShape on C-API
pentschev fa7fd75
DOC: fix versionadded for *_like() shape argument
pentschev d57e6d3
STY: add missing spaces in array initializers
pentschev f1b3e91
ENH: empty_like raises ValueError
pentschev d24ac10
TST: test for exception of *_like() functions
pentschev ec26417
DOC: release note for shape argument in *_like() functions
pentschev 40e7e9e
DOC: fix *_like() documentation on raises
pentschev 928952d
BUG: *_like() raises for non-C/F-layout arrays
pentschev e394356
TST: change *_like() shapes to prevent NPY_RELAXED_STRIDE_DEBUG=1 fai…
pentschev d938fb9
Move empty_like() exception to C implementation
pentschev 99a55e0
Merge branch 'master' into add-shape-to-like-array-creation
pentschev c087801
Update *_like() ValueError documentation
pentschev db7614b
Merge remote-tracking branch 'upstream/master' into add-shape-to-like…
pentschev 669ea71
Rearrange stride computation for *_like() if new shape and order='K'
pentschev f53afbe
Change handling of order= for *_like()
pentschev c3ac08e
Merge branch 'master' into add-shape-to-like-array-creation
pentschev 695b836
Fix *_like() tests
pentschev File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
8000 Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1183,28 +1183,37 @@ PyArray_NewFromDescrAndBase( | |
flags, obj, base, 0, 0); | ||
} | ||
|
||
/*NUMPY_API | ||
/* | ||
* Creates a new array with the same shape as the provided one, | ||
* with possible memory layout order and data type changes. | ||
* with possible memory layout order, data type and shape changes. | ||
* | ||
* prototype - The array the new one should be like. | ||
* order - NPY_CORDER - C-contiguous result. | ||
* NPY_FORTRANORDER - Fortran-contiguous result. | ||
* NPY_ANYORDER - Fortran if prototype is Fortran, C otherwise. | ||
* NPY_KEEPORDER - Keeps the axis ordering of prototype. | ||
* dtype - If not NULL, overrides the data type of the result. | ||
* ndim - If not 0 and dims not NULL, overrides the shape of the result. | ||
* dims - If not NULL and ndim not 0, overrides the shape of the result. | ||
* subok - If 1, use the prototype's array subtype, otherwise | ||
* always create a base-class array. | ||
* | ||
* NOTE: If dtype is not NULL, steals the dtype reference. On failure or when | ||
* dtype->subarray is true, dtype will be decrefed. | ||
*/ | ||
NPY_NO_EXPORT PyObject * | ||
PyArray_NewLikeArray(PyArrayObject *prototype, NPY_ORDER order, | ||
PyArray_Descr *dtype, int subok) | ||
PyArray_NewLikeArrayWithShape(PyArrayObject *prototype, NPY_ORDER order, | ||
PyArray_Descr *dtype, int ndim, npy_intp *dims, int subok) | ||
{ | ||
PyObject *ret = NULL; | ||
int ndim = PyArray_NDIM(prototype); | ||
|
||
if (dims == NULL) { | ||
ndim = PyArray_NDIM(prototype); | ||
dims = PyArray_DIMS(prototype); | ||
} | ||
else if (order == NPY_KEEPORDER && (ndim != PyArray_NDIM(prototype))) { | ||
order = NPY_CORDER; | ||
} | ||
|
||
/* If no override data type, use the one from the prototype */ | ||
if (dtype == NULL) { | ||
|
@@ -1237,7 +1246,7 @@ PyArray_NewLikeArray(PyArrayObject *prototype, NPY_ORDER order, | |
ret = PyArray_NewFromDescr(subok ? Py_TYPE(prototype) : &PyArray_Type, | ||
dtype, | ||
ndim, | ||
PyArray_DIMS(prototype), | ||
dims, | ||
NULL, | ||
NULL, | ||
order, | ||
|
@@ -1246,11 +1255,10 @@ PyArray_NewLikeArray(PyArrayObject *prototype, NPY_ORDER order, | |
/* KEEPORDER needs some analysis of the strides */ | ||
else { | ||
npy_intp strides[NPY_MAXDIMS], stride; | ||
npy_intp *shape = PyArray_DIMS(prototype); | ||
npy_stride_sort_item strideperm[NPY_MAXDIMS]; | ||
int idim; | ||
|
||
PyArray_CreateSortedStridePerm(PyArray_NDIM(prototype), | ||
PyArray_CreateSortedStridePerm(ndim, | ||
PyArray_STRIDES(prototype), | ||
pentschev marked this conversation as resolved.
Show resolved
Hide resolved
|
||
strideperm); | ||
|
||
|
@@ -1259,14 +1267,14 @@ PyArray_NewLikeArray(PyArrayObject *prototype, NPY_ORDER order, | |
for (idim = ndim-1; idim >= 0; --idim) { | ||
npy_intp i_perm = strideperm[idim].perm; | ||
strides[i_perm] = stride; | ||
stride *= shape[i_perm]; | ||
stride *= dims[i_perm]; | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My 1.ii proposal would look like changing everything between the new line 1261 and here to: if (PyArray_NDIM(prototype) >= ndim) {
int leading_dims = PyArray_NDIM(prototype) - ndim;
/* Use only the trailing strides */
PyArray_CreateSortedStridePerm(ndim,
PyArray_STRIDES(prototype) + leading_dims,
strideperm);
/* Build the new strides */
stride = dtype->elsize;
for (idim = ndim-1; idim >= 0; --idim) {
npy_intp i_perm = strideperm[idim].perm;
strides[i_perm] = stride;
stride *= shape[i_perm];
}
}
else {
int leading_dims = ndim - PyArray_NDIM(prototype);
/* Use all the strides */
PyArray_CreateSortedStridePerm(PyArray_NDIM(prototype),
PyArray_STRIDES(prototype),
strideperm);
/* Build the new trailing strides */
stride = dtype->elsize;
for (idim = PyArray_NDIM(prototype)-1; idim >= 0; --idim) {
npy_intp i_perm = strideperm[idim].perm + leading_dims;
strides[i_perm] = stride;
stride *= shape[i_perm];
}
/* Create remaining leading strides as C order */
for (idim = leading_dims; idim >= 0; --idim) {
strides[idim] = stride;
stride *= shape[idim];
}
} There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Or trading branching for state: /* not sure if this is clearer via min/max */
int leading_src_dims = 0; // max(src.ndim - dst.ndim, 0)
int leading_dst_dims = 0; // max(dst.ndim - src.ndim, 0)
int shared_dims; // min(src.ndim, dst.ndim
if (PyArray_NDIM(prototype) >= ndim) {
shared_dims = ndim;
leading_src_dims = PyArray_NDIM(prototype) - ndim;
}
else {
shared_dims = PyArray_NDIM(prototype);
leading_dst_dims = ndim - PyArray_NDIM(prototype);
}
/* Use only the trailing strides from the source */
PyArray_CreateSortedStridePerm(shared_dims,
PyArray_STRIDES(prototype) + leading_src_dims,
strideperm);
/* Build the destrination trailing strides */
stride = dtype->elsize;
for (idim = ndim-1; idim >= 0; --idim) {
npy_intp i_perm = strideperm[idim].perm + leading_dst_dims;
strides[i_perm] = stride;
stride *= shape[i_perm];
}
/* Create remaining leading strides as C order */
for (idim = leading_dst_dims; idim >= 0; --idim) {
strides[idim] = stride;
stride *= shape[idim];
} |
||
|
||
/* Finally, allocate the array */ | ||
ret = PyArray_NewFromDescr(subok ? Py_TYPE(prototype) : &PyArray_Type, | ||
dtype, | ||
ndim, | ||
shape, | ||
dims, | ||
strides, | ||
NULL, | ||
0, | ||
|
@@ -1276,6 +1284,29 @@ PyArray_NewLikeArray(PyArrayObject *prototype, NPY_ORDER order, | |
return ret; | ||
} | ||
|
||
/*NUMPY_API | ||
* Creates a new array with the same shape as the provided one, | ||
* with possible memory layout order and data type changes. | ||
* | ||
* prototype - The array the new one should be like. | ||
* order - NPY_CORDER - C-contiguous result. | ||
* NPY_FORTRANORDER - Fortran-contiguous result. | ||
* NPY_ANYORDER - Fortran if prototype is Fortran, C otherwise. | ||
* NPY_KEEPORDER - Keeps the axis ordering of prototype. | ||
* dtype - If not NULL, overrides the data type of the result. | ||
* subok - If 1, use the prototype's array subtype, otherwise | ||
* always create a base-class array. | ||
* | ||
* NOTE: If dtype is not NULL, steals the dtype reference. On failure or when | ||
* dtype->subarray is true, dtype will be decrefed. | ||
*/ | ||
NPY_NO_EXPORT PyObject * | ||
PyArray_NewLikeArray(PyArrayObject *prototype, NPY_ORDER order, | ||
PyArray_Descr *dtype, int subok) | ||
{ | ||
return PyArray_NewLikeArrayWithShape(prototype, order, dtype, 0, NULL, subok); | ||
} | ||
|
||
/*NUMPY_API | ||
* Generic new array creation routine. | ||
*/ | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just noticed order and subok both default to None rather than the ('K' and True) values documentation proposes and other _like() functions have. Which one is the correct? It seems to me that the function definition should be order='K' and subok=True, am I correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a little confusing, but the arguments for this function (the dispatcher) need to default to None. We actually have a test that verifies this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we then update the documentation to reflect the real defaults?