8000 ENH: Add support for copy modes to NumPy by czgdp1807 · Pull Request #19173 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

ENH: Add support for copy modes to NumPy #19173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 65 commits into from
Nov 15, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
496bd1a
Added np.CopyMode
czgdp1807 Jun 5, 2021
3a3d31b
initial work for supporting never_copy
czgdp1807 Jun 5, 2021
be5823f
Addressed reviews and PyArray_CopyConverter defined
czgdp1807 Jun 7, 2021
e16f3ff
np.CopyMode.NEVER gives error if copy cannot be avoided in np.array.a…
czgdp1807 Jun 7, 2021
645fadb
Updated API ready for formal testing
czgdp1807 Jun 7, 2021
8538720
tests for astype added
czgdp1807 Jun 7, 2021
8c00223
Added tests for np.array
czgdp1807 Jun 7, 2021
4e59da1
Testing for copy argument for existing APIs complete
czgdp1807 Jun 7, 2021
6f0f6ac
resolved conflicts
czgdp1807 Jun 7, 2021
6bff3dd
Added some notes for future commits
czgdp1807 Jun 8, 2021
a90880a
RuntimeError -> ValueError
czgdp1807 Jun 8, 2021
5481a8a
enum.IntEnum -> enum.Enum
czgdp1807 Jun 8, 2021
f2e7a81
aligned astype with main branch
czgdp1807 Jun 8, 2021
e828f15
Add tests for scalars
czgdp1807 Jun 8, 2021
1a92ae9
Merge branch 'main' into never_copy
czgdp1807 Jun 8, 2021
91d6693
resolved linting issues
czgdp1807 Jun 8, 2021
7b53a02
Fixed blank line linting issue
czgdp1807 Jun 8, 2021
eedb300
Apply suggestions from code review
czgdp1807 Jun 9, 2021
bc8903d
renaming complete
czgdp1807 Jun 9, 2021
3268a48
fixed failures due to Python 3.8.2
czgdp1807 Jun 9, 2021
ab01330
Merge branch 'main' into never_copy
czgdp1807 Jun 9, 2021
782d823
fixed PYPY errors
czgdp1807 Jun 10, 2021
342e0c5
removed complicated checks for copy_mode
czgdp1807 Jun 10, 2021
a09929b
Merge branch 'main' into never_copy
czgdp1807 Jun 15, 2021
30e3472
interface shifted
czgdp1807 Jun 15, 2021
a6caf9c
Apply suggestions from code review
czgdp1807 Jun 15, 2021
3dcf3a9
resolved conflicts
czgdp1807 Aug 6, 2021
321e028
Shifted to CopyMode to np.array_api
czgdp1807 Aug 7, 2021
844883c
corrected linting issues
czgdp1807 Aug 7, 2021
d995ecc
resolved conflicts
czgdp1807 Aug 7, 2021
0f8d4c5
fixed linting issues
czgdp1807 Aug 7, 2021
698fb6d
Made _CopyMode private
czgdp1807 Aug 18, 2021
b341e4c
Fixed linting issues
czgdp1807 Aug 18, 2021
781d0a7
resolved conflicts
czgdp1807 Sep 3, 2021
45dbdc9
CopyMode added to np.array_api
czgdp1807 Sep 3, 2021
a39312c
fixed linting issues
czgdp1807 Sep 3, 2021
c2acd5b
fixed linting issues
czgdp1807 Sep 3, 2021
56647dd
Addressed reviews
czgdp1807 Sep 4, 2021
c04509e
resolved conflicts
czgdp1807 Nov 2, 2021
790f927
Addressed reviews
czgdp1807 Nov 2, 2021
2677788
Fixed type check
czgdp1807 Nov 2, 2021
d9a9785
Addressed reviews and increased code coverage
czgdp1807 Nov 4, 2021
5cb2d64
Addressed reviews and increased code coverage
czgdp1807 Nov 9, 2021
8b939c9
Intentional RuntimeError
czgdp1807 Nov 9, 2021
7658ad9
Removed dead code
czgdp1807 Nov 10, 2021
2cf561b
Prohibited calling ``__array__`` method in never copy mode
czgdp1807 Nov 10, 2021
37cd05e
Fixed warning and updated docs
czgdp1807 Nov 10, 2021
9f9a348
Apply suggestions from code review
czgdp1807 Nov 10, 2021
946ab24
Updated docs
czgdp1807 Nov 10, 2021
2c51b0a
Merge branch 'never_copy' of https://github.com/czgdp1807/numpy into …
czgdp1807 Nov 10, 2021
5ede7eb
Added release notes entry
czgdp1807 Nov 10, 2021
d1cb662
L[0]->L[False], L[1]->L[True]
czgdp1807 Nov 10, 2021
6c01915
Deleted release notes entry
czgdp1807 Nov 10, 2021
5f1965f
do_copy -> allow_copy
czgdp1807 Nov 10, 2021
8000
05ff102
Apply suggestions from code review
czgdp1807 Nov 10, 2021
68dfa4b
Merge branch 'never_copy' of https://github.com/czgdp1807/numpy into …
czgdp1807 Nov 10, 2021
2db65c9
Addressed reviews
czgdp1807 Nov 10, 2021
d0d75f3
Reverted dead code removal
czgdp1807 Nov 10, 2021
eccb8df
Merge branch 'main' into never_copy
rgommers Nov 12, 2021
4b2cd27
STY: Small style fixups for never-copy changes
seberg Nov 12, 2021
84951a6
MAINT: Remove private CopyMode enum to private header
seberg Nov 12, 2021
f31c4a6
MAINT,BUG: Refactor __array__ and never-copy to move check later
seberg Nov 12, 2021
dea4055
MAINT: Rename `allow_copy` to `never_copy` in never-copy machinery
seberg Nov 12, 2021
9fee0f8
DOC: Slightly extend to docs to note that we assume no-copy buffer pr…
seberg Nov 12, 2021
f058aea
BUG: Fix refcounting issue and missed scalar special case for never-c…
seberg Nov 12, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
enum.IntEnum -> enum.Enum
  • Loading branch information
czgdp1807 committed Jun 8, 2021
commit 5481a8ac389962f855d15c91ba0fa365b3f06c90
2 changes: 1 addition & 1 deletion numpy/__init__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -3642,7 +3642,7 @@ trunc: _UFunc_Nin1_Nout1[L['trunc'], L[7], None]

abs = absolute

class CopyMode(enum.IntEnum):
class CopyMode(enum.Enum):

ALWAYS: L[1]
IF_NEEDED: L[0]
Expand Down
2 changes: 1 addition & 1 deletion numpy/_globals.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ def __repr__(self):

_NoValue = _NoValueType()

class CopyMode(enum.IntEnum):
class CopyMode(enum.Enum):

ALWAYS = 1
IF_NEEDED = 0
Expand Down
24 changes: 19 additions & 5 deletions numpy/core/src/multiarray/conversion_utils.c
Original file line number Diff line number Diff line change
Expand Up @@ -174,15 +174,29 @@ PyArray_CopyConverter(PyObject *obj, PyNpCopyMode_Enum *copymode) {
}

int int_copymode = -1;
PyArray_PythonPyIntFromInt(obj, &int_copymode);
PyObject* numpy_CopyMode = NULL;
npy_cache_import("numpy._globals", "CopyMode", &numpy_CopyMode);
if( numpy_CopyMode != NULL ) {
if(PyObject_IsInstance(obj, numpy_CopyMode)) {
PyObject* mode_value = PyObject_GetAttrString(obj, "value");
PyArray_PythonPyIntFromInt(mode_value, &int_copymode);
}
}
Copy link
Member
@eric-wieser eric-wieser Jun 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if( numpy_CopyMode != NULL ) {
if(PyObject_IsInstance(obj, numpy_CopyMode)) {
PyObject* mode_value = PyObject_GetAttrString(obj, "value");
PyArray_PythonPyIntFromInt(mode_value, &int_copymode);
}
}
if (numpy_CopyMode == NULL) {
return NPY_FAIL;
}
if (PyObject_IsInstance(obj, numpy_CopyMode)) {
PyObject* mode_value = PyObject_GetAttrString(obj, "value");
if (mode_value == NULL) {
return NPY_FAIL;
}
if (!PyArray_PythonPyIntFromInt(mode_value, &int_copymode)) {
return NPY_FAIL;
}
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there reasons against treating them as singletons?

if value is np.CopyMode.NEVER:
    ...
elif value is np.CopyMode.IF_NEEDED:
   ...
elif value is np.CopyMode.FORCE:
   ...
elif None:
   return  # (not sure we allow None for default)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, what happens in the above code is, first we try to verify if the copy argument is an instance of np.CopyMode. If it is then its value extracted and stored in int_copymode. If that doesn't work, then we follow the conventional path i.e., extract value of obj node via, PyArray_PythonPyIntFromInt. This helps in two ways, i) User can pass, np.CopyMode.NEVER and ii) They can also pass the values currently supported in main branch like, True, False, etc. In other words, the new changes extend the API and don't break anything existing.

Now, in the if-elif ladder in your comment, may break for values of copy argument which are currently supported since, we are just checking whether value is same as one of the options under np.CopyMode.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the last should just have been else: current_behaviour. But I think in Python we would write is also? And in that case it seems nicer to me here as well (you can add the isinstance/type-check check, but if its a 3-way if that should not be worthwhile).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One another way to simplify the checks would be to first import, np.CopyMode.ALWAYS, np.CopyMode.IF_NEEDED, np.CopyMode.NEVER, np.True_, np.False_ and then check whether the object is equal to any of these values, True or False using, PyObject_RichCompareBool. What do you say?


// If obj is not an instance of numpy.CopyMode then follow
// the conventional assumption that it must be value that
// can be converted to an integer.
if( int_copymode < 0 ) {
PyArray_PythonPyIntFromInt(obj, &int_copymode);
}

if( int_copymode != NPY_ALWAYS &&
int_copymode != NPY_IF_NEEDED &&
int_copymode != NPY_NEVER ) {
PyErr_Format(PyExc_ValueError,
"Unrecognized copy mode %d. Please choose one of "
"np.CopyMode.ALWAYS, np.CopyMode.IF_NEEDED, np.CopyMode.NEVER.",
int_copymode);
PyErr_SetString(PyExc_ValueError,
"Unrecognized copy mode. Please choose one of "
"np.CopyMode.ALWAYS, np.CopyMode.IF_NEEDED, np.CopyMode.NEVER, "
"True/np.True_, False/np.False_");
return NPY_FAIL;
}

Expand Down
0