8000 NEP: Adjust NEP-35 to make it more user-accessible by pentschev · Pull Request #17093 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

NEP: Adjust NEP-35 to make it more user-accessible #17093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Sep 7, 2020
Merged
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
7d7b46c
NEP: Adjust NEP-35 to make it more user-accessible
pentschev Aug 14, 2020
9b660e4
NEP: Simplify NEP-35 further with reviewer's suggestions
pentschev Aug 17, 2020
68fd054
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
61dcb63
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
3cf7b6b
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
52d9c74
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
615f19f
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
69e3e71
NEP: Improve NEP-35 abstract per @mattip's suggestion
pentschev Aug 19, 2020
cde3543
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
1017007
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
a82cc4b
NEP: Move NumPy users comment to top of NEP-35 Usage and Impact
pentschev Aug 19, 2020
17620c2
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
f1d1562
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
57d6bab
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
67c9733
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
974c023
Update doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
pentschev Aug 19, 2020
b5f5577
NEP: Clarify NEP-35 C implementation details.
pentschev Aug 19, 2020
2e30534
NEP: Clarify Dask intent with `my_dask_pad` function name
pentschev Aug 19, 2020
3d527ea
NEP: Improve grammar on NEP-35 reference to Dask's objects
pentschev Aug 19, 2020
b6f2c16
NEP: Fix some grammar and formatting in NEP-35
pentschev Aug 19, 2020
57f78df
ENH: Clarifies meta_from_array function in NEP-35
pentschev Aug 19, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
ENH: Clarifies meta_from_array function in NEP-35
  • Loading branch information
8000 @pentschev
pentschev committed Aug 19, 2020
commit 57f78df125892eaa4539913759f3ac29ff4aef6a
32 changes: 26 additions & 6 deletions doc/neps/nep-0035-array-creation-dispatch-with-array-function.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,12 +56,14 @@ experience, all using NumPy syntax. For example, say we have some CuPy array
write the following:

.. code:: python

x = cupy.identity(3)

Instead, the better way would be using to only use the NumPy API, this could now
be achieved with:

.. code:: python

x = np.identity(3, like=cp_arr)

As if by magic, ``x`` will also be a CuPy array, as NumPy was capable to infer
Expand Down Expand Up @@ -165,12 +167,30 @@ that are not of much use elsewhere.

To enable proper identification of the array type we use Dask's utility function
``meta_from_array``, which was introduced as part of the work to support
``__array_function__``, allowing Dask to handle ``_meta`` appropriately. That
function is primarily targeted at the library's internal usage to ensure chunks
are created with correct types. Without the ``like=`` argument, it would be
impossible to ensure ``my_pad`` creates a padding array with a type matching
that of the input array, which would cause a ``TypeError`` exception to
be raised by CuPy, as discussed above would happen to the CuPy case alone.
``__array_function__``, allowing Dask to handle ``_meta`` appropriately. Readers
can think of ``meta_from_array`` as a special function that just returns the
type of the underlying Dask array, for example:

.. code:: python

np_arr = da.arange(5)
cp_arr = da.from_array(cupy.arange(5))

meta_from_array(np_arr) # Returns a numpy.ndarray
meta_from_array(cp_arr) # Returns a cupy.ndarray

Since the value returned by ``meta_from_array`` is a NumPy-like array, we can
just pass that directly into the ``like=`` argument.

The ``meta_from_array`` function is primarily targeted at the library's internal
usage to ensure chunks are created with correct types. Without the ``like=``
argument, it would be impossible to ensure ``my_pad`` creates a padding array
with a type matching that of the input array, which would cause a ``TypeError``
exception to be raised by CuPy, as discussed above would happen to the CuPy case
alone. Combining Dask's internal handling of meta arrays and the proposed
``like=`` argument, it now becomes possible to handle cases involving creation
of non-NumPy arrays, which is likely the heaviest limitation Dask currently
faces from the ``__array_function__`` protocol.

Backward Compatibility
----------------------
Expand Down
0