8000 Addressing review comments · numpy/numpy@79b6702 · GitHub
[go: up one dir, main page]
Skip to content

Commit 79b6702

Browse files
melissawmmelissawm
committed
Addressing review comments
1 parent 7736826 commit 79b6702

File tree

3 files changed

+85
-70
lines changed

3 files changed

+85
-70
lines changed

‎doc/source/reference/arrays.classes.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ however, of why your subroutine may not be able to handle an arbitrary
4242
subclass of an array is that matrices redefine the "*" operator to be
4343
matrix-multiplication, rather than element-by-element multiplication.
4444

45+
.. _special-attributes-and-methods:
4546

4647
Special attributes and methods
4748
==============================

‎doc/source/reference/arrays.interface.rst

Lines changed: 19 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -4,18 +4,18 @@
44

55
.. _arrays.interface:
66

7-
*******************
8-
The Array Interface
9-
*******************
7+
****************************
8+
The array interface protocol
9+
****************************
1010

1111
.. note::
1212

13-
This page describes the numpy-specific API for accessing the contents of
14-
a numpy array from other C extensions. :pep:`3118` --
13+
This page describes the NumPy-specific API for accessing the contents of
14+
a NumPy array from other C extensions. :pep:`3118` --
1515
:c:func:`The Revised Buffer Protocol <PyObject_GetBuffer>` introduces
1616
similar, standardized API to Python 2.6 and 3.0 for any extension
1717
module to use. Cython__'s buffer array support
18-
uses the :pep:`3118` API; see the `Cython numpy
18+
uses the :pep:`3118` API; see the `Cython NumPy
1919
tutorial`__. Cython provides a way to write code that supports the buffer
2020
protocol with Python versions older than 2.6 because it has a
2121
backward-compatible implementation utilizing the array interface
@@ -81,7 +81,8 @@ This approach to the interface consists of the object having an
8181
===== ================================================================
8282
``t`` Bit field (following integer gives the number of
8383
bits in the bit field).
84-
``b`` Boolean (integer type where all values are only True or False)
84+
``b`` Boolean (integer type where all values are only ``True`` or
85+
``False``)
8586
``i`` Integer
8687
``u`` Unsigned integer
8788
``f`` Floating point
@@ -141,11 +142,11 @@ This approach to the interface consists of the object having an
141142
must be stored by the new object if the memory area is to be
142143
secured.
143144

144-
**Default**: None
145+
**Default**: ``None``
145146

146147
**strides** (optional)
147148
Either ``None`` to indicate a C-style contiguous array or
148-
a Tuple of strides which provides the number of bytes needed
149+
a tuple of strides which provides the number of bytes needed
149150
to jump to the next array element in the corresponding
150151
dimension. Each entry must be an integer (a Python
151152
:py:class:`int`). As with shape, the values may
@@ -156,26 +157,26 @@ This approach to the interface consists of the object having an
156157
memory buffer. In this model, the last dimension of the array
157158
varies the fastest. For example, the default strides tuple
158159
for an object whose array entries are 8 bytes long and whose
159-
shape is ``(10, 20, 30)`` would be ``(4800, 240, 8)``
160+
shape is ``(10, 20, 30)`` would be ``(4800, 240, 8)``.
160161

161162
**Default**: ``None`` (C-style contiguous)
162163

163164
**mask** (optional)
164-
None or an object exposing the array interface. All
165+
``None`` or an object exposing the array interface. All
165166
elements of the mask array should be interpreted only as true
166167
or not true indicating which elements of this array are valid.
167168
The shape of this object should be `"broadcastable"
168169
<arrays.broadcasting.broadcastable>` to the shape of the
169170
original array.
170171

171-
**Default**: None (All array values are valid)
172+
**Default**: ``None`` (All array values are valid)
172173

173174
**offset** (optional)
174175
An integer offset into the array data region. This can only be
175176
used when data is ``None`` or returns a :class:`buffer`
176177
object.
177178

178-
**Default**: 0.
179+
8000 **Default**: ``0``.
179180

180181
**version** (required)
181182
An integer showing the version of the interface (i.e. 3 for
@@ -243,6 +244,11 @@ flag is present.
243244
returning the :c:type:`PyCapsule`, and configure a destructor to decref this
244245
reference.
245246

247+
.. note::
248+
249+
:obj:`__array_struct__` is considered legacy and should not be used for new
250+
code. Use the :py:doc:`buffer protocol <c-api/buffer>` instead.
251+
246252

247253
Type description examples
248254
=========================

‎doc/source/user/basics.interoperability.rst

Lines changed: 65 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33
Interoperability with NumPy
44
***************************
55

6-
NumPy’s ndarray objects provide both a high-level API for operations on
6+
NumPy's ndarray objects provide both a high-level API for operations on
77
array-structured data and a concrete implementation of the API based on
8-
`strided in-RAM storage <https://numpy.org/doc/stable/reference/arrays.html>`__.
8+
:ref:`strided in-RAM storage <arrays>`.
99
While this API is powerful and fairly general, its concrete implementation has
1010
limitations. As datasets grow and NumPy becomes used in a variety of new
1111
environments and architectures, there are cases where the strided in-RAM storage
@@ -29,44 +29,39 @@ Using arbitrary objects in NumPy
2929

3030
When NumPy functions encounter a foreign object, they will try (in order):
3131

32-
1. The buffer protocol, described `in the Python C-API documentation
33-
<https://docs.python.org/3/c-api/buffer.html>`__.
32+
1. The buffer protocol, described :py:doc:`in the Python C-API documentation
33+
<c-api/buffer>`.
3434
2. The ``__array_interface__`` protocol, described
35-
:ref:`in this page <arrays.interface>`. A precursor to Python’s buffer
35+
:ref:`in this page <arrays.interface>`. A precursor to Python's buffer
3636
protocol, it defines a way to access the contents of a NumPy array from other
3737
C extensions.
38-
3. The ``__array__`` protocol, which asks an arbitrary object to convert itself
39-
into an array.
38+
3. The ``__array__()`` method, which asks an arbitrary object to convert
39+
itself into an array.
4040

4141
For both the buffer and the ``__array_interface__`` protocols, the object
4242
describes its memory layout and NumPy does everything else (zero-copy if
43-
possible). If that’s not possible, the object itself is responsible for
43+
possible). If that's not possible, the object itself is responsible for
4444
returning a ``ndarray`` from ``__array__()``.
4545

46-
The array interface
47-
~~~~~~~~~~~~~~~~~~~
46+
The array interface protocol
47+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4848

49-
The :ref:`array interface <arrays.interface>` defines a protocol for array-like
50-
objects to re-use each other’s data buffers. Its implementation relies on the
51-
existence of the following attributes or methods:
49+
The :ref:`array interface protocol <arrays.interface>` defines a way for
50+
array-like objects to re-use each other's data buffers. Its implementation
51+
relies on the existence of the following attributes or methods:
5252

5353
- ``__array_interface__``: a Python dictionary containing the shape, the
5454
element type, and optionally, the data buffer address and the strides of an
5555
array-like object;
5656
- ``__array__()``: a method returning the NumPy ndarray view of an array-like
5757
object;
58-
- ``__array_struct__``: a ``PyCapsule`` containing a pointer to a
59-
``PyArrayInterface`` C-structure.
6058

61-
The ``__array_interface__`` and ``__array_struct__`` attributes can be inspected
62-
directly:
59+
The ``__array_interface__`` attribute can be inspected directly:
6360

6461
>>> import numpy as np
6562
>>> x = np.array([1, 2, 5.0, 8])
6663
>>> x.__array_interface__
6764
{'data': (94708397920832, False), 'strides': None, 'descr': [('', '<f8')], 'typestr': '<f8', 'shape': (4,), 'version': 3}
68-
>>> x.__array_struct__
69-
<capsule object NULL at 0x7f798800be40>
7065

7166
The ``__array_interface__`` attribute can also be used to manipulate the object
7267
data in place:
@@ -96,21 +91,20 @@ We can check that ``arr`` and ``new_arr`` share the same data buffer:
9691
array([1000, 2, 3, 4])
9792

9893

99-
The ``__array__`` protocol
94+
The ``__array__()`` method
10095
~~~~~~~~~~~~~~~~~~~~~~~~~~
10196

102-
The ``__array__`` protocol acts as a dispatch mechanism and ensures that any
103-
NumPy-like object (an array, any object exposing the array interface, an object
104-
whose ``__array__`` method returns an array or any nested sequence) that
105-
implements it can be used as a NumPy array. If possible, this will mean using
106-
``__array__`` to create a NumPy ndarray view of the array-like object.
107-
Otherwise, this copies the data into a new ndarray object. This is not optimal,
108-
as coercing arrays into ndarrays may cause performance problems or create the
109-
need for copies and loss of metadata.
97+
The ``__array__()`` method ensures that any NumPy-like object (an array, any
98+
object exposing the array interface, an object whose ``__array__()`` method
99+
returns an array or any nested sequence) that implements it can be used as a
100+
NumPy array. If possible, this will mean using ``__array__()`` to create a NumPy
101+
ndarray view of the array-like object. Otherwise, this copies the data into a
102+
new ndarray object. This is not optimal, as coercing arrays into ndarrays may
103+
cause performance problems or create the need for copies and loss of metadata,
104+
as the original object and any attributes/behavior it may have had, is lost.
110105

111-
To see an example of a custom array implementation including the use of the
112-
``__array__`` protocol, see `Writing custom array containers
113-
<https://numpy.org/devdocs/user/basics.dispatch.html>`__.
106+
To see an example of a custom array implementation including the use of
107+
``__array__()``, see :ref:`basics.dispatch`.
114108

115109
Operating on foreign objects without converting
116110
-----------------------------------------------
@@ -121,7 +115,11 @@ Consider the following function.
121115
>>> def f(x):
122116
... return np.mean(np.exp(x))
123117

124-
We can apply it to a NumPy ndarray object directly:
118+
Note that `np.exp` is a :ref:`ufunc <ufuncs-basics>`, which means that it
119+
operates on ndarrays in an element-by-element fashion. On the other hand,
120+
`np.mean` operates along one of the array's axes.
121+
122+
We can apply ``f`` to a NumPy ndarray object directly:
125123

126124
>>> x = np.array([1, 2, 3, 4])
127125
>>> f(x)
@@ -149,9 +147,13 @@ The ``__array_ufunc__`` protocol
149147
A :ref:`universal function (or ufunc for short) <ufuncs-basics>` is a
150148
“vectorized” wrapper for a function that takes a fixed number of specific inputs
151149
and produces a fixed number of specific outputs. The output of the ufunc (and
152-
its methods) is not necessarily an ndarray, if all input arguments are not
150+
its methods) is not necessarily an ndarray, if not all input arguments are
153151
ndarrays. Indeed, if any input defines an ``__array_ufunc__`` method, control
154-
will be passed completely to that function, i.e., the ufunc is overridden.
152+
will be passed completely to that function, i.e., the ufunc is overridden. The
153+
``__array_ufunc__`` method defined on that (non-ndarray) object has access to
154+
the NumPy ufunc. Because ufuncs have a well-defined structure, the foreign
155+
``__array_ufunc__`` method may rely on ufunc attributes like ``.at()``,
156+
``.reduce()``, and others.
155157

156158
A subclass can override what happens when executing NumPy ufuncs on it by
157159
overriding the default ``ndarray.__array_ufunc__`` method. This method is
@@ -169,9 +171,7 @@ is safe and consistent across projects.
169171

170172
The semantics of ``__array_function__`` are very similar to ``__array_ufunc__``,
171173
except the operation is specified by an arbitrary callable object rather than a
172-
ufunc instance and method. For more details, see `NEP 18
173-
<https://numpy.org/neps/nep-0018-array-function-protocol.html>`__.
174-
174+
ufunc instance and method. For more details, see :ref:`NEP18`.
175175

176176
Interoperability examples
177177
-------------------------
@@ -223,7 +223,7 @@ Example: PyTorch tensors
223223

224224
`PyTorch <https://pytorch.org/>`__ is an optimized tensor library for deep
225225
learning using GPUs and CPUs. PyTorch arrays are commonly called *tensors*.
226-
Tensors are similar to NumPy’s ndarrays, except that tensors can run on GPUs or
226+
Tensors are similar to NumPy's ndarrays, except that tensors can run on GPUs or
227227
other hardware accelerators. In fact, tensors and NumPy arrays can often share
228228
the same underlying memory, eliminating the need to copy data.
229229

@@ -251,13 +251,22 @@ explicit conversion:
251251
Also, note that the return type of this function is compatible with the initial
252252
data type.
253253

254-
**Note** PyTorch does not implement ``__array_function__`` or
255-
``__array_ufunc__``. Under the hood, the ``Tensor.__array__()`` method returns a
256-
NumPy ndarray as a view of the tensor data buffer. See `this issue
257-
<https://github.com/pytorch/pytorch/issues/24015>`__ and the
258-
`__torch_function__ implementation
259-
<https://github.com/pytorch/pytorch/blob/master/torch/overrides.py>`__
260-
for details.
254+
.. admonition:: Warning
255+
256+
While this mixing of ndarrays and tensors may be convenient, it is not
257+
recommended. It will not work for non-CPU tensors, and will have unexpected
258+
behavior in corner cases. Users should prefer explicitly converting the
259+
ndarray to a tensor.
260+
261+
.. note::
262+
263+
PyTorch does not implement ``__array_function__`` or ``__array_ufunc__``.
264+
Under the hood, the ``Tensor.__array__()`` method returns a NumPy ndarray as
265+
a view of the tensor data buffer. See `this issue
266+
<https://github.com/pytorch/pytorch/issues/24015>`__ and the
267+
`__torch_function__ implementation
268+
<https://github.com/pytorch/pytorch/blob/master/torch/overrides.py>`__
269+
for details.
261270

262271
Example: CuPy arrays
263272
~~~~~~~~~~~~~~~~~~~~
@@ -271,7 +280,8 @@ with Python. CuPy implements a subset of the NumPy interface by implementing
271280
>>> x_gpu = cp.array([1, 2, 3, 4])
272281

273282
The ``cupy.ndarray`` object implements the ``__array_ufunc__`` interface. This
274-
enables NumPy ufuncs to be directly operated on CuPy arrays:
283+
enables NumPy ufuncs to be applied to CuPy arrays (this will defer operation to
284+
the matching CuPy CUDA/ROCm implementation of the ufunc):
275285

276286
>>> np.mean(np.exp(x_gpu))
277287
array(21.19775622)
@@ -307,8 +317,7 @@ implements a subset of the NumPy ndarray interface using blocked algorithms,
307317
cutting up the large array into many small arrays. This allows computations on
308318
larger-than-memory arrays using multiple cores.
309319

310-
Dask supports array protocols like ``__array__`` and
311-
``__array_ufunc__``.
320+
Dask supports ``__array__()`` and ``__array_ufunc__``.
312321

313322
>>> import dask.array as da
314323
>>> x = da.random.normal(1, 0.1, size=(20, 20), chunks=(10, 10))
@@ -317,8 +326,10 @@ Dask supports array protocols like ``__array__`` and
317326
>>> np.mean(np.exp(x)).compute()
318327
5.090097550553843
319328

320-
**Note** Dask is lazily evaluated, and the result from a computation isn’t
321-
computed until you ask for it by invoking ``compute()``.
329+
.. note::
330+
331+
Dask is lazily evaluated, and the result from a computation isn't computed
332+
until you ask for it by invoking ``compute()``.
322333

323334
See `the Dask array documentation
324335
<https://docs.dask.org/en/stable/array.html>`__
@@ -328,13 +339,10 @@ and the `scope of Dask arrays interoperability with NumPy arrays
328339
Further reading
329340
---------------
330341

331-
- `The Array interface
332-
<https://numpy.org/doc/stable/reference/arrays.interface.html>`__
333-
- `Writing custom array containers
334-
<https://numpy.org/devdocs/user/basics.dispatch.html>`__.
335-
- `Special array attributes
336-
<https://numpy.org/devdocs/reference/arrays.classes.html#special-attributes-and-methods>`__
337-
(details on the ``__array_ufunc__`` and ``__array_function__`` protocols)
342+
- :ref:`arrays.interface`
343+
- :ref:`basics.dispatch`
344+
- :ref:`special-attributes-and-methods` (details on the ``__array_ufunc__`` and
345+
``__array_function__`` protocols)
338346
- `NumPy roadmap: interoperability
339347
<https://numpy.org/neps/roadmap.html#interoperability>`__
340348
- `PyTorch documentation on the Bridge with NumPy

0 commit comments

Comments
 (0)
0