8000 Support Block (Chunk) Indexing (#1428) · keller-mark/zarr-python@98f74d5 · GitHub
[go: up one dir, main page]

Skip to content

Commit 98f74d5

Browse files
tasansalAltay Sansal
andauthored
Support Block (Chunk) Indexing (zarr-developers#1428)
* add .venv to `.gitignore` * add block indexing capabilities * add release notes * fix docstrings * update tutorial * fix missing codecov hit for read-only arrays * add block selection to array tests * lint * move release notes to unreleased section * update block indexing "as of" to 2.16 --------- Co-authored-by: Altay Sansal <altay.sansal@tgs.com>
1 parent ac89782 commit 98f74d5

File tree

8 files changed

+558
-13
lines changed

8 files changed

+558
-13
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ __pycache__/
88
# Distribution / packaging
99
.Python
1010
env/
11+
.venv/
1112
build/
1213
develop-eggs/
1314
dist/

docs/api/core.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ The Array class (``zarr.core``)
1010
.. automethod:: set_basic_selection
1111
.. automethod:: get_mask_selection
1212
.. automethod:: set_mask_selection
13+
.. automethod:: get_block_selection
14+
.. automethod:: set_block_selection
1315
.. automethod:: get_coordinate_selection
1416
.. automethod:: set_coordinate_selection
1517
.. automethod:: get_orthogonal_selection

docs/release.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,9 @@ Bug fixes
2323

2424
* Add ``__contains__`` method to ``KVStore``. By :user:`Christoph Gohlke <cgohlke>` :issue:`1454`.
2525

26+
* **Block Indexing**: Implemented blockwise (chunk blocks) indexing to ``zarr.Array``.
27+
By :user:`Altay Sansal <tasansal>` :issue:`1428`
28+
2629
.. _release_2.15.0:
2730

2831
2.15.0

docs/tutorial.rst

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -641,6 +641,84 @@ orthogonal indexing is also available directly on the array:
641641
>>> all(z.oindex[[0, 2], :] == z[[0, 2], :])
642642
True
643643

644+
Block Indexing
645+
~~~~~~~~~~~~~~
646+
647+
As of version 2.16.0, Zarr also support block indexing, which allows
648+
selections of whole chunks based on their logical indices along each dimension
649+
of an array. For example, this allows selecting a subset of chunk aligned rows and/or
650+
columns from a 2-dimensional array. E.g.::
651+
652+
>>> import zarr
653+
>>> import numpy as np
654+
>>> z = zarr.array(np.arange(100).reshape(10, 10), chunks=(3, 3))
655+
656+
Retrieve items by specifying their block coordinates::
657+
658+
>>> z.get_block_selection(1)
659+
array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
660+
[40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
661+
[50, 51, 52, 53, 54, 55, 56, 57, 58, 59]])
662+
663+
Equivalent slicing::
664+
665+
>>> z[3:6]
666+
array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
667+
[40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
668+
[50, 51, 52, 53, 54, 55, 56, 57, 58, 59]])
669+
670+
671+
For convenience, the block selection functionality is also available via the
672+
`blocks` property, e.g.::
673+
674+
>>> z.blocks[1]
675+
array([[30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
676+
[40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
677+
[50, 51, 52, 53, 54, 55, 56, 57, 58, 59]])
678+
679+
Block index arrays may be multidimensional to index multidimensional arrays.
680+
For example::
681+
682+
>>> z.blocks[0, 1:3]
683+
array([[ 3, 4, 5, 6, 7, 8],
684+
[13, 14, 15, 16, 17, 18],
685+
[23, 24, 25, 26, 27, 28]])
686+
687+
Data can also be modified. Let's start by a simple 2D array::
688+
689+
>>> import zarr
690+
>>> import numpy as np
691+
>>> z = zarr.zeros((6, 6), dtype=int, chunks=2)
692+
693+
Set data for a selection of items::
694+
695+
>>> z.set_block_selection((1, 0), 1)
696+
>>> z[...]
697+
array([[0, 0, 0, 0, 0, 0],
698+
[0, 0, 0, 0, 0, 0],
699+
[1, 1, 0, 0, 0, 0],
700+
[1, 1, 0, 0, 0, 0],
701+
[0, 0, 0, 0, 0, 0],
702+
[0, 0, 0, 0, 0, 0]])
703+
704+
For convenience, this functionality is also available via the ``blocks`` property.
705+
E.g.::
706+
707+
>>> z.blocks[:, 2] = 7
708+
>>> z[...]
709+
array([[0, 0, 0, 0, 7, 7],
710+
[0, 0, 0, 0, 7, 7],
711+
[1, 1, 0, 0, 7, 7],
712+
[1, 1, 0, 0, 7, 7],
713+
[0, 0, 0, 0, 7, 7],
714+
[0, 0, 0, 0, 7, 7]])
715+
716+
Any combination of integer and slice can be used for block indexing::
717+
718+
>>> z.blocks[2, 1:3]
719+
array([[0, 0, 7, 7],
720+
[0, 0, 7, 7]])
721+
644722
Indexing fields in structured arrays
645723
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
646724

0 commit comments

Comments
 (0)
0