8000 add Empty chunks section to tutorial.rst · jni/zarr-python@5091df0 · GitHub
[go: up one dir, main page]

Skip to content

Commit 5091df0

Browse files
committed
add Empty chunks section to tutorial.rst
1 parent a042c8a commit 5091df0

File tree

1 file changed

+19
-0
lines changed

1 file changed

+19
-0
lines changed

docs/tutorial.rst

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1302,6 +1302,25 @@ bytes within chunks of an array may improve the compression ratio, depending on
13021302
the structure of the data, the compression algorithm used, and which compression
13031303
filters (e.g., byte-shuffle) have been applied.
13041304

1305+
.. _tutorial_chunks_empty_chunks:
1306+
1307+
Empty chunks
1308+
~~~~~~~~~~~~
1309+
1310+
As of version 2.11, it is possible to configure how Zarr handles the storage of
1311+
chunks that are "empty" (i.e., every element in the chunk is equal to the array's fill value).
1312+
When creating an array with ``write_empty_chunks=False`` (the default),
1313+
Zarr will check whether a chunk is empty before compression and storage. If a chunk is empty,
1314+
then Zarr does not store it, and instead deletes the chunk from storage
1315+
if the chunk had been previously stored.
1316+
1317+
This optimization prevents storing redundant objects and can speed up reads, but the cost is
1318+
added computation during array writes, since the contents of
1319+
each chunk must be compared to the fill value, and these advantages are contingent on the content of the array.
1320+
If you know that your data will form chunks that are almost always non-empty, then there is no advantage to the optimization described above.
1321+
In this case, creating an array with ``write_empty_chunks=True`` will instruct Zarr to write every chunk without checking for emptiness.
1322+
1323+
13051324
.. _tutorial_rechunking:
13061325

13071326
Changing chunk shapes (rechunking)

0 commit comments

Comments
 (0)
0