@@ -758,7 +758,7 @@ databases. The :class:`zarr.storage.RedisStore` class interfaces `Redis <https:/
758
758
(an in memory data structure store), and the :class: `zarr.storage.MongoDB ` class interfaces
759
759
with `MongoDB <https://www.mongodb.com/ >`_ (an object oriented NoSQL database). These stores
760
760
respectively require the `redis-py <https://redis-py.readthedocs.io >`_ and
761
- `pymongo <https://api.mongodb.com/python/current/ >`_ packages to be installed.
761
+ `pymongo <https://api.mongodb.com/python/current/ >`_ packages to be installed.
762
762
763
763
For compatibility with the `N5 <https://github.com/saalfeldlab/n5 >`_ data format, Zarr also provides
764
764
an N5 backend (this is currently an experimental feature). Similar to the zip storage class, an
@@ -897,6 +897,18 @@ The second invocation here will be much faster. Note that the ``storage_options`
897
897
have become more complex here, to account for the two parts of the supplied
898
898
URL.
899
899
900
+ It is also possible to initialize the filesytem outside of Zarr and then pass
901
+ it through. This requires creating an :class: `zarr.storage.FSStore ` object
902
+ explicitly. For example::
903
+
904
+ >>> import s3fs * doctest: +SKIP
905
+ >>> fs = s3fs.S3FileSystem(anon=True) # doctest: +SKIP
906
+ >>> store = zarr.storage.FSStore('/zarr-demo/store', fs=fs) # doctest: +SKIP
907
+ >>> g = zarr.open_group(store) # doctest: +SKIP
908
+
909
+ This is useful in cases where you want to also use the same fsspec filesystem object
910
+ separately from Zarr.
911
+
900
912
.. _fsspec : https://filesystem-spec.readthedocs.io/en/latest/
901
913
902
914
.. _supported by fsspec : https://filesystem-spec.readthedocs.io/en/latest/api.html#built-in-implementations
@@ -1306,18 +1318,18 @@ filters (e.g., byte-shuffle) have been applied.
1306
1318
1307
1319
Empty chunks
1308
1320
~~~~~~~~~~~~
1309
-
1321
+
1310
1322
As of version 2.11, it is possible to configure how Zarr handles the storage of
1311
1323
chunks that are "empty" (i.e., every element in the chunk is equal to the array's fill value).
1312
- When creating an array with ``write_empty_chunks=False ``,
1324
+ When creating an array with ``write_empty_chunks=False ``,
1313
1325
Zarr will check whether a chunk is empty before compression and storage. If a chunk is empty,
1314
- then Zarr does not store it, and instead deletes the chunk from storage
1315
- if the chunk had been previously stored.
1326
+ then Zarr does not store it, and instead deletes the chunk from storage
1327
+ if the chunk had been previously stored.
1316
1328
1317
- This optimization prevents storing redundant objects and can speed up reads, but the cost is
1318
- added computation during array writes, since the contents of
1319
- each chunk must be compared to the fill value, and these advantages are contingent on the content of the array.
1320
- If you know that your data will form chunks that are almost always non-empty, then there is no advantage to the optimization described above.
1329
+ This optimization prevents storing redundant objects and can speed up reads, but the cost is
1330
+ added computation during array writes, since the contents of
1331
+ each chunk must be compared to the fill value, and these advantages are contingent on the content of the array.
1332
+ If you know that your data will form chunks that are almost always non-empty, then there is no advantage to the optimization described above.
1321
1333
In this case, creating an array with ``write_empty_chunks=True `` (the default) will instruct Zarr to write every chunk without checking for emptiness.
1322
1334
1323
1335
The following example illustrates the effect of the ``write_empty_chunks `` flag on
@@ -1329,7 +1341,7 @@ the time required to write an array with different values.::
1329
1341
>>> from tempfile import TemporaryDirectory
1330
1342
>>> def timed_write(write_empty_chunks):
1331
1343
... """
1332
- ... Measure the time required and number of objects created when writing
1344
+ ... Measure the time required and number of objects created when writing
1333
1345
... to a Zarr array with random ints or fill value.
1334
1346
... """
1335
1347
... chunks = (8192,)
@@ -1368,8 +1380,8 @@ the time required to write an array with different values.::
1368
1380
Random Data: 0.1359s, 1024 objects stored
1369
1381
Empty Data: 0.0301s, 0 objects stored
1370
1382
1371
- In this example, writing random data is slightly slower with ``write_empty_chunks=True ``,
1372
- but writing empty data is substantially faster and generates far fewer objects in storage.
1383
+ In this example, writing random data is slightly slower with ``write_empty_chunks=True ``,
1384
+ but writing empty data is substantially faster and generates far fewer objects in storage.
1373
1385
1374
1386
.. _tutorial_rechunking :
1375
1387
0 commit comments