Compatibility policy for zarr-python v3 #2097
Replies: 1 comment 2 replies
-
Short answer, if we wanted to preserve code-level compatibility, we would be targeting a 2.x release and not a 3.x release. The longer answer is that supporting zarr version 2 and zarr version 3 and preserving code compatibility with zarr-python v2 (which closely models zarr version 2) is not an easy path. So far, zarr-python 3.x has optimized for supporting zarr version 2 and zarr version 3, at the cost of changing zarr-python 2.x APIs. I think these are the right priorities (but others might disagree). Keep in mind that zarr v3 is a different format from zarr v2. Expecting old APIs to work for a new format seems like a stretch. That doesn't mean we should wantonly break stuff, and we are open to a variety of solutions for minimizing the changes. The simplest thing is for users who depend on zarr-python 2.x APIs to pin their dependencies to that version of zarr-python. More complicated solutions could be including all of zarr-python 2.x inside a v2 namespace in zarr-python 3.x, or publishing zarr-python 2.x as a stand-alone |
Beta Was this translation helpful? Give feedback.
-
|
It's an interesting thought exercise for what has a longer tail: software or data files? :) Overall I think I agree: Prioritize APIs that make sense for Zarr v3, but don't wantonly break stuff. To make this a bit more concrete I published #2098 as a draft of what it takes to get |
Beta Was this translation helpful? Give feedback.
-
|
I have created #2102 to help address some of these questions. Contributions / feedback on that PR is welcome. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Apologies if this has been discussed already, but is there policy written down anywhere with respect to code compatibility written against zarr 2.x? Concretely, I wonder if making this snippet using xarray is a design goal of zarr-python v3?
That runs fine with zarr 2.18.2; Should it work with zarr-python 3.x without modification to xarray? To give you an idea of the kinds of changes needed to make this work:
storeis passed as a positional arg, while it's keyword only on v3 (scikit-learn had a nice decorator for deprecating positional arguments, so if we wantstoreto be keyword only we can deprecate it gracefully without too much effort)Attributes.put(dict). Easy to alias to.update(with a deprecation if we want)Groupobject (store, read_only, synchronizer). No idea if these still make sense in the v3 codebase.Group.createpreviously took some keyword arguments (shape, dtype, fill_value, chunks) that seem more like Array arguments. Reading throughzarr.v2.Group.create, I guess that creates an Array in the group, rather than a new group. Is that deliberate?There's some additional examples of this in some issues (#2023, #2021)
One point worth discussing: is there value in code-level compatibility if zarr-python 3.x is (presumably) writing zarr-v3 by default? IMO yes, but perhaps not.
Beta Was this translation helpful? Give feedback.
All reactions