8000 [REGRESSION] to_netcdf doesn't accept dtype=S1 encoding anymore · Issue #2149 · pydata/xarray · GitHub
[go: up one dir, main page]
Skip to content

[REGRESSION] to_netcdf doesn't accept dtype=S1 encoding anymore #2149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
crusaderky opened this issue May 17, 2018 · 5 comments · Fixed by #2158
Closed

[REGRESSION] to_netcdf doesn't accept dtype=S1 encoding anymore #2149

crusaderky opened this issue May 17, 2018 · 5 comments · Fixed by #2158

Comments

@crusaderky
Copy link
Contributor

In xarray 0.10.4, the dtype encoding in to_netcdf has stopped working, for all engines:

>>> import xarray
>>> ds = xarray.Dataset({'x': ['foo', 'bar', 'baz']})
>>> ds.to_netcdf('test.nc', encoding={'x': {'dtype': 'S1'}})
[...]

xarray/backends/netCDF4_.py in _extract_nc4_variable_encoding(variable, raise_on_invalid, lsd_okay, h5py_okay, backend, unlimited_dims)
    196         if invalid:
    197             raise ValueError('unexpected encoding parameters for %r backend: '
--> 198                              ' %r' % (backend, invalid))
    199     else:
    200         for k in list(encoding):

ValueError: unexpected encoding parameters for 'netCDF4' backend:  ['dtype']

I'm still trying to figure out how the regression tests didn't pick it up and what change introduced it.

@shoyer I'm working on this as my top priority. Do you agree this is serious enough for an emergency re-release? (0.10.4.1 or 0.10.5, your choice)
@crusaderky crusaderky changed the title [REGRESSION] to_netcdf doesn't accept dtype encoding anymore [REGRESSION] to_netcdf doesn't accept dtype=S1 encoding anymore May 17, 2018
@shoyer
Copy link
Member
shoyer commented May 17, 2018 via email

@crusaderky
Copy link
Contributor Author
  1. this is limited to dtype=S1. There's auto tests implemented for dtype=f4 and dtype=int16 (https://github.com/pydata/xarray/blob/master/xarray/tests/test_backends.py#L729-L756), and those work fine.
ds = xarray.Dataset({'x': [1, 2]})
ds.to_netcdf('test.nc', encoding={'x': {'dtype': 'f4'}})
  1. the bug can be worked around by setting the dtype on the variable directly - for which there is an automated test specific for S1 (https://github.com/pydata/xarray/blob/master/xarray/tests/test_backends.py#L333-L343):
ds = xarray.Dataset({'x': ['foo', 'bar', 'baz']})
ds['x'].encoding['dtype'] = 'S1'
ds.to_netcdf('test.nc')

@crusaderky crusaderky mentioned this issue May 17, 2018
Closed
@crusaderky
Copy link
Contributor Author
crusaderky commented May 17, 2018 •
edited
Loading

@shoyer found the culprit: 3c8935e
Still need to understand why if you set the encoding directly on the variable it works though.

@crusaderky
Copy link
Contributor Author

@shoyer I'm struggling to understand your change - help would be appreciated.

@shoyer
Copy link
Member
shoyer commented May 17, 2018

I just got off an airplane (without internet), but I'll try to look into this shortly.

I guess I shouldn't be surprised that 3c8935e was the source of this bug -- it was pretty big refactor :(.

@shoyer shoyer mentioned this issue May 18, 2018
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@shoyer @crusaderky
0