8000 np.load() "invalid shape in fixed-type tuple" in NumPy 1.16.0 · Issue #13431 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

np.load() "invalid shape in fixed-type tuple" in NumPy 1.16.0 #13431

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jzwinck opened this issue Apr 30, 2019 · 1 comment · Fixed by #13433
Closed

np.load() "invalid shape in fixed-type tuple" in NumPy 1.16.0 #13431

jzwinck opened this issue Apr 30, 2019 · 1 comment · Fixed by #13433

Comments

@jzwinck
Copy link
Contributor
jzwinck commented Apr 30, 2019

In NumPy 1.16.0, np.load() fails to load some files created by np.save(), saying "ValueError: invalid shape in fixed-type tuple."

The problem seems to be that #12358 changed np.lib.format._read_array_header(). It used to pass the descr to np.dtype(), but now it uses a new function descr_to_dtype() which fails for some valid inputs.

We have significant amounts of data stored on disk with formats which NumPy 1.16.0 cannot load, so this is a showstopper for upgrading beyond 1.15.

Reproducing code example:

This fails in NumPy 1.16.0, but works in all prior versions:

import numpy as np
a = np.zeros(1, np.dtype([('c', ('<f8', (5,)), (2,))]))
np.save('a.npy', a)
np.load('a.npy')

Error message:

Traceback (most recent call last):
  File "numpy/lib/npyio.py", line 440, in load
    pickle_kwargs=pickle_kwargs)
  File "numpy/lib/format.py", line 678, in read_array
    shape, fortran_order, dtype = _read_array_header(fp, version)
  File "numpy/lib/format.py", line 563, in _read_array_header
    dtype = descr_to_dtype(d['descr'])
  File "numpy/lib/format.py", line 282, in descr_to_dtype
    dt = numpy.dtype((descr_to_dtype(descr_str), shape))
  File "numpy/lib/format.py", line 282, in descr_to_dtype
    dt = numpy.dtype((descr_to_dtype(descr_str), shape))
ValueError: invalid shape in fixed-type tuple.

Numpy/Python version information:

NumPy 1.16.0
Python 3.6.6

@mattip
Copy link
Member
mattip commented Apr 30, 2019

It seems these two dtypes are equivalent, we only handled the second in the refactoring

np.zeros(1, dtype=[('c', ('<f8', (5,)), (2,))])
np.zeros(1, dtype=[('c', '<f8', (2, 5))])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants
0