8000 np.load() "invalid shape in fixed-type tuple" in NumPy 1.16.0 · Issue #13431 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

np.load() "invalid shape in fixed-type tuple" in NumPy 1.16.0 #13431

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jzwinck opened this issue Apr 30, 2019 · 1 comment · Fixed by #13433
Closed

np.load() "invalid shape in fixed-type tuple" in NumPy 1.16.0 #13431

jzwinck opened this issue Apr 30, 2019 · 1 comment · Fixed by #13433
Labels
06 - Regression

Comments

@jzwinck
Copy link
Contributor
jzwinck commented Apr 30, 2019

In NumPy 1.16.0, np.load() fails to load some files created by np.save(), saying "ValueError: invalid shape in fixed-type tuple."

The problem seems to be that #12358 changed np.lib.format._read_array_header(). It used to pass the descr to np.dtype(), but now it uses a new function descr_to_dtype() which fails for some valid inputs.

We have significant amounts of data stored on disk with formats which NumPy 1.16.0 cannot load, so this is a showstopper for upgrading beyond 1.15.

Reproducing code example:

This fails in NumPy 1.16.0, but works in all prior versions:

import numpy as np
a = np.zeros(1, np.dtype([('c', ('<f8', (5,)), (2,))]))
np.save('a.npy', a)
np.load('a.npy')

Error message:

Traceback (most recent call last):
  File "numpy/lib/npyio.py", line 440, in load
    pickle_kwargs=pickle_kwargs)
  File "numpy/lib/format.py", line 678, in read_array
    shape, fortran_order, dtype = _read_array_header(fp, version)
  File "numpy/lib/format.py", line 563, in _read_array_header
    dtype = descr_to_dtype(d['descr'])
  File "numpy/lib/format.py", line 282, in descr_to_dtype
    dt = numpy.dtype((descr_to_dtype(descr_str), shape))
  File "numpy/lib/format.py", line 282, in descr_to_dtype
    dt = numpy.dtype((descr_to_dtype(descr_str), shape))
ValueError: invalid shape in fixed-type tuple.

Numpy/Python version information:

NumPy 1.16.0
Python 3.6.6
@mattip
Copy link
Member
mattip commented Apr 30, 2019

It seems these two dtypes are equivalent, we only handled the second in the refactoring

np.zeros(1, dtype=[('c', ('<f8', (5,)), (2,))])
np.zeros(1, dtype=[('c', '<f8', (2, 5))])

@mattip mattip mentioned this issue Apr 30, 2019
Merged
@charris charris added this to the 1.16.4 release milestone Apr 30, 2019
@charris charris mentioned this issue May 14, 2019
Merged
@kousaa kousaa mentioned this issue Nov 28, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
06 - Regression
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

BUG: Handle subarrays in descr_to_dtype mattip/numpy
3 participants
@charris @mattip @jzwinck
0