8000 np.asarray - unexpected behaviour · Issue #14221 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

np.asarray - unexpected behaviour #14221

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
deviganesan-coder opened this issue Aug 7, 2019 · 10 comments
Closed

np.asarray - unexpected behaviour #14221

deviganesan-coder opened this issue Aug 7, 2019 · 10 comments
Labels
00 - Bug component: numpy._core

Comments

@deviganesan-coder
Copy link
deviganesan-coder commented Aug 7, 2019 •
edited by seberg
Loading

When I try to do np.asarray of a list of arrays, it results in different behaviour depending on the dimensions of the arrays. When the list of arrays is of dimensions like (n,n) and (n,) it results in error.

Reproducing code example:

import numpy as np
a1 = [array([[-0.00491985,  0.00491965],
        [-0.00334969,  0.00334955],
        [-0.00136081,  0.00136076]], dtype=float32),
 array([-0.00104678,  0.00104674], dtype=float32)]

a2 = [array([[-0.00334969,  0.00334955],
        [-0.00136081,  0.00136076]], dtype=float32),
 array([-0.00104678,  0.00104674], dtype=float32)]

print(np.asarray(a1))   #works fine
print(np.asarray(a2))  #throws error
print(np.asarray([a.reshape(-1, 2) for a in a2]))  #works fine

Error message:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-51-3060768e9016> in <module>()
----> 1 np.asarray(a2)

/home/devi/.local/lib/python3.5/site-packages/numpy/core/numeric.py in asarray(a, dtype, order)
    536 
    537     """
--> 538     return array(a, dtype, copy=False, order=order)
    539 
    540 

ValueError: could not broadcast input array from shape (2,2) into shape (2)

Numpy/Python version information:

('1.16.3', '2.7.12 (default, Nov 12 2018, 14:36:49) \n[GCC 5.4.0 20160609]')
@zjpoh
Copy link
Member
zjpoh commented Aug 30, 2019

Another example that raises the same error.

np.asarray([np.ones((n, m)), np.ones((n))])

I suspect the error is because when you have a list of arrays which have the same first dimension of the shape, then the np.asarray is trying to make it to a single array instead of an array of array.

Similarly error also occurs for multi-dimensional arrays.

I can take a look at this if nobody else is working on it.

@eric-wieser
Copy link
Member

I can take a look at this if nobody else is working on it

Feel free too, but I'll warn you that this has almost certainly come up before, and is rather tricky to fix.

@zjpoh
Copy link
Member
zjpoh commented Aug 30, 2019

😱I'll take a look and see if I can figure out something

@leonardbinet
Copy link
leonardbinet commented Sep 1, 2019 •
edited
Loading

I also have unexpected behavior with csr matrices:

import numpy as np
from scipy.sparse import csr_matrix

a = np.array([[1, 0, 2, 2], [1, 4, 2, 4]])
m = csr_matrix(a)

m.shape
Out: (2, 4)

m.todense()
Out:
matrix([[1, 0, 2, 2],
        [1, 4, 2, 4]])

m_2 = np.asarray(m)
# weird shape
m_2.shape
Out: ()

# whereas it seems to be conserved somehow
m_2
Out:
array(<2x4 sparse matrix of type '<type 'numpy.int64'>'
	with 7 stored elements in Compressed Sparse Row format>, dtype=object)


m_2.todense()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-12-3ca00cd97e3f> in <module>()
----> 1 m_2.todense()

AttributeError: 'numpy.ndarray' object has no attribute 'todense'

versions:
1.17.1 3.7.2 (default, Feb 12 2019, 08:16:38)
[Clang 10.0.0 (clang-1000.11.45.5)]

 python: 3.7.2 
 scipy: 1.3.1

@leonardbinet leonardbinet mentioned this issue Sep 1, 2019
Merged
@zjpoh
Copy link
Member
zjpoh commented Sep 11, 2019

I'm driving into this. Do you know what should be the expected behavior? Should it return an array of arrays? Or should it reshape and return a single np arrays.

@rgommers
Copy link
Member

The sparse matrix thing is a different issue.

Do you know what should be the expected behavior? Should it return an array of arrays? Or should it reshape and return a single np arrays.

In [21]: np.asarray([np.ones((5, 3)), np.ones((4))])                               
Out[21]: 
array([array([[1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.]]),
       array([1., 1., 1., 1.])], dtype=object)

In [22]: np.asarray([np.ones((5, 3)), np.ones((3))])                               
Out[22]: 
array([array([[1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.]]),
       array([1., 1., 1.])], dtype=object)

In [23]: np.asarray([np.ones((3, 5)), np.ones((3))])                               
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-23-596edbc3970a> in <module>
----> 1 np.asarray([np.ones((3, 5)), np.ones((3))])

~/anaconda3/lib/python3.7/site-packages/numpy/core/numeric.py in asarray(a, dtype, order)
    536 
    537     """
--> 538     return array(a, dtype, copy=False, order=order)
    539 
    540 

ValueError: could not broadcast input array from shape (3,5) into shape (3)

The consistent thing for this particular issue would be to not get the ValueError but make it return the same object-array-of-arrays as the other case.

Of course such behavior is probably still not what the user intended ...... if we'd want to change it, that should probably raise an exception. I don't really know, it's tricky.

@WarrenWeckesser
Copy link
Member

FYI: The somewhat surprising automatic conversion of the asarray arguments to an object array when the arguments have non-conformable shapes was discussed in #5303.

@zjpoh
Copy link
Member
zjpoh commented Sep 11, 2019

@rgommers Thanks. I agree on your point on consistency. But I also agree that it is probably not the intended behavior and making it consistent might hide the problem.

Yeap. The sparse matrix should be a separate issue. Let me move that over.

@WarrenWeckesser Thanks for pointing that out. This is a really tricky issue.

@zjpoh zjpoh mentioned this issue Sep 11, 2019
Closed
@eric-wieser
Copy link
Member

@zaraasghar, your issue is completely unrelated, and looks like a user error. I'd recommend asking on stack overflow.

@seberg
Copy link
Member
seberg commented Nov 19, 2022

All of the examples now correctly throw an error.

@seberg seberg closed this as completed Nov 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug component: numpy._core
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@seberg @rgommers @WarrenWeckesser @eric-wieser @leonardbinet @zjpoh @deviganesan-coder
0