8000 BUG: `numpy.array` brings different results with `dask.array.array` · Issue #28924 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

BUG: numpy.array brings different results with dask.array.array #28924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
apiqwe opened this issue May 8, 2025 · 3 comments
Closed

BUG: numpy.array brings different results with dask.array.array #28924

apiqwe opened this issue May 8, 2025 · 3 comments
Labels
00 - Bug

Comments

@apiqwe
Copy link
apiqwe commented May 8, 2025

Describe the issue:

I found that numpy.array will bring different results with dask.array.array as follow.
I wonder whether numpy's results are correct?

Reproduce the code example:

import numpy as np
import dask.array as da

print(np.array([(1,2),(3,4)],dtype=[('x','i4'),('y','i4')]))
print(da.array([(1,2),(3,4)],dtype=[('x','i4'),('y','i4')]).compute())

Error message:

[(1, 2) (3, 4)]
[[(1, 1) (2, 2)]
 [(3, 3) (4, 4)]]

Python and NumPy Versions:

numpy 2.2.4
python 3.10.0 (default, Mar 3 2022, 09:58:08) [GCC 7.5.0]

Runtime Environment:

[{'numpy_version': '2.2.4',
'python': '3.10.0 (default, Mar 3 2022, 09:58:08) [GCC 7.5.0]',
'uname': uname_result(system='Linux', node='rog', release='6.11.0-25-generic', version='#25~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 17:20:50 UTC 2', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2',
'AVX512F',
'AVX512CD',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL'],
'not_found': ['AVX512_KNL', 'AVX512_KNM']}},
{'architecture': 'SkylakeX',
'filepath': '/home/xxx/anaconda3/envs/dask/lib/python3.10/site-packages/numpy.libs/libscipy_openblas64_-6bb31eeb.so',
'internal_api': 'openblas',
'num_threads': 16,
'prefix': 'libscipy_openblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.28'}]

Context for the issue:

No response
@apiqwe apiqwe added the 00 - Bug label May 8, 2025
@seberg
Copy link
Member
seberg commented May 9, 2025

Yes, the NumPy results are correct, tuples are considered scalars in this context if the dtype is a structured dtype.

Is that simple? No, but it is what it is and I believe always has been. So closing, can reopen if you have an argument why this should be considered wrong/should be changed.

@seberg seberg closed this as not planned Won't fix, can't repro, duplicate, stale May 9, 2025
@apiqwe apiqwe mentioned this issue May 9, 2025
Open
@fjetter
Copy link
fjetter commented May 9, 2025 •
edited
Loading

The problem we're stumbling over is rather the difference in behavior between using dtype in the constructor vs casting an array using astype (see also dask/dask#11936 (comment))

>>> import numpy as np
>>> np.array([(1,2),(3,4)], dtype=[('x', '<i4'), ('y', '<i4')])
array([(1, 2), (3, 4)], dtype=[('x', '<i4'), ('y', '<i4')])

>>> np.array([(1,2),(3,4)]).astype([('x','i4'),('y','i4')])
array([[(1, 1), (2, 2)],
       [(3, 3), (4, 4)]], dtype=[('x', '<i4'), ('y', '<i4')])

is this expected?

@seberg
Copy link
Member
seberg commented May 9, 2025

Yes, it's expected unfortunately. The reason is that, asarray(..., dtype=dtype) is sensitive to what the dtype is.
I.e. tuples are scalars, but effectively only if the dtype is a structured dtype.

So once you do asarray(...) without dtype first, then the tuple will be unpacked and after that the cast could fail I guess (because it is a bit strange maybe), but it works per-element which means broadcasting to all structured dtype elements.

So basically, the shape of [(1, 2), (2, 3)] is not fully defined before you know the dtype, and I don't think there is something to do about it really, unless you want to make it hard to initialize structured arrays from lists...

(PS: happy to re-open, the issue. I am less trigger happy if there is a context given ;))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug
Projects
None yet
Milestone
No milestone
336F
Development

No branches or pull requests
3 participants
@seberg @fjetter @apiqwe
0