10000 BUG: broadcast_to throws errors when adding length=1 dimensions to the end of an array · Issue #28182 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

BUG: broadcast_to throws errors when adding length=1 dimensions to the end of an array #28182

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub 8000

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ben1sheff opened this issue Jan 17, 2025 · 4 comments
Closed

BUG: broadcast_to throws errors when adding length=1 dimensions to the end of an array #28182

ben1sheff opened this issue Jan 17, 2025 · 4 comments
Labels
33 - Question Question about NumPy usage or development 57 - Close? Issues which may be closable unless discussion continued

Comments

@ben1sheff
Copy link
ben1sheff commented Jan 17, 2025 •
edited by WarrenWeckesser
Loading

Describe the issue:

Broadcast rules seem to indicate you can add dimensions of length one basically anywhere in an array shape. In particular, for adding dimensions to the end of the shape, the following work fine

import numpy as np
test = np.array([2, 3])
print(test[:, None])
print(np.expand_dims(test, axis=1)

and each approach gives an array that looks like [[2], [3]], as expected. I wanted to do this with broadcast_to for more generic shapes, and it causes errors

Reproduce the code example:

import numpy as np
test = np.array([2, 3])
np.broadcast_to(test, (2, 1))

Error message:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/mamba/mambaforge/lib/python3.11/site-packages/numpy/lib/_stride_tricks_impl.py", line 410, in broadcast_to
    return _broadcast_to(array, shape, subok=subok, readonly=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/mamba/mambaforge/lib/python3.11/site-packages/numpy/lib/_stride_tricks_impl.py", line 349, in _broadcast_to
    it = np.nditer(
         ^^^^^^^^^^
ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (2,)  and requested shape (2,1)

Python and NumPy Versions:

2.2.0
3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0]

Runtime Environment:

[{'numpy_version': '2.2.0',
'python': '3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) '
'[GCC 12.3.0]',
'uname': uname_result(system='Linux', node='Luthien', release='5.15.167.4-microsoft-standard-WSL2', version='#1 SMP Tue Nov 5 00:21:55 UTC 2024', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_KNM',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL',
'AVX512_SPR']}},
{'architecture': 'Zen',
'filepath': /path/to/mamba/mambaforge/lib/libopenblasp-r0.3.27.so',
'internal_api': 'openblas',
'num_threads': 6,
'prefix': 'libopenblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.27'}]

Context for the issue:

I wanted to multiply an array across its first one or two dimensions, broadcasting out along the other dimensions as usual, but automatic broadcasting wants the same number of dimensions in each array before multiplying. I could manage this with transpositions, but broadcast_to should be able to natively handle this.
@seberg seberg added 33 - Question Question about NumPy usage or development and removed 00 - Bug labels Jan 18, 2025
@mhvk
Copy link
Contributor
mhvk commented Jan 18, 2025

Thanks for reporting. This is not a bug: the idea of broadcasting is that one can emulate what would happen automatically in things like a + b, where the arguments are also brought to a common shape. But in broadcasting, one never adds dimensions at the end or in the middle. What would work is np.reshape(a, (2, 1)), but not sure this will necessarily generalize correctly for you.

@seberg
Copy link
Member
seberg commented Jan 21, 2025

I am not sure what you want. Adding dimensions that aren't at the start is certainly nothing broadcast_to does. Broadcasting with another operand is also not the same as broadcasting to a specific shape: What braodcast_to does is well defined.

What I could imagine is allowing np.broadcast_to(arr, (3, None)) or np.broadcast_to(arr, (3, -1)) to indicate that one dimension is considered broadcastable (and thus irrelevant).

but automatic broadcasting wants the same number of dimensions in each array before multiplying

As Marten, I read that as not asking for what I said above, but rather asking for an operation that is unrelated to broadcasting.
Maybe you are even just looking for np.multiply.outer, I am not sure.

@seberg seberg added the 57 - Close? Issues which may be closable unless discussion continued label Jan 21, 2025
@mhvk
Copy link
Contributor
mhvk commented Jan 21, 2025

@ben1sheff - I realized that there is perhaps an easy example to show that it your suggestion is difficult to implement: for your array, with shape (2,), what should np.broadcast(test, (2, 2)), do? Right now it is well-defined, since there is only one choice.

@ben1sheff
Copy link
Author

@mhvk I think your initial message answered my concern, and reshape actually does work fine here. Your and @seberg's comments make sense for why this isn't something broadcasting should do. In principle, there wouldn't be ambiguity for adding irrelevant, broadcastable dimensions, but it wouldn't generalize well as mentioned in the last comment, and isn't really what broadcast_to is for.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
33 - Question Question about NumPy usage or development 57 - Close? Issues which may be closable unless discussion continued
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@seberg @mhvk @ben1sheff
0