8000 view does not work on indexed structured arrays · Issue #13765 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

view does not work on indexed structured arrays #13765

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
degrootc opened this issue Jun 12, 2019 · 3 comments
Open

view does not work on indexed structured arrays #13765

degrootc opened this issue Jun 12, 2019 · 3 comments
Labels
00 - Bug 50 - Duplicate

Comments

@degrootc
Copy link

When attempting to create a view of a indexed structured array, in order to create a a standard ndarray, the view method does not appropriately resize the output array. It appears to instead preserve the size of the full structured array.

I have two examples in the code:

  1. Uses the prescribed methodology from https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.view.html
  2. Uses column stack to combine each column slice

As you can see, example #1 is the incorrect size and has garbage from memory in the middle column. Example #2 works as expected.

Reproducing code example:

import numpy as np
test_arr = np.array([(1, 2, 3), (4, 5, 6), (7, 8, 9), (10, 11, 12)],
                    dtype=[('a', np.int8), ('b', np.int8), ('c', np.int8)])

temp_arr = (test_arr[['a', 'c']]
            .copy()
            .view(np.int8)
            .reshape(test_arr.shape + (-1,)))

print('using view:\n', temp_arr)

temp_arr = None
for col_name in ['a', 'c']:
    temp_field_arr = test_arr[col_name]

    if temp_arr is None:
        temp_arr = temp_field_arr.copy()
    else:
        temp_arr = np.column_stack((temp_arr,
                                    temp_field_arr.copy()))

print('using column_stack:\n', temp_arr)

Numpy/Python version information:

python 3.7.3
numpy 1.16.4

import sys, numpy; print(numpy.version, sys.version)
1.16.4 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0]
@seberg
Copy link
Member
seberg commented Jun 12, 2019

I think this is a duplicate of the discussion in gh-13299, in that you expect the copy removes the unnecessary padding (and thus view to work). Although the truth is pretty complex with respect to view itself, since sometimes we may not want to remove all padding.

View has to work with the raw itemsize really probably, since otherwise it would need to absorb the dtype shape somehow? I could think about adding a warning if it views previously inaccessible data, but not sure that is a solution either.

@degrootc
Copy link
Author

I agree that this is similar. However, if you remove the copy you still get unexpected behavior. The view will return the full contents of the structured array if copy is removed. The behavior is unexpected since the array index occurs first in precedence.

I will also add that this seems to be a recent change to the behavior of numpy. I need to track down the exact version when this changed but the older version of numpy had correct behavior.

@seberg
Copy link
Member
seberg commented Jun 12, 2019

@degrootc no need to track it down, the view semantics of indexing were relatively recently changed to do exactly this. This fixes some other issues with assignment. The copy discussion is mostly around the fact that the need for a logic which removes the padding is increased by that behaviour change. And the logic place to do remove it would probably be within copy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug 50 - Duplicate
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@seberg @degrootc
0